Preheat the oven to 325°F. Butter and lightly flour two 8-inch round cake pans; line the bottoms with lightly buttered parchment.
Place the walnuts and half the breadcrumbs in the food processor and pulse until walnuts are finely ground. The breadcrumbs prevent the nuts from turning into a paste.
Using an electric hand-held mixer, beat the egg yolks in a large bowl until pale and thick.
Slowly add the sugar and continue to beat until well blended. With a sturdy spoon, add the walnuts, breadcrumbs, baking powder and 1 teaspoon vanilla. Mix well - it will be as thick as mud!
Beat egg whites until stiff but not dry. Lighten the walnut batter by gently mixing in a third of the beaten whites. Carefully fold in the remaining whites, divide among the two cake pans, and bake for 30 minutes, or until a cake tester comes out clean. Let cakes cool in the pans for 10 minutes, then turn out onto racks.
When fully cool, use a long serrated knife to cut each of the two layers in half horizontally. Whisk the jam to loosen, and divide between the bottom half of each cake, spreading evenly. Replace the top half of each cake. Beat crème fraîche, 2 tablespoons of confectioner's sugar, and remaining 1 teaspoon vanilla until very thick. Spread all of the cream on top of one of the cake layers, then gently place remaining cake layer gently on top so that the cream does not squish out. Cover and chill for at least an hour before serving. Dust the top with confectioner's sugar just before serving.