In a large mixing bowl, whisk together the sugar, cornstarch, and egg yolks until well combined. Whisk in 1 cup of the milk to the sugar mixture.

Place 2 cups of milk into a medium saucepan and bring it up to a boil.

Slowly whisk the hot milk into the sugar mixture. Transfer the mixture back into the pot and cook over medium heat until it thickens enough to coat the back of a spoon, stirring the whole time. Remove the pot from the stove, stir in the butter, and half the vanilla. Allow the pudding to cool.

Spread 1/3 of the pudding into the bottom of a 1 1/2 quart casserole pan or an 8x8 pan. Add a layer of vanilla wafers and a layer of sliced bananas. Repeat this another time, ending with an extra layer of pudding.

In a medium mixing bowl, beat the heavy cream, powdered sugar, and the remaining vanilla until you have medium peaks.

Spread the whipped cream over the top of the pudding and sprinkle the top with some crushed wafers and banana slices.