While cake is baking prepare pudding mix. Put milk in a 1 quart size bowl, and add pudding mix. Beat with a wire whisk or electric mixer for two minutes. Refrigerate.
Cool baked cake rounds by putting them each on a plate lined with wax or parchment paper and putting them in the freezer for 30 to 45 min.
Making sure cake is very cold, arrange 1/2 of the sliced bananas on top of bottom layer of cake. Spoon 1/2 of the pudding over the bananas. Add second layer cake and repeat. Cake can be frosted on the sides or not. Make sure no pudding has seeped out the sides of the cake before you frost.