In a medium saucepan, mix together the sugar, flour, and salt. Gradually whisk in the milk and bring the mixture to a boil over medium heat, stirring constantly.
Once it starts to bubble, cook for an additional 2 minutes until it thickens.
In a small bowl, whisk the egg yolks. Slowly add a few tablespoons of the hot milk mixture into the egg yolks to temper them, then whisk the yolks back into the saucepan.
Cook for another 2-3 minutes until the pudding is thickened, then remove from heat.
Stir in the vanilla extract and butter until fully melted and combined. Allow the pudding to cool slightly.
In a large serving dish, layer half of the vanilla wafers, followed by half of the banana slices, and then half of the pudding. Repeat the layers.
Chill the assembled pudding in the refrigerator for at least 4 hours, or overnight for the best results.
Before serving, top with a generous layer of whipped cream and sprinkle some crushed vanilla wafers on top for added texture.