For the graham cracker crust: Preheat the oven to 350°F. Blend the graham crackers in a food processor until they turn into fine crumbs. Pour the crumbs into a large mixing bowl.
Add the brown sugar and gluten-free flour and mix until well-combined. Add the melted butter and stir until fully combined and all of the crumbs are moistened.
Press the crumbs firmly into the bottom and up the sides of a 9×1.5-inch pie pan. Bake for 8–10 minutes. Remove from the oven and set aside to cool for 10 minutes while you make the caramel.
For the caramel: In a heavy-bottomed medium saucepan over medium heat, combine the butter, brown sugar and salt. Whisk constantly until the butter is melted and everything is well-combined, then continue whisking until the mixture starts to boil.
Once the mixture reaches a boil, add the sweetened condensed milk, whisking constantly, and bring back to a boil.
Reduce the heat to a simmer and continue whisking. Cook for about 5 minutes, until the caramel turns a rich golden color, thickens and starts to pull away from the sides of the pan as you stir.
Remove from heat and stir in the vanilla.
Pour the caramel over the cooled crust and spread it evenly with a rubber spatula.
Refrigerate for 1 hour.
For the whipped cream topping: In the bowl of a stand mixer fitted with the whisk attachment, or using a hand mixer, beat together the heavy cream, powdered sugar and vanilla on medium-high speed until stiff peaks form, about 4–5 minutes.
Arrange the banana slices over the caramel in an even layer (or two layers if you love bananas). Spread the whipped cream on top.
Garnish with chocolate shavings and serve immediately.