Preheat the oven to 375°F. Spray a 10x15-inch jelly roll pan with cooking spray.
In a large bowl, whisk together the gluten-free flour, baking soda, baking powder, salt, cinnamon and nutmeg.
In a small bowl or large measuring cup, combine the mashed banana, milk and vanilla extract.
Add the eggs, granulated sugar and brown sugar to the bowl of a stand or hand mixer. Mix on medium-high speed for 2-3 minutes, until fluffy and pale in color. With the mixer running on low, slowly drizzle in the oil. Mix until combined.
With the mixer running on low, add ½ of the dry ingredients. Mix until combined, then add ½ of the banana mixture. Repeat with the rest of the dry ingredients and banana mixture, mixing well after each addition. Scrape the bottom of the bowl to make sure everything is well-combined.
Pour the batter into the prepared pan and spread into an even layer.
Bake for 25-30 minutes, or until a toothpick inserted into the center of the bars comes out clean.
For the brown butter frosting: You want to frost the cake when it’s still warm, so start making the frosting as soon as the banana bars come out of the oven. The cake should cool for 10-15 minutes before adding the frosting.
Melt the butter in a medium skillet over medium-low heat, stirring occasionally.
Once the butter is melted and begins to foam, stir continuously to prevent the milk solids from sticking to the skillet. The butter will change from yellow to golden brown quickly and will have a nutty aroma.
Pour in the milk and bring to a simmer. Once the mixture begins to simmer, remove it from the heat and whisk in the powdered sugar, vanilla extract and a pinch of salt. Whisk until completely smooth.
Immediately pour the frosting over the warm cake. Smooth into an even layer. Sprinkle with walnuts, if desired. Cool completely before slicing and serving.