Line a standard muffin pan with paper baking cups.
In a large bowl, whisk together the gluten-free flour, sugar, baking powder, baking soda, salt and poppy seeds.
In a medium bowl, whisk together the milk, butter, oil, eggs, sour cream, lemon zest, lemon juice, vanilla extract, lemon extract and almond extract.
Add the wet ingredients to the dry ingredients and stir with a rubber spatula until just combined.
Using a kitchen scoop or large spoon, portion the batter evenly into the prepared muffin pan.
Refrigerate the muffins for 20 minutes before baking. Preheat the oven to 425°F.
Bake the muffins for 10 minutes at 425°F, then reduce the oven temperature to 375°F and continue to bake for 10-15 minutes, until a toothpick inserted into the center comes out clean.
Let the muffins cool in the pan for 5 minutes, then transfer to a wire rack to cool completely.
For the glaze: In a small bowl, mix together the powdered sugar and lemon juice until smooth. Drizzle or dip each muffin top with the glaze. Let the glaze set before serving.