Preheat the oven to 350°F (175°C) and line a muffin tin with paper liners.
In a large bowl, whisk together the flour, granulated sugar, baking powder, salt, and ground cinnamon.
In another bowl, combine the melted butter, egg, milk, and vanilla extract. Mix well.
Pour the wet ingredients into the dry ingredients and stir until just combined. Do not overmix.
In a small bowl, mix together the brown sugar, ground cinnamon, and softened butter to create the cinnamon filling.
Spoon half of the muffin batter into the prepared muffin cups, filling each about halfway. Add a teaspoon of the cinnamon filling on top of the batter in each cup, then cover with the remaining muffin batter.
Bake for 18-20 minutes or until a toothpick inserted into the center comes out clean.
While the muffins cool, prepare the glaze by whisking together the powdered sugar, milk, and vanilla extract until smooth.
Drizzle the glaze over the cooled muffins before serving.