Combine the milk and cereal in an air tight container. Cover and refrigerate for one hour. Strain.
Preheat your oven to 350 degrees. Line 1 12-cup muffin tin with paper liners and set aside.
Whisk together the flour, cornstarch, baking powder, and baking soda. Set aside.
Combine the butter and sugar in the bowl of an electric mixer. Beat on medium speed until light and fluffy. Add the eggs, one at a time, until fully incorporated. Stir in the vanilla and yogurt.
Stir in the flour and cereal milk, alternating between the two, until both are mixed into the batter.
Divide between your prepared cupcake liners, filling 3/4 full.
Bake 15-18 minutes or until a toothpick comes out clean. Allow to cool completely.
Whisk together the butter and powdered sugar with an electric mixer. Stir in the vanilla extract and 2-3 tablespoons of cereal milk until your desired consistency is reached. Spread onto your cooled cupcakes and sprinkle with marshmallows.