Preheat the oven to 350°F with a rack in the center. Line a standard muffin pan with paper liners.
Beat the softened cream cheese and ¼ cup granulated sugar until smooth. Mix in the flour, milk, and vanilla until combined. Transfer to a piping bag or zip-top bag and set aside while you prepare the batter.
In a medium bowl, whisk together the flour, baking powder, baking soda, pumpkin pie spice, cinnamon, and salt.
In a large bowl, whisk together the brown sugar, granulated sugar, pumpkin purée, eggs, oil, and vanilla until well combined.
Add the dry ingredients to the wet ingredients. Fold gently with a rubber spatula just until no dry pockets of flour remain. Do not overmix.
Divide the batter among the prepared liners, filling each about three-quarters full before adding the filling. If your pan’s cups are smaller, use additional lined wells in a second pan rather than overfilling.
Snip the end of the piping bag or a corner of the zip-top bag. Insert the opening just beneath the surface of the batter in the center of each muffin. Squeeze gently while lifting upward, leaving a visible dollop on top. Divide all the filling evenly among the muffins; avoid piping it near the bottom.
Bake for approximately 18–25 minutes, beginning to check at 18 minutes. The muffin tops should feel set, and a toothpick inserted into the pumpkin portion, away from the cream cheese, should come out clean or with a few moist crumbs—not wet batter. The filling should look set rather than liquid.
Let the muffins cool in the pan for 5 minutes, then carefully transfer them to a wire rack. Cool for at least 10 more minutes before serving; the filling will be hot straight from the oven.