Add the brownie mix, cream cheese, and 4 tablespoons milk to a mixing bowl. Beat until smooth with a hand mixer. Add more milk 1 Tbsp at a time until the mixture is your desired consistency. (You want to be able to dip graham crackers into it without them breaking.)
Transfer the brownie batter to a cast-iron skillet or oven-safe baking dish and spread it into an even layer.
Sprinkle the graham crackers crumbs over the brownie batter. Then arrange the marshmallows on top of the graham crackers covering the surface of the dip entirely.
Broil the marshmallows until golden brown (or more if you like them kind of charred).
Remove the dip from the oven. Serve immediately with graham crackers and sugar wafers for dipping while the marshmallows are still gooey.
Notes
If you want your dip to be fluffy: Mix one 8-ounce tub of thawed Cool Whip into the brownie batter mixture before spreading it into the skillet or baking dish. This will also increase the yield to about 24 servings.