Beat softened butter and sugar with a heavy-duty stand mixer fitted with a paddle attachment on medium speed until creamy, about 2 minutes. Add vanilla, and beat until just blended. Stir together flour, cocoa, and salt in a bowl; gradually add to butter mixture, beating on low speed until just combined, about 1 minute.

Divide dough in half; pat each half into a 5-inch disk. Wrap each disk in plastic wrap; chill 2 hours.

Preheat oven to 325°F. Unwrap 1 dough disk; place on a lightly floured work surface. Roll disk to 1/4-inch thickness. Using a 2 1/2-inch heart-shaped cutter, cut 12 cookies from dough, rerolling scraps as needed. Using a 1-inch heart-shaped cutter, cut holes in centers of 6 of the cookies (leaving remaining 6 cookies whole).

Arrange the 6 whole cookies on a baking sheet lined with parchment paper. Arrange the 6 cut-out cookies on a separate baking sheet lined with parchment. Bake in preheated oven until bottoms are slightly browned, 9 to 11 minutes (cut-out cookies will bake a little quicker). Cool on baking sheets 2 minutes; transfer to wire racks to cool completely, 20 minutes. (You can bake the unused centers from the cut-out hearts 6 to 8 minutes.)

Meanwhile, repeat Steps 3 and 4 using remaining dough disk.

Divide marshmallow crème evenly between 2 small bowls. Tint crème in 1 bowl with 2 drops of pink food coloring, and stir to incorporate. Leave crème in second bowl untinted. Spread pink crème evenly among the centers of 6 of the whole cookies; spread untinted crème over remaining 6 whole cookies. Place crème-covered cookies on a baking sheet; decorate with sprinkles or sanding sugar as desired. Top crème-covered cookies with cut-out cookies to make 12 cookie sandwiches.