How to Make It
Make Cookies: Cream together butter and sugar until light and fluffy, 2-3 minutes.
In a medium sized bowl, whisk together flour, baking powder, and salt. Set aside.
To the butter mixture, add the egg, milk, vanilla extract, and almond extract. Scrape down sides of the bowl, and beat on medium-high until fully incorporated, about 1 minute.
Scrape down sides of the bowl again, and add dry ingredients. Mix on low until flour is fully incorporated, about 1 minute. Wrap and refrigerate dough for 1 hour.
Preheat oven to 350°F. Roll dough out to 1/4-inch thickness. Using an egg shaped cookie cutter, cut out cookie dough, re-rolling scraps to cut out more cookies. Place on a parchment lined baking sheet and bake until just beginning to take on color, about 9 minutes. Remove to cooling rack for 1 hour.
Make Royal Icing: Combine egg whites, powdered sugar, and vanilla in a stand mixer fitted with the whip attachment, and whip on medium-high until glossy, about 5 minutes. Add water or flavored waters until the consistency is slightly thinner than toothpaste (when you lift the whip attachment, the icing should ribbon back into the bowl and slowly dissolve into at the rest of the icing). Divide royal icing between individual small bowls, one for each food coloring you have selected, and set aside 1/2 cup of white icing in an airtight container. Press plastic wrap onto the surface of the white icing and place in the refrigerator. To the bowls, add the desired amount of food coloring and stir until well combined.
Decorate Cookies: Place each colored icing into a piping bag and cut a small opening in the end of the bag. Pipe 1/8 inch away from the edge of the cookie and pipe a line around the perimeter. Carefully fill in the oval with more icing until it is one solid oval of icing. Repeat with other cookies and other colors. Allow to sit for 12 hours or overnight uncovered.
Place white royal icing in a piping bag, and pipe festive decorations across the face of each cookie. Allow to dry for 1 hour before serving.