Place yeast in a large bowl. Add water and honey; let stand until foamy, 5 to 10 minutes. Add milk, kosher salt, and 1/4 cup of the oil to yeast mixture; stir to combine. Add flour; stir with a wooden spoon until no dry spots remain and a shaggy dough forms. (You may need to knead gently with your hands to help the dough stick together.) Cover bowl with a clean dish towel or plastic wrap; set aside in a warm place free from drafts until doubled in size, about 1 hour.

Preheat oven to 425°F with oven rack in middle of oven. Place 1/4 cup of the oil in a rimmed baking sheet. Turn dough out onto prepared pan; turn to coat dough in oil. Using your fingertips, gently spread dough toward edges of pan. (Dough will not reach all the way to edges.) Let stand 15 minutes.

Using your fingertips, press dough gently toward edges of pan. (Dough should almost fit.) Sprinkle dough evenly with rosemary leaves and flaky sea salt. Bake in preheated oven until golden brown, 18 to 20 minutes. Transfer focaccia to a wire rack; brush with remaining 2 tablespoons oil. Cut into squares, or tear into pieces. Serve with Stewed Grapes and Olives.