Bring a large saucepan of salted water to the boil over high heat, add quail eggs and cook for 3 minutes, then drain and cool under cold running water. Peel 12 quail eggs.
For pickling mixture, combine all ingredients plus 2 tsp sea salt and ¼ cup water in a large saucepan over medium heat and bring to the boil. Simmer for 7 minutes, reduce heat, add peeled eggs and simmer for 5 more minutes. Cool, then transfer to a sterilised jar. Refrigerate until ready to use.
Combine celery seed and 2 tsp salt in a mortar and, using a pestle, pound until coarsely ground. Transfer to a small bowl.
To serve, place pickled quail eggs in a bowl. Place remaining 12 quail eggs on a plate and serve with celery salt and buttered bread.