Preheat oven to 180C. For potato and celeriac gratin, combine butter, garlic and thyme in a small saucepan and cook over low heat until melted and fragrant (3-4 minutes), season to taste and set aside. Trim potato and celeriac to form oval shapes of similar size, then thinly slice on a mandolin. Layer potato, celeriac and onion in a 20cm-diameter springform pan, overlapping slices, drizzling with butter and seasoning generously between layers. Place pan on a piece of aluminium foil and wrap to enclose base and sides of pan (this is to catch any leakage) and place on an oven tray lined with foil. Gradually add veal stock, then place on lowest rack of oven and cook until gratin is golden and vegetables are very tender (1½-1¾ hours). Stand for 20 minutes before removing sides of pan (excess liquid may drain out) and serve hot.
Meanwhile, beat butter in an electric mixer until pale and fluffy (5-6 minutes), add mustard, garlic and horseradish, season to taste and beat until well combined.
Pat veal dry with absorbent paper to remove any moisture, then rub mustard mixture thickly over. Place on a lightly oiled wire rack, then place directly into oven over gratin (juices will drip into gratin) and roast until cooked to your liking (35-40 minutes for medium-rare). Rest in a warm place for 20 minutes, press herbs into veal and serve hot with gratin.