The real reasons the spoon-licking Duke of Hastings, played by Regé-Jean Page, left Bridgerton (don't worry, he's back on our screens soon)