Are Meghan Markle and Prince Harry leaving Kensington Palace? The royal couple didn’t waste any time starting a family after they tied the knot in a gorgeous royal wedding back in May. The couple is currently expecting their first child together, and since their little royal family is expanding, it would make sense that they would be interested in moving into a bigger place. Now, according to the Daily Mail, Meghan and Harry might leave their home in Kensington Palace in order to find a bigger place to raise their growing family.

According to columnist Richard Eden, royal insiders expected that Queen Elizabeth’s cousin, the Duke of Gloucester, 74, and his wife, Birgitte, 72, would move out of their lavish 21-room apartment in the castle so that Meghan and Harry can move in right next to Prince William and Kate Middleton. But there’s just one issue — Birgitte does not want to budge.

“Birgitte doesn’t want to leave,” one of her friends told Eden. “And why should she? She and her husband [Prince Richard] are still full-time working members of The Firm and they were given their apartment by Her Majesty.”

She kind of does have a point. It seems rude to kick Birgitte and Prince Richard out of their home to make for a new royal couple. But if there isn’t another royal property large enough to suit Meghan and Prince Harry’s taste — they currently live in the two-bedroom Nottingham Cottage on the palace grounds — insiders said that the Duke and Duchess of Sussex might have to leave Kensington Palace altogether.

Neil Mockford/GC Images

“Harry and Meghan want to move and need more space, but they don’t want to live next door to William and Catherine,” a source said. “They will soon have separate offices and separate lives, so it would make sense for them to live further afield. There is no reason why their London home has to be at Kensington Palace. There are plenty of other options, including using their own money to buy their own place.”