King Charles is kicking off the Christmas celebrations at his annual pre-Christmas Palace lunch, but not everyone will be in attendance.
PEOPLE understands that Kate Middleton, Prince William and their three children, Prince George, 11, Princess Charlotte, 9, and Prince Louis, 6, will not be present at this year's festivities.
The Prince and Princess of Wales were never expected to attend the lunch at Buckingham Palace on Thursday, Dec. 19. The couple and their children are already in Norfolk, England, more than 100 miles away from London, ahead of the royal family's private Christmas celebrations at the Sandringham Estate.
PEOPLE understands that William and Kate are looking forward to catching up with family at Sandringham over Christmas as usual.
While Kate and William may not have joined in the festivities, Charles, 76, still hosted several members of the royal family for Christmas lunch, including his younger brother Prince Edward and his wife Sophie, the Duchess of Edinburgh, who were seen arriving at Buckingham Palace on Thursday.
Charles' brother Prince Andrew and Sarah Ferguson canceled their appearance at the Christmas luncheon in the wake of backlash around the Duke of York's suspected connection to an alleged Chinese "spy."
PEOPLE understands that the Duke of York, 64, reached the decision after discussing the matter with his ex-wife, the Duchess of York, who convinced him that this was the right move. The same is true of Prince Andrew's decision to skip the royal family's Christmas gathering at Sandringham next week on Dec. 25.
Royal biographer Ingrid Seward tells PEOPLE that Andrew will have also spoken things over with his daughters, Princess Beatrice and Princess Eugenie, who are protective and loyal towards him.
"Despite [the Duchess'] missteps in the past, she does know how things will play out socially. She would tell him it’s best to stay away. And Beatrice would have backed her up in that," says Seward.
Prince Andrew and Ferguson, who is popularly known as Fergie, divorced in 1996 after a decade of marriage, but remain close today and continue to live at Royal Lodge in Windsor.
The royals' Christmas lunch is a private family gathering, and a tradition King Charles has continued from his late mother Queen Elizabeth's reign. Dozens of family members convene about a week before Christmas for the upbeat holiday meal at Buckingham Palace in London. From there, the late Queen would head to Sandringham House, her private estate in the Norfolk countryside, and step out for church at St. Mary Magdalene on Christmas Day with her family in tow.
The royal family's Christmas plans have been under unusual scrutiny this year amid fallout around Andrew's alleged links to the "spy" scandal. Charles' younger brother has been under harsh spotlight since Dec. 13, when it emerged that he reportedly had close professional ties to a Chinese national, alleged to be a spy. The story made headlines when a British court upheld an existing decision to ban the man from the U.K. on national security grounds following a special immigration investigation, drawing attention also to his reported relationship with Andrew.
Looking ahead to the royal family's Christmas plans on Dec. 25, PEOPLE understands that Prince Andrew, Fergie, Princess Beatrice, Princess Eugenie, and their respective families will not join King Charles at Sandringham because of the spy scandal. Though Andrew hasn't held a working royal role for years, he continues to join his family for major ceremonial events and holiday celebrations.
The York family had been widely expected to stay at Wood Farm on the Sandringham estate for Christmas like they did last year, but the plans have apparently pivoted.
PEOPLE understands that Beatrice, 36, and Eugenie, 34, had invitations to spend Christmas with their respective in-laws some time ago and have decided to do so. Their attendance at Sandringham had been in question, an insider says, while their parents will likely keep a low profile at Royal Lodge, their home on grounds of Windsor Castle.