Diana's brother Charles Spencer turned down a request from The Crown team

Season 4 of The Crown imagines the moment Prince Charles first met a teenage Diana Spencer at her family's ancestral home, Althorp.

However, the hit Netflix drama had to find an alternative venue for the scenes after the late princess' younger brother Charles Spencer knocked back a filming request.

RELATED: Everything members of the royal family have said about The Crown

Charles, the ninth Earl Spencer, has revealed he was approached by The Crown team about filming at Althorp House, of which he is custodian.

"I said, 'Obviously not'," he recalled during an appearance on Love Your Weekend with Alan Titchmarsh.

For a daily dose of 9Honey, sign up here to receive our top stories straight to your inbox

Part of Earl Spencer's reasoning was a desire to honour Princess Diana's memory, something he feels is his "role".

"I feel it is my duty to stand up for her when I can ... We grew up together, you know if you grow up with somebody they are still that person, it doesn't matter what happens to them later," he revealed.

He is also concerned people who watch The Crown overwhelmingly "forget that it is fiction", rather than an imagining of what goes on behind palace walls.

"They assume, especially foreigners — I find Americans tell me they have watched The Crown as if they have taken a history lesson. Well, they haven't," he said.

WATCH: 9Honey speaks to The Crown's Prince Charles and Princess Diana. (Post continues.)

"There is a lot of conjecture and a lot of invention, isn't there? You can hang it on fact, but the bits in between are not fact."

Althorp House was built in 1508 and has been in the Spencer family for 19 generations. Located in Northamptonshire, the estate covers 13,000 acres of countryside.

RELATED: Charles Spencer determined to pursue inquiry into Panorama interview

For the scenes set at Althorp — namely Charles encountering a 16-year-old Diana, dressed in a tree costume, while he was visiting her older sister, Sarah — The Crown crew used another stately home, Ragley Hall.

The property, which is the ancestral seat of the Hertford family, is located in Warwickshire and sits on thousands of acres of ground.

Ragley Hall isn't the only historic home in the UK to score a role in the Netflix series.

As the British royals have no involvement in the show's production, its creators have used other properties as stand-ins for various royal palaces and castles. You can see them here.