Harlaxton Manor passed through the hands of numerous influential families. Constructed by Edmund de Swynford in the 14th Century, it served as a hunting lodge for John of Gaunt, the Duke of Lancaster and son of King Henry III. Much later in 1619, Daniel de Ligne, a Flemish refugee later knighted by James I, purchased the Manor. Sir Daniel de Ligne’s great, great granddaughter, Ann Orton, inherited the Manor after marrying George Gregory, thus beginning the line of Gregory Gregory. After the decline of Swynford’s Harlaxton Manor, Gregory Gregory would build the current Harlaxton Manor house, likely in part as an unsuccessful attempt at gaining a title from the king.
George Gregory, a lawyer for the Harlaxton family, married Ann Orton, historically considered to be the great, great granddaughter of Daniel de Ligne. Claiming she was the newly discovered heir, Gregory became Lord of the Manor.
The family resided mostly at Rempstone Hall, their secondary residence, and in a holiday apartment in London, leaving the original Manor desolate. Landed gentry families often amassed multiple estates through marriage and inheritance.
George de Ligne Gregory, the first son of George Gregory, built Hungerton Hall. In his will, George de Ligne Gregory left the Harlaxton and Nottingham estates to his three brothers. William, the next oldest, inherited Rempstone and Denton and changed his name to William Gregory Williams.
Gregory Gregory Williams, son of William Gregory Williams, inherited the estates. Changing his name to Gregory Gregory, he sold off plots of land in Lenton and Radford to help fund the new Harlaxton Manor. Gregory Gregory visited Bramshill, Hardwick, Hatfield, Knole, Burghley, Wollaton, Kirby, Longleat, and Temple Newsham for inspiration for his new country house.