The house is the seat of Lord Saye and Sele, having been in the Fiennes family since the reign of Henry VII.
Fiennes married (1), Elizabeth, daughter of the famous parliamentarian Sir John Eliot, by whom he had one son, afterwards 3rd Viscount Saye and Sele; and (2), Frances, daughter of Richard Whitehead of Tuderley, Hants, by whom he had three daughters.
He was ordained and became vicar of Fawsley in 1637, but soon resigned and became chaplain successively to Lord Saye and Sele, Lord Berkeley, and Prince Charles Louis, nephew of Charles I.
He now showed that he had not by his charities wronged his relations by settling on his greatnephew and heir Thomas Wykeham, whom he had educated at Winchester and New College, Broughton Castle and estates, still held by his descendants in the female line, the family of Twisleton-Wykeham-Fiennes (peerage of Saye and Sele).
About £88,000 was thus raised, and in 1622 William Fiennes, 1st Viscount Saye and Sele, was imprisoned for six months for protesting.