Descended from Ralph Stafford (1354-1410), the younger son of Sir John Stafford of Hooke and Margaret Stafford, daughter of the 1^st Earl of Stafford, the Grafton Manor was obtained by Ralph via marriage to Maud Hastings. He was Steward of the Household to Edward III.

On his death in 1410 the title passed to his son Humphrey (1370-1419) who fought for Henry V at Agincourt in 1415. Humphrey was succeeded by his son John who inherited Upton Warren Manor by marriage.

John Stafford died in 1422 without issue and his possessions were inherited by his brother Sir Humphrey (1400 -1450) which were further enhanced in1426 by Sir Humphrey’s marriage to Alianora de Aylesbury who brought with her Blatherwyke Manor, Milton Keynes. They had three sons Richard, Humphrey (1427-1486) and Thomas.

In 1436, Sir Humphrey held the office of Royal Commissioner and the Governor of Calais.

In May 1448, Sir Humphrey, his eldest son Richard and a number of his retainers were making for their Inn at Coventry. On the road they met Sir Robert Harcourt and his attendants. Sir Humphrey and Harcourt, a supporter of the Duke of Suffolk and the King’s favourite, had long been embroiled in a personal feud and they immediately fell to blows.

Harcourt struck Richard on the head with his sword, not seriously, for the young man made at Harcourt with his dagger. He stumbled and one of Harcourt's men fatally stabbed him in the back. Sir Humphrey was also struck from behind and fell from his horse. A general melee ensued in which the Stafford retainers killed two of Harcourt's men.

The following day the city coroner charged Sir Robert with murder and he was detained in Chester Castle. However as a direct consequence of pressure applied to the local Sheriff by Suffolk under a writ of the Privy Seal, he was soon released and had still not been brought to trial a year later.

Tired of waiting for satisfaction by judicial means, the Sir Humphrey assembled some “two hundred friends and tenants” and marched overnight to Stanton Harcourt in Oxfordshire. Sir Robert had enough warning to make for the tower of the parish church to which Stafford’s laid siege to for more than six hours, loosing off over a thousand arrows and killing one Harcourt retainer. Despite threatening to set fire to the tower Harcourt was fortunate and managed to hold out until Sir Humphrey was forced to withdraw as he was unequipped for a long siege.

Harcourt was killed in 1469 by Sir Humphrey’s second son, Humphrey (1427-1486).

In 1450 dissatisfaction with the King and his Government was rife and the men of Kent rose up in rebellion under the leadership of Jack Cade claiming that the King’s system of tax collection was unjust and extortive. By June some 20,000 rebels had gathered at Blackheath, south of London to petition the King

The King replied two days later by commanding the gathering to disperse. The King then marched his army out of London to give battle to the rebels only to find their camp deserted.

The King then foolishly divided his army ordering half, under the command of Sir Humphrey Stafford and his brother William to hunt down the rebels who by now had now fallen back to Sevenoaks. The road into the town narrowed through a wood and soon the Royal army found it’s self caught in a bottleneck. Rebels, concealed on either side of the road, sprung their trap, firing volleys of arrows into the stranded men followed up by a fierce charge in which both Sir Humphrey Stafford and his brother were killed.

Hearing the news, the King panicked and withdrew from London, leaving the city undefended; Cade saw his opportunity and swiftly marched on London making camp at the White Hart Inn, Southwark. Fearing the consequences if they resisted the Lord Mayor allowed Cade to cross London Bridge unopposed and as the gates were opened he rode through 'like a lordly captain' wearing armour and spurs taken from the body of Sir Humphrey Stafford. Despite Cade’s assurances, the rebels looted the city and several prominent citizens were executed before calm was restored and the rebels withdrew to Southwark for the night. At 10.00pm, Cade again attempted to enter the city but as they crossed the bridge they were attacked by a force led by Lord Scales and comprising the Tower Garrison, well-to-do citizens and a group of archers, recently returned from France. Fierce fighting quickly erupted and as the rebels were pushed back they also came under cannon fire from the Tower. In desperation, Cade set fire to the houses near the drawbridge and as the flames swept across the bridge the two sides were forced apart. The following morning protracted negotiation took place which culminating in Cade being duped into accepting the 'Charters of Pardon'. Cade was subsequently outlawed, betrayed and killed on 12 July 1450.

The Grafton Estates now passed to the second son, Sir Humphrey (1427-1486)

In 1483 Sir Humphrey helped suppress the rebellion of his cousin, the Duke of Buckingham by blockading the immediate exits across the Upper Severn by destroying the bridges. Prevented from raising rebellion the Duke’s forces began to disperse and he was forced to flee alone before being betrayed by one of his servants and executed on 2 November 1483.

When Henry Tudor invaded in 1485, Sir Humphrey again rallied to the King and along with his brother, Thomas, was present at the Battle of Bosworth on 22 August. With Tudor triumphant and the King dead, Sir Humphrey Stafford, his brother and Francis Lord Lovell and others fled the field and escaped to Sanctuary at St. John near Colchester in Essex.

In 1486 Sir Humphrey and Lovell came out of hiding in a failed attempt to raise the country against Henry VII in support of the imprisoned Earl of Warwick (infant son to the Duke of Clarence). With the rebellion crushed, Sir Humphrey again fled to Sanctuary, but shortly after was forcibly removed and subsequently executed for treason at Tyburn in 1486 and buried at Greyfriars' in Leicester.

The Grafton possessions were granted to Sir Edward Poynings in 1488 before transferring back to the Stafford family in 1521