The question is frequently asked: "When was 'the Abbey' built?" No documentary evidence is known for the date of the building of the manor house, and no drawing of it is known before Kip's 'View' of 1708. A small sketch of it is included in an estate survey of 1726, and there is a drawing in the Bodleian Library dated 1804. The name Wytham Abbey was not used until about 1850; previously it was called Wytham House. There has never been an abbey or any kind of religious house on this site. At best there is only a legend in the Abingdon Abbey Chronicle about some nuns at Wytham in the seventh or eighth centuries, and some think Wittenham was their abode, not Wytham at all.
Until the arrival of the 5th Earl the house was surrounded with a moat, and it had a large yard with outbuildings in front of it on the east side. It was built round two courtyards, with a turreted gatehouse leading into the southern and larger one. On the south side of this courtyard was the hall with dais and oriel window at the west end, and a lantern in the roof to let in light (probably also, originally, to let out smoke). The gatehouse and hall and chambers round the southern courtyard were very probably built about 1480 when Richard Harcourt obtained possession of the manor. The gatehouse is all that remains visible of his work. It has three storeys, a battlemented roof and an octagonal staircase turret on each side. There are three oriel windows. The buttresses are set at an angle like those of 'Pope's Tower', probably built by William Orchard (architect of Magdalen College) for Sir Robert Harcourt at Stanton Harcourt in 1450. In Lyson's Berkshire, 1806, it is stated that a Harcourt coat of arms was then to be seen on a ceiling, but its position is not given. The next owner, Sir John Williams, doubtless improved the house. He set glass in the great west window of the hall about 1550 with Tudor coats of arms, his own monogram. and ER for Edwardus Rex. Some of this remains (1965) in the windows of the south room on the ground floor. During the time of the Norrises, perhaps in the early seventeenth century, the 'domestic offices' were built round the north courtyard on their present scale. Although the house was partly refitted inside during the eighteenth century, the 1804 drawing, showing the house from the south-east angle of the moat, does not display any external changes.
The 5th Earl made considerable alterations. He pulled down the hall and built drawing rooms, etc., in a Georgian-gothic style. He covered the southern courtyard, and used part as an entrance hall, and part to house a staircase, so that rooms could be entered without going outside as was previously necessary. (It may be as well to state that a legend has grown up recently that this staircase came from Cumnor Place and is haunted by Amy Robsart. There is no foundation of fact for this legend.) The Earl continued his work in the layout of the grounds round the house. He filled in the moat, and took down the outbuildings and battlemented wall. The stepped gable from one of these outbuildings is at the south end of the present stable, facing the Park; and parts of the battlemented wall separate the churchyard from the grounds of the house. He completed his work by rebuilding the church, taking part of the churchyard into his grounds, and by making a new drive leading straight up to the gatehouse from the south end of the village. This drive ceased to be used after Hazel ffennell's death in 1939. It is now grassed over, and the east end of it has become part of the Dower House garden.
After this there were few further changes. The 6th Earl partly rebuilt the south side of the house, and extended the 'domestic offices'. The ffennell's made the arched porch which is now a striking feature of the south side. In recent years there have been internal re-arrangements as a result of the conversion of the building into flats. Externally it remains more or less as finished by the 5th Earl about 1812, and, perhaps because of his conservative taste, it is still in general appearance not unlike the seventeenth century gothic buildings of several Oxford colleges.
Wytham Abbey is now privately owned and is never open to the public.
[Text taken from "The Parish Of Wytham" by Margaret Sparks, written and sold for the benefit of Wytham Church]