A building recorded on Strachey's map of the 1730s and Locke's suvey of 1777 is
labelled in both as 'castle'.
This structure was a manor house constructed by Henry Harvey, a successful lawyer, in the 1630s.
Three illustrations survive of this building, a watercolour from 1761:
And two images by John Chubb made sometime around 1800:
All three reveal the structure in different stages of ruin, and all are depicted from the West.
The 1761 watercolour reveals what was a two-turreted building, although one of these had evidently collapsed or been demolished.
The third image, Chubb's second, shows the structure after a wall and the remaining turret have been removed, revealing and early and very high quality
gothic door within the building confirming that this manor house was an adapted castle building.
From these maps and images a reconstruction of the East facade of Harvey's house can be attempted.
Reconstruction of the Caroline Manor House
John Strachey remarked in the 1730s that this building, referred to as 'the castle' in his map,
was only about a hundred years old. However, the evidence from the images suggests that the house has incorporated
features of an earlier castle building.
Further evidence for this comes from the historian Collinson in the eighteenth century,
who asserted that Henry Harvey had converted his mansion from the old castle gatehouse, which was 'in the form of a Roman B', in 1638.
The date Collinson suggests is suspicious, being the date given in civil war documents relating to Harvey's other house by the river,
although the detail that the house was converted from a gatehouse seems to be plausible.
Airs' study of Elizabethan and Jacobean manor houses highlights the piecemeal approach
of construction and the gradual renewal of older buildings, rather than total replacement;
even if the older medieval structure was demolished the foundations would often be reused.
Conversion is likely to have been the case with this two turreted manor house, as it is a very unusual shape for a house of this time.
Two drawbridges are recorded for Bridgwater Castle. Rather confusingly these are described as an
'inner bridge facing the tower within the ditch' [which]
led to a second drawbridge and the inner bailey.'
The main gate and bridge was almost certainly on the Cornhill. The gatehouse Harvey converted into a manor house
was probably fronted on the King Square side by the second drawbridge and led into the Inner Bailey on the Castle Street side.
Its sheer size might suggest that it was something akin to the great thirteenth-century 'Keep-Gatehouses' of Caerphilly Castle.
Place, C., 'Castle Street, Bridgwater, Somerset' (2009) in the Somerset Historic Environment Record p.23 
Collinson, A History of Somerset, Vol III (London, 1791) p.77 
Airs, M., The Tudor and Jacobean Country House, (Sutton, Stroud, 1995) pp.25-26 
Dunning, R., 'Bridgwater Castle' and 'Otterhampton' in A History of the County of Somerset: Vol. 6 (Victoria County History, London. 1992) pp.206-207