The adjoining lane probably took its name from the Inn, rather than the other way around.
A possible explination for the name may be that it was owned by the Tynte Family of Halswell House. In 1689 Sir Halswell Tynte starting building Halswell House and they owned a number of properties about the town.
Landlords included Thomas Shattock (c.1780), Robert Washer (1886-1894) and Charles Sparks (1938-1961).
The inn closed in 1961 and the building pulled down soon afterwards.
Sydney Gardnor Jarman, A History of Bridgwater, (London, 1889)
David Williams, Bridgwater Inns Past and Present, (Abbey Press, 2nd edn. 1997)