Born in Birmingham, he moved to Suffolk just before the closure of the Mid-Suffolk Light Railway branch line from Haughley to Laxfield and it was this that is said to have inspired the novel Love on a Branch Line.
After leaving school Hadfield worked as an editor at the publishing firm J. M. Dent in London. During the Second World War he was a Book Officer for the British Council and formed a unit translating books into Arabic. After the war he founded the Cupid Press, which specialised in limited-edition anthologies of verse. In 1957 he published A Book of Britain, an anthology of words and pictures covering 500 years of art, articles and poems celebrating the best of British culture.