Halfpenny was born on 9 October 1748 at Bishopthorpe in Yorkshire, where his father was gardener to the Archbishop of York. He was apprenticed to a house-painter, and practised house-painting in York for some years. He afterwards raised himself to the position of an artist and a teacher of drawing. He acted as clerk of the works to John Carr the architect who was restoring York Minster, and repaired some of its old decoration.

Halfpenny was twice married, and was survived by two daughters, Margaret and Charlotte. He died at his house in Gillygate, York, on 11 July 1811, and was buried in the churchyard of St Olave’s Church.