Henry John Fice Lee was born in Yealmpton in 1861 where he became a baker. Later, he moved to Ivybridge with his wife Amelia. Together they had two sons, Sydney Herbert and Henry John, and a daughter, Caroline who married Naval Officer Lt. William Frank Smallbone in 1918. He later became Commander Smallbone and they lived at Uplands on Exeter Road in Ivybridge. Henry and Amelia Lee lived initially at Clare Street and later, following the success of business, at Greenwood on Western Road. Apparently, he had told his wife he would one day own a mill to supply his bakery business.
During the mid-1890s Henry Lee took his first step in fulfilling his aspirations. Samuel Head, the proprietor of a tan yard in the centre of Ivybridge advertised a corn mill ‘To be Let’. Although a date is not documented it is known that Mr Lee took up the lease of the Union Mills, a former cooperative mill which ceased operating in 1893. The Devonport Union Mill Society, a wholesale bread cooperative from Plymouth, had purchased the old manorial corn mills at Ivybridge way back in 1821 and had operated them with great success during this period. Mr Lee appeared to be a very successful businessman and soon needed to expand. In 1901 he publicised in the newspaper that he was ‘taking into partnership Mr. James Scoble in the Baking and Confectionery Department’. Mr Scoble resided at 5 Western Road whilst Henry Lee stated that his corn and forage business would be carried out under his personal supervision at the Union Mills in Fore Street.
At this time, the neighbouring paper mill, operated by the Holman family was falling on hard times. With poor trading conditions, largely the result of cheap imported paper, the mill was put on the market in 1903. At the auction, which took place at The London Hotel, the successful bidder was none other than Henry J.F. Lee. Over the next few years he converted the premises into a corn and provender mill. He modernised the building suitable for milling and provided a new facade facing Fore Street. A plaque on the exterior read ‘H. J. F. Lee Ivybridge Mills 1905’. With continuing success, he extended his operations at Yealmpton where he founded Lee Bros with his brother Samuel Fice Lee.