The Stone family emigrated from England in 1848. They sailed to America on the Sailor Prince, and arrived in New Orleans on 20 November 1848. His father appears to have died by 21 July 1850 because his wife, Hannah Bradley Stone, remarried in St. Louis, Missouri on that date. Joseph does not appear with his mother and siblings in the 1850 Missouri Census, but he does appear in the household of Ralph B. Douglass (his sister Jane's future husband) in the 1850 Utah Census.
His mother, Hannah Bradley Stone Bardsley, came to Utah with her husband John Bardsley in 1855.
Further research is needed to determine which company Joseph traveled to Utah with.