He was born on July 16, 1781, in Derryfield, New Hampshire, which is now Manchester. His brother baptized and confirmed him on January 9, 1832. He was also ordained an elder. Later, on June 3, 1833, in Kirtland, Ohio, he was ordained a high priest by Lyman Wight. Smith served a mission to the eastern states in 1836 and on September 3, 1837, he was sustained as assistant counselor to the First Presidency, which he held until the death of the Prophet Joseph Smith in 1844. In January 10, 1844, Smith was ordained a patriarch by Joseph Smith while he lived and presided over the Saints in Macedonia, Illinois. He was also appointed a member of the Council of Fifty by May 3, 1844.
During his lifetime, he and his family lived in Far West, Adam-ondi-Ahman, and Nauvoo. When the Saints were expelled, he traveled to Winter Quarters and then on to the Great Salt Lake Valley by September 23, 1847.
He married Clarissa Lyman in 1815 and they were the parents of four children. He eventually practiced plural marriage. He was the great-grandfather of George Albert Smith.