Little is known about the life of Alexander Gardner, and even less is known about his younger brother, James, including the date of his death. Like Alexander, James was born in Scotland and immigrated to the United States with the rest of the clan in 1850. They settled initially in Iowa but then moved east to New York City. Both brothers worked for Mathew Brady. James left with Alexander when he split away to found his own firm in 1863.