James Wadsworth was born in New York in 1807. After graduating from Harvard University he studied law. An opponent of slavery, Wadsworth was member of the Free Soil Party before joining the Republican Party.

On the outbreak of the American Civil War Wadsworth joined the Union Army. He served under Major General Irvin McDowell at Bull Run (July, 1862) before being commissioned as a brigadier general in the Army of the Potomac. Wadsworth commanded the 2nd Brigade (October, 1861 to March, 1862) before being appointed as military governor of Washington.

Wadsworth returned to active service in January, 1863 and took part in the battle of Chancellorsville (May, 1863) and Gettysburg (July, 1863). He was leading the 4th Division, V Corps, when he was mortally wounded at the Wilderness. James Wadsworth died two days later on 8th May, 1864.