Phelan, a member of the Democratic Party, worked as a banker before becoming mayor of San Francisco (1897-1902). He also served as president of the Relief and Red Cross Funds after the 1906 earthquake.
Phelan was elected to the United States Senate as was chairman of the Committee on Railroads and Committee on Irrigation and Reclamation of Arid Lands during his time in Congress (1915-1921).
After leaving the Senate he was chairman of the United Bank & Trust Company. James Phelan died in Saratoga, California, on 7th August, 1930.