A strong centre-half, Allen became an important member of the team that finished in 9th place in the First Division of the 1932-33 season. Portsmouth also reached the 1934 FA Cup Final. However, Manchester City won the game 2-1.
Allen won his first international cap for England against Northern Ireland on 14th October 1933. The England team that day also included Eddie Hapgood, Wilf Copping, Cliff Bastin and Eric Brook. England won the game 3-0. He also played in the 2-1 loss to Wales on 15th November 1933.
Aston Villa was relegated from the First Division at the end of the 1935-36 season. However, the following season Allen won a Second Division championship-winning medal.
During the Second World War Allen guested for Chelsea, Birmingham City, Southampton, Portsmouth and Fulham. He retired from professional football in May 1944 after suffering a serious injury. He has scored 3 goals in 160 games for Aston Villa.
James Allen died in Southsea in February 1995.