Allan had developed an interest in football while at university but discovered that rugby was the predominant winter team sport in the North-East. As Roger Hutchinson points out in Into the Light (1999) "Allan uncovered a group of other teachers in the area who shared his interest in righting this wrong, and at a meeting in Norfolk Street in the October of 1879 the Sunderland and District Teachers' Association Football Club was formed."
The team originally played at Blue House Field that was close to the Hendon Board School where James Allan was employed. The captain was Robert Singleton, the headmaster of Gray School in Sunderland. Other school teachers in the side included John Graystone and Walter Chappell.
Allan and his friends rented the ground at Hendon for £10 a year. They also had to pay the travelling cost of taking a team to away games throughout the North-East. At a meeting in October 1880 they discussed the possibility of closing the club. However, it was eventually decided to raise the money by opening it up to non-teaching members. As a result the club changed its name to Sunderland Association Football Club.
As the author of Sunderland: The Official History points out: "The club was formed not by shipbuilders or miners, but by school teachers, local school master James Allan having taken the initiative in organising such a venture. More surprisingly still, the teachers not only formed the club, but made up the entire team too, and the club's original name - Sunderland and District Teachers' Association Football Club - reflected this."
James Allan was a talented centre-forward with a good goalscoring record. In 1883 Allan's goals helped Sunderland reach the final of the Northumberland & Durham Cup. The following year the club played Darlington in the first final of the Durham Cup. Sunderland won 4-0 but Darlington complained that the 2,000 fans were guilty of intimidating their players. The Football Association ordered the final to be replayed and sent Major Francis Marindin to referee the game. This time Sunderland won the game 2-0.
In 1884 Allan was one of the six Sunderland players selected for the Durham County side which played Northumberland.
On 20th December, 1885, Sunderland beat Castletown 23-0 in the first round of the Durham Association Challenge Cup. James Allan, who played on the left-wing, scored 12 of the goals. This remains a Sunderland club record.
In an attempt to improve the team Sunderland began recruiting players from Scotland. The Football Association decided that professionalism should be allowed, but with restrictions. Any professionals had either to have been born in the town they represented, or to have lived within six miles of their club's headquarters for the previous two years.
In 1887 Sunderland beat Middlesbrough 4-2 in an early round of the FA Cup. Middlesbrough protested that three of Sunderland's players (Monaghan, Hastings and Richardson) were living in Scotland and was lodged at the Royal Hotel at the club's expense. In January 1888, the Football Association examined the Sunderland books and discovered "a payment of thirty shillings in the cash book to Hastings, Monaghan and Richardson for train fares from Dumfries to Sunderland". Sunderland was kicked out of the FA Cup and ordered to pay the expenses of the inquiry. The three players concerned were each suspended from football in England for three months.
Allan, who disliked the idea of recruiting professional players, resigned from the club. On 13th March 1888 he organized a meeting at the Empress Hotel. The meeting decided to create Sunderland Albion. Allan managed to persuade six first-team players from Sunderland to join him.
Sunderland and Sunderland Albion now became bitter rivals. The two clubs were drawn together in the 1888-89 FA Cup. However, Sunderland withdrew rather than allow Albion to benefit from the increased gate receipts.
Allan ensured that Sunderland Albion were founder members of the Football Alliance in 1889. Sunderland joined the Football League in 1890. Albion responded by joining the Northern League but the success of the Football League meant that they had difficulty keeping the loyalty of their supporters. When Sunderland won the Football League title in 1892 Albion accepted defeat and closed down.