Peter Kyle was born in Glasgow and his early football was played in Scotland. He had a trial with Clyde but was rejected because of his fierce temper. He was also rejected by Hearts but in 1899 he joined Liverpool. This was not a success and he played briefly for Leicester City before signing for West Ham in 1901. After only one league and two cup games he was involved in a straight swap for Welsh international full-back Bill Jones from Kettering Town. Kyle scored 19 goals in 41 appearances before being sacked in 1905 for an incident during training. In April 1906 he joined Woolwich Arsenal. That season, he scored 13 goals in 29 league appearances and only Charles Satterthwaite, the former West Ham player, scored more for Arsenal that season. Kyle also played for Sheffield United before being sacked after another training incident.