Colossus Computer
In September 1942, Max Newman joined the Government Code and Cypher School at Bletchley Park and was placed in charge of a research section that was known as the "Newmanry". (1) He was given the problem of dealing with the Lorenz SZ40 machine that was used to encrypt communications between Adolf Hitler and his generals. It operated in a similar way to the Enigma Machine, but the Lorenz was far more complicated, and it provided the Bletchley codebreakers with an even greater challenge. In 1943 Newman came up with a way to mechanise the cryptanalysis of the Lorenz cipher and therefore to speed up the search for wheel settings. (2)
Alan Turing suggested that Newman worked with Tommy Flowers, a young telephone engineer, who had helped in building a decoder in 1941. (3) Flowers explained the objective of Newman's machine: "The purpose was to find out what the positions of the code wheels were at the beginning of the message and it did that by trying all the possible combinations and there were billions of them. It tried all the combinations, which processing at 5,000 characters a second could be done in about half an hour. So then having found the starting positions of the cipher wheels you could decode the message." (4)
Colossus Mark I
The initial machine designed by Newman kept on breaking down. Flowers later recalled: "I was brought in to to make it work, but I very soon came to the conclusion that it would never work. It was dependent on paper tape being driven at very high speed by means of spiked wheels and the paper wouldn't stand up to it." Flowers suggested that Newman used valves instead of the old-fashioned electromechanical relay switches that had been used in Turing's machines. He claimed valves would do the same job much faster without the need for the synchronisation of the two tapes.
Gordon Welchman, a colleague at Bletchley Park, pointed out: "Flowers seems to have realized at once that synchronization 44 punched-tape operations need not depend on the mechanical process of using sprocket holes. He used photoelectric sensing, and at that early date he had enough confidence in the reliability of switching networks based on electronic valves (tubes, in America), rather than electromagnetic relays, to risk using such techniques on a grand scale. From his prewar experience, Flowers knew that most valve failures occurred when, or shortly after, power was switched on, and he designed his equipment with this in mind. He proposed a machine using 1,500 valves." (5)
Tommy Flowers claimed that Max Newman and his team of codebreakers were highly sceptical of his suggestion: "They wouldn't believe it. They were quite convinced that valves were very unreliable. This was based on their experience of radio equipment which was carted around, dumped around, switched on and off, and generally mishandled. But I'd introduced valves into telephone equipment in large numbers before the war and I knew that if you never moved them and never switched them off they would go on forever. They asked me how long it would take to produce the first machine. I said at least a year and they said that was terrible. They thought in a year the war could be over and Hitler could have won it so they didn't take up my idea." (6)
The project was now shelved. However, Flowers was so convinced that he could get Newman machine to work effectively he continued building the machine. At the Post Office Research Station at Dollis Hill, Flowers took Newman's blueprint and spent ten months turning it into the Colossus Computer, which he delivered to Bletchley Park on 8th December 1943, but was not fully operational until 5th February 1944. It consisted of 1,500 electronic valves, which were considerably faster than the relay switches used in Turing's machine. However, as Simon Singh, the author of The Code Book: The Secret History of Codes & Code-Breaking (2000) has pointed out than "more important than Colossus's speed was the fact that it was programmable. It was this fact that made Colossus the precursor to the modern digital computer." (7)
Newman's staff that operated the Colossus consisted of about twenty cryptanalysts, about six engineers, and 273 Women's Royal Naval Service (WRNS). Jack Good was one of the cryptanalysts working under Newman: "The machine was programmed largely by plugboards. It read the tape at 5,000 characters per second... The first Colossus had 1,500 valves, which was probably far more than for any electronic machine previously used for any purpose. This was one reason why many people did not expect Colossus to work. But it began producing results also immediately. Most of the failures of valves were caused by switching the machine on and off." (8) It was now possible to crack a Lorenz-encrypted message in hours rather than days. (9)
Harry Fensom later reported: "The Colossi were of course very large, hence their name, and gave off a lot of heat, ducts above them taking some of this away. However, we appreciated this on the cold winter nights, especially about two or three in the morning. When I came in out of the rain, I used to hang my raincoat on the chair in front of the hundreds of valves forming the rotor wheels and it soon dried off. Of course it was essential that the machines were never switched off, both to avoid damaging the valves and to ensure no loss of code-breaking time. So there was an emergency mains supply in the adjoining bay which took over automatically on mains failure." (10)
Peter Hilton, who worked with Max Newman on Colossus, claims that he was an outstanding leader of men: "He realised that he could get the best out of us by trusting to our own good intentions and our strong motivation and he made the thing always as informal as possible. For example, he gave us one week in four off. We would be just encouraged to do research on our cryptanalytical methods. Of course, the research work should always be related to the job and we always wrote down what we were thinking in a huge book so that it could be preserved and some of those ideas were adopted and became part of the procedure. So I think Newman was the model administrator." (11)
Colossus Mark II
In February, 1944, the Lorenz SZ40 machine was further modified in an attempt to prevent the British from decyphering it. With the invasion of Europe known to be imminent, it was a crucial period for the codebreakers, as it was vitally important for Berlin to break the code being used between Adolf Hitler in Berlin and Field Marshal Gerd von Rundstedt, the Commander-in-Chief of the German Army in western Europe. (12)
Max Newman and Tommy Flowers now began working on a more advanced computer, Colossus Mark II. Flowers later recalled: "We were told if we couldn't make the machine work by June 1st it would be too late to be of use. So we assumed that that was going to be D-Day, which was supposed to be a secret." The first of these machines went into service at Bletchley Park on 1st June 1944. It had 2,400 valves and could process the tapes five times as fast. "The effective speed of sensing and processing the five-bit characters on punched paper tape was now twenty-five thousand characters per second... Flowers had introduced one of the fundamental principles of the postwar digital computer - use of a clock pulse to synchronize all the operations of his complex machine." (13) It has been pointed out that the speed of the Mark II was "comparable to the first Intel microprocessor chip introduced thirty years later". (14)
When the night staff arrived for work just before midnight on 4th June, 1944 they were informed that tomorrow was D-Day: "They told us that D-Day was today and they wanted every possible message decoded as fast as possible. But then it was postponed because the weather was so bad and that meant we girls knew it was going to take place, so we had to stay there until D-Day. We slept where we could and worked when we could and of course then they set off on June 6, and that was D-Day." (15)
Winston Churchill and his commanders wanted to know if the deception plans for the D-Day landings had been successful. Developed by two agents, Tomás Harris and Juan Pujol: The key aims of the deception were: "(a) To induce the German Command to believe that the main assault and follow up will be in or east of the Pas de Calais area, thereby encouraging the enemy to maintain or increase the strength of his air and ground forces and his fortifications there at the expense of other areas, particularly of the Caen area in Normandy. (b) To keep the enemy in doubt as to the date and time of the actual assault. (c) During and after the main assault, to contain the largest possible German land and air forces in or east of the Pas de Calais for at least fourteen days." (16)
Harris devised a plan of action for Pujol (code name GARBO). He was to inform the Germans that the opening phase of the invasion was under way as the airborne landings started, and four hours before the seaborne landings began. "This, the XX-Committee reasoned, would be too later for the Germans to do anything to do anything to frustrate the attack, but would confirm that GARBO remained alert, active, and well-placed to obtain critically important intelligence." (17)
Christopher Andrew has explained how the strategy worked: "During the first six months of 1944, working with Tomás Harris, he (GARBO) sent more than 500 messages to the Abwehr station in Madrid, which as German intercepts revealed, passed them to Berlin, many marked 'Urgent'... The final act in the pre-D-Day deception was entrusted, appropriately, to its greatest practitioners, GARBO and Tomás Harris. After several weeks of pressure, Harris finally gained permission for GARBO to be allowed to radio a warning that Allied forces were heading towards the Normandy beaches just too late for the Germans to benefit from it." (18)
Tommy Flowers had a meeting with General Dwight D. Eisenhower on 5th June. He was able to tell Eisenhower that Adolf Hitler was not sending any extra troops to Normandy and still believed that the Allied troops would land east of the Pas de Calais. Flowers was also able to report that Colossus Mark II had decoded message from Field Marshal Erwin Rommel that one of the drop sites for an US parachute division was the base for a German tank division. As a result of this information the drop site was changed.
Jean Thompson later explained her role in the operation in the book, Station X: The Codebreakers of Bletchley Park (1998): "Most of the time I was doing wheel setting, getting the starting positions of the wheels. There would be two Wrens on the machine and a duty officer, one of the cryptanalysts - the brains people, and the message came in on a teleprinted tape. If the pattern of the wheels was already known you put that up at the back of the machine on a pinboard. The pins were bronze, brass or copper with two feet and there were double holes the whole way down the board for cross or dot impulses to put up the wheel pattern. Then you put the tape on round the wheels with a join in it so it formed a complete circle. You put it behind the gate of the photo-electric cell which you shut on it and, according to the length of the tape, you used so many wheels and there was one moveable one so that could get it taut. At the front there were switches and plugs. After you'd set the thing you could do a letter count with the switches. You would make the runs for the different wheels to get the scores out which would print out on the electromatic typewriter. We were looking for a score above the random and one that was sufficiently good, you'd hope was the correct setting. When it got tricky, the duty officer would suggest different runs to do." (19)
Primary Sources
(1) Michael Smith, Station X: The Codebreakers of Bletchley Park (1998)
Max Newman, one of the mathematicians working in the Testery, became convinced that, using the principles advocated by Turing in his pre-war treatise on a computing machine, it would be possible to build a machine that, once the patterns of the wheels had been worked out in the Testery, would find the settings of the first row of wheels, thereby making the codebreakers' task immeasurably easier.
Newman, an academic from Cambridge University, took his ideas to Travis and received the necessary backing and promise of funding to set up his own section, which became known as the Newmanry. He then went to Wynn-Williams at the Telecommunications Research Establishment in Malvern and asked him to design the machine.
It was known as 'Robinson', after Heath Robinson, the cartoonist designer of fantastic machines, and the first version was delivered to Bletchley Park in May 1943. It worked on the principle that although the encyphering letters were supposed to be random, they were not. No machine can generate a truly random sequence of letters. Robinson compared a piece of teleprinter tape carrying the encyphered text with a piece of tape on which the wheel patterns had been punched to look for statistical evidence that would indicate what the wheel-settings were.Robinson was designed to keep the two paper tapes in synchronisation at a thousand characters a second but at that speed the sprocket wheels kept ripping the tapes. Turing who, while working on the bombe, had been impressed by the abilities of a bright young telephone engineer at Dollis Hill called Tommy Flowers, suggested to Newman that he might be just the man to get Robinson to work.
'I came into the project when the Robinson machine didn't work properly, because it was made almost entirely of telephone parts, telephone switching parts, which was my area,' Flowers said. 'I was brought in to make it work, but I very soon came to the conclusion that it would never work. It was dependent on paper tape being driven at very high speed by means of spiked wheels and the paper wouldn't stand up to it.'
Tester recalled long conversations with Flowers, Turing, Tutte and Newman over what should be done. Flowers, who had been developing telephone exchanges containing valves instead of the old-fashioned relays used in Robinson, suggested that he could make an electronic machine built with valves that would do the same job much faster without the need for the synchronisation of the two tapes. The data on the wheel patterns would be generated electronically using ring circuits while the tape reading the cypher text would be read photo-electrically and could be run on smooth wheels rather than sprockets so it wouldn't rip....
Colossus was the first practical application of a large-scale program controlled computer and as such the forerunner of the post-war digital computer. Although it had a specialised function, it showed that Turing's theory could be turned into practice. The sequence of operations was determined mainly by setting of external switches and plugboards, which were controlled by Wrens on the orders of the Newmanry codebreakers. Just like the Robinson, it was looking for sequences that were not random. "The work on Tunny divided roughly between what you might call the machine work and the hand work," Halton recalled. "Machine work was of course putting messages on to Colossus which was a process whereby we would determine the starting positions of the first set of five wheels which were involved in making up the key. Then after that process had taken place, that is to say a process based on statistics, mathematics, the message shorn of part of its key would come to the codebreakers in the Testery who would then have the job of using their knowledge of expected pieces of text in order to set the remaining wheels."
(2) Gordon Welchman, The Hut Six (1982)
Flowers seems to have realized at once that synchronization 44 punched-tape operations need not depend on the mechanical process of using sprocket holes. He used photoelectric sensing, and at that early date he had enough confidence in the reliability of switching networks based on electronic valves (tubes, in America), rather than electromagnetic relays, to risk using such techniques on a grand scale. From his prewar experience, Flowers knew that most valve failures occurred when, or shortly after, power was switched on, and he designed his equipment with this in mind. He proposed a machine using 1,500 valves, nearly twice the number used in the pioneering ACE computer built in England after the war...
Flowers and his group built the first Colossus in eleven months. Its photoelectric punched-tape reader operated at five thousand characters per second, a remarkable speed for those days. Flowers was a pioneer in the redesign of the electronic decision-making circuits that had been invented before the war. The first "string and sealing wax" (Flowers' own description) version of the Colossus was a tremendous success. Again Flowers and Radley anticipated a future demand and made preliminary arrangements for production. Around March 1944 Dollis Hill received an urgent request from Bletchley Park for more Colossi. They produced them, thanks to their preliminary arrangements, and the effective speed of sensing and processing the five-bit characters on punched paper tape was now twenty-five thousand characters per second. Moreover, as Brian Randell points out, Flowers had introduced one of the fundamental principles of the postwar digital computer-use of a clock pulse to synchronize all the operations of his complex machine.
(3) Tommy Flowers, quoted by Michael Smith, the author of Station X: The Codebreakers of Bletchley Park (1998) page 148-49
They wouldn't believe it. They were quite convinced that valves were very unreliable. This was based on their experience of radio equipment which was carted around, dumped around, switched on and off, and generally mishandled. But I'd introduced valves into telephone equipment in large numbers before the war and I knew that if you never moved them and never switched them off they would go on forever.
They asked me how long it would take to produce the first machine. I said at least a year and they said that was terrible. They thought in a year the war could be over and Hitler could have won it so they didn't take up my idea. They decided they would proceed hopefully with the Robinson, which is what they did, and they left the question of whether the valve-based machine would be constructed or not to me.
I was so convinced that Robinson would never work that we developed the new machine on our own at Dollis Hill. We made the first prototype in ten months, working day and night, six-and-a-half days a week, twelve hours a day sometimes. We started with the design of what was to be called Colossus in February 1943 and we had the first prototype machine working at Bletchley Park on 8 December.
The purpose of the Colossus was to find out what the positions of the code wheels were at the beginning of the message and it did that by trying all the possible combinations and there were billions of them. It tried all the combinations, which processing at 5000 characters a second could be done in about half an hour. So then having found the starting positions of the cypher wheels you could decode the message.
When we'd made it for them and they saw it work, they were really astounded. It had about 1500 valves in it, which horrified Bletchley Park. But one of the disadvantages of Robinson was that it didn't always give you the right answer. The answer that they got from the machine was in numbers, a counter counted the number of times that certain letters appeared and the counter was a bit unreliable so they didn't always get the same count.
What they did with Colossus, the first day they got it, was to put a problem on it to which they knew the answer. It took about half an hour to do the run. They let it run for about four hours, repeating the processes every half hour, and to their amazement, it gave the same answer every time. They really were amazed. It was that reliable, extremely reliable.