GCHQ

Hitler’s decision to turn east was a fabulous stroke of luck for Britain. At a time when its forces were struggling, this was a most welcome redirection of the main German war effort. Taken together with Pearl Harbor at the end of the year, it is right to regard 1941 as nothing less than the fulcrum of the war. However, Bletchley Park now faced a new problem. Should it pass sensitive intelligence derived from Ultra to the new Soviet ally, which had been a dedicated enemy of Britain since 1917? The idea that two of Britain’s adversaries were about to fight to the death filled most military intelligence officers with ill-disguised glee. Many argued that passing sigint to the Soviets was pointless, since few expected them to hold out later than 1942. Others insisted that not even Ultra could penetrate the fog of self-deception with which Stalin had surrounded himself.13

In the event, Bletchley Park did develop a precarious sigint liaison with the Soviets. When the British Chiefs of Staff despatched a military mission to Moscow, the code-breakers decided to work through it to find out what the Soviets were doing. They began cautiously, asking about ‘low-grade material only’, notably German Air Force three-letter tactical codes. They intended to send an officer from Bletchley, and in the long term even hoped to persuade the Soviets to accept a British Y unit, or forward listening station, that would intercept German tactical messages on their front. In late 1941 the Soviets agreed to a visit from Squadron Leader G.R. Scott-Farnie, who worked on Britain’s Y interception system in the Middle East.14

Scott-Farnie gave the Soviets a good deal of information on low-grade German Air Force systems, but quickly came up against a different culture of intelligence exchange.15 The Soviets adored captured documents, and did not attach much credence to any information that was not supported by such evidence. Once the game of document exchange began, Scott-Farnie discovered that the Soviet approach ‘was precisely that of a horse dealer who enjoys the poste and riposte of a bargain, and they looked at the exchange of documents on an eye for an eye basis’.16

Bletchley now had to decide whether to follow up the Scott-Farnie Mission. Alastair Denniston was ‘full of hesitation because of the continued Soviet retreat before the German onslaught’, but the intelligence directorates of Britain’s three armed services thought it worthwhile. Josh Cooper, who had reviewed the exchanged material, concluded that the Soviets were ‘absolute beginners’ in their work on the German Air Force, but thought they should be shown the RAF Y stations at Kingsdown in Kent and Cheadle in Cheshire to point them in the right direction. If the Soviets were impressed, he added, they might allow a British Y unit to be sent to the Soviet Union.17 In the end, Edward Crankshaw, an Army Y Service officer, was sent out, armed with more barter material in the form of documents. This was to be ‘swapped’ with the Soviet interceptors, since Bletchley Park thought ‘there is no better analogy than the schoolboy with his stamp collection’. By the spring of 1942 Crankshaw was established in the Soviet Union, and was trading his wares.18 However, the greatest success in the Soviet Union was achieved by the Royal Navy. It was running supply convoys to the Russian port of Murmansk, and this justified the setting up of a radio station at the nearby town of Polyarnoe. A small naval Y intercept party was soon attached to it, and began cooperating with the Soviets on low-level German naval communications. This kept going until December 1944, and yielded good material on subjects such as the movements of the German battleship Tirpitz in northern waters.19

The main worry about giving Ultra to the Soviets was the insecurity of their own cyphers – in 1942, Bletchley Park was increasingly aware of the German ability to read a great deal of Soviet operational military traffic in the field. Frederick Winterbotham, who worked on sigint distribution, argued that Moscow simply had to be told about the weak security of its cyphers. However, Winterbotham’s colleagues insisted that it was ‘impossible’ to tell the Soviets, even though he had ‘invented a good cover story’ to explain how they knew.20 The secret truth was that Bletchley Park was collecting second-hand sigint. The Germans were sending their own sigint from the Eastern Front back to Berlin using an Enigma key code-named ‘Mustard’, which in turn was being read by the British. Although much of the sigint obtained from the Soviets was operational, the British also noted that ‘first grade traffic can be read – at least in part’ by the Germans. Some of the German successes had stemmed from a Soviet codebook, ‘OKK–5’, known to have been captured by the Finns and given to the Germans. While the British had struggled to break these codes in the 1930s, the Germans were having more success.21 On 16 June 1942, Nigel de Grey, the Deputy Director at Bletchley Park, stepped in and settled the argument. He noted that Edward Crankshaw, the GC&CS liaison with the Soviets, would soon be returning from Moscow for another visit. He would be ordered to give the Soviets the details of their compromised cyphers and ‘the methods of reading’. This decision probably reflected the fact that, against all predictions, the Soviet forces were hanging on impressively and looked as if they were going to be in the war for some time to come.22

In August 1942, Crankshaw briefed the Soviets on their appalling lack of security, typified by their alarming tendency to use low-grade cyphers for high-grade secrets.23 There was abundant evidence of this in German Air Force Enigma, but Crankshaw only hinted at it by ‘somewhat tenuous means’. Predictably, the Soviets would not accept his warnings because ‘direct evidence was not forthcoming’. Depressed, he went back to Bletchley Park in February 1943, never to return to Moscow. He joined the staff at Bletchley Park and tried to keep the relationship going at a distance, ‘but the temperature was falling’. The Director, Commander Edward Travis, was only willing to allow the relationship to continue ‘if it is a solid gain for us’. The Polyarnoe naval listening station continued to function, but with the Soviets turning the tide on the Eastern Front they seemed to feel no need for further cooperation, and other contacts ‘petered out’.24 On 9 February 1944, London discussed the possibility of a visit to Britain by Soviet cypher experts and decided against it.25

Bletchley Park’s heated debate on what information to give to the Soviets was academic. All along, one of the KGB’s top agents, John Cairncross, had been working at Bletchley. Although Cairncross studied at Cambridge in the early 1930s, he was not recruited by Anthony Blunt, one of the key KGB talent scouts there, who found him both quarrelsome and arrogant. Instead, after Cairncross joined the Foreign Office in 1936, he was persuaded to work for Soviet intelligence by James Klugman, a prominent British Communist, who later served in the wartime Special Operations Executive. Although Cairncross was fearsomely intelligent, his difficult personality ensured that he was always being moved on. At the outbreak of the war with Germany he was sent to the Cabinet Office to work for the Cabinet Secretary, Lord Hankey. There he saw some of the early British thinking on the development of the atomic bomb. In 1941 he was moved to Bletchley Park, labouring in Hut Three on the Luftwaffe order of battle. His moment of triumph came in early 1943 when he was able to warn his KGB controller of the impending German armoured offensive at Kursk. Code-named ‘Operation Citadel’, this was the last great German push on the Eastern Front. It proved to be the largest tank battle of the Second World War, and the information provided by Cairncross proved to be important in launching an early attack upon the German tactical air force, much of which was destroyed on the ground. Stalin later awarded him the Order of the Red Banner in recognition of his achievement.26

Soon after Kursk, Cairncross moved again. He now returned to London and ended up in Section V, the counter-intelligence section of SIS, working alongside Kim Philby. Although he worked with Philby, Guy Burgess and indeed Donald Maclean, Cairncross was unaware of their common allegiance to Moscow, and believed he was the sole high-grade KGB agent in Whitehall. Bizarrely, he was caught in 1951 because of an official note in his handwriting found in the flat of Guy Burgess after Burgess had fled to Moscow with Maclean. Cairncross had given Burgess this quite innocently in the course of official business, without knowing he was a fellow spy. Once the note was found, Cairncross was followed, and MI5 surveillance believed they had caught him trying to meet with his KGB controller. Without hard evidence he could not be prosecuted, and he was merely asked to resign. Ironically, the Ultra material that Cairncross passed to the KGB was taken more seriously by Moscow precisely because it was stolen. Had the British handed it willingly to their ally, Stalin’s suspicious mind would almost certainly have devalued it.27

Cairncross was not the only KGB agent with access to Ultra. In late 1942, Anthony Blunt, another high-grade Soviet agent, was designated one of the two MI5 liaison officers who worked closely with Bletchley Park.28 Anxiety about KGB agents and subversion was yet another reason that the British kept working on Soviet traffic. Monitoring stations, notably the Metropolitan Police intercept station at Denmark Hill in south London, reported an upswing in traffic between Moscow and secret agents in Britain. There was also a British field unit, called the Radio Security Service, that hunted for illegal agent radio transmissions, and it told the same story, although the agent traffic could not be broken.29 John Croft, who worked at the GC&CS diplomatic code-breaking centre at Berkeley Street in London, was one of those who soldiered on with Soviet material. Croft was engaged on wartime Comintern traffic in Europe, known as ‘Iscot’, which could be read. Although circulated only to a very select group of individuals within Whitehall, this material mostly revealed a dutiful Soviet struggle against their shared enemy, Nazi Germany. There is no indication that this material was exchanged with Washington.30

Early British cooperation with the American code-breakers was also tentative. Again, the obstacle was obsessive security. Security problems existed on several different levels. The British and the Americans had cooperated on sigint during the First World War, but this had bequeathed a legacy of doubt and anxiety, even distrust. In November 1940, when reviving sigint cooperation with the Americans was discussed, Alastair Denniston was quick to point out that after the First World War the ‘notorious’ American code-breaker Herbert O. Yardley had published a tell-all book about his experiences. The very name ‘Yardley’ caused a shudder in British code-breaking circles. Yardley was now working for the Canadians, and GC&CS insisted that they sack him summarily before they were allowed to join the wartime sigint club.31 Indeed, the Canadians were told that other agencies ‘would not touch Yardley with a ten foot pole’.32

The British, and especially Sir Stewart Menzies, the Chief of SIS, were frosty towards the Americans, and regarded them as fundamentally insecure. By contrast, the Americans generously opted to share the secret of their spectacular code-breaking success against the Japanese ‘Magic’ diplomatic cypher with the British as early as January 1941, even handing over precious examples of their copies of the Japanese cypher machine. The British were ‘flabbergasted’. They did not expect the Americans to ‘simply walk in and plonk down their most secret cryptanalytical machine’.33 Yet the British remained reticent, and did not initially reciprocate fully with their knowledge of Enigma. The jibe about American insecurity had a certain irony, since the British chose to send one of the priceless American copies of the ‘Purple’ machine out to their naval base at Singapore shortly before it fell to the Japanese. The machine was delivered by ship just as the Japanese invasion of Malaya began, and disappeared into the chaos of battle. To this day its fate is unknown.34

Collaboration with Washington was also hard because American sigint was a house divided. Although William Friedman, the US Army’s best cryptologist, was busy advocating sigint cooperation with the British in early 1940, the US Navy’s chief code-breaker, Commander Laurance Safford, was adamantly set against working with allies. But after pressure from President Franklin D. Roosevelt the Navy had been won round, and the Americans sent a team of technical experts to Britain in early 1941. Known as the ‘Sinkov Mission’, they spent several weeks touring Bletchley Park and visiting outlying intercept stations. The British were willing to receive them because they knew the main focus of American attention was Japan. At this stage the British were keen to keep discussions focused on Japan, because this allowed them to hide the extent of their knowledge of the German Enigma system. Both Sir Stewart Menzies and Sir Alexander Cadogan were adamant that the Ultra secret would not be shared with the Americans.35 Laurance Safford later represented the first Anglo-American exchanges of late 1940 and early 1941 as a one-way street in which the Americans handed over their precious ‘Magic’ material on Japan but got nothing in return. In fact this is far from the case. Prescott Currier, one of the Americans who came to Bletchley in early 1941, recalled: ‘All of us were permitted to come and go freely and to visit and talk with anyone in any area that interested us.’36 Later that year, a select circle of American code-breakers were also given more details about Enigma.37

The hottest issue was the distribution of sigint to the policy-makers. In late May 1941, Brigadier Raymond Lee, the American Military Attaché in London, conveyed an American request for comprehensive intelligence exchange in the Far East. There followed painfully slow and complex discussions about who would get sigint with what levels of security: ‘The whole thing has been so tangled up,’ he complained.38 Sigint was also very confused in Washington. Unlike Britain’s GC&CS, American signals intelligence was less centrally organised, resulting in great rivalry between the armed services.39 Because the American wartime sigint organisation was divided between the Army and the Navy, one of the great problems for the British was cooperating with one without upsetting the other. Famously, the Americans solved the tussle over who would decrypt Japanese codes by agreeing that the Army would decode the material on the even days of the month, and the Navy on the odd days. A more ludicrous system for the division of labour would have been hard to devise.40

GC&CS might have been more centralised than the Americans, but it had less money. Expanded cooperation with America on Japan allowed GC&CS to shed some difficult code-breaking tasks. High-grade Japanese Army cyphers had proved impenetrable for a decade. By 1941, Bletchley Park was too busy with the European war, while its Far Eastern code-breakers were struggling to cope with the mass of material on Japanese espionage derived from low-level consular intercepts in South-East Asia. On 22 August 1941, Anglo-American cooperation lifted this task from their shoulders. During talks in Washington, Alastair Denniston persuaded the US Army that it should ‘take over investigation of Japanese main army cipher soon as priority commitment’. Shortly after, Captain Geoffrey Stevens from Singapore travelled to Washington carrying all the British material on the Japanese main army cypher.41 The British were glad to see the back of it. At the end of the war approximately 2,500 Americans would still be working on this one Japanese cypher to no avail.

All the while, Britain was also decyphering some American traffic. Amongst the decrypts selected for the personal perusal of Winston Churchill were those of many Allied and neutral countries. GC&CS was clearly working successfully on the American diplomatic code ‘Grey’ until December 1941.42 Remarkably, there was no embarrassment about this. In June 1941, while discussing comprehensive sharing of Far East intelligence, the British asked the US Military Attaché, General Raymond Lee, for his opinion on the security of American cyphers. This was the conduit through which sigint would pass between London and Washington. Lee replied tartly that the GC&CS already knew a great deal about this matter. He recorded in his diary:

The talk then turned again on the question of security. They wanted to know whether my despatches went by radio or cable and were relieved to hear that they went by cable, and were further relieved to hear that we have a direct wire straight into the War Department. However, I pointed out that this wire was subject to interception by their people here in England [GC&CS] and I had no doubt they had taken our messages and attempted to decipher them.

He added that it was now very much in the interests of GC&CS to be honest about the security of American cypher systems, ‘because the stuff that is going over it is more vital to them than to us’. Lee’s frank exchange with the British underlines one of the hidden benefits of cooperation between the Allied code-breakers. Once they began to share their most precious assets, ‘Magic’ and then eventually Ultra, improved communications security became paramount. London and Washington now had a vested interest in the impenetrability of each other’s messages. After all, if GC&CS could break American codes, then so, perhaps, could the Germans.43

Churchill eventually wrote to Roosevelt and owned up to British work on American diplomatic codes. ‘From the moment we became allies,’ he explained, ‘I gave instructions that this work should cease. However, danger of our enemies having achieved a measure of success cannot, I am advised, be dismissed.’ In fact, it is unlikely that all work on American traffic ceased. In areas such as the Middle East, Britain had a considerable incentive to continue to work on American commercial traffic, much of which was in commercial code or plain text. Indeed, a close reading of Churchill’s assurance to Roosevelt suggests that it might have related to diplomatic traffic only.44 Some GC&CS staff recall work on the traffic of American commercial attachés throughout the war, although as yet no documents have been released.45 Predictably, clear traffic from American oil companies was being intercepted in 1944 as they began to look for new markets in Europe.46

For this very reason, the US Army and Navy were agreed that nothing should be passed to the British about American code-making procedures, such as the Sigaba cypher machine. General George Marshall, the US Army Chief of Staff, specifically forbade any such exchange in September 1940.47 Anxiety about protecting national cypher systems persisted through the war on both sides of the Atlantic. In February 1945 Britain’s newly formed Cypher Policy Board debated a proposal by its Secretary, Captain Edmund Wilson, for ‘free and complete interchange’ with the Americans on cypher machine development, together with scrambler phones and secure speech.48 This horrified both Edward Bridges, the Cabinet Secretary, and Sir Stewart Menzies, Chief of SIS, and the idea was rejected. Cooperation on communications security would focus on machines specially designed for combined use.49

The gradual collapse of the British monopoly over Ultra intelligence paved the way for closer Anglo-American sigint cooperation. As we have seen, Bletchley’s initial idea for wartime cooperation was that the Americans would continue their pre-war focus on Japanese traffic; meanwhile the British would handle the work on Enigma, dispensing its product to the Americans as they saw fit. Although they had informed the Americans about Enigma in 1941, some precise details of processing had been withheld. Bletchley was determined to prevent the Americans working on Enigma in parallel, even though the Battle of the Atlantic gave Washington a legitimate need for Ultra intelligence. However, once the German Navy introduced an improved Enigma machine with four rotors, the British could not produce enough ‘bombes’ to deal with the increased number of tests required to break it.50 In September 1942, Joseph Wenger, who led the US Navy code-breakers, proposed spending $2 million to acquire no fewer than 230 four-wheel ‘bombes’. This was ten times the number available to Bletchley. John Tiltman, Britain’s Soviet code specialist, realised that American sigint was beginning to operate on an industrial scale, and that for Bletchley Park the game of ‘Ultra monopoly’ was surely up.51

In September 1942, Edward Travis and the head of Bletchley Park’s Naval Section, Frank Birch, travelled to Washington and concluded the ‘Holden Agreement’, which established full and integrated collaboration on German naval traffic, including Enigma. This was a key part of the emerging Anglo-American sigint relationship, and a constituent part of the secret alliance which still exists to this day.52 Travis’s hand was strengthened by the remarkable fact that the US Navy breathed not a word about the Holden Agreement to the US Army. The British therefore persisted in their hopes of keeping control over the processing of Ultra material derived from Luftwaffe and German Army traffic. Nigel de Grey, the Deputy Director of Bletchley Park, was apoplectic at the possibility of the Americans being allowed to duplicate further British work on Enigma. However, a US Army code-breaker based at Bletchley, Colonel Telford Taylor, suggested a tactful way forward. He advised his superiors in Washington that all they needed for the time being was a small ‘foothold’ in the work on Enigma, which would allow them to gain experience. More level-headed organisational types at Bletchley Park, such as Gordon Welchman, could see that the ability of the Americans to procure unlimited numbers of bombes was crucial, adding, ‘We certainly need help.’53 The result was the BRUSA agreement, a further crucial landmark in the construction of the Anglo-American sigint relationship. On 17 May 1943, Bletchley agreed to American participation in work on German Army and Air Force traffic. A second Holden Agreement on naval sigint followed in 1944. These treaties were of enormous importance, and paved the way for more ambitious post-war sigint alliances.

The exigencies of war had broken Britain’s cryptographic monopoly on Ultra. However, Ultra was a military system, representing the core work of Bletchley Park. There is no evidence that Britain and the United States concluded an overarching treaty on diplomatic or commercial sigint, the material that GC&CS worked on at Berkeley Street. In 1942, Alastair Denniston, who had been moved sideways to manage diplomatic sigint, arranged for cooperation on a number of specific countries such as Italy, France, Spain, Portugal, Japan and, of course, Germany. However, this was done on an ad hoc basis. There was no diplomatic BRUSA agreement. It seems that the Americans were not intercepting and working on a range of materials that would have prompted a wider deal. Typically, Denniston told John Tiltman, with evident relief, ‘They do no work on any of the Near Eastern governments.’54

Denniston’s main point of contact in the United States was William Friedman and the US Army code-breakers, who dominated American work on diplomatic systems. The Americans were keen to cooperate, since up until 1941 the US Army had been intensely focused on the diplomatic cyphers of Japan. In 1940 the Americans lost access to Japan’s diplomatic cypher, and it was only recovered as the result of a prodigious effort by a team under Frank Rowlett. By contrast, British code-breakers were working on the diplomatic cyphers of some twenty-six different countries.55 Therefore, when the Americans offered access to ‘Magic’, the British reciprocated with a wide range of diplomatic material, including high-grade Italian systems. Then, in March 1942, John Tiltman visited Washington and brought with him Spanish and Vichy French cyphers. Given the arrival of American forces in the Mediterranean, this was valuable material. By 1944 the Americans had received more material from the British on diplomatic cyphers used by the Greeks, Hungarians, Iranians and Iraqis. However, the processing went on behind a curtain. Denniston asked at one point, ‘Do they actually work on the stuff which we send them, or do they simply put it in their library?’ Diplomatic cyphers from countries that the British considered to be client states, such as Egypt, were withheld.56