Surely, the grimmest part of the Second World War was the Holocaust (or Shoah). This entailed the systematic and wholesale destruction of European Jewry and other groups such as Slavs, Poles, and Romany (Gypsies), among others, which the Nazis had deemed “inferior” and then slated for destruction because of race, blood, or disability. In fact, one of the major war aims of Nazi Germany was the extermination of global Jewry. During the war years, Europe’s landscape was scarred by the presence of concentration, labor, and death camps. Einsatzgruppen (operations groups), and numerous German Police units roamed the western Soviet Union in the wake of the Wehrmacht, slaughtering Jews, Slavs, and Bolsheviks. Collaborationist regimes of nations allied to or conquered by the Axis powers cooperated with the Nazi security forces in extinguishing national or resident refugee Jewish populations. The darkness that overwhelmed Nazi-occupied Europe and threatened other nations in the world was only slightly lessened by individual acts of courageous opposition and the example of the nation of Denmark, which smuggled virtually its entire Jewish population to safety in Sweden. By the end of the war, it has been estimated that Europe’s Jewish population had been reduced to somewhere between a third to a quarter of its 1939 level.1
In the years following the war, a number of histories, memoirs, and specialized studies about the Holocaust were published. These works were based on a variety of private and official sources that were then available to the public. For the longest period, one element that was largely missing in the historical accounts was the records of the various Western intelligence agencies, such as the American Office of Strategic Services (OSS) and Britain’s M.I.6. Also absent was the intelligence gathered by the wartime Allied communications intelligence (COMINT) agencies, most notably the British Government Code and Cypher School (GC&CS), the U.S. Navy’s OP-20-G, and the U.S. Army’s Signals Intelligence Service (SIS). The revelations in the mid-1970s of the Allied code-breaking successes – the deep and persistent exploitation of Axis codes, ciphers, and communications, popularly referred to as “Ultra” – only whetted the appetite of Holocaust researchers. Once it was known that Allied codebreakers had pierced the Reich’s deepest secrets, the question posed was how much more information would be available for researchers?
Beginning in the mid-1970s, scholars of the Holocaust who had wanted to utilize the archived material of the wartime code-breaking agencies focused their research on the then available records at the United States’ National Archives and Records Administration (NARA) and the United Kingdom’s Public Record Office (renamed the National Archives in 2003). During the first few years after the Ultra revelations, these scholars discovered little significant information about the Holocaust in the wartime records of the British GC&CS and the American SIS. These records, by the way, were stored in the record groups of their modern successor agencies. The records of the SIS were to be found in Record Group (RG) 457, the records of the National Security Agency (NSA). The GC&CS records were placed in Group HW, the records of the Government Communications Headquarters (GCHQ). The actual number of records in either group was not large by any measure; nor were they particularly revealing. For example, in NARA by 1990, there were about 150 translations of intercepted wartime Japanese, German, and Vichy diplomatic messages that referred to the Holocaust, while at the PRO there may have been fewer.2 Considering the nature and scope of the Holocaust – every country in Europe and many of their colonial holdings were affected in some fashion by what the Nazis were doing – the number of publicly available records seemed meager.
At the same time, the records that were available seemed to have large gaps in the subject matter that they covered. Topics of enormous importance to understanding the Holocaust, such as the depredations of the police and SS units in Russia, the operations of the death camps, and the roundup of the Hungarian Jews, barely were mentioned in the extant material. Especially when compared to the large body of records from these and other events during the Holocaust, that there were so few items from communications intelligence sources in the British and American archives during the 1970s seemed to invite disbelief, ridicule, or suspicion.
Researchers also could construe the absence of significant archival holdings of the wartime records of the Allied code-breaking agencies to mean that further, still unreleased, caches of records existed. These “absent” records were believed to be in either one of two forms: finished intelligence in the form of reports still classified and therefore withheld from the public, or there existed troves of “raw,” or undecrypted, Axis messages at these agencies. Exacerbating the situation was the publication, during the decade of the 1980s, of the official multivolume History of British Intelligence in the Second World War.3 This enormous history referenced British records of intercepted Nazi messages about the Holocaust, mainly those of the German Police who were one of the primary agents for the massacres of hundreds of thousands of Jews and others in the western Soviet Union. The history also referred to intercepted messages from the SS concerning the slave labor populations at a number of concentration camps.4 Ironically, the actual records used to write the history still were not accessible by the public.
In a way these scholars were correct in their observations. Even by the late 1980s, the British and Americans still had much World War II cryptologic material to release. It was not until a number of further significant releases of wartime records of the Signal Intelligence Service and the Government Code and Cypher School through the decade of the 1990s that the amount of COMINT material available to researchers of the Holocaust dramatically increased.5 By early 2004, at NARA, over 600 translations and decrypts of various intercepted messages about the Holocaust could be found in the record groups of the National Security Agency. These included some decrypts of German Police messages that reported the massacres of Jews and other groups. Others are messages from mostly diplomatic sources that bear witness to such events as the roundup of Jews in Hungary and other countries in occupied Europe. At the PRO, the complete set of German Police and SS decrypts were available to be reviewed by the public.6
Even with the releases of the 1990s, the U.S. government still held back significant collections of U.S. government records about the Holocaust. But the remaining wartime records, and those from the postwar period that relate to the Holocaust and to Nazi and other Axis power war crimes will soon be declassified and released thanks to the efforts of the United States Interagency Working Group on Nazi War Crimes (IWG). Established in January 1999 in accordance with the Nazi War Crimes Disclosure Act (P.L. 105-246), the IWG was charged with locating, inventorying, and recommending for declassification all classified Nazi war criminal records held by the United States government. The act was amended in 2000 to include declassification of U.S. government records pertaining to Japanese war crimes and war criminals.7 Many of the records that were released under the aegis of the Disclosure Act were from U.S. intelligence agencies. Hopefully, the release of these records will help dispel those claims and charges made over the years by some scholars and Holocaust survivors that “there had to be more records” or that the intelligence agencies were “holding back records.”8
To this historian, the problems that researchers and scholars of the Holocaust had had over the years with the number of available COMINT records appeared to lay elsewhere than just with the paucity of this material. Similar reactions to the later several releases of wartime records to the NARA and the PRO suggested to me that the issue was a fundamental one: That researchers and scholars still misunderstood the basic operation of the Allied wartime COMINT system. Many people did not understand how the wartime code-breaking agencies procured, processed, and disseminated communications intelligence. They did not realize that there were technical and institutional constraints and limitations under which GC&CS and SIS operated. Also, it was not generally well understood that there were priorities established for collection and decryption of Axis and neutral communications and that higher authorities in Allied intelligence and military operations had set these.
Many people also did not know that the operational needs of these agencies largely determined what wartime records were retained after the war, how the existing records were controlled, where the relevant records resided in various national archival collections, and who was responsible for their release. In short, the story of COMINT records relating to the Holocaust is much more than a simple matter of the number of pages available to the public at various national archives.
In considering all of the above, I determined that a historical guide would
be useful for researchers, scholars, and the general public. Such a guide
could help Holocaust researchers gain a better understanding of how Allied
communications intelligence reported intelligence on the Holocaust. It would
explain the variety of material that would be encountered in the records
of the wartime cryptologic agencies.
This guide, then, will concentrate on three topics that would be of
interest and utility to scholars and the general public. First, it
explains how the Western communications intelligence system operated
during the war.
It will consider how well the system operated and what were its limitations.
This latter point is important when considering how Western COMINT
handled intelligence about the Holocaust. Second, the guide describes
how the wartime
records of the SIS and GC&CS currently are organized in the national
archives of Great Britain and the United States, where these records
can be found, and the various formats they come in. Third, the guide summarizes
what information is available from SIGINT records about the Holocaust.
This
summary consists of both a general chronology of the Holocaust and
selected incidents for which significant communications intelligence records
are
available.
Despite the scope and detail of some of the material contained in this guide, it is not intended as a narrative history of the Holocaust based on the records of Western communications intelligence agencies. The major reason is that the archived COMINT records cannot sustain such a history. There are too many important parts of the history of the Holocaust for which no communications intelligence was collected. As will be demonstrated later in this work, communications intelligence could not reveal high-level Nazi policy deliberations regarding the Jews and other groups. On occasion, communications intelligence could “tip off’ an impending action by Nazi security forces, as in Italy in the fall of 1943. But this advantage was rare. More often, COMINT was best as a chronicle of some campaigns that already were under way such as the massacres carried out by the German Police units in the western USSR in 1941 and the roundup of the Hungarian Jews in mid-1944.
Although something of a historical narrative of the Holocaust is presented in the last chapter of this guide, it is meant to be a selected summary of the available information from COMINT records. It is beyond the scope and means for historians of cryptology to rewrite the story of the dreadful events of the Holocaust. Their mission is to discover the relevant records and write the history of cryptology and place that story within the context of larger events of the Second World War. It remains for historians of the Holocaust to utilize completely within their narratives the historical information provided by the records of the Allied code-breaking agencies.
This guide will limit its focus to the two major Western COMINT agencies that produced intelligence about the Holocaust during the war: the British GC&CS and the U.S. Army’s SIS. Early in the war, the U.S. Navy’s cryptologic element, OP-20-G, contributed some intercept of diplomatic communications, but by mid-1942, it ceded this work completely to the SIS and concentrated almost exclusively on Axis naval communications. A number of smaller Allies contributed to the overall Western radio intelligence work. These included Canada, Australia, New Zealand, and others. However, the American and British security concern to protect Ultra sometimes circumscribed the contribution of these smaller allies. Among these, refugee Polish cryptologists contributed major intercept and code-breaking efforts against German Police communications. Their work will be discussed later in the guide.
It is important to mention that the British had a limited COMINT relationship with the Soviet Union. Among other things, this included an exchange of technical information on German Police ciphers. (But it stopped short of revealing the Enigma breakthrough.) The Russians certainly were in a position to intercept more police messages than the British (or the Poles). And they were able to read the same German Police messages as the British. But Western researchers do not know to what extent and for how long the Russians were able to exploit German Police radio traffic. Also, it is not known if the Soviets retained these decrypts in their archives after the war. These uncertainties mean that this guide will forego any consideration of the Soviet contribution to communications intelligence about the Holocaust. This subject must await future researchers gaining access to the appropriate archives in Moscow.9
The background chapter to this guide offers brief summaries of the history of anti-Semitism in the West and the early Nazi policies in Germany, as well as a short review of the limited body of historical and memoir literature prior to 1997 that pertains to both cryptology and the Holocaust. Chapter 3 will describe the general system by which communications intelligence was produced by the Allies during World War II. This description will encompass the system from the establishment of collection priorities, through the intercept of targeted Axis and neutral communications, next to the processing or analysis of the intercept for intelligence, and finally to the dissemination of the produced intelligence. Just as importantly, this section includes observations on how the nature of the communications intelligence process affected the collection of information concerning the Holocaust. Chapter 4 will list the various locations for relevant records of the American and British cryptologic agencies held by the National Archives and Records Administration and the Public Record Office. This chapter will also include a description and some examples of other smaller relevant records holdings. Chapter 5 will briefly review the available COMINT material that is part of the historical narrative of the Holocaust. This chapter includes a brief overview of the course of the Holocaust and somewhat more detailed descriptions of specific topics that include the refugee problem and Palestine, Vichy and the Jews, the roundup of the Hungarian Jews, the situation of the Jews in the Far East under the Japanese, and German-Swiss trade and financial transactions during the war. Finally, Chapter 6 considers some important general observations about cryptology and the Holocaust. In a way, these observations are a summation of the material presented in the guide.
A Note on Terminology
Since the first revelations of the Ultra secret in the mid-1970s,
the public has been exposed to a number of arcane terms associated with
the business of making or breaking codes. Unfortunately, there has been
a tendency among many scholarly and popular writers and reporters to confuse
or mix terms. This misuse of terms often led to some inaccuracies in their
texts such as referring to the German Enigma machine as a “code
machine.” Although most of these terms are not relevant to this
work, a few necessarily have to be used to accurately describe various
activities and items of the Allied communications intelligence system.
So I will define the most important ones and explain how they are used
in this monograph.
COMINT is the acronym for communications intelligence and can be defined as measures taken to intercept, analyze, and report intelligence derived from all forms of communications. This definition describes most accurately the entire Western system to exploit Axis communications. The COMINT system included the code-breaking centers at Bletchley Park in Great Britain and the American centers at Arlington Hall, Virginia, and Nebraska Avenue in Washington, D.C. It also includes the monitoring stations manned by Allied radio operators that listened in and copies Axis and neutral radio transmissions. It further covers the work of the various Allied staffs and units that took the analyzed messages, picked out the intelligence that mattered and forwarded it to whatever Allied command, ministry, department, or leader that would need it.
A similar term, signals intelligence, or SIGINT, is often used synonymously with communications intelligence in many histories of wartime intelligence. Signals intelligence is a term that encompasses a much broader category of electromagnetic emissions than just those used for communications. This category includes such emissions as radar and navigation beacons. During World War II, Western technical intelligence took an active interest in collecting such signals used by the Axis so that countermeasures could be developed against them. The famous British “window,” or chaff (strips of aluminum that reflected radar signals and created interference on German radar screens) was an effective weapon against German warning radars during the Allied bomber offensives. The British employed technical deceptive measures to defeat the Germans navigational beacons used by the Luftwaffe to guide its bombers during night raids against British cities.10
Cryptology is defined as the study of the making and breaking of codes and ciphers. Cryptography is the development of codes and ciphers. A code is defined as a method in which arbitrary groups of letter, number, or other symbols replace words, phrases, letters, or numbers for the purposes of concealment or brevity. To encode is to transform plaintext into a code. To decode is to break a code back to its underlying plaintext. A cipher is a method of concealing plaintext by transposing letters or numbers or by substituting other letters or numbers according to a key. A key is a set of instructions, usually in the form of letters or numbers, which controls the sequence of the encryption of text and the decryption of cipher back to the original plaintext. Transforming plaintext into cipher is called encryption. Breaking the cipher back to plaintext is called decryption. Cryptanalysis is the analytic method whereby code or cipher is broken back to the underlying plaintext. Traffic analysis is the method by which intelligence is derived from the analysis of the communications activity and elements of messages short of the actual cryptanalysis of a message.
Two examples of famous ciphers from World War II are the Axis cipher machines,
the German Enigma and the Japanese Purple (known to the Japanese as the
97-shiki O-bun In-ji-ki, or Alphabetical Typewriter ’97). Both machines
substituted letters for plaintext elements according to daily settings (key)
for each machine. Interestingly, most ciphers used by all sides during the
war overwhelmingly were manual in nature. That is, they involved the use
of paper charts and key. Such a manual cipher was the double transposition
cipher used by German Police units to encrypt their reports about the massacres
of Jews, partisans, prisoners, and Soviet commissars to Police headquarters
in Berlin.
Codes used during the war usually came in the form of a book. On each
page of the codebook, a plaintext word or phrase was aligned opposite its
code group equivalent. Examples of a code include that used by the Soviets
for its espionage messages known through the Allied cover name as Venona
and the Japanese operational naval code, JN-25. Both codes utilized books
of code values for plaintext, but added an additional element: key, in the
form of groups of numbers that were used to encrypt the code groups, further
concealing the “true” code groups. This practice made decoding
even more difficult: before a cryptanalyst could recover the plaintext value
behind a code group, he or she first had to recover the true code group.
An example of an ordinary code used during the war was the so-called “Black
Code” used by the United States Army military attachés. This
system was exploited by both Italy and Germany to great effect in 1941 and
was probably Field Marshall Erwin Rommel’s best source of intelligence
during the battle for North Africa.11
To facilitate the understanding of readers who may be uncertain of the above jargon or uncomfortable with it, I will use terms like “cryptology,” “communications intelligence,” “COMINT,” “signals intelligence,” and “SIGINT” interchangeably either as adjectives or as nouns by which to describe or identify the overall system the Allies used to exploit Axis cryptography and communications. Using these five terms as general descriptors will not sacrifice accuracy and probably will make the text more readable. Other terms from cryptology, used once, will be defined as they are encountered in the text.
Acknowledgments
This guide took over five years to complete, far longer than originally
planned when I began in late 1998. Why did it take so long to finish?
Initially, like many other historians of cryptology, I believed that there
was not much material available and felt therefore that a guide could
be published within a relatively short time. In 1999 I previewed the guide
in a paper delivered at the Cryptologic History Symposium held at Fort
George G. Meade, Maryland. At the time I felt that relevant record sources
mostly had been examined by scholars. A review of the symposium paper
in an intelligence journal had emphasized my remark about the “scanty” amount
of communications intelligence records.”12 But I was wrong. The
Interagency Working Group (IWG) on Nazi War Crimes (that soon included
Imperial Japanese Imperial Records), which I joined in early 2001 as a
technical advisor to its staff, already was deep into an extensive review
all the records of Nazi crimes held by the many agencies and departments
of the United States government, including its intelligence agencies.
The records that were unearthed by the various intelligence agencies,
and the clues they provided to supplementary finds in the British Public
Record Office and the U.S. National Archives, indicated that there were
many more pages of cryptologic records than earlier believed. Every further
meeting of the IWG and the associated Historical Advisory Panel I attended
had uncovered more records. I realized I had to delay the completion of
this guide until the search by the IWG neared completion. The wait, I
believe, was well worth it.
Within the National Security Agency, there were many people who contributed their efforts, directly and indirectly, to the publication of this Guide. First of all, the Publications Team of the Center for Cryptologic History (CCH) did its usual magic and turned my initial “heap of words” into the presentable form it now is. Many thanks go to the tireless efforts of Lula Greenwood and Barry Carleen. Within the CCH, Dave Mowry, Sharon Maneki, and David Hatch reviewed my manuscript and offered many useful points. Elsewhere, I am grateful for the insights from Robert L. Benson and Rona Lerner. There were others who contributed to this guide from the NSA, yet they cannot be mentioned by name. I wish to thank them publicly for their help.
Because of the scope of the information in this guide, it was necessary
to consult with many individuals outside of the NSA. Although personal research
provided much historical background to the Holocaust, the many months that
I spent reviewing the literature of the Holocaust could not provide me with
that astute insight that is gained only from a deep and sustained professional
study of a subject. So I queried a number of academic historians, independent
scholars, and researchers familiar with the Holocaust for help. In all of
my correspondence with them, I encountered nothing but gracious and full
cooperation. My questions were patiently and quickly answered, requests
for material were filled promptly, and several people took time from their
busy schedules to review the manuscript. I hope that our effort was a mutually
rewarding learning experience. Their cooperation suggests that history is
more than a mere academic discipline; at times it can be a true community
of effort, even, if at times, it can be a bit contentious. With this experience
in mind, I wish to thank the following people for their incisive and useful
comments on my manuscript: Gerhard Weinberg, Richard Breitman, Rebecca Boehling,
Colin “Brad” Burke, David Alvarez, Christopher Lovett, Sebastian
Laurent, Miriam Kleiman, and Ron Zweig. A very special thanks go to Stephen
Tyas, Ralph Erskine, and Rebecca Ratcliff for both their comments on the
manuscript and for sharing their historical research from the Public Record
Office. And a final thanks to the archivists of the U.S. Holocaust Memorial
Museum in selecting images from their photographic archives.
If there are any errors in this guide, then, in the eloquent words
of the Qur’an: “If I err, I err just on my own.”
Robert J. Hanyok
Fort George G. Meade, MD
2005
Notes
Chapter 1
Background
The Context of European and Nazi Anti-Semitism
Anti-Semitism has been part of European history perhaps as early as
Alexander’s Empire in the 4th Century B.C.E. What has varied over
the centuries since then has been the intensity of the animosity towards
Jews, as well as the basis for it, whether it was grounded in political,
religious, cultural, or ethnic differences. The early Roman emperors generally
had a difficult relationship with Jews in the Empire. There was a major
problem over the ritual of emperor worship, which Jews refused to perform.
Imperial adjudication was required to settle disputes between Jewish and
Gentile communities in such cities as Alexandria and Rome. After Constantine’s
reign (337 C.E.), the Christian emperors and church leaders placed administrative
and legal restrictions on the Jewish population of the Roman Empire, although,
for religious reasons, Jews were tolerated.1
In early Medieval Europe there was popular hostility against Jews, but it was unsystematic and was charged with a clearly religious tone – more anti-Judaic than anti-Semitic. This attitude changed, though, in the 11th and 12th centuries when the zeal and intolerance of the Crusades spawned more virulent and violent anti-Semitic atrocities. During the First Crusade in 1096, Jewish communities, mostly in the Rhineland, were plundered and their inhabitants sometimes massacred by crusader armies on their way to the Holy Land. The following two centuries saw the growth of official policies that established ghettoes, discriminatory laws, and financial victimization. In 1215 the Catholic Church’s Fourth Lateran Council prescribed absolute ghettoization for urban Jewish communities and decreed that Jews wear a yellow label as a sign of their pariah status.2
The religious wars of the reformation only brought more massacres and mistreatment for the Jews, especially in Germany. It was not until the Enlightenment and the French Revolution that conditions ameliorated somewhat when civil rights and citizenship were granted Jews. But in the latter part of the nineteenth century, anti-Semitism resurged throughout Europe. The centuries-old religiously based anti-Semitism continued to attract adherents. This old version was joined by a new strain that grew out the nationalistic fervor that dominated European politics well into the twentieth century. It was based on pseudo-scientific social and biological theories of racial differences that emphasized cultural, ethnic, and stereotyped physical differences. In some nationalist lexicons, Jews were now classified as members of the Semite “race.”
The racial core of Nazi ideology was obvious from the earliest days of the movement in Munich. Early proponents of Nazism also hinted at the future destruction of Jewry. When the Nazis came to power in 1933, some anti-Semitic measures were adapted, but the initial policy was relatively unstructured. This changed in September 1935, when Hitler announced the passage of the Nuremberg Laws that, among others, prohibited relations and marriages between Jews and other Germans, forbade Jews to fly the German flag, and deprived them of citizenship. In later years during the war, American intercepts of German consular radio and cable traffic recorded how the effect of these laws was extended to overseas German Jews. These intercepts reported incidents in which individuals as far away as Argentina and China were deprived of German citizenship because their Jewish parentage or denied state pensions unless a Jewish spouse was divorced.3
In the first years of the Nazi regime, the principal method for removing Jews from Germany was emigration. To “encourage” Jews to leave, the Nazis instituted a number of discriminatory measures that included the “aryanization” of the German economy, and, after the Anschluss with Austria, the placement of the mandatory “J” stamp in German passports held by Jews. By the beginning of the war, more than half of German and Austrian Jews had emigrated abroad. Some aspects of the early Jewish emigration from Germany appeared in the occasional Japanese diplomatic message from Berlin intercepted by the United States.4
In 1939, for European Jewry, two important events occurred that pushed their fate in the direction of the “Final Solution.” The first was in January 1939, when the control of anti-Semitic policy in Germany was delegated to the SS (Schutz Staffel). Operational control was placed specifically in the SS’s Reich Main Security Office (Reichssicherheitshauptamt or RSHA) under SS General Reinhard Heydrich. Heydrich controlled all Reich security services, including the Security Service (Sicherstheitsdienst or SD) and the Gestapo (Geheime Staats Polizei or Secret State Police). Later, Adolf Eichmann was placed in charge of Department IVA4b of the RSHA, which was responsible for administering the “Jewish problem.”
The second occurred in September 1939 when Germany invaded Poland. England
and France, honoring a defense pact with Poland, declared war on Nazi Germany.
With these actions, the Second World War began, and the fate of European
Jewry was changed for the worse. The war ended Germany’s original
plan to rid itself of its Jewish population through emigration and expulsion.
As more countries fell under its sway, the number of Jews under Nazi control
grew dramatically. The early policy of enforced expulsion was useless; there
was nowhere to send them, even to the colonies of the nations they had conquered
as with the bizarre scheme to create a Jewish reservation on the French-held
island of Madagascar off the Southeast African coast.
When exactly the mass murder of the Jews was ordered as policy is
not absolutely certain. In late July 1941 Reich Marshall Herman Goering
signed the order that Heydrich had drafted calling for a “final solution” (Endloesung)
to the Jewish presence in German-occupied Europe. Two things about this
order are significant. First, it was signed after massacres of Jews and
other groups in Russia had begun in the wake of the German invasion. Second,
this order also needs to be understood as part of Hitler’s position
regarding Jews and their association with Bolshevism. In January 1939 Hitler
had threatened the Jews with annihilation, but he made this threat within
the context of a potential war, and then as part of the struggle against
the “Bolshevikization [sic] of the earth.”5 In whatever terms
or context it was stated, though, the destruction of Europe’s Jews
was a prime Nazi goal during the war.6 During the final phases of the planning
for the invasion of the Soviet Union, the SS and the German General Staff
approved plans for killing Jews as part of the policy to liquidate all “undesirables,” which
included Soviet political, military, and security officials.7 Following
on the heels of the victorious Wehrmacht, Einsatzgruppen, along with numerous
German Police battalions and SS units, massacred almost three-quarters of
a million Soviet Jews in the first ten months of the invasion.
In late 1941 and into early 1942, the decisions and the first steps to exterminate all other European Jews were taken. In January 1942 Heydrich met with senior officials from the SS, the Police, the Foreign Office, the Nazi Party, and Reich Chancellery and Ministry of Justice in the Berlin suburb of Wannsee to map out the campaign of mass murder. As a guide to future plans, Heydrich referred to the earlier order from Goering for the Final Solution he claimed he had drafted. Like the earlier order signed by Goering, the decision at the conference to exterminate Europe’s Jews was taken after several actions already had begun. These included the operations of the first death camp at Chelmo in western Poland that had started already in December 1941. A month earlier, in Kovno, Lithuania, police units began murdering German Jews who had been deported there. And the construction of Auschwitz-Birkenau, begun in December 1941, was accelerated. The time of “night and fog” (Nacht und Nebel) over Nazi-occupied Europe had arrived.
The Holocaust differed in two significant ways from other examples of twentieth-century genocide. First of all, the Nazis did not simply indulge themselves in an orgy of massacres and other atrocities. Instead, to facilitate their policy of extermination, the Nazis borrowed from the panoply of twentieth century science and technology. They adopted techniques, equipment, and processes from engineering and basic sciences such as chemistry. They also adapted modern business methods, technology, and techniques from the fields of accounting, administration, transport, and bureaucratic organization. The Nazi and SS hierarchy took an avid interest in the progress of the extermination and demanded constant reports from subordinates, units in the field, and the death camps. While not always consistent and efficient, the Nazi machine was organized along methods that allowed for measurements of progress and any necessary “improvements” to the existing system.
Secondly, the Nazis moved to eliminate Jews from countries they had overrun during the war. The extranational nature of the Holocaust was the aspect that most differentiated it from other examples of genocide in the twentieth century, such as that perpetrated on ethnic Armenians by the Ottoman Turks in 1915-16 and the Khmer Rouge depredations in Cambodia from 1975 to 1979. As the Germans conquered Europe, they immediately established the administrative and enforcement machinery to seize and kill Jews living in the occupied countries. In this task various collaborationist regimes and local fascist organizations sympathetic to the Nazis helped the Germans in a number of ways. Some of these regimes murdered their own Jews or shipped them to the concentration camps in Eastern Europe. Other countries expelled non-national Jews and ethnic groups acquired through territorial acquisition, as happened in Bulgaria, or rounded up foreign Jews and shipped them to their deaths in the east as did France. The continental scope and bureaucratic nature of the extermination program compelled the Nazi apparatus to coordinate its efforts over an extensive region, from the Atlantic to the steppes of the western Soviet Union.
Both of these aspects of the Final Solution forced the Nazis to rely on modern communications to facilitate their efforts – both to satisfy the SS command in Berlin for progress reports and to coordinate their far-flung efforts over the European continent. Where underground cable systems existed, the Nazi authorities made use of them for their communications. The Allies could not monitor the ground cable system in continental Europe. However, the cable system’s reach was limited. Where the local system had been destroyed during combat operations or had not yet been constructed, and this was common in most of the regions of the western USSR, the command in Berlin had to rely on radio communications to control the particular missions of the dispersed police units and Einsatzgruppen, as well as receive reports on their “progress.” Meanwhile, diplomatic missions and nongovernmental organizations around Europe reported on the roundups and massacres they witnessed or about which they had received information. These communications also made it possible for Allied signals intelligence agencies to intercept, exploit, and disseminate information to their leadership about the Holocaust.
Previous Histories and Articles
For nearly thirty years after World War II, the United States and
Great Britain generally succeeded in keeping completely secret from the
public the story of the exploitation of the German Enigma and many other
Axis codes and ciphers.8 This success was largely possible because of
the willingness of the tens of thousands of wartime employees of the Western
cryptologic services to keep silent about the secret. However, there were
a few tears in the shroud over this period.
In 1967 a Polish historian, Wladyslaw Kozaczuk, published an account of the Polish breakthrough against Enigma, Bitwa o tajemnice (The Secret Battle).9 Kozaczuk was the first to state that the Poles had solved the German Enigma. This book, though, received little attention outside of Poland. In 1973 Gustave Bertrand, the former chief of the French Army’s radio intelligence branch, published his memoirs, Enigma: Ou la Plus Grande Enigme de la Guerre, 1939-1945 (Enigma: Or the Greatest Mystery of the War), which revealed some more about the early Allied exploitation of Enigma. 10 Bertrand had played a minor, if not unimportant role in the Polish breakthrough against Enigma. He had developed a contact in the German Ministry of War who turned over to him technical information and keying material for the Enigma. Bertrand, whose own country’s code-breaking capability had deteriorated in the years after the First World War, passed the material to the Polish Cipher Bureau. Like Kozaczuk’s work, Bertrand’s book made little impression. German historians, who had reviewed Kozazcuk’s book, tended to dismiss the claims that the Poles or any of the Allies had broken Enigma. The secret still stood into the 1970s.
Therefore, outside of intelligence circles, only a handful of people, mostly scholars, knew about the Allied cryptanalytic success against Enigma when, in 1974, F.W. Winterbotham’s book The Ultra Secret was published.11 Winterbotham was a Royal Air Force officer who formed the Special Liaison Units that distributed Ultra to military commands. But he was not involved in actual codebreaking and other cryptologic activities. Winterbotham’s book created a shock wave in the intelligence and historical communities. The British government contemplated legal action against him, but finally declined to prosecute him. The Ultra Secret went on to become a best seller. Many historians scrambled, perhaps prematurely, to incorporate Winterbotham’s revelations into the narrative of World War II. Winterbotham’s book, though, was full of errors, distortions, and omissions. Among other things, he attributed the solution to the Enigma to a spy who smuggled a complete machine out of Germany. This ignored the Polish contribution. He also claimed that the British won the Battle of Britain solely because of Ultra’s contribution. This claim ignored the far more critical roles played by Britain’s early warning radar network and Fighter Command’s centralized command and control system.
There was no mention in Winterbotham’s book about what information there might have been about the Holocaust from Ultra sources. Interestingly, prior to the publication of The Ultra Secret, there was some information already available about codebreaking and the Holocaust. The first inkling of what SIGINT records might hold about the Holocaust had appeared in Gustave Bertrand’s Enigma. In the course of describing the operations of a small, multinational, Allied covert communications monitoring site in southern France, Bertrand revealed that this site had intercepted and decrypted German Police messages that contained information about massacres in the western Soviet Union. Bertrand stated that the site, over the course of a little more than two years, had intercepted over three thousand German Police messages. He related the substance of a police message from 21 August 1941 that reported that 5,130 Jews had been shot by SS and police units.12 Most of Bertrand’s story was repeated in Wladyslaw Kozaczuk’s Enigma, published in Polish in 1979.13 The information about the German Police decrypts in the books from France and Poland largely went unnoticed. This was probably because the works were slow to be translated into English, and the rather meager information they contained about the massacres was subsumed within the interest generated by the much broader revelations about the effect of the exploitation of Enigma on the course of the war. Another problem was that Bertrand’s reference to the police decrypts was based largely on his recollections and personal notes. There were no copies of the intercepts or decrypts available.
The first account in English that discussed Ultra information in connection with the Holocaust was Walter Laqueur’s The Terrible Secret, first published in 1980. This book tracked the knowledge of the Holocaust among various groups such as German civilians, international Jewish organizations, officials of the major Allied and neutral countries, the leaders of the Jewish communities in Europe and America, and nongovernment organizations such as the International Red Cross. His account carried some additional details over the previous histories that had referred to the decrypts of the German Police messages. He also mentioned that the British were reading the Enigma messages of the German railway service, and, as a result, this suggested that the British had the capability to track the movement of trains throughout occupied Europe, which would have included the trains transporting Jews to the death camps in the east. It appears that Laqueur had some access or knowledge of British code-breaking efforts against the German Police, SS, and railway Enigma ciphers. He refers to some specific details from the police decrypts, but it is not clear if he had seen specific documents from GC&CS because they had not been released to the public.14 Laqueur also interviewed Peter Calvocoressi, an important figure in the operations of one of the analytic centers at Bletchley Park. He also may have been a source of Laqueur’s information.
In the same year as Laqueur’s book, Calvocoressi published his wartime memoir, Top Secret Ultra.15 Calvocoressi had a more detailed knowledge of the British code-breaking effort than Winterbotham. In this book, he painted a more detailed and technically accurate picture of the operations of the GC&CS’ center at Bletchley Park. Calavocoressi also added a gruesome detail that tied in Engima cryptanalysis with the Holocaust. He claimed that German cryptographers used the numbers of the slave labor population from the daily SS concentration camp reports, encrypted in a “medium grade cipher,” to set the keys (wheel settings) for their Enigma machines.16 But like the previous works, Calvocoressi used no source documents. They remained classified.
The publication that provided the first detailed description of what allied communications intelligence knew of the Holocaust was Sir Harry Hinsley’s five-volume series, British Intelligence in the Second World War, published between 1979 and 1990. Volume II, published in 1981, contained an appendix on the police decodes which described the operations of the Einsatzgruppen and police units on the Eastern Front and the SS messages about the death and labor camps.17 Hinsley, who had worked on the German naval intelligence section at Bletchley Park during the war, and his editorial and writing staff had been given access to GC&CS records, decrypts, and translations of intercepted Axis messages for the purpose of writing the history. Again, the material, while cited in the work, itself remained classified and inaccessible to the public and would remain so some fifteen years after the first volume of Hinsley’s history had been published.
Hinsley’s work also was based partly on a multivolume history of wartime activities produced by the GC&CS in the years immediately after the war. Volume XIII of the Air and Military History series concentrated on the exploitation of the communications and encrypted messages of the German Police and other Reich security organizations. It contained a short description of intelligence derived from concentration camps and an appendix that listed individual massacres on the Eastern Front contained in selected decrypted police messages.18 Many of these volumes would be released to the British archives in the 1990s.
In 1984 a monograph was published in the Journal of Historical Review that reviewed the published literature of wartime intelligence, including the Police decrypts, which carried information about the massacres and the concentration camps.19 The article called into question what the intelligence actually revealed about the Nazi’s ultimate plan for the elimination of Europe’s Jews. Unfortunately, the journal in which this article appeared was a well-known forum for that faction of scholars and researchers associated with a movement known as “Holocaust denial.”20 Rather than discuss the intelligence about the Holocaust and how Allied officials differed over its meaning, or review the Nazi program of silence and obfuscation about the Final Solution, the author claimed that the gaps in Allied intelligence suggested that many aspects of the Holocaust, such as the gassings at Auschwitz, were a fiction. However, the amassed evidence from captured records and the testimony of Holocaust victims and perpetrators overwhelms the article’s contention. Later releases of Police decrypts to the PRO would illustrate how the missing intelligence was attributable to greater German security measures and the limitations in the communications intelligence system.
Two shorter pieces that dealt with the German Police and the Holocaust appeared in the mid-1990s. The first was a monograph, SIGINT and the Holocaust, by Robert L. Benson, a senior researcher at the National Security Agency, known for his major role behind the release of the NKVD espionage messages referred to as VENONA. The other was a short piece in the New York Times Magazine, “The Holocaust Was No Secret,” by William J. Vanden Heuvel. The article in the Times probably was published in the wake of news stories, featured initially in early November 1996 in the Washington Post, about the discovery of decrypts of German Police messages in the NSA Record Group 457 at NARA.21 Both Vanden Heuvel’s and Benson’s articles focused on the revelations contained in the decrypts of the German Police and SS messages about the massacres in Russia. Both authors also grappled with the question of why Allied leaders, especially the British prime minister, Winston Churchill, who knew of the atrocities, did not do anything to publicize, impede, or stop them? As the two authors pointed out, there was really nothing that the Allies could have done to stop or appreciably retard the slaughter except to totally defeat the Nazis as soon as possible.22
As has been seen above, the critical element absent from all of these early works was the source material itself – in this case, the collection of decrypts of the German Police and SS messages. It also has been mentioned that an incomplete set of these decrypts had been discovered in RG 457 by American University history professor Richard Breitman in June 1996. This set of police decrypts was part of a larger 1996 NSA release of wartime records to the National Archives that numbered some 1.3 million pages. The set of German Police messages had been obtained by NSA from GCHQ in the mid-1980s and had been incorporated into the large release. The set that NSA had acquired was only a small portion of a much larger set of German Police messages that were still classified and resided in British archives. (See Chapter 3, page 62, about the background to the acquisition of these messages by NSA.) The publicity around the discovery of the police messages in the U.S. National Archives helped generate sufficient public pressure for the release to the public of the complete set of police decrypts by the Public Record Office in May 1997.
It should be pointed out, though, that since the early to mid 1980s, translations of Axis and Vichy diplomatic messages that contained limited information about the Holocaust had been available at the National Archives in the United States. These documents were located in the NSA Record Group 457. Beginning in 1979, NSA had turned over “redacted” (that is, copies of documents with the still-classified portions blacked out) copies of translations of Axis and Vichy diplomatic messages to the National Archives. Unfortunately, there was no topical index for the collections, which totaled almost 150,000 pages. Those individual translations that were relevant to the Holocaust lay buried away in their archival boxes and remained largely untouched by researchers for years.23
The first full use of any of these records in a historical study of the Holocaust did not occur until 1998 with the publication of Official Secrets by the same Professor Richard Breitman of American University in Washington, DC. The book contained a description of how the police messages were collected and processed by GC&CS. The largest part of Professor Breitman’s work looked at the response of the Allied governments to the course of the Holocaust, especially in light of information available from intelligence sources including the decrypts of the German Police and SS units. Using these, he was able, among other insights, to accord a far more significant role to the police units in the Holocaust, as well as reaffirm the importance of the Nazi policies of secrecy and deception that disguised the nature and scope of the Holocaust.24 His book also detailed the information contained in the SS concentration camp reports mentioned earlier by Calvocoressi.
It took nearly twenty-five years from the publication of Winterbotham’s book about the Ultra secret for many of the official records of the American and British wartime cryptologic agencies to be released to the public. Among the records were the collections of decrypts of translations of Axis and neutral messages, primarily German, which dealt with the Holocaust. The delayed release of the communications intelligence records did not impede scholarly research and understanding of the overall Holocaust phenomenon. Mountains of records had been available that documented Nazi planning and execution of the Final Solution. At worst, the lack of cryptologic material probably affected Holocaust scholarship of specific incidents. For example, the lack of police decrypts and translations may have contributed to scholars underestimating for decades the major role of the German Police in the massacres and other atrocities committed in the western USSR.25
Notes
Chapter 2
Overview of the Western Communications Intelligence System during
World War II
During the Second World War, the United States and Great Britain operated the principal Western Allied code-breaking agencies. The Commonwealth countries of Canada, Australia, and New Zealand, and Britain’s colony of India provided substantial support with personnel and material, especially in the Middle East, Pacific and Asia theaters of combat. There were lesser contributions from small European national detachments that had escaped from Poland, France, and the Netherlands East Indies. By war’s end, this multinational effort had brought to a high-level of proficiency a worldwide system that intercepted, decrypted, translated, and disseminated intelligence derived from Axis and neutral communications.1
A popular perception about Western communications intelligence and its subsidiary code- breaking function, often generally referred to as “Ultra,” was that this information was available from all Axis and neutral sources at all times to the Allied commanders and leaders. This view is simply not true. The war between the Axis cryptographers to devise and emplace systems to protect their communications and the Allied cryptologists to collect and exploit those same communications (and the mirrored struggle between Allied cryptographers and Axis cryptologists) was marked by victories and defeats on both sides. Although there were major successes by the Allies early in the war, notably the exploitation of some versions of the German Enigma and the Japanese diplomatic machine system, known as Purple, the struggle for cryptologic supremacy was not settled until the midpoint of the war when the full resources of the Allied cryptologic effort finally achieved a general and consistent inroad into most, but still not all, major Axis cryptographic systems. Even then, there remained gaps and shortcomings in the overall Allied capability that produced unpleasant tactical military surprises for the Allies later in the war, such as at the German Second Ardennes Offensive in December 1944. Several Axis and neutral cryptographic systems were never exploited due to a paucity of Allied intercept, the strength of the particular Axis code or cipher, or the late date of a cryptographic system’s introduction into operation. In short, the Ultra success by the Allies was never total or constant when measured in terms of the total number of enemy or neutral codes and ciphers that could be broken or the duration of their exploitation. Where the Allies succeeded was in the exploitation of those Axis communications and cryptographic systems that were critical to the conduct of certain battles and campaigns. This ability also allowed the Allies the ability to gauge the strategic intentions of Berlin and Tokyo in a depth that otherwise would have been lacking if there had been no recourse to the information from Ultra, Magic, and the host of other systems.2
Trying to describe or graphically represent the entire Western COMINT system can be difficult. Some initial attempts were made by the U.S. services. One of the more popular early graphic efforts was a chart devised by the War Department’s G-2 (Intelligence), “A Message from Originator to MIS.” This version was taken from the postwar Army history The Achievements of the Signal Security Agency in World War II and is reproduced here.3 It illustrated the system that started with the transmission of a message by an originating entity. It then detailed several intermediate analytic steps to the point where the translation of the original intercept was disseminated to the War Department.
The system this chart illustrated was limited to those cryptologic functions from intercept to translation that were performed by the Army’s Signal Intelligence Service at Arlington Hall. The chart failed, though, to describe why a certain message was to be intercepted in the first place. There were numerous Axis and neutral radio terminals and networks to intercept. What was the process that determined which one was monitored? Similarly, the chart also left out what happened to the intelligence after it reached the Pentagon. To whom did this information go and how did it get there? Was it handled any differently than intelligence from other sources such as captured documents? In short, the chart failed to explain the place and role of cryptology within the context of the activities of the larger Allied intelligence system.
The production of Allied communications intelligence during World War II was a multistep system. It began with the determination of a hierarchical priority of intercept of Axis communications and was completed with the dissemination of the intelligence derived from it. This system can be likened to a “closed cycle” in which all steps were interrelated. Within this process a significant change to one step affected all the others. For example, a reordering of requirements due to a change in capability or a crisis in a military theater’s situation would affect what was collected and processed. The rise or decline of cryptanalytic effectiveness, efficiency, or advances in technology or techniques could affect what terminals would be monitored in the future and what intelligence gained from them would be disseminated and to whom it would be sent.4
Besides being distinguished by the interdependent nature of its operations, the COMINT system, especially as practiced by the British, was noteworthy for the operational interaction of its analysts That is, individuals at all points in the system contributed information that others could use in their separate jobs. This interaction deve-loped partly from the background of a large number of the individuals who worked in the cryptologic agencies during the war. Many people who worked at Bletchley Park and Arlington Hall were current students, graduates, or faculty recruited from universities and other schools that prized cooperation and an intellectual detachment. And a majority of these hires were women.5 More so, the leadership of the cryptologic agencies encouraged the continuation of this free flow of ideas and information. Even those who merely logged in intercepted messages were encouraged to contribute insights and observations.6 One high-ranking American observer, who visited Bletchley Park in May 1943, noted that the British personnel approach was “ bold and forward-looking… an imaginable conception of the possibilities of obtaining results from attention to details and infinite pains.”7 The result of this approach was a system, which was described by William F. Friedman as “unified at the top and operationally intimate below.”8
The wartime Western Allied communications intelligence system consisted of the following steps: setting requirements, priorities, and division of effort; intercepting messages; processing the intercept; and disseminating the resulting intelligence. Each step consisted of a number of subordinate processes that contributed to its completion, though each process was not employed against every intercept or cryptographic system. Also, each step was affected by a number of technical and institutional constraints, as well as political/strategic influences or contingencies that further determined how effectively a step was carried out. Many of the subordinate processes and the constraints and outside influences for each step will be described in the proceeding sections.
Two other aspects of the Allied COMINT system need to be considered before we proceed with the description. First of all, when we talk of an Allied system, we are really describing two distinct national systems, that of the United States and Great Britain. While both countries carried out similar communications intelligence activities, we shall see that there were a number of differences between them in organization, security restrictions, equipment, training, and even technical jargon. Like much of the rest of the special relationship between the United States and Great Britain, the wartime Anglo-American COMINT relationship was marked by a search for ways to make the two national systems work in a more cooperative fashion.
Another characteristic of the wartime allied cryptologic agencies was the phenomenal growth experienced by both as the war progressed. In 1939 cryptologic organizations that barely numbered a few hundred were, by war’s end, staffed by tens of thousands of civilians and military personnel. As a representative register of this massive growth, consider that the U.S. Army’s signals intelligence arm was producing about forty translations a week in January 1942. By the time Japan surrendered, it was turning out about 1,025 translations a week.9 Also, during the war, both the GC&CS and the SIS grew organically. They had to design, build, refine, and modify their organizations, equipment, structure, practices, and procedures while they also worked against Axis cryptography and communications. In 1939 American and British COMINT were local cottage workshops. By 1945 they had become a joint global industrial concern.
What follows is a description of the communications intelligence system with a particular eye to the ways in which its operations influenced how it obtained and distributed intelligence about the Holocaust.
Step 1: Setting the Requirements, Priorities, and the Division of Effort
For the Allies, perhaps the most difficult step in the COMINT process
was simply to decide what Axis communications to intercept, decipher,
and report. To visualize the potential size of the Axis communications
target is to grasp the scope and nature of the problem facing the Allied
cryptologic agencies, especially early in the war: thousands of radio
terminals on hundreds of radio networks around the world supporting Axis
military, naval, diplomatic, security, intelligence, and commercial entities.
All of these utilized hundreds of cryptographic systems from simple hand
ciphers to complex book codes and intricate machines such as Enigma, Purple,
and Tunny.10 Added to this initially uncharted wilderness were the hundreds
of military, diplomatic, and commercial communications networks of important
neutral and Axis-friendly countries, notably Vichy France, Turkey, Sweden,
Spain, Portugal, Switzerland, the Vatican, and those of Latin America.
These networks, too, had potential as sources of intelligence and could
not be ignored; many already were targets of Allied cryptologists.
Early in the war, the Allied communications intelligence effort could not adequately monitor the existing Axis communications networks, much less even hope to cover it completely. The British, beginning in September 1939, and later the Americans in late 1941, lacked the facilities, the personnel, and the technology to adequately monitor all of the Axis terminals that mattered. It would take time for the influx of money, training, and the development of administrative and technical support organizations to create the impressive Allied COMINT structure that existed by the end of the war.
Meanwhile, the disparity in capabilities between the Allied cryptologic
agencies and the Axis communications networks forced the intelligence staffs
in Washington and London to prioritize the Axis communications terminals
and cryptographic systems that had to be attacked. This prioritization was
determined by a number of general factors. First of all, there had to be
either an already demonstrated or a reasonably high expectation that intelligence
of value to the war effort could be derived from a terminal’s messages.
Another factor, which could override all others at times, was the perceived
current strategic or tactical military needs of the moment. For all practical
purposes, these factors were military-oriented. They dictated target listings
throughout the war and dominated all prioritization schemes from strategic
priorities to those in individual theaters of operation.
The COMINT agencies, too, had their own considerations to add to the
calculations for target priority. Usually, these considerations reflected
the agency’s own measure of its technical ability to intercept or
to exploit cryptanalytically a particular Axis terminal or a general target
category such as “Japanese military.” The feeling within the
SIS leadership, for example, was that tasking of radio links was far more
complex than to be left to the whims of intelligence officers. In early
1943, the SIS disagreed with the idea that the War Department’s G-2
could just hand it a list of radio terminals to go after. One memorandum
stated that there were other factors “in the chain of events leading
from G-2’s desires to actual results.” Rather, it continued,
the priority of a target radio terminal should be determined first by whether
a monitoring station could hear it and then whether any cryptographic systems
used in its traffic could be read or were being studied.11
As an example of how requirements were levied and how they changed with the war, it is useful to look at the experience of the SIS. For the overall U.S. COMINT effort, the SIS was responsible for intercepting and processing Axis military, weather, and air force communications traffic. It also was responsible for exploiting international diplomatic communications. This was a carryover from its success with the Japanese diplomatic cipher machine known as Purple. In April 1942, G-2 gave the Signal Security Service (SSS), as the Signal Intelligence Service was soon to be called, a set of priorities for collection and processing (which included cryptanalysis and translation).12 The first priority included all German, Japanese, and Italian military traffic. Second priority was given to all Axis military attaché communications. Axis diplomatic traffic among their respective capital cities was in the third priority. The fourth priority was for all so-called German “administrative” radio nets, which probably referred to German illicit intelligence and security radio networks. The diplomatic messages of other countries, minor Axis and neutrals, were spread across priorities five to eight.13
Nearly a year later, in March 1943, the priority list had changed somewhat. The priority list for collection and analysis was organized now into groups identified by letters that ranged from “A,” the highest, to “G,” the lowest. For the SSS, the Japanese Army was now the most important target. This was tagged “#1 Special Research Project,” which probably referred to the main Japanese army code that was still unexploited at the time.14 Weather traffic was also part of Group A, as were certain diplomatic links between Tokyo and Moscow, Berlin, and Rome. German military had fallen to the bottom of Group “B,” along with Japanese diplomatic messages to Europe (other than other Axis capitals) and “Security” (intelligence, espionage and security elements such as police) messages.15 The latter reference to “Security” traffic is interesting because the SSA still lacked the means to intercept consistently the communications of the German security agencies such as the Abwehr (military intelligence), the SD, and the German Police. The Americans had few monitoring sites that could intercept such communications in the European theater. There were some U.S. Army radio intercept companies in England and North Africa, but they were tasked with collecting tactical German military radio traffic. A note attached to the tasking for these intercept units indicated that the coverage of the German security elements was primarily for the purpose of tracking the volume of traffic and reporting this to Army counterintelligence.16
The SIS already was somewhat familiar with German Police communications and cipher systems. In January 1941, then SIS Major Abraham Sinkov, one of William F. Friedman’s original cryptanalytic staff, had headed a small technical exchange mission to GC&CS. It was this mission that provided the British with a working Purple and Red Japanese diplomatic cipher machines. During the ensuing technical discussions, one of Britain’s foremost codebreakers, John Tiltman, provided Sinkov with detailed information on how the so-called German Police cipher worked. He was informed that the descriptor “German Police” applied to the systems used by the Schutzpolizei, the SD, the SS, and the line Order Police battalions and regiments. Sinkov was told that the information in the police messages was useful for mapping associated German military units.17 However, at the time the U.S. had no way of intercepting these communications; the cryptanalytic information was useless except for training.
As for the remainder of the revamped 1943 SIS overall requirements, these consisted mostly of targeted diplomatic communications from most minor Axis, neutral, and some minor allied nations. As in the previous year’s priorities they remained spread across the lower priority Groups from “C” to “G.” For example, the tasking requirement for intercept of diplomatic messages between Washington and major neutrals was now at the Group “D” level. The requirements to intercept diplomatic messages from the Vatican, Latin America, and European countries to elsewhere other than Washington were covered in Groups “E” and “F.”18
In the various combat theaters of operation, cryptologic working arrangements between the British and Americans followed de facto national theater command responsibilities. In Europe the British were preeminent for the first two years of the war. A gradual cooperative effort with the United States Navy’s OP-20-G cryptologists developed during the lengthy U-boat campaign in the Atlantic. In the European Theater of Operations (ETO), the GC&CS, and the supporting British service units, remained the principal Allied cryptologic agency when it came to the Axis military and air force. In the Pacific theater, the American army and naval cryptologic organizations supervised allied intercept and code-breaking operations. In the China-Burma-India Theater, there were British stations in Ceylon and India that had American contingents. Also, Canadian, Australian, and New Zealand personnel served at various Pacific and Asian sites with the Americans and British.
These worldwide operations required that the United States and Great Britain eventually had to establish corresponding divisions of effort and responsibility for setting requirements, intercept, processing, and dissemination. Otherwise, there would be problems with redundant operations, and misallocated resources. Also, the rules, methods, material, and channels for technical and intelligence exchanges needed to be ironed out between the two countries. The two countries had to make a general agreement to govern operations around the world. For the first eighteen months of the war, the GC&CS and the SIS and OP-20-G had a number of separate and limited reciprocal interchanges. By mid-1943, these various working relationships were institutionalized in a series of agreements between the British and the U.S. War Department, which culminated in the so-called BRUSA (Britain-United States of America) Agreement of June 1943.19 (The U.S. Navy would sign a separate, more limited, agreement. It was known as the Extension Agreement and applied to an earlier 1942 exchange arrangement known as the Holden Agreement.)
The main provisions of the BRUSA Agreement were the exchange of technical intelligence (sources and methods) and a division of effort in the daily activities of collection, analysis, and dissemination of intelligence. The two Allies formally agreed to the exchange of finished intelligence. There was no exchange of “raw” (undecrypted) intercepts, except for U-boat messages, and possibly some examples for training purposes.20 The monitoring sites, though, would send their “raw” intercept to the national center responsible for processing it. The agreement did not include so-called “nonservice” (nonmilitary) intercepts. This uncovered category principally entailed Axis and neutral diplomatic and intelligence-related (illicit espionage and internal security organization) messages. Both countries continued individually to decrypt and exchange diplomatic translations, though a later mechanism for exchange between GC&CS and Arlington Hall was established in August 1943. As a result of this later agreement, the two Allied cryptologic agencies targeted the diplomatic traffic of every major Axis power, minor Axis ally, minor Allied power, and significant neutral, though the total amount collected against each country varied. As for the category of Axis intelligence messages, another division of effort was arranged. The British collected and processed German intelligence and security-related messages in occupied Europe: the United States, principally the United States Coast Guard, collected and processed Axis overseas illicit and overt espionage radio traffic, notably Abwehr messages from Latin America, North Africa, and the Far East.
In essence, the BRUSA Agreement formalized the realities of the division of effort in the combat theaters that had existed since 1942. The Americans took the responsibility for Japanese service and nonservice communications, while the British oversaw that of the German and Italian military and security forces. This agreement did not preclude either country from collection and analysis of each other’s missions. In fact, both sides exchanged personnel who were then integrated fully into each other’s efforts. Three special SSA units were established in Britain to work alongside the British in the areas of collection, analysis, and security. American cryptologists could be found at various British intercept and analytic sites, including Bletchley Park. U.S. radio intercept units stationed in England and North Africa intercepted and analyzed Axis communications in the European and North African theaters, but they passed the take to the British analytic centers.21
Great Britain retained authority over the production of communications intelligence in Europe, while the Americans controlled similar activity in the Pacific. Because of this division of effort, German military communications remained a lower priority for the U.S. Army codebreakers at Arlington Hall Station for the rest of the war.22 Tactical communications intelligence, that is, intercept and analysis of plaintext messages or those encrypted by low-level codes and ciphers that were sent by Axis tactical or operational-level combat units, was produced by Allied cryptologic elements attached to Allied military commands or units in Europe and North Africa. Theater or local commanders controlled the operations of these units and were the main recipient of that type of communications intelligence.
As mentioned above, the priorities for collection and processing shifted during the war according to current strategic needs and capabilities. This shift was especially obvious when it came to diplomatic targets. A good example of the shifting priorities of an individual target country’s communications can be seen in the Allied effort against Switzerland’s diplomatic radio traffic. During previous European conflicts, Switzerland had remained neutral. As a result, that nation had taken on certain roles such as representing the foreign interests of belligerents. These included monitoring the conditions in prisoner-of-war and internee camps, supporting International Red Cross relief activities, and allowing Swiss locales to be used for unofficial contacts between combatants. Considering these historic roles, Switzerland’s diplomatic communications naturally were a target of interest for the Allied cryptologic agencies.
The GC&CS had been analyzing Swiss diplomatic messages since 1939, and the Americans began their separate attack in December 1942. But both agencies produced few translations during this period. There were some technical reasons for this paucity of communications intelligence. One was that for their European message traffic the Swiss relied mostly on the European cable network that was inaccessible to Allied communications monitoring sites. In addition, the Swiss used a large number of codes and ciphers – over ten systems, mostly manual ciphers and codes, and an early version of the Enigma cipher device for their diplomatic traffic. Yet, the most important reason for the small output of translations of Swiss traffic was that, early in the war, the messages to and from Bern appeared to carry little intelligence of value.23 The intercept of Swiss diplomatic messages continued for the purposes of training and cryptanalytic continuity, but not for reporting intelligence. By early 1943, the Americans downgraded the intercept priority of various Swiss diplomatic terminals to fourth and fifth priority on its list.24
This situation changed in the summer of 1944 when the Allies discovered that the Swiss diplomatic cables now carried important information on the conditions in the Balkan capitals that were being overrun or threatened by the advancing Red Army. Of particular interest was the situation in Budapest, Hungary. The several changes in governments and the German-supported coup in October 1944 had kept the Hungarians in the conflict. Furthermore, the Germans were in the midst of a roundup, which had begun in the summer, of the sizable Jewish population in Hungary for transport to the death camps and slave labor details in German war industries. A review of the diplomatic translations from this period issued by the SIS shows a marked increase in the number of published translations of Swiss diplomatic messages. Allied interest in the Swiss diplomatic messages continued late into the war as concern grew over the possible fate of prisoners and internees under Japanese control. The Swiss, working in concert with the International Red Cross, reported their findings on camp and prisoner conditions throughout Asia. Another point of interest for the Allies concerned the activities of German banking officials who were negotiating payments and commercial accounts with the Swiss for German purchases of war material and currency exchanges. (See pages 104-110 for more details on this last point.)
One of the problems that bedeviled Allied cryptologists who tried to set requirements was that the intelligence value of the information carried on various communications networks or individual terminals could not always be predicted. The above example of Switzerland shows how reality did not always match expectations. On the other hand, useful intelligence could come from unexpected sources. For example, one of the best sources for insight into Nazi Germany’s strategic plans and Hitler’s own appreciation of possible Allied moves came from the Purple decrypts of Japan’s ambassador in Berlin, Baron Oshima Hiroshi. Oshima had developed a rapport with most of the Nazi hierarchy, especially Hitler and Joachim von Ribbentrop, the German foreign minister. They confided to Oshima some of the Reich’s most important secrets. These personal connections made his long reports to Tokyo a gold mine of information on a number of topics, from German development of new weapons to Hitler’s own appreciation of the Allied troop strength and strategic options for landing in Western Europe in 1944.25 Oshima reported nothing on the concentration camps or massacres in the western Soviet Union, though a few of messages to Tokyo suggest he was aware of the situation of the Jews. (See pages 100-101)
In a similar fashion, British codebreakers were probably surprised by the first bits of information about the massacres and camps that appeared on certain German Police radio networks, such as those servicing the police units in the Soviet Union and SS and SD units in occupied Europe. The Allied intercept and processing of German Police and SS communications originally was performed as a supplementary source to intelligence about the German military, principally to gather administrative data and order of battle information. The police and SS communications also contained intelligence on German civilian morale and other domestic security concerns, information about the results of Allied air raids, and escapes by Allied POWs.26 The police messages were also targeted since the manual cryptographic system used was similar to systems used by Wehrmacht units.27
However, London and Washington would never make gathering intelligence about the fate of Europe’s Jews and other groups targeted for destruction by the Nazis a major requirement for agencies like GC&CS and SIS. Both would collect and process some information about the roundups, massacres, and the camps from Police and diplomatic communications. Yet, this information was a byproduct, perhaps even incidental to the coverage security and diplomatic nets. The primary purpose of the COMINT requirements placed on Bletchley Park and Arlington Hall was to gain intelligence dictated by the military exigencies of the war. Whatever advantage was gained from the intercept of Axis communications related to the Nazi atrocities was of little use to the current Allied prosecution of the war, except in a limited way for propaganda against the Axis.28 Later, the British would gather it with the intent to use intelligence as evidence in proposed postwar war crimes tribunals. (See pages 50-51 for eventual fate of this intelligence.)
Step 2: Intercepting the Messages
After the requirements had been set, the next step was intercepting
the radio terminals that were designated the highest priority targets.
It was at this juncture in the communications intelligence system that
the general requirements had to be translated from the vague entries such
as “German military” to a list of targeted radio stations
that could be monitored by Allied radio intercept personnel. Before discussing
the allotment of target stations to monitoring stations, it is necessary
to describe the communications the Allies monitored. There were two major
types of communications media that were targeted: cable (or wire) and
radio. Both had their advantages and shortcomings.
The first of these, cable intercept, proved to be a useful but limited source of communications intelligence. At the war’s outset, both the United States and Great Britain invoked wartime information contingencies and imposed restrictions on telegram and cable traffic to, through, and from both countries. All such wire traffic – personal, commercial, and diplomatic – was subjected to censorship review. Each resident foreign diplomatic mission was required to submit a copy of each message it sent, even in its encrypted form, to the appropriate government censorship office. The main reason for this was to control the flow of information out of the country that was critical to the war effort.29 In England, the General Post Office oversaw this program, sometimes referred to as “traffic blanketing.”30 The British also implemented censorship practices at all overseas Commonwealth cable terminals located in such places as Malta, New Delhi, Gibraltar, and Barbados.31 In the United States, the government, through the Office of Censorship run by Byron Price and staffed by civilians and contingents from the Army and Navy, received copies of all such wire traffic (before transmission overseas or upon receipt from an overseas terminal) from the various cable and wireless telegram companies such as Mackay Radio, Western Union, Radio Corporation of America, and Global Wireless.32 These companies had terminals in New York, Washington, DC, New Orleans, Louisiana, Galveston, Texas, and San Francisco, California. Copies of cable traffic that passed through overseas U.S. and British terminals on to other countries were also collected under the wartime restrictions. This coverage affected mostly diplomatic, personal, and commercial cable traffic between Europe, the United States, Latin America, and Africa.
This cable intercept had the primary advantage of simplifying the intercept problem. All messages flowing through an American or Commonwealth terminal were accessible to Allied cryptologists. Another advantage was the errorless copy of the intercepted cable message; each message usually was free of transmission garbles and missed cipher groups that could hinder exploitation, except for errors by the originator’s code clerk. The collection of cable messages was inexpensive for the cryptologic agencies since the national censorship offices performed the task.
On the other hand, there were limitations to this collection method. For example, only cables sent between Washington (and other U.S. terminals with overseas connections such as San Francisco, New Orleans, and New York) and an overseas terminal were available to the Americans. Messages sent on cables not routed through American- or Commonwealth-controlled terminal stations were beyond intercept. There were few means then to obtain these cables except through covert means, such as placing taps or suborning cable clerks. These methods risked eventual exposure and were difficult to hide and continue without discovery by foreign security agencies.
Another difficulty was that some of the targeted countries knew or suspected that the Allies probably were reading their cable traffic. To defeat the intercept and possible exploitation of their cables, some diplomats were warned by their capitals to take certain precautions when sending cables, even encrypted ones. For example, Axis diplomats in Europe and the United States were told to be careful when they included excerpts of public speeches and newspaper stories in their messages. They were warned not to quote them verbatim because the excerpts were also available to Allied codebreakers and could be used as “cribs” to break the enciphered messages. 33 To defeat the Allied cable censors, some countries sent sensitive correspondence by diplomatic pouch or courier. Of course, the courier pouches could be searched covertly (“tossed” was the expression) by Allied agents. But this technique was difficult to arrange and, if discovered, was a possible source of diplomatic embarrassment. It appears that Allied agents probably did this anyway. In early 1942, the U.S. Office of Strategic Services, and its predecessor, the Office of the Coordinator of Information, received copies of diplomatic messages from the British Security Coordination. A number of these messages appear to have been obtained in this manner.34 (See pages 65-6).
In occupied Europe, Nazi military and occupation authorities made use of cable telephone or telegram systems where available, thus denying the Allies access to these communications. Also, during the years of the war, the Germans, especially the military, expanded, repaired, and improved the European cable backbone network in occupied Europe in order to support their operations. In four years, the German army nearly doubled the size of the European cable trunk network to 6,900 miles.35 This cable expansion denied more intercept to the Allies.
The other and much larger and lucrative source of Allied intercept was radio communications. Since its introduction at the turn of the century, radio communications had expanded dramatically. As technology advanced and industry was able to mass-produce components and radio sets, the speed and distances that information could be transmitted grew. Many advanced countries had developed complete national-level communications networks that served private and public functions. When the war began, governments quickly took over many facilities and converted them to military or other official uses. The use of these in place systems presented more problems for Allied intercept stations since they had to match Axis capabilities.
Many Axis and neutral diplomatic missions, not needing or trusting the Allied-controlled cable systems, transmitted their diplomatic messages by long-range, high frequency (HF) radio.36 Axis naval and air force units and organizations relied on long-range HF radio for their communications, unless ground-based cable systems were available. Some communications operated on lower frequency bands similar to those used by commercial broadcast stations. Many Axis units used low-power high frequency radios to communicate with their local commands that reduced the chance for interception by Allied monitoring stations.
The Axis used different modes of communications on the many frequency bands. The most common mode was manual morse, the famous “dit-dah” or on-off keying of a continuous wave signal. This mode appeared on all frequencies, and all service and government elements used it. The Axis also used teleprinter systems for high- volume traffic among command centers, though this appeared late in the war. Voice communications was also heavily used. Most often this appeared at higher frequency ranges. The most common users were aircraft pilots communicating among themselves and with ground controllers. Some aircraft and ground stations used long-range HF voice systems, though this was rare because of the lack of security.37
German Police units operating in the western USSR used both long-range high frequency and the shorter-distance low frequency and medium frequency manual morse communications to send reports about their activities from major command centers in Minsk and Kiev to headquarters back in Berlin. Police battalions received radio or courier reports from companies and detachments in the field. The battalions, in turn, sent their radio reports over HF radio to regimental headquarters in the region in which they operated.38
For the Allies, intercepting Axis and neutral radio communications of interest presented a myriad of problems. The most obvious problem, as mentioned earlier, was simply the number and location of Axis radio terminals – perhaps several thousand. In order just to hear these radio stations meant that the Allies had to develop extensive corps of thousands of radio intercept operators to man the monitoring stations to collect the Axis and neutral communications for intelligence.
The major Allied radio intercept organizations were, for the British, the Radio Security Service (belonging to M.I.6), the General Post Office, and those units of the various services of the British armed forces, known as the “Y-Service.” For the Americans, the radio intercept personnel were assigned to the Signal Intelligence Service (Army) and organized under the Second Service Battalion, OP-20-G (Navy), the United States Coast Guard, the Office of Strategic Services, and the Federal Communication Commission. Canada operated the Examination Unit of the Canadian National Research Council. A number of men and women from other Commonwealth countries also performed as intercept operators. These included personnel from Australia, New Zealand, and India. Commonwealth military personnel who performed intercept services were organized into units known as Special Wireless Groups. Additionally, contingents from smaller Allied forces also intercepted Axis communications.
All of these personnel were stationed at monitoring facilities located around the world. These facilities ranged from large field stations, such as at Beaumanor, Leicestershire, England, or Vint Hill Farms in Virginia, to small direction finding huts in Alaska or Scotland. It is difficult to get a precise total number of Allied monitoring sites. Many stations did exist for the duration of the war. Some served only to support a campaign. Many stations were limited to a direction finding function only. Some sites, such as the U.S. Navy station at Muirkirk, Maryland, originally were commercial radio stations that were taken over by the military. American sites, including those of the Army, Navy, Coast Guard, OSS, and FCC numbered over seventy-five. British and Commonwealth stations numbered around sixty and were located in the British Isles and overseas in Australia, Ceylon (Sri Lanka), Egypt, Palestine, Malta, Iraq, Kenya, and India.
Generally, Allied overseas monitoring stations supported the commands in the theater of operation in which they were located. For example, in early 1942 U.S. Army radio intercept companies that were assigned to locations in the British Commonwealth, such as in Northern Ireland, or in the Atlantic region, at sites in Iceland and Newfoundland, intercepted German military, naval, or espionage communications.39 In the Pacific and Asian theaters, some field stations were manned by a mix of personnel from the United States and Commonwealth countries, such as the large sites at Central Bureau Brisbane and the Wireless Experimental Center in India. Other facilities were manned almost exclusively by personnel from one nation, such as the Americans at the U.S. Navy radio intercept site in Hawaii and the British in Ceylon. On the other hand, in the Southwest Pacific area of operations, Australian detachments accompanied SIS units to forward intercept sites in New Guinea and the Philippines to support General MacArthur’s advance.40
Most field sites were controlled and manned by one service element: army, navy, air force, or security service, such as M.I.6 or the OSS. This arrangement was not absolute; other service contingents or nationalities could be present. Also, intercept missions could cross service boundaries and responsibilities. For example, early in the war, U.S. Navy sites did collect diplomatic communications. But service priorities usually prevailed when it came to mission assignment. For example, a tentative 1943 intercept assignment list for the major SIS monitoring station #1, located at Vint Hill Farms, Virginia, was dominated by international diplomatic and commercial terminals. The number one assignment block included all government traffic between Tokyo and Rome, Bern, Vichy, Ankara, and Stockholm. The second and third tier priority was both German and Italian weather traffic (known as “WX”). The fourth priority was all government traffic to and from Berlin to Manchukuo, Madrid, occupied China, and