The Code-Breakers

For those with a serious interest in the history and practice of cryptography, David Khan’s The Code-Breakers: The Comprehensive History of Secret Communication from Ancient Times to the Internet provides an enormous mass of knowledge. The scope of the 1200 page book is vast: covering everything from the earliest ciphers to the origins of public key cryptography in detail. It is probably fair to say that the period best covered is that between the Middle Ages and the Second World War, though the sections covering the decoding of Egyptian hieroglyphics and Linear B are also detailed and skilfully written. Those interested primarily in the contemporary practice of cryptography – or those seeking a more concise text – would be well advised to consider the books by Simon Singh and Bruce Schneier on the topic.

Khan’s book excels in actually describing how various cryptographic systems work, as well as how they were broken. For the most part, his analysis is factual and dispassionate. The sole exception is in the period covering the Cold War, in which his ire against the Soviet Union and those Americans who turned into traitors for it are acute. At times, the book gets into an excessive amount of detail about the bureaucratic organization of different cipher bureaus: including lengthy sections about how various wartime bodies were reorganized. In most cases, the book does not provide much biography on the men and women involved, though exceptions exist in the case of some of the most eminent or interesting cryptographers. The book does provide an interesting discussion of the history of writing on cryptography, including the impact that major publications had on the development of the field and its comprehension within society at large. Kahn also does a good job of debunking some of the many spurious claims that have been made about ‘revolutionary’ and ‘unbreakable’ cryptosystems that people have invented: stressing how the making of cryptographic systems is a realm of abstract mathematics, while the breaking of such systems is a gritty and practical exercise.

In addition to covering the techniques of cryptography and cryptanalysis themselves, the book covers many related security issues: including physical security, invisible inks, elements of spycraft, decisions about how to use information gleaned through cryptanalysis, and the use of broken cryptographic systems to transmit fake or confusing information. The book also covers the relationships between cryptographic work and the activities it is supporting. An especially intriguing section details the efforts of the American navy to combat rum smuggling during the prohibition era. Ships with floating cryptoanalytical laboratories provided vital intelligence to interception vessels, just as other cryptanalysis had helped re-direct U-boats away from German submarines during the Second World War. The book covers an enormous variety of code systems, ranging in use and sophistication. These include diplomatic and commercial systems, high level military systems used between major installations, systems for vehicles, trench codes for those on the front lines, and more. The most abstract section of the book contemplates communication between human beings and extraterrestrials, covering questions about how we could recognize alien communication, as well as mathematical steps through which a comprehensible discourse could potentially be established.

For those interested in actually breaking codes and ciphers themselves, the book provides detailed information on techniques including frequency analysis, factorization attacks of the kind used against polyalphabetic substitution ciphers, and the index of coincidence. It also provides a lot of information on the weak ways in which cryptography is often used and the kinds of errors that have allowed for key breaks into previously unreadable cryptosystems. While it would not be especially useful for attacking modern computerized cryptographic systems, it would provide some guidance for those seeking to break into amateur or puzzle-type cryptographic challenges.

The Code-Breakers may well be the most comprehensive cryptographic history available, though it is far less detailed in its description of post-Cold War cryptosystems than some of its more concise recent contemporaries. For those wishing to gain an appreciation for how cryptography emerged, the role it played for most of human history, and the techniques that have been employed to guard and attack messages, this is an ideal place to turn.

Author: Milan

In the spring of 2005, I graduated from the University of British Columbia with a degree in International Relations and a general focus in the area of environmental politics. In the fall of 2005, I began reading for an M.Phil in IR at Wadham College, Oxford. Outside school, I am very interested in photography, writing, and the outdoors. I am writing this blog to keep in touch with friends and family around the world, provide a more personal view of graduate student life in Oxford, and pass on some lessons I've learned here.

20 thoughts on “The Code-Breakers

  1. Homomorphic cryptosystems are ones where mathematical operations on the ciphertext have regular effects on the plaintext. A normal symmetric cipher — DES, AES, or whatever — is not homomorphic. Assume you have a plaintext P, and you encrypt it with AES to get a corresponding ciphertext C. If you multiply that ciphertext by 2, and then decrypt 2C, you get random gibberish instead of P. If you got something else, like 2P, that would imply some pretty strong nonrandomness properties of AES and no one would trust its security.

    The RSA algorithm is different. Encrypt P to get C, multiply C by 2, and then decrypt 2C — and you get 2P. That’s a homomorphism: perform some mathematical operation to the ciphertext, and that operation is reflected in the plaintext. The RSA algorithm is homomorphic with respect to multiplication, something that has to be taken into account when evaluating the security of a security system that uses RSA.

    This isn’t anything new. RSA’s homomorphism was known in the 1970s, and other algorithms that are homomorphic with respect to addition have been known since the 1980s. But what has eluded cryptographers is a fully homomorphic cryptosystem: one that is homomorphic under both addition and multiplication and yet still secure. And that’s what IBM researcher Craig Gentry has discovered.

  2. History of the US-USSR hotline
    Posted by Cory Doctorow, July 20, 2009 12:19 PM

    Here’s a pieced-together social and technical history of the Kremlin-White House hotline, a fascinating story of crypto, diplomacy and wicked hardware:

    The method to be used was one-time tape. Section 4 of the annex to the memorandum stated: “The USSR shall provide for preparation and delivery of keying tapes to the terminal point of the link in the United States for reception of messages from the USSR. The United States shall provide for the preparation and delivery of keying tapes to the terminal point of the link in the USSR for reception of messages from the United States. Delivery of prepared keying tapes to the terminal points of the link shall be effected through the Embassy of the USSR in Washington (for the terminal of the link in the USSR) and through the Embassy of the United States in Moscow (for the terminal of the link in the United States).

    For its one-time tape hardware, the US would employ the ETCRRM II, or Electronic Teleprinter Cryptographic Regenerative Repeater Mixer II. One of many ‘one-time’ tape mechanisms sold by commercial firms, it was produced and sold for about $1,000 by Standard Telefon Kabelfabrik of Oslo, the Norwegian subsidiary of International Telephone and Telegraph Corporation, the same company which installed the American terminal in the National Military Command Center deep within the Pentagon. It has four teleprinters — two with English alphabet and two with Russian — and four associated ETCRRM II’s . In Moscow, the terminus was installed in the Kremlin, near the office of the Premier”.

  3. Thousands call for Turing apology

    Thousands of people have signed a Downing Street petition calling for a posthumous government apology to World War II code breaker Alan Turing.

    Writer Ian McEwan has just backed the campaign, which already has the support of scientist Richard Dawkins.

    In 1952 Turing was prosecuted for gross indecency after admitting a sexual relationship with a man. Two years later he killed himself.

    The petition was the idea of computer scientist John Graham-Cumming.

  4. “Alan Turing was the greatest computer scientist ever born in Britain. He laid the foundations of computing, helped break the Nazi Enigma code and told us how to tell whether a machine could think.

    He was also gay. He was prosecuted for being gay, chemically castrated as a ‘cure’, and took his own life, aged 41.

    The British Government should apologize to Alan Turing for his treatment and recognize that his work created much of the world we live in and saved us from Nazi Germany. And an apology would recognize the tragic consequences of prejudice that ended this man’s life and career.”

  5. An apology to Turing would be a very positive thing for the British government to do.

  6. 200-Year-Old Cipher Finally Cracked

    “A code expert just cracked a code used by a friend of Thomas Jefferson in a letter written to Jefferson some 200 years ago. This code is fairly easy to crack using a computer, but extremely difficult without one. I think it would have been much harder if the author had not included an indication as to what code algorithm he used in the letter accompanying the coded message.”

  7. 60 Years of Cryptography, 1949-2009

    “2009 marks 60 years since the advent of modern cryptography. It was back in October 1949 when mathematician Claude Shannon published a paper on Communication Theory of Secrecy Systems. According to his employer at the time, Bell Labs, the work transformed cryptography from an art to a science and is generally considered the foundation of modern cryptography. Since then significant developments in secure communications have continued, particularly with the advent of the Internet and Web. CIO has a pictorial representation of the past six decades of research and development in encryption technology. Highlights include the design of the first quantum cryptography protocol by Charles Bennett and Gilles Brassard in 1984, and the EFF’s ‘Deep Crack’ DES code breaker of 1998.”

  8. AES explained by stick figures
    Posted by Cory Doctorow, September 23, 2009 3:05 AM

    If you’ve always wondered how AES — the Advanced Encryption Standard, the gold-standard for crypto — works, and if you enjoy explanations in stick-figure cartoon form, you are in luck, for Moserware’s “A Stick Figure Guide to the Advanced Encryption Standard (AES)” is funny, lucid and fascinating.

  9. Despite the drastic layoffs, the small cadre of US and British codebreakers excelled against the new “main enemy,” as Russia became known. The joint US-British effort deciphered tens of thousands of Russian army and navy messages during the mid-to-late 1940s. But on October 29, 1948, as President Truman was about to deliver a campaign speech in New York, the party was over. In what became known within the crypto world as “Black Friday,” the Russian government and military flipped a switch and instantly converted to new, virtually unbreakable encryption systems and from vulnerable radio signals to buried cables. In the war between spies and machines, the spies won. The Soviets had managed to recruit William Weisband, a forty-year-old Russian linguist working for the US Army, who informed them of key cryptologic weaknesses the Americans were successfully exploiting. It was a blow from which the codebreakers would never recover. NSA historians called it “perhaps the most significant intelligence loss in US history.”

  10. Pingback: The Secret Sentry
  11. Thursday, June 22, 2006
    Speak friend and enter

    CSE and other agencies of the Canadian government have spent more than $42 million over the last three years buying updated secure telephone units and other equipment from the NSA, reports Elizabeth Thompson (“No bidding war for U.S. spy agency: Big buyers won’t say what they’re paying for,” Montreal Gazette, 21 June 2006). The secure telephones are replacing an earlier generation of STU-III secure telephones that were also bought from the United States. Users who are equipped with the phones can carry on an encrypted conversation with another person who has one of the phones that cannot (in practical terms) be decrypted by an eavesdropper. The units enable government ministers and high officials to discuss matters up to and including Top Secret Codeword classified material over the phone.

    The U.S. uses the same kind of phones for its own secure conversations; they are very likely the best available. There is a risk involved in buying such equipment from another country, however. As the Gazette story suggests, it is not at all impossible that our close ally has put a “back door” in the phones (and the other communications security equipment) that it has sold to us—for that rainy day when it feels the need to know what the Canadian government is really thinking on some trade issue, or border security matter, or foreign policy question. The neat thing about having a back door into the system—from the NSA’s point of view—is that it would enable the U.S. government, but nobody else, to listen in to Canada’s “secure” conversations.

  12. Al Qaeda Secret Code Broken
    By Bruce Schneier

    For six months, the code-breakers worked around the clock deciphering the code the three terrorists created.

    Between them, the code-breakers speak all the dialects that form the basis for the code. Several of them have high-value skills in computer technology. The team worked closely with the U.S. National Security Agency and its station at Menwith Hill in the north of England. The identity of the code-breakers is so secret that not even their gender can be revealed.

    “Like all good codes, the one they broke depended on substituting words, numbers or symbols for plain text. A single symbol could represent an idea or an entire message,” said an intelligence source.

    The code the terrorists devised consists of words chosen from no fewer than 20 dialects from Afghanistan, Iran, Pakistan, Yemen and Sudan.

    Inserted with the words ­ either before or after them ­ is local slang. The completed message is then buried in Islamic religious tracts.

  13. This goes to show how the algorithm is rarely the weakest part of a practical encryption system: “Quantum Encryption Implementation Broken

    Professor Johannes Skaar’s Quantum Hacking group at NTNU have found a new way to break quantum encryption. Even though quantum encryption is theoretically perfect, real hardware isn’t, and they exploit these flaws. Their technique relies on a particular way of blinding the single photon detectors so that they’re able to perform an intercept-resend attack and get a copy of the secret key without giving away the fact that someone is listening. This attack is not merely theoretical, either. They have built an eavesdropping device and successfully attacked their own quantum encryption hardware.”

  14. “In 1978, the CalTech mathematician Robert McEliece developed a cryptosystem based on the (then) new idea of using asymmetric mathematical functions to create different keys for encrypting and decrypting information. The security of these systems relies on mathematical steps that are easy to make in one direction but hard to do in the other. Today, popular encryption systems such as the RSA algorithm use exactly this idea. But in 1994, the mathematician Peter Shor dreamt up a quantum algorithm that could factorise much faster than any classical counterpart and so can break these codes. As soon as the first decent-sized quantum computer is switched on, these codes will become breakable. Since then, cryptographers have been hunting for encryption systems that will be safe in the post quantum world. Now a group of mathematicians have shown that the McEliece encryption system is safe against attack by Shor’s algorithm and all other known quantum algorithms. That’s because it does not depend on factorisation but gets its security from another asymmetric conundrum known as the hidden subgroup problem which they show is immune to all known quantum attacks.

  15. NSA Adds Kahn Collection To Cryptologic Museum

    “The Baltimore Sun reports that as recently as the late 1960s, the very existence of the National Security Agency was a closely held secret until a New York newspaper reporter named David Kahn published The Codebreakers, a 1,200-page blockbuster that would establish Kahn as the world’s leading expert on the history of cryptology, the art and science of making and breaking codes. ‘According to my editor, the NSA director flew up to New York to say it would be dangerous to national security, and unpatriotic, to publish it,’ says Kahn. Fast forward 43 years and now the NSA has announced it has added the David Kahn Collection to the library of its public anteroom, the National Cryptologic Museum — complete with more than 130,000 pages of original interview notes and 2,800 books. ‘For those who care about cryptology — what it is, how it works, where it fits into world history and culture — at some point, [they’d] want to look at the Kahn collection,’ says curator Patrick Weadon. ‘It’s an eclectic cornucopia of all things cryptological.'”

  16. As America struggled to stem the Japanese advance across the Pacific following the disaster of Pearl Harbour, military codes—cumbersome and weak—were proving a fatal weakness. But Philip Johnston, a missionary’s son raised on a Navajo reservation, hit on the idea of using a language that the Japanese could not crack. Native American tongues had been used for battlefield messages in the first world war, (Hitler had even dispatched spies to America in the 1930s to study them in case they would be used again). But Navajo had not been written down, and almost no outsiders spoke it fluently.

    Moreover, to be safe, the code that Mr Nez and his fellow-Navajo volunteers in the secret 382nd Platoon helped devise was a complex one. The letter A was represented by any of three Navajo words: “ant”, “apple” or “axe”. Common military terms had words of their own: a fighter plane was a hummingbird, (da-he-tih-hi), a battleship was a whale (lo-tso), a destroyer a shark, (ca-lo). A hand grenade was a potato, and America was Ne-he-mah (“our mother”). The Japanese did eventually capture (and torture) a Navajo—but he was not a code-talker. He could not make head or tail of the messages.

    Marine commanders were initially sceptical. But a message that took an hour to encrypt, transmit and decrypt on the existing mechanical Shackle system could be transmitted orally by code-talkers in just 40 seconds. Even America’s own code-crackers failed to break it.

Leave a Reply

Your email address will not be published. Required fields are marked *