New developments in spam

Remarkably, it seems that 70% of the world’s spam emails were originating from an American firm called McColo. On November 11th, two American internet service providers cut them off from the web, leading to the huge drop in the global volume of spam. It is estimated that 90% of spam messages are actually sent by computers that have been compromised by viruses, which makes it a bit surprising that such a drop could be generated by disconnecting one firm. Clearly, it is a network that needed central direction to operate. Those that emerge as successors will probably be more robust, located in more unpoliced jurisdictions, or both.

While the respite is likely to be temporary, the situation may reveal some useful information on the practice and economics of spam. This unrelated paper (PDF) examines the latter. The researchers infiltrated a segment of the Storm Botnet and monitored its activity and performance. On the basis of what they observed and estimates of the rest, they concluded that the botnet earned about 3.5 million dollars a year by selling pharmaceuticals. While that isn’t an inconsiderable sum, I suspect it is less than is being spent by companies combatting the flood of spam messages themselves.

Celebrating soldiers, celebrating peace

The problematic nature of Remembrance Day has been covered twice here already, in 2006 and 2007. My question for today is this: would it be better to have two separate holidays, one of which is unambiguously pacifist and committed to recognizing the horrible character of war, and another in which the sacrifices of veterans are marked?

The first occasion would mostly be about civilians, since theirs is the primary experience of contemporary war. The second would still need to address difficult questions about why sacrifices on one side were more noble than those on the other, as well as what kind of conduct we should consider acceptable or laudable in war.

Spying on North American weather

Most weather systems in the Atlantic move from west to east. As a result, the Allies had a tactical advantage during the Second World War. Their weather stations in North America provided information that was useful for making plans in the Atlantic and European theatres of war.

The Germans made a creative effort to alter that balance by secretly planting a weather station in Labrador. The automated station was transported by U-boat and installed under cover of fog. Unfortunately for the Germans, the station only operated for a few days and the U-boat sent to repair it got sunk.

You can see the weather station on display at the Canadian War Museum, which is free on Thursday evenings.

Protecting the new president

Alongside general jubilation about the Obama victory, a number of my friends have expressed their concern about Obama’s personal safety. That certainly seems like a legitimate concern. Four American presidents have been assassinated while in office, and every president since Nixon has faced at least one attempt (though levels of credibility vary). During his victory speech on election night, most people probably noticed the transparent bulletproof barriers set up around the podium.

The Secret Service is certainly taking the threat seriously. According to a Stratfor briefing, Obama got a security detail earlier than any other candidate and, by the end of his campaign, it had grown to the size of a full presidential protection team – unprecedented for a candidate, and a significant strain on the manpower of the service.

Given the likelihood that Obama will be targeted by white supremacists or others – as well as the colossal impact his assassination would likely have – I certainly don’t envy the Secret Service at this point in time. While they have plenty of resources to provide physical protection, as well as identify and break up conspiracies, the risk from disciplined and capable lone wolf operators is impossible to eliminate while maintaining public appearances. Even with the assistance of the FBI, CIA, NSA, etc, there will be a measure of luck involved in ensuring that future close contact with the public does not produce disaster.

NIST hash competition

Several times, the American government has held open competitions to create new cryptographic standards. Important examples include the Data Encryption Standard (DES) selected in 1976 and the Advanced Encryption Standard (AES) chosen in 2001. As mentioned before, the hunt is now on for a new hash function. These are one-way forms of encryption that play a number of vital roles, such as making it possible to save only encrypted versions of passwords in password databases that might be compromised.

Bruce Schneier, who made an unsuccessful bid for his TwoFish cipher to be accepted as the AES, is now part of the team that has created the Skein Hash Function for the ongoing National Institute of Standards and Technology competition. The function is based around a successor to TwoFish called, unsurprisingly, Threefish. All entries must be submitted by tomorrow and will be publicly scrutinized over the next four years or so. The result should be a more secure successor to the SHA hash functions.

Massive anti-terror database contemplated in the UK

British Transport Secretary Geoff Hoon has been saying some worrisome things about terrorism, security, and civil liberties. He is backing a plan to create a massive database of mobile and internet communications, for purposes of fighting terrorism. One worrisome aspect is the suggestion that it would be used to deal with “terrorists or criminals.” Technologies initially justified as an extreme measure necessary to fight terrorism will always spread to more banal uses, with a greater scope for abuses.

Indeed, that is the biggest issue that needs to be weighed against the possible terror-fighting capacity of such databases. They will inevitably be abused. Furthermore, governments are far more dangerous than terrorists, both when they are acting in malicious ways and when they are trying to be benign. Modern history certainly demonstrates that, while the power of terrorists to inflict harm is considerable, the ability of states to do so is extreme.

Previously:

Martin Hellman on the risk of nuclear war

Despite the end of the Cold War, there remains some possibility of a major nuclear exchange between some combination of those world powers with more than a couple of hundred nuclear weapons. Such an outcome could arise through accident or miscalculation, unauthorized launch, or simply through the progressive stressing of the situation, in a manner akin to the Cuban Missile Crisis in 1962, the Yom Kippur War of 1973, of the Able Archer exercise in 1983.

Martin Hellman – one of the three civilian inventors of public key cryptography – has written a piece describing some statistical ways through which we could contemplate the risk of global nuclear war, as well as evaluate it relative to other threats. As a near-term nightmare scenario, the massive use of nuclear weapons surely exceeds the threat posed by climate change: climatic change across a decade is highly abrupt, whereas the time between the decision to use nuclear weapons and the generation of mass casualties would likely be only minutes.

Based on the frequency with which near misses have taken place, Hellman argues that the perpetuation of the current global nuclear situation carries a 1% per year risk of mass nuclear exchange. He estimates that this exceeds the risk of living beside a nuclear power plant by 1000 to 1 and has a clever rhetorical device for making that concrete:

Equivalently, imagine two nuclear power plants being built on each side of your home. That’s all we can fit next to you, so now imagine a ring of four plants built around the first two, then another larger ring around that, and another and another until there are thousands of nuclear reactors surrounding you. That is the level of risk that my preliminary analysis indicates each of us faces from a failure of nuclear deterrence.

Surely, if his estimate is anywhere near correct, all the ongoing concern about new nuclear power plants should be superseded more than one thousandfold by concern about the state of security in the face of nuclear war. After all, everybody lives with the risk associated with global thermonuclear war and nuclear winter. Only those living fairly close to nuclear power plants bear acute risks associated with meltdowns.

Hellman’s warning is akin to the one repeatedly sounded by former US Defense Secretary Robert McNamara, who himself revised the American nuclear warplan for the Kennedy administration in 1963. In both cases, the suggestions are similar: work to reduce the number of weapons, increase the time required for anybody to use them, and avoid the complacent belief that the lack of explosive accidents or attacks since the Second World War proves them to be impossible.

1958 climate change video

Boing Boing came up with quite a find today: a video from 1958 that is both amusing and full of relatively accurate information about climate change. Entitled “The Unchained Goddess,” it was produced as an episode of the Bell Telephone Hour.

As I have described before, the idea that climate change only entered the realm of scientific knowledge within the last few years is quite mistaken. Notice also how the announcer in the video is concerned about emissions of “six billion tonnes per year of carbon dioxide.” The figure today is closer to forty billion.

The Code-Breakers

For those with a serious interest in the history and practice of cryptography, David Khan’s The Code-Breakers: The Comprehensive History of Secret Communication from Ancient Times to the Internet provides an enormous mass of knowledge. The scope of the 1200 page book is vast: covering everything from the earliest ciphers to the origins of public key cryptography in detail. It is probably fair to say that the period best covered is that between the Middle Ages and the Second World War, though the sections covering the decoding of Egyptian hieroglyphics and Linear B are also detailed and skilfully written. Those interested primarily in the contemporary practice of cryptography – or those seeking a more concise text – would be well advised to consider the books by Simon Singh and Bruce Schneier on the topic.

Khan’s book excels in actually describing how various cryptographic systems work, as well as how they were broken. For the most part, his analysis is factual and dispassionate. The sole exception is in the period covering the Cold War, in which his ire against the Soviet Union and those Americans who turned into traitors for it are acute. At times, the book gets into an excessive amount of detail about the bureaucratic organization of different cipher bureaus: including lengthy sections about how various wartime bodies were reorganized. In most cases, the book does not provide much biography on the men and women involved, though exceptions exist in the case of some of the most eminent or interesting cryptographers. The book does provide an interesting discussion of the history of writing on cryptography, including the impact that major publications had on the development of the field and its comprehension within society at large. Kahn also does a good job of debunking some of the many spurious claims that have been made about ‘revolutionary’ and ‘unbreakable’ cryptosystems that people have invented: stressing how the making of cryptographic systems is a realm of abstract mathematics, while the breaking of such systems is a gritty and practical exercise.

In addition to covering the techniques of cryptography and cryptanalysis themselves, the book covers many related security issues: including physical security, invisible inks, elements of spycraft, decisions about how to use information gleaned through cryptanalysis, and the use of broken cryptographic systems to transmit fake or confusing information. The book also covers the relationships between cryptographic work and the activities it is supporting. An especially intriguing section details the efforts of the American navy to combat rum smuggling during the prohibition era. Ships with floating cryptoanalytical laboratories provided vital intelligence to interception vessels, just as other cryptanalysis had helped re-direct U-boats away from German submarines during the Second World War. The book covers an enormous variety of code systems, ranging in use and sophistication. These include diplomatic and commercial systems, high level military systems used between major installations, systems for vehicles, trench codes for those on the front lines, and more. The most abstract section of the book contemplates communication between human beings and extraterrestrials, covering questions about how we could recognize alien communication, as well as mathematical steps through which a comprehensible discourse could potentially be established.

For those interested in actually breaking codes and ciphers themselves, the book provides detailed information on techniques including frequency analysis, factorization attacks of the kind used against polyalphabetic substitution ciphers, and the index of coincidence. It also provides a lot of information on the weak ways in which cryptography is often used and the kinds of errors that have allowed for key breaks into previously unreadable cryptosystems. While it would not be especially useful for attacking modern computerized cryptographic systems, it would provide some guidance for those seeking to break into amateur or puzzle-type cryptographic challenges.

The Code-Breakers may well be the most comprehensive cryptographic history available, though it is far less detailed in its description of post-Cold War cryptosystems than some of its more concise recent contemporaries. For those wishing to gain an appreciation for how cryptography emerged, the role it played for most of human history, and the techniques that have been employed to guard and attack messages, this is an ideal place to turn.

Resource types and the resource curse

As discussed before, the ‘resource curse’ hypothesis holds that the presence of valuable resources can sometimes reduce the security of states, since it offers up a prize to anyone capable of seizing them. A bit of recent research has added nuance to the picture. By looking at the long-running civil war in Columbia, the authors were able to look at periods when coffee (a labour intense crop) and oil (a capital intense crop) rose and fell in value:

Using newspaper reports of violent skirmishes in 950 Colombian municipalities between 1988 and 2005, Dube and Vargas find that when coffee prices went up, violence went down in locations where a large fraction of land area was under coffee cultivation. When coffee prices fell, however, as they did by almost 70 percent in the late 1990s, violence in coffee areas rose dramatically. The researchers estimate that an additional 500 deaths may have resulted from the increased conflict that came from lower coffee prices. The opposite was true for oil: It was higher prices that intensified conflict in areas with productive oil wells or pipelines. (Since both coffee and oil prices are traded in global markets, it is unlikely that price increases were caused by panicking commodities traders spooked by increased civil-war violence in Colombia.)

One suggestion that arises is not unfamiliar: establish strong governance regimes in states with capital intensive resources. It is far better to be like Norway, using resource income transparently and putting aside a share of the oil revenues for the benefit of future generations, than like Nigeria, long mired in conflict as different groups compete for resource wealth.

On the labour intensive side, the proposal is a bit more novel: provide international aid to stabilize commodity prices in conflict-prone states. There are those who argue that a 50% drop in coffee prices helped cause the Rwandan genocide. Surely, the economic cost of temporarily bolstering commodity prices in delicate states is less than the probable cost of re-establishing security and resuming development after an internal conflict. The difference between the economic cost and the moral cost of inaction is probably greater still.