From the September/October 2004 issue of Foreign Affairs.

As the war on terrorism continues, statistics on terrorist attacks are becoming as important as the unemployment rate or the GDP. Yet the terrorism reports produced by the U.S. government do not have nearly as much credibility as its economic statistics, because there are no safeguards to ensure that the data are as accurate as possible and free from political manipulation. The flap over the error-ridden 2003 Patterns of Global Terrorism report, which Secretary of State Colin Powell called "a big mistake" and which had to be corrected and re-released, recently brought these issues to the fore. But they still have not been adequately addressed.

Now-common practices used to collect and disseminate vital economic statistics could offer the State Department valuable guidance. Not long ago, economic statistics were also subject to manipulation. In 1971, President Richard Nixon attempted to spin unemployment data released by the Bureau of Labor Statistics (BLS) and transferred officials who defied him. This meddling prompted the establishment of a series of safeguards for collecting and disseminating economic statistics. Since 1971, the Joint Economic Committee of Congress has held regular hearings at which the commissioner of the BLS discusses the unemployment report. More important, in the 1980s, the Office of Management and Budget issued a directive that permits a statistical agency's staff to "provide technical explanations of the data" in the first hour after principal economic indicators are released and forbids "employees of the Executive Branch" from commenting publicly on the data during that time.

The State Department should adopt similar protections in the preparation and dissemination of its reports. In addition to the global terrorism report, the State Department is required by Congress to report annually on international bribery, human rights practices, narcotics control, and religious freedom. Gathering and reporting data for congressional oversight is presently a low-level function at the State Department. The department rarely relies on high-quality, objective data or on modern diagnostic tests to distinguish meaningful trends from chance associations. Adopting safeguards against bias, both statistical and political, would enable Congress to better perform its constitutional role as the White House's overseer and allow the American public to assess the government's foreign policy achievements.

A PATTERN OF ERRORS

Congress requires that the State Department provide each year "a full and complete report" that includes "detailed assessments with respect to each foreign country ... in which acts of international terrorism occurred which were, in the opinion of the Secretary, of major significance." The global terrorism reports are intended to satisfy this requirement, but, over time, they have become glossy advertisements of Washington's achievements in combating terrorism, aimed as much at the public and the press as at congressional overseers.

The 2003 global terrorism report was launched at a celebratory news conference in April. Deputy Secretary of State Richard Armitage and Ambassador J. Cofer Black, the State Department coordinator for counterterrorism, outlined some remaining challenges, but principally they announced the Bush administration's success in turning the terrorist tide. Black called the report "good news," and Armitage introduced it by saying, "You will find in these pages clear evidence that we are prevailing in the fight." The document's first paragraph claimed that worldwide terrorism dropped by 45 percent between 2001 and 2003 and that the number of acts committed last year "represents the lowest annual total of international terrorist attacks since 1969." The report was transmitted to Congress with a cover letter that interpreted the data as "an indication of the great progress that has been made in fighting terrorism" after the horrific events of September 11.

But we immediately spotted errors in the report and evidence contradicting the administration's claims. For example, the chronology in Appendix A, which lists each significant terrorist incident occurring in the year, stopped on November 11-an unusual end to the calendar year. Clearly, this was a mistake, as four terrorist attacks occurred in Turkey between November 12 and the end of 2003. Yet it was impossible to tell whether the post- November 11 incidents were inadvertently dropped off the chronology and included in figures in the body of the report or completely overlooked.

More important, even with the incomplete data, the number of significant incidents listed in the chronology was very high. It tallied a total of 169 significant events for 2003 alone, the highest annual count in 20 years; the annual average over the previous five years was 131. How could the number of significant attacks be at a record high, when the State Department was claiming the lowest total number of attacks since 1969? The answer is that the implied number of "nonsignificant" attacks has declined sharply in recent years. But because nonsignificant events were not listed in the chronology, the drop could not be verified. And if, by definition, they were not significant, it is unclear why their decrease should merit attention.

On June 10, after a critical op-ed we wrote in The Washington Post, a follow-up letter to Powell from Representative Henry Waxman (D-Calif.), and a call for review from the Congressional Research Service, the State Department acknowledged errors in the report. "We did not check and verify the data sufficiently," spokesman Richard Boucher said. "... [T]he figures for the number of attacks and casualties will be up sharply from what was published."

At first, Waxman accused the administration of manipulating the data to "serve the Administration's political interests." Powell denied the allegation, insisting that "there's nothing political about it. It was a data collection and reporting error." Although there is no reason to doubt Powell's explanation, if the errors had gone in the opposite direction-making the rise in terrorism on President George W. Bush's watch look even greater than it has been-it is a safe bet that the administration would have caught them before releasing the report. And such asymmetric vetting is a form of political manipulation.

Critical deficiencies in the way the report was prepared and presented compromised its accuracy and credibility. Chief among these were the opaque procedures used to assemble the report, the inconsistent application of definitions, insufficient review, and the partisan release of the report. These deficiencies resulted in a misleading and unverifiable report that appeared to be tainted by political manipulation.

It is unclear exactly how the report was assembled. The report notes that the U.S. government's Incident Review Panel (IRP) is responsible for determining which terrorist events are significant. It says little, however, about the panel's members: how many there are, whether they are career employees or political appointees, or what affiliations they have. Nor does it describe how they decide whether an event is significant. Do they work by consensus or majority rule? What universe of events do they consider?

The State Department announced a decline in total terrorist attacks, which resulted from a decline in nonsignificant events. But without information about the nonsignificant events, readers were essentially asked to blindly trust the nameless experts who prepared it.

The report's broad definitions, moreover, are sometimes too blunt to help classification. Terrorism is defined as "premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents, usually intended to influence an audience." The report specifies that an international terrorist attack is an act committed by substate actors from one nation against citizens or property of another. An incident "is judged significant if it results in loss of life or serious injury to persons, major property damage, and/or is an act or attempted act that could reasonably be expected to create the conditions noted."

But hardly any explanation was provided about how the IRP distinguishes significant from nonsignificant events. When is property damage too minor for an event to be significant? How are nonsignificant events identified? Is the IRP responsible for making these determinations too? Has the source and scope of their information changed over time? The corrected 2003 report, the first to list individual nonsignificant acts, defines as "major" property damage that exceeds $10,000. It does not indicate, however, whether that criterion applied to previous reports.

Admittedly, measuring international terrorism is no easy task. Even scholarly reckonings are not free from subjective judgment, and there are inevitably close calls to be made. The most one can hope for in many cases is consistent application of ambiguous definitions.

Unfortunately, in the global terrorism reports the rules have been applied inconsistently. Many cross-border attacks on civilians in Africa have not been included in the reports, for example, even though similar attacks in other regions have been. The report for 2002, moreover, counts as significant a suicide attack by Chechen shaheeds (Islamist martyrs) against a government building in Moscow that killed 72 people. Yet none of the numerous suicide attacks by the Chechen "black widows" that terrorized Russia and killed scores in 2003 was tallied as an international terrorist attack in the latest report. After one such attack, Russian President Vladimir Putin said, "Today, after a series of recent terrorist attacks, we can say that the bandits active in Chechnya are not just linked with international terrorism, they are an integral part of it." If the State Department considers such attacks domestic, rather than international, it should do so consistently from one year to the next.

Another problem is that the staff that prepared the 2003 global terrorism report did not participate in releasing it; in fact, they have yet to be identified. High-level Bush administration officials presented the report to the media, using it to support White House policies and take credit for the alleged decline in terrorism. Even after the report's flaws were recognized, they continued to spin the figures. When the corrected version was released, Black repeated that "we have made significant progress," despite being pressed to acknowledge that last year the number of significant attacks reached a 20-year high. Given the war on terrorism's central role in the upcoming presidential election, such presentation gives the appearance that the report is being manipulated for political gain.

The State Department has tried to explain the report's flaws using language eerily reminiscent of the Bush administration's justification of the failure to find weapons of mass destruction in Iraq. Spokesman Boucher told reporters that previous claims that the war on terrorism was succeeding had been based "on the facts as we had them at the time [and] the facts that we had were wrong." Even Powell partook in the spinning. On the one hand, he announced that "the [original] narrative is sound and we're not changing any of the narrative." On the other hand, he acknowledged, "We will change the narrative wherever the narrative relates to the data."

To his credit, Powell instructed those responsible for preparing the report to brief Waxman's staff on the procedures they had used and the origins of their mistakes. Based on a summary of the briefing by Waxman's staff, much has come to light. Authority for compiling the list of attacks was shifted from the CIA to the Terrorist Threat Integration Center (TTIC), an organization created in May 2003 to "merge and analyze all threat information in a single location." The TTIC provided information to the IRP, which, it was disclosed, consists of representatives from the CIA, the Defense Intelligence Agency, the National Security Agency, and the State Department's Bureau of Intelligence and Research. A TTIC representative chaired the meetings and could cast a vote to break ties on the classification of an event as significant or nonsignificant.

At least this year, chaos prevailed. The IRP's members changed from meeting to meeting-when they attended the meetings at all. The CIA employee responsible for the database left but was never replaced; in mid-process, an outside contractor who entered data was replaced by another contractor. Because of technical incompetence, the report relied on the wrong cutoff date.

Arithmetic errors were rampant. Larry Johnson, a retired CIA and State Department professional, discovered that the total number of fatalities in the chronology exceeded the number listed in the statistical review in Appendix G. According to Black, the errors resulted from "a combination of things: inattention, personnel shortages and database that is awkward and is antiquated and needs to have very proficient input be made in order for to be sure that the numbers will spill then to the different categories that are being captured [sic]." The debacle is more like an episode of the Keystone Kops than a chapter from Machiavelli, but even that analogy is not very comforting.

SETTING THE RECORD STRAIGHT

Despite the data's limitations, the chronology of significant events in the 2003 global terrorism report yields important information about terrorism's trends, its geographical characteristics, and its magnitude.

Time-series analysis, which seeks to discern trends in given phenomena over time, requires a consistent approach to collecting data. The State Department's terrorism report presents time-series analysis, but by focusing on the total number of attacks it misleadingly combines verifiable data on significant events with nonverifiable data on insignificant ones. And because, as TTIC director John Brennan admitted, "many nonsignificant events occur throughout the world that are not counted in the report," one must also be concerned about consistency in the measurement of the total number of terrorist events. Even if the nonsignificant events were listed (and thus could be verified), trends in significant events are more relevant because they track events that, by definition, are more important. Accurately measuring these trends is a prerequisite for understanding the factors that underlie them and the policies that shape them. In fact, an analysis of the revised report reveals that the number of significant attacks increased from 124 to 175, or by 41 percent, from 2001 to 2003-a significant fact indeed.

The detailed chronology also allows analysts to cumulate terrorist events for each country and cross-classify them according to the country where they occurred and the perpetrators' country of origin. These figures can then be related to the countries' characteristics, yielding information that can help policymakers devise strategies to address terrorism's root causes. Using the global terrorism reports for the years 1997-2002, the authors of this article have previously found that terrorists tend to come from nondemocratic countries, both rich and poor, and generally target nationals from rich, democratic countries.

The State Department has rightly emphasized that the threat of terrorism remains serious, but a close examination of its data helps put the magnitude of the threat in perspective. In 2003, a total of 625 people--including 35 Americans--were killed in international terrorist incidents worldwide. Meanwhile, 43,220 died in automobile accidents in the United States alone, and three million died from AIDS around the world. Comparative figures, particularly when combined with forecasts of future terrorism trends, can help focus debate on the real costs people are willing to bear--in foregone civil liberties and treasure--to reduce the risk posed by terrorism.

CHANGING TRACKS

The State Department currently uses, and Congress accepts, nineteenth-century methods to analyze a twenty-first-century problem. To prevent errors of the type that riddled the 2003 global terrorism report, Congress has two alternatives. It could reassign the State Department's reporting responsibilities to a neutral research agency, such as the GAO (the General Accounting Office, recently renamed the Government Accountability Office) which routinely uses appropriate statistical practices. The problem is that the GAO has little foreign policy expertise and does not necessarily have access to the (sometimes classified) information that goes into the reports. Alternatively, Congress could keep the reports within the State Department's purview but demand that its practices for data collection and analysis be improved and that the reports be insulated from partisan manipulation.

If responsibility remains within the State Department, Congress should establish a statistical bureau in the department to ensure that scientific standards are respected in all reports, thereby elevating the status of data-gathering and statistics there. The bureau would promote consistency, statistical rigor, and transparency. When appropriate, it could seek input from the scientific community. And, while respecting classified sources, it could also insist that sufficient information be released to independent analysts for verification.

To overcome conflicts of interest facing political appointees who issue government reports, the State Department should adopt rules similar to those that govern the production and dissemination of key economic indicators. Career staff who prepare the reports should be given an hour to brief the media on technical aspects of the data, during which time political appointees would be precluded from making public comments. (After the hour elapses, it is expected that political appointees would offer their interpretations.) Career staff should be protected so they can prepare mandated reports without interference from political appointees and then present them for review by the statistics bureau. Once the reports are finalized, but before they are publicly released, they should be circulated to designated political appointees who need to prepare for their release. Disclosure dates should be announced long in advance to prevent opportunistic timing by political appointees.

Last October, in a candid memorandum to top aides that was leaked to the press, Secretary of Defense Donald Rumsfeld admitted, "Today, we lack metrics to know if we are winning or losing the global war on terror. Are we capturing, killing, or deterring and dissuading more terrorists every day than the madrassas [Islamic schools] and the radical clerics are recruiting, training, and deploying against us?" The statement was a stinging acknowledgment that the government lacks both classified and unclassified data to make critical policy decisions. It is also a reminder that only accurate information, presented without political spin, can help the public and decision-makers know where the United States stands in the war on terrorism and how best to fight it.

President Bush's 2002 nuclear posture differs sharply from its predecessors and is relevant to the President's recently repeated assertion that he will strike first against any country that might pose a threat of using weapons of mass destruction.

The main new trend in the posture is that the US will be prepared to use nuclear weapons in a much wider range of circumstances than before. Such an emphasis has not been seen since the days of "flexible response" forty or so years ago, when tactical nuclear weapons were deployed in Europe and elsewhere.

Yet, nuclear weapons don't help much with the kinds of missions the US prepares for, including the ones noted in the posture, such as digging out deep underground facilities that might contain bio-warfare agents. Deep underground facilities are difficult or impossible to destroy without large nuclear explosions that create large amounts of fallout. Nuclear weapons are more suited for use against shallow-buried facilities (of the order of ten meters deep) but even in those cases, Hiroshima-type yields are needed, and complete destruction of the bio-agents cannot be guaranteed. Other uses mentioned to justify the posture are even more marginal in their feasibility.

Given the overwhelming US conventional advantage and the relative invulnerability of the US to all but nuclear weapons, the US nuclear posture should aim at minimizing the chances of nuclear weapons spread rather than seeking marginal gains with tactical nuclear weapons. Nuclear weapons are equalizers. Why bring them back into the forefront of regional problems, whether in the Middle East or anywhere else?

Increasing the US nuclear threat will increase the motivation of adversaries, big or small, to improve and extend their own nuclear force, or to get one if they don't already have one. The US cannot subsequently be confident that it will be the only power to use or threaten to use nuclear weapons. There are now several demonstrations of the relative ease with which states can acquire nuclear weapons. North Korea, a poor nation of 17 million people, made and separated with little help enough plutonium for perhaps one or more weapons. South Africa made at least six weapons with essentially no help. Other cases tell the same tale.

The nuclear genie is long out of the bottle and the relative stability that characterized the Cold War is also gone. Instead, the US has been pursuing an aggressive strategy of military expansion around the world and ever closer to other states' vital interests. Quite apart from the wisdom of that strategy, is it wise to couple it with an increased nuclear threat to possible adversaries, as the posture does?

In the past, the existence of a real or putative nuclear threat has been a serious motivation for states to improve and extend their own nuclear force, or to get one if they didn't already have it. That was true of the US, USSR, China, and others. The US, as the world's strongest and least vulnerable major power, should pursue a strategy that minimizes the most serious risk rather than increase it for marginal, and questionable, benefits. The posture implies a strategy that does the opposite.

A nuclear posture better suited to our times would recognize these changes. It would lay the policy basis for the following difficult, long-term, but necessary steps:

1. Minimizing the demand for nuclear weapons, focusing on Asia. Asia contains most of the world's population and might, in a few decades, have most of its wealth. Three states there (four if Israel is included) have nuclear weapons; several more could readily have them. The US nuclear posture should provide US initiatives toward a more stable security order there, one in which peaceful states will not be threatened by nuclear or potentially nuclear rivals. The Non-Proliferation Treaty provides a basis -the only existing basis- for such an order, but it needs to be updated with more inducements in the way of technical cooperation and reassurance, and more clearly defined internationally agreed sanctions if the treaty is disregarded. The US nuclear posture in essence forswears the lead in this endeavor.

2. A pattern for nuclear arms reductions that would include eventually limitations on all arsenals. Openness here is as important as numbers. The US and Russia have most of the weapons but, after the first hundred or so survivable weapons, it matters less and less how many a state has. An internationally recognized framework is needed that can be applied to the regions of the world where nuclear rivalries threaten. Instead, the US has gone the other way, with a sketchy US-Russia agreement that delays the time scale for reductions and does not provide any precedent for international agreements on inspections.

3. A strategy for addressing the problem of nuclear terrorism. The most serious dimension of that problem--the possibility of a terrorist nuclear weapon --is closely related to the proliferation of nuclear weapons and capabilities. Any strategy to avoid that has an important international dimension. Hundreds of tons of weapons-grade uranium and plutonium, most of it surplus in the US and former Soviet Union from the Cold War, need to be better secured and accounted for. A solution to the problem of keeping nuclear weapons and materials out of the tens of millions of shipping containers that crisscross the world requires international cooperation on standards, procedures, cost sharing, and inspections. A good start has been made toward these goals, mainly through the Nunn-Lugar programs, but more money and agreements are needed. A modern nuclear posture should establish the policy basis for securing those resources and agreements. There is at present no comprehensive global strategy for securing such vital agreements and establishing the institutions to enforce them. Consistent, high-priority US participation is vital to secure other countries' participation.

4. A strategy for reducing the risks of accidental nuclear launch while at the same time maintaining invulnerability of the reduced deployments. The nuclear posture briefly mentions the "rigorous safeguards" on US weapons systems and proposes to deal with the problem of accidental or unauthorized launch of "certain foreign forces" via nuclear missile defense. That is at best a partial and certainly a distant remedy. Maintaining the human and financial infrastructure for nuclear weapons system will become more difficult in the US as well as elsewhere. Given the relationship among nuclear deterrent forces, the problem cannot be solved unilaterally. A program that would use US technical leadership to improve warning and control for all states threatened by nuclear weapons is also needed. It is needed now in South Asia. Later, it could help limit crises with or among Russia and China, and help prevent proliferation in the Middle East. President Reagan, with a portion of Star Wars, and, before him, President Eisenhower, with Open Skies, had something of the kind in mind. It is time to begin thinking about how this would look in modern form.

In summary, a nuclear posture for a world with more dispersed power centers and more widely available nuclear technology should have more, not less, emphasis on international agreements. President Eisenhower stated fifty years ago that "Only chaos will result from our abandonment of collective international security." That is even truer in today's world than it was then. The present administration seems to have a bias against such agreements, which are slow to bear fruit and do not win votes. That is shown in the posture itself, which states that arms control measures will not stand in the way of nuclear weapons development.

Yet these and other agreements are essential to deal with the dangers of proliferation to unstable states, with the possible use of international trade for terrorism, and with the risk of accidents and unauthorized launch. Nuclear deterrence continues to be needed, but the last thing a modern posture should do is to bring nuclear weapons back into the forefront of regional deterrence.

Ironically, when it has committed itself to the task, the US has used international agreements more effectively than any other nation. The Cold War-- better called a Cold Peace perhaps, since the military lines of demarcation never changed while the safeguarding of Western values and collapse of the Soviet Union were brought about mainly by economic and political instruments-- saw a rise in US power and influence in good part through the use of US-led international agreements in the areas of trade and security, areas that are necessarily related. Now is not the time to give up that approach, especially not in matters relating to nuclear weapons.

The United Nations High-Level Panel on Threats, Challenges and Change completed its comprehensive review of collective security, recommending historic changes to the U.N. in its report, "A more secure world: Our shared responsibility." Among the panel's 101 recommendations for the U.N. and member states are expansion of the U.N. Security Council and creation of a Peacebuilding Commission to advance proactive, preventive global security measures.

The report culminates a year-long project for which SIIS Senior Fellow Stephen J. Stedman served as research director. The 16-member panel, commissioned by U.N. Secretary-General Kofi Annan and chaired by former Thailand Prime Minister Anand Panyarachun, represents the U.N.'s most comprehensive effort to analyze collective security, since the founding of the international body in 1947. The select panel sought international input in an effort to honor the perspectives of all member states, as it analyzed current threats and identified specific security measures.

Nations are the "front line in today's combat," Annan said, introducing the report. He added, "The task of helping states improve their own capacities to deal with contemporary threats is vital and urgent. The United Nations must be able to do this better. The panel tells us how."

The report identifies six major threats to global security: war between states; violence within states, including civil wars, large-scale human rights abuses and genocide; poverty, infectious disease and environmental degradation; nuclear, chemical, biological and radiological weapons; terrorism; and transnational organized crime.

The panel proposed expanding the U.N. Security Council--for which it put forth two options--as well as creating a Peacebuilding Commission to help the Security Council pursue the recommended preventive security strategies. One proposal for Security Council expansion would appoint new permanent members, and the other would establish new long-term, renewable seats. Neither option creates any new vetoes.

In a cover letter to the secretary-general, Panyarachun thanked CISAC and Stedman for supporting the panel's work. CISAC Co-Director Christopher F. Chyba served on the panel's 30-member resource group, providing expertise on nuclear nonproliferation and bioterrorism. CISAC hosted a nuclear nonproliferation workshop at Stanford for the panel last March, and Panyarachun discussed security issues with representatives from China, India, Pakistan, Russia and the United States at CISAC's Five-Nation Project meeting in Bangkok last summer. Stedman's research staff included Bruce Jones, a former CISAC Hamburg Fellow, and Tarun Chhabra, a graduate of CISAC's undergraduate honors program.

Annan has asked Stedman to stay at the United Nations another year to help gain worldwide support in implementing the panel's recommendations.

The panel's report received prominent news coverage, including a front-page New York Times article ("Report urges big changes for the U.N.," by Warren Hoge, Dec. 4), and in the Economist an invited article by Annan ("Courage to fulfill our responsibilities," Dec. 4) as well as several other pieces in the Nov. 24 and Dec. 4 issues.

Addressing the fourth annual Foresight Conference on Molecular Nanotechnology in 1995, Adm. David Jeremiah, a former vice chairman of the Joint Chiefs of Staff, made a bold prediction. The "military applications of molecular manufacturing have even greater potential than nuclear weapons to radically change the [world] balance of power," he said.

The risk in trying to stop others from using the technology to gain a strategic advantage, Jeremiah continued, was that "the uninformed policy maker is likely to impose restrictions on [the] development of technology in such a way as to inhibit commercial development, ultimately beneficial to mankind, while permitting those operating outside the restrictive bounds to gain an irrevocable advantage."

It was an ominous, if not unfamiliar, statement to the assembled crowd, some of whom were undoubtedly excited by the potential of exploring the area of science Jeremiah was referring to, an area physicist Richard Feynman had labeled the "room at the bottom."

The "bottom" of which Feynman spoke and later wrote is the low end of the scale of matter--atoms and molecules. Twenty years later, researcher Eric Drexler coined the term "nanotechnology" for the science Feynman envisioned--building materials, structures, and machinery from the molecule up.

Absent the kind of regulation Jeremiah feared, scientists and businessmen have taken research into the construction and potential application of nanoscale materials and run with it. The consequence has been a growing diversity of products and processes integrated into both consumer and military realms.

Although the general public has begun to more broadly feel the impact of nanotechnology, researchers are just beginning to explore the potential for early nanotech developments and applications to harm human and animal health and the environment. In the age of terrorism, the potential of certain nanotechnologies and nanomaterials to be used to do both good and bad requires not just the exploration of the possibilities of this mushrooming field of science but also broad anticipation of possible threats.

(For complete article, follow the link below.)