From the September/October 2004 issue of Foreign Affairs.

As the war on terrorism continues, statistics on terrorist attacks are becoming as important as the unemployment rate or the GDP. Yet the terrorism reports produced by the U.S. government do not have nearly as much credibility as its economic statistics, because there are no safeguards to ensure that the data are as accurate as possible and free from political manipulation. The flap over the error-ridden 2003 Patterns of Global Terrorism report, which Secretary of State Colin Powell called "a big mistake" and which had to be corrected and re-released, recently brought these issues to the fore. But they still have not been adequately addressed.

Now-common practices used to collect and disseminate vital economic statistics could offer the State Department valuable guidance. Not long ago, economic statistics were also subject to manipulation. In 1971, President Richard Nixon attempted to spin unemployment data released by the Bureau of Labor Statistics (BLS) and transferred officials who defied him. This meddling prompted the establishment of a series of safeguards for collecting and disseminating economic statistics. Since 1971, the Joint Economic Committee of Congress has held regular hearings at which the commissioner of the BLS discusses the unemployment report. More important, in the 1980s, the Office of Management and Budget issued a directive that permits a statistical agency's staff to "provide technical explanations of the data" in the first hour after principal economic indicators are released and forbids "employees of the Executive Branch" from commenting publicly on the data during that time.

The State Department should adopt similar protections in the preparation and dissemination of its reports. In addition to the global terrorism report, the State Department is required by Congress to report annually on international bribery, human rights practices, narcotics control, and religious freedom. Gathering and reporting data for congressional oversight is presently a low-level function at the State Department. The department rarely relies on high-quality, objective data or on modern diagnostic tests to distinguish meaningful trends from chance associations. Adopting safeguards against bias, both statistical and political, would enable Congress to better perform its constitutional role as the White House's overseer and allow the American public to assess the government's foreign policy achievements.

A PATTERN OF ERRORS

Congress requires that the State Department provide each year "a full and complete report" that includes "detailed assessments with respect to each foreign country ... in which acts of international terrorism occurred which were, in the opinion of the Secretary, of major significance." The global terrorism reports are intended to satisfy this requirement, but, over time, they have become glossy advertisements of Washington's achievements in combating terrorism, aimed as much at the public and the press as at congressional overseers.

The 2003 global terrorism report was launched at a celebratory news conference in April. Deputy Secretary of State Richard Armitage and Ambassador J. Cofer Black, the State Department coordinator for counterterrorism, outlined some remaining challenges, but principally they announced the Bush administration's success in turning the terrorist tide. Black called the report "good news," and Armitage introduced it by saying, "You will find in these pages clear evidence that we are prevailing in the fight." The document's first paragraph claimed that worldwide terrorism dropped by 45 percent between 2001 and 2003 and that the number of acts committed last year "represents the lowest annual total of international terrorist attacks since 1969." The report was transmitted to Congress with a cover letter that interpreted the data as "an indication of the great progress that has been made in fighting terrorism" after the horrific events of September 11.

But we immediately spotted errors in the report and evidence contradicting the administration's claims. For example, the chronology in Appendix A, which lists each significant terrorist incident occurring in the year, stopped on November 11-an unusual end to the calendar year. Clearly, this was a mistake, as four terrorist attacks occurred in Turkey between November 12 and the end of 2003. Yet it was impossible to tell whether the post- November 11 incidents were inadvertently dropped off the chronology and included in figures in the body of the report or completely overlooked.

More important, even with the incomplete data, the number of significant incidents listed in the chronology was very high. It tallied a total of 169 significant events for 2003 alone, the highest annual count in 20 years; the annual average over the previous five years was 131. How could the number of significant attacks be at a record high, when the State Department was claiming the lowest total number of attacks since 1969? The answer is that the implied number of "nonsignificant" attacks has declined sharply in recent years. But because nonsignificant events were not listed in the chronology, the drop could not be verified. And if, by definition, they were not significant, it is unclear why their decrease should merit attention.

On June 10, after a critical op-ed we wrote in The Washington Post, a follow-up letter to Powell from Representative Henry Waxman (D-Calif.), and a call for review from the Congressional Research Service, the State Department acknowledged errors in the report. "We did not check and verify the data sufficiently," spokesman Richard Boucher said. "... [T]he figures for the number of attacks and casualties will be up sharply from what was published."

At first, Waxman accused the administration of manipulating the data to "serve the Administration's political interests." Powell denied the allegation, insisting that "there's nothing political about it. It was a data collection and reporting error." Although there is no reason to doubt Powell's explanation, if the errors had gone in the opposite direction-making the rise in terrorism on President George W. Bush's watch look even greater than it has been-it is a safe bet that the administration would have caught them before releasing the report. And such asymmetric vetting is a form of political manipulation.

Critical deficiencies in the way the report was prepared and presented compromised its accuracy and credibility. Chief among these were the opaque procedures used to assemble the report, the inconsistent application of definitions, insufficient review, and the partisan release of the report. These deficiencies resulted in a misleading and unverifiable report that appeared to be tainted by political manipulation.

It is unclear exactly how the report was assembled. The report notes that the U.S. government's Incident Review Panel (IRP) is responsible for determining which terrorist events are significant. It says little, however, about the panel's members: how many there are, whether they are career employees or political appointees, or what affiliations they have. Nor does it describe how they decide whether an event is significant. Do they work by consensus or majority rule? What universe of events do they consider?

The State Department announced a decline in total terrorist attacks, which resulted from a decline in nonsignificant events. But without information about the nonsignificant events, readers were essentially asked to blindly trust the nameless experts who prepared it.

The report's broad definitions, moreover, are sometimes too blunt to help classification. Terrorism is defined as "premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents, usually intended to influence an audience." The report specifies that an international terrorist attack is an act committed by substate actors from one nation against citizens or property of another. An incident "is judged significant if it results in loss of life or serious injury to persons, major property damage, and/or is an act or attempted act that could reasonably be expected to create the conditions noted."

But hardly any explanation was provided about how the IRP distinguishes significant from nonsignificant events. When is property damage too minor for an event to be significant? How are nonsignificant events identified? Is the IRP responsible for making these determinations too? Has the source and scope of their information changed over time? The corrected 2003 report, the first to list individual nonsignificant acts, defines as "major" property damage that exceeds $10,000. It does not indicate, however, whether that criterion applied to previous reports.

Admittedly, measuring international terrorism is no easy task. Even scholarly reckonings are not free from subjective judgment, and there are inevitably close calls to be made. The most one can hope for in many cases is consistent application of ambiguous definitions.

Unfortunately, in the global terrorism reports the rules have been applied inconsistently. Many cross-border attacks on civilians in Africa have not been included in the reports, for example, even though similar attacks in other regions have been. The report for 2002, moreover, counts as significant a suicide attack by Chechen shaheeds (Islamist martyrs) against a government building in Moscow that killed 72 people. Yet none of the numerous suicide attacks by the Chechen "black widows" that terrorized Russia and killed scores in 2003 was tallied as an international terrorist attack in the latest report. After one such attack, Russian President Vladimir Putin said, "Today, after a series of recent terrorist attacks, we can say that the bandits active in Chechnya are not just linked with international terrorism, they are an integral part of it." If the State Department considers such attacks domestic, rather than international, it should do so consistently from one year to the next.

Another problem is that the staff that prepared the 2003 global terrorism report did not participate in releasing it; in fact, they have yet to be identified. High-level Bush administration officials presented the report to the media, using it to support White House policies and take credit for the alleged decline in terrorism. Even after the report's flaws were recognized, they continued to spin the figures. When the corrected version was released, Black repeated that "we have made significant progress," despite being pressed to acknowledge that last year the number of significant attacks reached a 20-year high. Given the war on terrorism's central role in the upcoming presidential election, such presentation gives the appearance that the report is being manipulated for political gain.

The State Department has tried to explain the report's flaws using language eerily reminiscent of the Bush administration's justification of the failure to find weapons of mass destruction in Iraq. Spokesman Boucher told reporters that previous claims that the war on terrorism was succeeding had been based "on the facts as we had them at the time [and] the facts that we had were wrong." Even Powell partook in the spinning. On the one hand, he announced that "the [original] narrative is sound and we're not changing any of the narrative." On the other hand, he acknowledged, "We will change the narrative wherever the narrative relates to the data."

To his credit, Powell instructed those responsible for preparing the report to brief Waxman's staff on the procedures they had used and the origins of their mistakes. Based on a summary of the briefing by Waxman's staff, much has come to light. Authority for compiling the list of attacks was shifted from the CIA to the Terrorist Threat Integration Center (TTIC), an organization created in May 2003 to "merge and analyze all threat information in a single location." The TTIC provided information to the IRP, which, it was disclosed, consists of representatives from the CIA, the Defense Intelligence Agency, the National Security Agency, and the State Department's Bureau of Intelligence and Research. A TTIC representative chaired the meetings and could cast a vote to break ties on the classification of an event as significant or nonsignificant.

At least this year, chaos prevailed. The IRP's members changed from meeting to meeting-when they attended the meetings at all. The CIA employee responsible for the database left but was never replaced; in mid-process, an outside contractor who entered data was replaced by another contractor. Because of technical incompetence, the report relied on the wrong cutoff date.

Arithmetic errors were rampant. Larry Johnson, a retired CIA and State Department professional, discovered that the total number of fatalities in the chronology exceeded the number listed in the statistical review in Appendix G. According to Black, the errors resulted from "a combination of things: inattention, personnel shortages and database that is awkward and is antiquated and needs to have very proficient input be made in order for to be sure that the numbers will spill then to the different categories that are being captured [sic]." The debacle is more like an episode of the Keystone Kops than a chapter from Machiavelli, but even that analogy is not very comforting.

SETTING THE RECORD STRAIGHT

Despite the data's limitations, the chronology of significant events in the 2003 global terrorism report yields important information about terrorism's trends, its geographical characteristics, and its magnitude.

Time-series analysis, which seeks to discern trends in given phenomena over time, requires a consistent approach to collecting data. The State Department's terrorism report presents time-series analysis, but by focusing on the total number of attacks it misleadingly combines verifiable data on significant events with nonverifiable data on insignificant ones. And because, as TTIC director John Brennan admitted, "many nonsignificant events occur throughout the world that are not counted in the report," one must also be concerned about consistency in the measurement of the total number of terrorist events. Even if the nonsignificant events were listed (and thus could be verified), trends in significant events are more relevant because they track events that, by definition, are more important. Accurately measuring these trends is a prerequisite for understanding the factors that underlie them and the policies that shape them. In fact, an analysis of the revised report reveals that the number of significant attacks increased from 124 to 175, or by 41 percent, from 2001 to 2003-a significant fact indeed.

The detailed chronology also allows analysts to cumulate terrorist events for each country and cross-classify them according to the country where they occurred and the perpetrators' country of origin. These figures can then be related to the countries' characteristics, yielding information that can help policymakers devise strategies to address terrorism's root causes. Using the global terrorism reports for the years 1997-2002, the authors of this article have previously found that terrorists tend to come from nondemocratic countries, both rich and poor, and generally target nationals from rich, democratic countries.

The State Department has rightly emphasized that the threat of terrorism remains serious, but a close examination of its data helps put the magnitude of the threat in perspective. In 2003, a total of 625 people--including 35 Americans--were killed in international terrorist incidents worldwide. Meanwhile, 43,220 died in automobile accidents in the United States alone, and three million died from AIDS around the world. Comparative figures, particularly when combined with forecasts of future terrorism trends, can help focus debate on the real costs people are willing to bear--in foregone civil liberties and treasure--to reduce the risk posed by terrorism.

CHANGING TRACKS

The State Department currently uses, and Congress accepts, nineteenth-century methods to analyze a twenty-first-century problem. To prevent errors of the type that riddled the 2003 global terrorism report, Congress has two alternatives. It could reassign the State Department's reporting responsibilities to a neutral research agency, such as the GAO (the General Accounting Office, recently renamed the Government Accountability Office) which routinely uses appropriate statistical practices. The problem is that the GAO has little foreign policy expertise and does not necessarily have access to the (sometimes classified) information that goes into the reports. Alternatively, Congress could keep the reports within the State Department's purview but demand that its practices for data collection and analysis be improved and that the reports be insulated from partisan manipulation.

If responsibility remains within the State Department, Congress should establish a statistical bureau in the department to ensure that scientific standards are respected in all reports, thereby elevating the status of data-gathering and statistics there. The bureau would promote consistency, statistical rigor, and transparency. When appropriate, it could seek input from the scientific community. And, while respecting classified sources, it could also insist that sufficient information be released to independent analysts for verification.

To overcome conflicts of interest facing political appointees who issue government reports, the State Department should adopt rules similar to those that govern the production and dissemination of key economic indicators. Career staff who prepare the reports should be given an hour to brief the media on technical aspects of the data, during which time political appointees would be precluded from making public comments. (After the hour elapses, it is expected that political appointees would offer their interpretations.) Career staff should be protected so they can prepare mandated reports without interference from political appointees and then present them for review by the statistics bureau. Once the reports are finalized, but before they are publicly released, they should be circulated to designated political appointees who need to prepare for their release. Disclosure dates should be announced long in advance to prevent opportunistic timing by political appointees.

Last October, in a candid memorandum to top aides that was leaked to the press, Secretary of Defense Donald Rumsfeld admitted, "Today, we lack metrics to know if we are winning or losing the global war on terror. Are we capturing, killing, or deterring and dissuading more terrorists every day than the madrassas [Islamic schools] and the radical clerics are recruiting, training, and deploying against us?" The statement was a stinging acknowledgment that the government lacks both classified and unclassified data to make critical policy decisions. It is also a reminder that only accurate information, presented without political spin, can help the public and decision-makers know where the United States stands in the war on terrorism and how best to fight it.

In the first years of his presidency, Ronald Reagan labeled the Soviet Union the "evil empire" and went out of his way to avoid contact with such a regime.

Over time, however, Reagan charted a new course of dual-track diplomacy. He engaged Kremlin leaders (well before Gorbachev) in arms control, while also fostering contacts and information flow between the West and the Soviet people in the hope of opening them up to the possibilities of democracy.

In the long run, it was not arms control with the Soviets, but democratization within the Soviet Union, that made the United States safer.

If George W. Bush desires a foreign-policy legacy as grand as Reagan's, now is the time to think big and change course as dramatically as Reagan did.

Earlier this month, the so-called EU Three--Britain, France and Germany-- achieved an important victory for global security, convincing Iran to suspend its uranium enrichment and plutonium reprocessing activities pending further negotiation on its nuclear question. Though Iran claims that it does not desire a nuclear bomb, the West has long been skeptical of the oil-rich state's contention that it seeks a nuclear fuel cycle for energy purposes alone. Europe and the United States (and of course Israel) will sleep better knowing that Tehran is not pursuing enrichment activities, whatever their alleged purpose.

But the EU3 agreement, which fails to discuss consequences for Iran if it breaks the deal, is vulnerable to being undermined not only by Iran but also by the United States; both have already raised eyebrows in the wake of the accord. Iran raced to produce uranium hexafluoride, a gas that can be enriched into bomb fuel, before it began to observe the temporary suspension on Monday. And both President George W. Bush and outgoing Secretary of State Colin Powell have publicly aired their suspicions that Iran will continue its drive for nuclear weapons under cover of the deal.

At the moment, administration hawks are pressing to confront the mullahs at the United Nations Security Council, where economic sanctions could be considered; calls for using force and for regime change are likely to follow.

Military action is inadvisable at this point, because of a dearth of solid intelligence and the secretive, geographically diffuse nature of Iran's nuclear sites. If the issue reaches the Security Council with the United States and Europe continuing along divergent paths, the inevitable deadlock will deal a severe and lasting blow to international security. Therefore, the agreement must be fortified to keep the Iranians honest, the Europeans effectively engaged and the U.S. hawks bridled.

This can be achieved through a U.S.-European accord laying out trigger mechanisms for specified consequences if Iran violates certain benchmarks. For example, if Iran fails to allow inspectors the access accorded by the Nuclear Nonproliferation Treaty's additional protocol--which Iran is provisionally observing pending ratification--or resumes enrichment and centrifuge-building activities, it could face severe economic sanctions, censure by the Security Council (necessitating cooperation from Russia and China), or in the event of hostility, a forceful response.

We don't know yet whether Tehran will play by the rules. The regime has mastered the art of behaving badly and then seeking rewards for getting back into line. To date, the Europeans have played into its hands, offering carrots for compliance without wielding sticks to punish violations.

Therefore, the Bush administration's apparent comfort with a military option can serve as an important deterrent against Iranian cheating, arming the EU3 agreement with teeth that it would not otherwise have. Iran desires economic incentives but does not yet desperately need them; without a credible threat of U.S.-backed sanctions imposed by the international community, the mullahs can simply decide one day that the restrictions have ceased to be worth their while, and break any deal as though it were merely a business contract.

For the United States, accepting the EU3's carrot-based approach (provided the benchmarks are added) will show the world that it still supports negotiated diplomacy and multilateralism, even in cases where military threats loom. Participating in this framework will also send a message to Iran that the United States is not ruling out renewed relations. This would resonate with the largely pro-American Iranian populace, who despise their regime and are seeking inroads to break free of it.

But if the United States instead presents itself as a unilateralist maverick, it will hinder its own interests; the only thing Iranians disdain more than the mullahs is outside meddling with their deeply nationalistic desire for self-determination. The more overtly hostile the United States acts toward Iran, the more the mullahs are able to spin America's posture to alienate Iranians against the "Great Satan."

The way to keep the Iranian regime in check while speeding its demise is to insure the nuclear agreement through benchmarks and triggers, and then give the mullahs exactly what they ask for in terms of increased access to international institutions like the World Trade Organization.

Such carrots can also be Trojan Horses, allowing the forces of democratic reform within Iran to blossom by enabling pro-democracy elements to make global connections. The U.S. and Europe should saddle up those horses together.

We've been down this road before: A crisis threatens global security, and the international community is not coming together to deal with it. Hawks in the U.S. administration see the Europeans as too timid to use force and reliant on diplomacy to a fault, while many Europeans see the United States as trigger-happy and too impatient with negotiated settlements. This lack of cohesion damaged the legitimacy of the American-led war in Iraq and left U.S.-European relations in tatters. A similar disunity jeopardizes current attempts to manage Iran's nuclear aspirations, even though both sides agree that the threat posed by a nuclear Iran is grave and real.

Departing Deputy Secretary of State Richard L. Armitage positively spun the divergent U.S. and European approaches to Iran: "The [diplomatic] incentives of the Europeans," he said, "only work against the backdrop of the United States being strong and firm on this issue. In the vernacular, it's kind of a good cop/bad cop arrangement. If it works, we'll all have been successful." The problem with Armitage's hopeful outlook is that the good cop/bad cop strategy works only if pursued consciously and in coordination, and the U.S. and European approaches do not reflect that yet. In fact, they seem headed in opposite directions.

The good cops--Britain, France and Germany--recently persuaded Iran to suspend all uranium enrichment-related activities until they reach a final accord. If the mullahs cooperate, they will receive numerous economic carrots, including possible membership in the World Trade Organization (the U.S. would have to agree) and improved trade relations.

In October 2003, when the International Atomic Energy Agency was prepared to take its negative report on Iran's nuclear program to the U.N. Security Council, the mullahs cut a similar deal with the Europeans, promising to suspend all enrichment-related activities. But Iran soon grew impatient with the agreement and resumed efforts to produce the gas that feeds uranium enrichment. It similarly rushed to make as much of that gas as possible before the latest accord's deadline, undercutting confidence in the deal on both sides of the Atlantic. In another bad-faith move, Iran announced last week that it wanted to keep operating uranium-enrichment equipment for research purposes, backing off its pledge to freeze all such activities.

Enter the bad cop--the United States. It has pushed to refer the question of Iran's nuclear aspirations to the Security Council. When Secretary of State Colin L. Powell steps down, the hawkish voices in his department will probably intensify and gain influence, especially if the mullahs break the newest deal.

Armitage might be right that the discordant U.S.-European approaches will push the mullahs to hold to the deal. But the U.S. will be uncomfortable with an agreement that does not insist on any means of enforcement or verification, as is the case with the latest accord.

Iran knows that the war in Iraq colors U.S. conduct toward it. The worse Iraq gets, the less Iran worries--and the mullahs don't seem too worried at the moment. But if they break the accord with the Europeans and the Europeans respond timidly and U.S. resources are freed up as a result of an improving situation in Iraq, the U.S. could take on Iran alone--to everyone's detriment. To avoid this risk, the U.S. and Europe need to harmonize their approaches and develop a coordinated strategy for Iran. The best way to accomplish this is to agree in advance on the consequences Iran will face if it violates its commitments. For example, if the mullahs renege on the latest deal, frustrate the monitoring and verification efforts of IAEA inspectors or fail to ratify an addition to the Nuclear Non-Proliferation Treaty that allows for more invasive inspections, the U.S. and Europe should go to the Security Council, impose economic sanctions or, in the worst case, take military action.

Fortunately, diplomatic disunity over Iran does not run as deep as it did over Iraq, where even the nature of the threat was a bone of contention. Both the U.S. and Europe are worried about a nuclear Iran, and they feel strongly about enforcing the rules of nonproliferation. In June 2003, European foreign ministers required only 45 minutes to approve a document that endorsed U.N.-sanctioned use of force as a last resort against proliferators, as well as "political and diplomatic preventative measures."

If the Europeans agree to leave all responses on the table and to act decisively at the first sign of Iranian mischief, the United States would be foolish not to form a partnership with them. (It's also important that the U.S. set a better example as a member of the nonproliferation community by abandoning plans to build new mini-nuclear weapons and ratifying the Comprehensive Test Ban Treaty.)

The role the U.S. forges for itself in dealing with Iran will have significance beyond reinvesting in international order or responding to the mullahs' nuclear ambitions. Iran's despotic regime will collapse some day, and there will be a "morning after" similar to that in Iraq, where reconstruction efforts have floundered because U.S. planners underestimated the challenge of nation-building and the need for international support to make it work. When Iran makes its move toward a better government, the U.S. should be in a position to lead a coherent, collective international effort to help it get off the ground.

Yet since the severing of U.S.-Iranian ties in 1980, the U.S. has been slack in developing a viable Iran policy. Iran's nuclear ambition should be motive enough to reverse this inattention. U.S. policy toward Iran must cease to be reactive, as it is now.

In addition to working with the Europeans to curb the mullahs' nuclear efforts, the U.S. should begin crafting a strategy to work toward--and then with--a democratic Iran. Supporting a government that complies with its international obligations is certainly preferable to containing one that thwarts them. By getting involved now, the U.S. can do much to show Iranians that it will be a friend to a free Iran. A democratic Iran may still want a nuclear bomb as a matter of national pride. But a less threatening, pro-diplomacy U.S. would be in a stronger position to argue the benefits of membership in the nonproliferation community rather than life as a rogue power.

Participating in a multilateral approach to Iran's nuclear program is a great place to start. In doing so, the U.S. will signal to Iranians that its aggressive position does not reflect a desire to remake Iran in its own image but rather a desire to achieve, alongside Europe, a substantial victory for nonproliferation and international security.