From the September/October 2004 issue of Foreign Affairs.

As the war on terrorism continues, statistics on terrorist attacks are becoming as important as the unemployment rate or the GDP. Yet the terrorism reports produced by the U.S. government do not have nearly as much credibility as its economic statistics, because there are no safeguards to ensure that the data are as accurate as possible and free from political manipulation. The flap over the error-ridden 2003 Patterns of Global Terrorism report, which Secretary of State Colin Powell called "a big mistake" and which had to be corrected and re-released, recently brought these issues to the fore. But they still have not been adequately addressed.

Now-common practices used to collect and disseminate vital economic statistics could offer the State Department valuable guidance. Not long ago, economic statistics were also subject to manipulation. In 1971, President Richard Nixon attempted to spin unemployment data released by the Bureau of Labor Statistics (BLS) and transferred officials who defied him. This meddling prompted the establishment of a series of safeguards for collecting and disseminating economic statistics. Since 1971, the Joint Economic Committee of Congress has held regular hearings at which the commissioner of the BLS discusses the unemployment report. More important, in the 1980s, the Office of Management and Budget issued a directive that permits a statistical agency's staff to "provide technical explanations of the data" in the first hour after principal economic indicators are released and forbids "employees of the Executive Branch" from commenting publicly on the data during that time.

The State Department should adopt similar protections in the preparation and dissemination of its reports. In addition to the global terrorism report, the State Department is required by Congress to report annually on international bribery, human rights practices, narcotics control, and religious freedom. Gathering and reporting data for congressional oversight is presently a low-level function at the State Department. The department rarely relies on high-quality, objective data or on modern diagnostic tests to distinguish meaningful trends from chance associations. Adopting safeguards against bias, both statistical and political, would enable Congress to better perform its constitutional role as the White House's overseer and allow the American public to assess the government's foreign policy achievements.

A PATTERN OF ERRORS

Congress requires that the State Department provide each year "a full and complete report" that includes "detailed assessments with respect to each foreign country ... in which acts of international terrorism occurred which were, in the opinion of the Secretary, of major significance." The global terrorism reports are intended to satisfy this requirement, but, over time, they have become glossy advertisements of Washington's achievements in combating terrorism, aimed as much at the public and the press as at congressional overseers.

The 2003 global terrorism report was launched at a celebratory news conference in April. Deputy Secretary of State Richard Armitage and Ambassador J. Cofer Black, the State Department coordinator for counterterrorism, outlined some remaining challenges, but principally they announced the Bush administration's success in turning the terrorist tide. Black called the report "good news," and Armitage introduced it by saying, "You will find in these pages clear evidence that we are prevailing in the fight." The document's first paragraph claimed that worldwide terrorism dropped by 45 percent between 2001 and 2003 and that the number of acts committed last year "represents the lowest annual total of international terrorist attacks since 1969." The report was transmitted to Congress with a cover letter that interpreted the data as "an indication of the great progress that has been made in fighting terrorism" after the horrific events of September 11.

But we immediately spotted errors in the report and evidence contradicting the administration's claims. For example, the chronology in Appendix A, which lists each significant terrorist incident occurring in the year, stopped on November 11-an unusual end to the calendar year. Clearly, this was a mistake, as four terrorist attacks occurred in Turkey between November 12 and the end of 2003. Yet it was impossible to tell whether the post- November 11 incidents were inadvertently dropped off the chronology and included in figures in the body of the report or completely overlooked.

More important, even with the incomplete data, the number of significant incidents listed in the chronology was very high. It tallied a total of 169 significant events for 2003 alone, the highest annual count in 20 years; the annual average over the previous five years was 131. How could the number of significant attacks be at a record high, when the State Department was claiming the lowest total number of attacks since 1969? The answer is that the implied number of "nonsignificant" attacks has declined sharply in recent years. But because nonsignificant events were not listed in the chronology, the drop could not be verified. And if, by definition, they were not significant, it is unclear why their decrease should merit attention.

On June 10, after a critical op-ed we wrote in The Washington Post, a follow-up letter to Powell from Representative Henry Waxman (D-Calif.), and a call for review from the Congressional Research Service, the State Department acknowledged errors in the report. "We did not check and verify the data sufficiently," spokesman Richard Boucher said. "... [T]he figures for the number of attacks and casualties will be up sharply from what was published."

At first, Waxman accused the administration of manipulating the data to "serve the Administration's political interests." Powell denied the allegation, insisting that "there's nothing political about it. It was a data collection and reporting error." Although there is no reason to doubt Powell's explanation, if the errors had gone in the opposite direction-making the rise in terrorism on President George W. Bush's watch look even greater than it has been-it is a safe bet that the administration would have caught them before releasing the report. And such asymmetric vetting is a form of political manipulation.

Critical deficiencies in the way the report was prepared and presented compromised its accuracy and credibility. Chief among these were the opaque procedures used to assemble the report, the inconsistent application of definitions, insufficient review, and the partisan release of the report. These deficiencies resulted in a misleading and unverifiable report that appeared to be tainted by political manipulation.

It is unclear exactly how the report was assembled. The report notes that the U.S. government's Incident Review Panel (IRP) is responsible for determining which terrorist events are significant. It says little, however, about the panel's members: how many there are, whether they are career employees or political appointees, or what affiliations they have. Nor does it describe how they decide whether an event is significant. Do they work by consensus or majority rule? What universe of events do they consider?

The State Department announced a decline in total terrorist attacks, which resulted from a decline in nonsignificant events. But without information about the nonsignificant events, readers were essentially asked to blindly trust the nameless experts who prepared it.

The report's broad definitions, moreover, are sometimes too blunt to help classification. Terrorism is defined as "premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents, usually intended to influence an audience." The report specifies that an international terrorist attack is an act committed by substate actors from one nation against citizens or property of another. An incident "is judged significant if it results in loss of life or serious injury to persons, major property damage, and/or is an act or attempted act that could reasonably be expected to create the conditions noted."

But hardly any explanation was provided about how the IRP distinguishes significant from nonsignificant events. When is property damage too minor for an event to be significant? How are nonsignificant events identified? Is the IRP responsible for making these determinations too? Has the source and scope of their information changed over time? The corrected 2003 report, the first to list individual nonsignificant acts, defines as "major" property damage that exceeds $10,000. It does not indicate, however, whether that criterion applied to previous reports.

Admittedly, measuring international terrorism is no easy task. Even scholarly reckonings are not free from subjective judgment, and there are inevitably close calls to be made. The most one can hope for in many cases is consistent application of ambiguous definitions.

Unfortunately, in the global terrorism reports the rules have been applied inconsistently. Many cross-border attacks on civilians in Africa have not been included in the reports, for example, even though similar attacks in other regions have been. The report for 2002, moreover, counts as significant a suicide attack by Chechen shaheeds (Islamist martyrs) against a government building in Moscow that killed 72 people. Yet none of the numerous suicide attacks by the Chechen "black widows" that terrorized Russia and killed scores in 2003 was tallied as an international terrorist attack in the latest report. After one such attack, Russian President Vladimir Putin said, "Today, after a series of recent terrorist attacks, we can say that the bandits active in Chechnya are not just linked with international terrorism, they are an integral part of it." If the State Department considers such attacks domestic, rather than international, it should do so consistently from one year to the next.

Another problem is that the staff that prepared the 2003 global terrorism report did not participate in releasing it; in fact, they have yet to be identified. High-level Bush administration officials presented the report to the media, using it to support White House policies and take credit for the alleged decline in terrorism. Even after the report's flaws were recognized, they continued to spin the figures. When the corrected version was released, Black repeated that "we have made significant progress," despite being pressed to acknowledge that last year the number of significant attacks reached a 20-year high. Given the war on terrorism's central role in the upcoming presidential election, such presentation gives the appearance that the report is being manipulated for political gain.

The State Department has tried to explain the report's flaws using language eerily reminiscent of the Bush administration's justification of the failure to find weapons of mass destruction in Iraq. Spokesman Boucher told reporters that previous claims that the war on terrorism was succeeding had been based "on the facts as we had them at the time [and] the facts that we had were wrong." Even Powell partook in the spinning. On the one hand, he announced that "the [original] narrative is sound and we're not changing any of the narrative." On the other hand, he acknowledged, "We will change the narrative wherever the narrative relates to the data."

To his credit, Powell instructed those responsible for preparing the report to brief Waxman's staff on the procedures they had used and the origins of their mistakes. Based on a summary of the briefing by Waxman's staff, much has come to light. Authority for compiling the list of attacks was shifted from the CIA to the Terrorist Threat Integration Center (TTIC), an organization created in May 2003 to "merge and analyze all threat information in a single location." The TTIC provided information to the IRP, which, it was disclosed, consists of representatives from the CIA, the Defense Intelligence Agency, the National Security Agency, and the State Department's Bureau of Intelligence and Research. A TTIC representative chaired the meetings and could cast a vote to break ties on the classification of an event as significant or nonsignificant.

At least this year, chaos prevailed. The IRP's members changed from meeting to meeting-when they attended the meetings at all. The CIA employee responsible for the database left but was never replaced; in mid-process, an outside contractor who entered data was replaced by another contractor. Because of technical incompetence, the report relied on the wrong cutoff date.

Arithmetic errors were rampant. Larry Johnson, a retired CIA and State Department professional, discovered that the total number of fatalities in the chronology exceeded the number listed in the statistical review in Appendix G. According to Black, the errors resulted from "a combination of things: inattention, personnel shortages and database that is awkward and is antiquated and needs to have very proficient input be made in order for to be sure that the numbers will spill then to the different categories that are being captured [sic]." The debacle is more like an episode of the Keystone Kops than a chapter from Machiavelli, but even that analogy is not very comforting.

SETTING THE RECORD STRAIGHT

Despite the data's limitations, the chronology of significant events in the 2003 global terrorism report yields important information about terrorism's trends, its geographical characteristics, and its magnitude.

Time-series analysis, which seeks to discern trends in given phenomena over time, requires a consistent approach to collecting data. The State Department's terrorism report presents time-series analysis, but by focusing on the total number of attacks it misleadingly combines verifiable data on significant events with nonverifiable data on insignificant ones. And because, as TTIC director John Brennan admitted, "many nonsignificant events occur throughout the world that are not counted in the report," one must also be concerned about consistency in the measurement of the total number of terrorist events. Even if the nonsignificant events were listed (and thus could be verified), trends in significant events are more relevant because they track events that, by definition, are more important. Accurately measuring these trends is a prerequisite for understanding the factors that underlie them and the policies that shape them. In fact, an analysis of the revised report reveals that the number of significant attacks increased from 124 to 175, or by 41 percent, from 2001 to 2003-a significant fact indeed.

The detailed chronology also allows analysts to cumulate terrorist events for each country and cross-classify them according to the country where they occurred and the perpetrators' country of origin. These figures can then be related to the countries' characteristics, yielding information that can help policymakers devise strategies to address terrorism's root causes. Using the global terrorism reports for the years 1997-2002, the authors of this article have previously found that terrorists tend to come from nondemocratic countries, both rich and poor, and generally target nationals from rich, democratic countries.

The State Department has rightly emphasized that the threat of terrorism remains serious, but a close examination of its data helps put the magnitude of the threat in perspective. In 2003, a total of 625 people--including 35 Americans--were killed in international terrorist incidents worldwide. Meanwhile, 43,220 died in automobile accidents in the United States alone, and three million died from AIDS around the world. Comparative figures, particularly when combined with forecasts of future terrorism trends, can help focus debate on the real costs people are willing to bear--in foregone civil liberties and treasure--to reduce the risk posed by terrorism.

CHANGING TRACKS

The State Department currently uses, and Congress accepts, nineteenth-century methods to analyze a twenty-first-century problem. To prevent errors of the type that riddled the 2003 global terrorism report, Congress has two alternatives. It could reassign the State Department's reporting responsibilities to a neutral research agency, such as the GAO (the General Accounting Office, recently renamed the Government Accountability Office) which routinely uses appropriate statistical practices. The problem is that the GAO has little foreign policy expertise and does not necessarily have access to the (sometimes classified) information that goes into the reports. Alternatively, Congress could keep the reports within the State Department's purview but demand that its practices for data collection and analysis be improved and that the reports be insulated from partisan manipulation.

If responsibility remains within the State Department, Congress should establish a statistical bureau in the department to ensure that scientific standards are respected in all reports, thereby elevating the status of data-gathering and statistics there. The bureau would promote consistency, statistical rigor, and transparency. When appropriate, it could seek input from the scientific community. And, while respecting classified sources, it could also insist that sufficient information be released to independent analysts for verification.

To overcome conflicts of interest facing political appointees who issue government reports, the State Department should adopt rules similar to those that govern the production and dissemination of key economic indicators. Career staff who prepare the reports should be given an hour to brief the media on technical aspects of the data, during which time political appointees would be precluded from making public comments. (After the hour elapses, it is expected that political appointees would offer their interpretations.) Career staff should be protected so they can prepare mandated reports without interference from political appointees and then present them for review by the statistics bureau. Once the reports are finalized, but before they are publicly released, they should be circulated to designated political appointees who need to prepare for their release. Disclosure dates should be announced long in advance to prevent opportunistic timing by political appointees.

Last October, in a candid memorandum to top aides that was leaked to the press, Secretary of Defense Donald Rumsfeld admitted, "Today, we lack metrics to know if we are winning or losing the global war on terror. Are we capturing, killing, or deterring and dissuading more terrorists every day than the madrassas [Islamic schools] and the radical clerics are recruiting, training, and deploying against us?" The statement was a stinging acknowledgment that the government lacks both classified and unclassified data to make critical policy decisions. It is also a reminder that only accurate information, presented without political spin, can help the public and decision-makers know where the United States stands in the war on terrorism and how best to fight it.

The post-1970 crisis in the Islamic world and Pakistan's role

It is possible to trace today's crisis in the Islamic world back to the time of the Prophet (pbuh) and the four Righteous Caliphs. Many Salafists among Muslims and many so-called Orientalists among Westerners do just that. Opposed in every other way, they both believe in an Islamic "essence" unchanged since then. Others go back to the 19th century CE, to the onset of Western domination over much of the Muslim 'umma. Yet it seems to me that to understand today's crisis adequately we need go no further back than the years around 1970 in Islam's old Arab and Iranian heartlands. Admittedly a number of factors had to come together to produce the dilemmas we still live with.

The 20th century struggle against colonialism raised high hopes that the departure of the colonisers would usher in a new era of dignity and prosperity for Muslims. The main ideology of these hopes was the kind of republican nationalism associated with Gamal 'Abd al-Nasser in Egypt and Muhammad Mossadeq in Iran. By about 1970 these hopes had collapsed.

Not only had Israel persisted as a reminder that decolonisation did not mean an end to subordination, but the 1967 Six Days' War was such a catastrophe that its casualties were not just military: it discredited the republican nationalist ideology as well. The Arab world was rent by rivalries between republicans and monarchists, with the Cold War protagonists egging them on and paying them rents for friendship. Worst of all, the postcolonial regimes turned out to be authoritarian and corrupt.

Nor was that the whole story. There had also been much economic and social development, yet it was of very special kinds. State-led industrialisation had been based mainly on oil and gas, and oil and gas are special commodities. The iron and steel that drove earlier Western growth had created new middle and working classes; oil and gas do not, and their profits are easily captured by sitting elites. To pay for industry, moreover, states ran down agriculture. Within decades this drove millions from farms and small towns into cities that then exploded their infrastructures. The states offered education, particularly at higher levels - at one point Egypt was producing 75,000 graduates a year - but beginning about 1970 states were withdrawing from the economy and turning responsibility for growth over to captive and anaemic private sectors. So more and more first-generation graduates were entering increasingly slack economies with no real prospects for jobs or dignity.

All this was a recipe for political radicalism, and the ideological vacuum left behind by discredited republican nationalism was filled by the dream of recreating the unity and purity of the original 'umma in the 7th century CE. That dream had been part of Islamic discourse almost from the beginning, but it had mainly appealed to the 'umma's fringes, the Bedouin soldiers of the Khariji movement, the small townsmen of Islam's middle years who had then become Shi'a or Sufis. Now, around 1970, the dream had been modernised by thinkers like Sayyid Qutb in the Arab lands, 'Ali Shariati in Iran, and Maulana Abu-l-'Ala Maududi in this country, and in that form it entered the Islamic mainstream. It became the chief ideology of opposition to the authoritarian and corrupt postcolonial regimes.

The result has been thirty years of savage and bloody civil war among Muslims. It has struck Westerners and Israelis too, but most of the victims have been Muslim, because the regimes were now headed by Muslims. When Syrian leader Hafez al-Assad retook the city of Hama from Sunni insurrectionists in 1982, he killed at least 10,000 people, three times the casualties of September 11.

What would it take for Muslims to transcend this crisis? Time after time in their history they have overcome huge challenges by creating marvellous new syntheses of thought and feeling and practice. I have no doubt that they have the spiritual and intellectual and physical resources to do so once again. But what would be the elements of renewal at this new stage?

Some elements have already been moving into place.

As the civil war has proceeded, there has been covert movement on both sides toward a new centre. Regimes have been Islamising themselves. They have been introducing some Islamic law and some Islamic practice into their governance. Conversely, Islamists have been entering the political system. They now run for election; they enter cabinets; they serve in parliaments; they function as (more or less) loyal oppositions.

The process has been drenched in bad faith on both sides, but movement has been real.

Concurrently, more and more Muslims who might have become Islamist political revolutionaries two decades ago are now forsaking politics for community action in the 'umma. Rather than bombs and guns, the name of the game is now schools, clinics, charities, and the Islamic piety of individual Muslims and their families.

Moreover, with the end of the Cold War sitting regimes can no longer collect rents from the USSR, and they find it harder to collect rents from the US now that competition with the USSR is over. Even the new rents the US is paying since September 11 will never match Cold War largesse. There will never again be enough official assistance to keep regimes in power by sustaining their growth rates.

Now they must rely instead on private foreign direct investment (PFDI). This is because all over the world production of knowledge is replacing production of things as the engine of economic growth. PFDI flows mainly on economic grounds. It is not attracted by the archaic, state-dominated, information-shy economies of the Arab Middle East and Iran. Their share of world PFDI has fallen from 12 percent in 1990 to 3-4 percent today. To attract it, they need reforms that will make them less rigid, less state-dominated, and less information-shy. Such economic reforms typically lead to demands for political reforms too. That is their quandary.

Such pressures will not end Islamist radicalism. The conditions that give it birth are often still there. But such pressures do tend to force radicalism to the margins of the 'umma once again. Osama is a perfect example: through the 1990s he was forced step by step back to the only place in the world where he now had a double layer of protection and hence the space and time needed to mount an operation like September 11.

Nor will such pressures automatically generate the new Islamic synthesis the planet needs. But they do create a new opportunity for Muslims to fashion an authentically Islamic modernity that is adequate to their history and their hopes.

I would argue that September 11 did not change this basic picture. It came as a shock to most Muslims, and even Islamists asked themselves whether Osama's methods were the best path to the common goal. Iraq, of course, has been much more problematic. There military defeat was so rapid and complete that it rekindled the usual Arab feelings of helplessness and rage, and the botched aftermath has given these feelings time to swell and take political form. Radicalism is reconstituting itself, but - it should be noted - on a new basis.

For Osama, for Dr Ayman al-Zawahiri, Islam may still be the banner of revolutionary overthrow. For younger Muslims, Islam is increasingly the badge of membership in national communities. It is no longer just an ideology for outsiders. More and more it is the ideology of outsiders and deprived or threatened ruling ethnic elites: Sunni Tikritis in Iraq, Pushtuns in Afghanistan. Driven toward the margins by repression, cooptation or military defeat, Islamism is re-entering the body politic through the service entrance of Islamo-nationalism.

The consequences can be unhealthy. If only Muslims should be citizens, Christians and Jews are excluded in ways quite novel in Islamic experience, and quite dangerous. But there may also be a new and exciting opening for an Islamic legitimation of the modern nation-state that is valid for Sunnis.

So far, the only place in the Islamic heartlands to produce such a legitimation has been Iran. Not long before he died in 1989, Imam Khomeini ruled on religious grounds that in emergencies national interests can take precedence over the shari'a. It helps explain how Iran has emerged from the charismatic phase of Islamic rule without widespread violence. But Iran's special Shi'i traditions make it hard to transpose to Sunni-majority societies. Taliban rule in Afghanistan was perhaps an effort to create a version for Sunnis, but it ended before it succeeded. In both cases, moreover, the effort took place within a theocratic framework, direct rule by 'ulema.

Theocracy is not a mainstream Islamic tradition and will not appeal in most Muslim countries. A broader version of religious legitimation of the nation-state could be taking shape now in Iraq. It may be that the Americans are needed both as a parameter and as a target. But the outcome is very uncertain, the circumstances very special. And Iraq too has a majority of Shi'a.

Where does Pakistan fit in this picture? I see some similarities and more differences.

Like some Arab states, Pakistan inherited a postcolonial security threat that has absorbed disproportionate resources and has thereby reinforced older socio-political structures and a traditional sense of political irresponsibility: someone else is always to blame.

Although Pakistan was founded as an Islamic nation-state by modern means and modern people, here too modernity is so associated with the West that it must be denied as un-Islamic.

And Pakistan too has been stranded by the end of the Cold War and the onset of the IT era in economics. New rents from the war on terrorism will not restore the levels of official assistance Pakistan attracted before 1990, and private foreign direct investment has not rushed in to fill the gap.

But Pakistan is also different from the Arab world and Iran in relevant ways. Some are counterintuitive; most are to Pakistan's advantage.

First, Pakistan is not dependent on oil and gas, and can be better off for it. Pakistan is dependent on cotton, and compared to oil and gas, cotton and cotton textile production makes for larger middle and working classes, better attuned to modern political and economic needs than Middle Eastern elites.

Second, Pakistan is less developed than the old Islamic heartlands - more agricultural, less urbanised, less educated - and that too can help. It has not destroyed its agriculture. Except for Karachi, rural outmigration has not exploded its cities, and even there civil war has been on an ethnic and not a religious basis. And the graduating cohorts entering the limp economy have been relatively small. In other words Pakistan has not yet produced the conditions that brought Islamist radicalism to the centre of Middle East politics. It therefore has a window of opportunity to create better structures less conducive to civil war.

Third, Pakistanis have been struggling for over half a century to bring religion and politics together in a functioning system of governance. The need to experiment came with Pakistan's original mandate; it has led through the Ahmedi riots, the Objectives Resolution, the MRD in 1977 (sic: PNA is meant), and various Islamisation steps thereafter. Certainly, however, experimentation has been particularly intense since 2002. Its outcome is also quite uncertain.

What this means, though, is that Pakistanis have a wealth of lived experience wrestling with issues that are newer and more destructive in other Muslim societies, and of doing so mainly without violence. They should therefore be better able to integrate the religious impulse into a basically democratic political system without first establishing theocracy. If they can, it will be a first version of religious legitimation for the modern nation-state in a society with a recognisably Sunni majority. Where Pakistan fits in todayís Islamic world is as a major test case. Not for Americans: for Pakistanis. And for all the other members of the 'umma.

*Footnote: This essay draws on themes from the writer's book on Islam and a talk he gave at the Administrative Staff College in Lahore on May 24, 2004.