John Diamond has been a media fellow at Stanford University's Hoover Institution and is an associate in the consulting firm Technology Growth Partners, which advises high-tech startup firms. His recent book, 'The CIA and the Culture of Failure: US Intelligence From the End of the Cold War to the Invasion of Iraq,' published by Stanford University Press, was the subject of a recent talk to the World Affairs Council in San Francisco.

THE short drive from the White House to the headquarters of the Central Intelligence Agency can begin on Constitution Avenue. The route crosses the Theodore Roosevelt Bridge into Virginia and merges onto the George Washington Memorial Parkway.

After tracing the big bend in the Potomac River, the parkway begins to climb. In a few hundred yards, the spires of Georgetown University, academic home to many a former intelligence officer and recruiting ground of many a future one, come into view over the cliffs on the opposite bank. Not far from the parkway, in the Rosslyn section of Arlington, CIA officials met in 1991 with United Nations weapons inspectors to develop a plan for uncovering and destroying Iraq's weapons of mass destruction (WMD).

On the approach to the Agency entrance in Langley, Virginia, a sign marked "WARNING: Restricted U.S. Government Installation" interrupts the pretty suburban scenery. Occasionally, a small cross and bouquet of flowers can be seen placed along the median by the turn lane into the Agency grounds. These tokens of love and mourning mark the scene of a double murder in early 1993 when a lone Pakistani man, enraged by the presence of American troops in Muslim holy lands, opened fire with an AK-47 assault rifle on a line of cars carrying CIA employees and contractors awaiting the turn signal into the Agency grounds.

  IN the post-9/11 world, with the awareness that the hatred of one killer signaled the intent of a global jihadist movement, the drive into the CIA grounds is a multi-step procedure. Trees, shrubs and lawns designed to give the Agency headquarters a campus feel do not hide the barbed-wire fencing, hydraulic metal barriers and concrete stanchions that are more evocative of a medium-security prison.

On a lawn closer to the headquarters stands a chunk of the Berlin Wall, gift of a reunited Germany in recognition of the years during which Berlin had been the central arena of the Cold War. Spray-painted graffiti on the concrete slab includes what appears to be part of a line from a Jimi Hendrix song: ". . . and the wind cries . . ." Other graffiti, also, oddly, in English, include the words "Freedom" and "Democracy" and a paraphrase of President Ronald Reagan's June 12, 1987, speech in Berlin: "tear down the wall."

The words evoke a yearning for freedom from tyranny by people in a place where the CIA fought communism over four decades. In fact, the paint was applied not in Berlin but at Langley, to dress up an otherwise non-descript slab of concrete. As such, it stands as the CIA's last propaganda gambit against the East Bloc, a final psychological operation, an inside joke cooked up to enhance a souvenir of the Cold War.

Here, in January of 2003, Secretary of State Colin Powell, CIA Director George Tenet and a cluster of senior intelligence analysts put together the now-infamous briefing for the United Nations Security Council in which Powell, citing information since discredited as almost entirely erroneous, presented the case that Iraq's arsenal of weapons of mass destruction warranted immediate military action.

From about the time that unpainted slab of the Berlin Wall arrived at Langley to the day when Colin Powell and a room full of harried intelligence analysts assembled a case for war in Iraq—the CIA experienced a series of tremors that weakened its foundation, though the cracks were not always visible at the time. Many of these took on the label "intelligence failure"—sometimes because the CIA had failed, other times because the White House or Congress found it expedient to make it seem as if it had.

These controversies—allegations that the CIA missed the collapse of the Soviet Union, the CIA's performance before and during the Persian Gulf War, the Aldrich Ames spy scandal, the belated realization of the al-Qaeda threat, battles over the capabilities of "rogue state" adversaries, the intensifying struggle against Islamic terror and the near-total misjudgment of Iraq's arsenal—unfolded during a decade of transition and distraction for the United States.

The economy, the Republican takeover of Congress, crises in Somalia, the Balkans and Rwanda, and the Monica Lewinsky scandal and impeachment of President Clinton all deflected the country's attention from the gradual weakening of the intelligence community during a time when its services were most urgently needed. Problems at the CIA seemed to be matters for Washington. The consequences of failure seemed detached from the lives of average Americans.

The 9/11 attacks and the course of the war in Iraq would change that.

THE CIA reports to Congress, the military and other arms of the executive branch, but most of all to the White House, and specifically the president. On most mornings, the first thing a president reads comes from the CIA.

These arms of government are the CIA's "customers," in the fashionable term, and the president is first among them. This allusion to the CIA as something akin to a business, with customers who have both needs and demands that must be served, gained in currency during the business-centric 1990s. Intelligence reports freighted with caveats, footnotes and minority views that ate up the time of busy executives were out, as were estimates of future developments that predicted so many different possibilities as to predict nothing at all.

Though the term intelligence "customer" or "consumer" remains in use to this day, the idea that the CIA served only a rarified group of cleared officials within the government was shattered by the 9/11 attacks and by the disastrous miscalculations in the run-up to war in Iraq. More than at any time since the Japanese attack on Pearl Harbor, intelligence failure reached directly into the lives of average Americans.

Throughout the years between the end of the Cold War and 9/11, warning signs of serious trouble in the intelligence community had been accumulating. Budget cuts were an important ingredient, but there were other stresses as well. Severe turbulence beset the Agency's executive suite, where five directors held the top post in the first seven years of the 1990s. A series of events beginning with the CIA's flawed performance leading up to and during the Gulf War, its failure to predict the collapse of the Soviet Union and the eruption of the Aldrich Ames spy scandal led to a pronounced drop in confidence in the CIA among both Democratic and Republican leaders.

For most of the CIA's history, disclosure of its misdeeds or failures has alternately titillated the public, served various political agendas, or fueled conspiracy theories. Some Americans still believe the CIA had something to do with the assassination of President Kennedy and the crack cocaine epidemic of the mid-to-late-1980s. The political left spent the latter years of the Cold War focused on the CIA's role in propping up corrupt and abusive Latin American juntas as a bulwark against communism and welcoming hosts for American business interests. The admissions in the mid-1970s about CIA spying on Americans and CIA involvement in a plot to assassinate Fidel Castro gave political traction to the liberal criticism of the agency. On the right, anti-Soviet hard-liners argued that the reining in of the CIA following the congressional investigations of 1975 had defanged U.S. intelligence, leaving it ill-equipped to deal with an aggressive, expansionist adversary.

One offshoot of this argument was the Reagan administration's backing of a massive CIA operation to funnel weapons and training to Mujahadin fighters in Afghanistan.

THE intelligence lapses prior to the 9/11 attacks and the realization that the war in Iraq had been sold on a foundation of faulty intelligence brought the public into the national debate about intelligence in an unprecedented fashion.

No official body has unequivocally said that better intelligence could have prevented the 9/11 attacks, but an overwhelming consensus formed around the idea that massive intelligence failure had preceded 9/11 and that massive intelligence reform must follow it. The general public could see in the 9/11 attacks and the war in Iraq a straight-line connection between intelligence failure and the deaths of Americans.

The lapses that allowed the 9/11 plotters to keep their deadly plan secret, the gross overestimation of the threat posed by Saddam's supposed weapons of mass destruction and underestimation of the danger of insurgency and civil war that might follow the collapse of his regime, contributed to events that cost thousands of lives and hundreds of billions of dollars.

These devastating events were not caused by the CIA. Osama bin Laden and al-Qaeda launched the 9/11 attacks, and the CIA had warned, generally, that we were in danger; Saddam Hussein adopted the policy of strategic ambiguity that complicated efforts to accurately assess the strength of his armaments; President Bush, not the CIA, made the decision to invade Iraq; some CIA analysts had warned, tentatively, of civil unrest in Iraq after the fall of Saddam, and so on. But the integral role of intelligence in these events and their impact on average Americans brought the public into the debate over how the United States steals secrets and interprets intelligence in a way unimaginable just a few years ago.

THE seemingly limited and transient consequences of the earlier post–Cold War intelligence failures and the profound consequences of the failures related to September 11 and the war in Iraq are connected, however. A steady erosion of intelligence capability contributed to a series of intelligence lapses and alleged lapses—and to a consequent decline of confidence in the intelligence community—that left the CIA critically weakened. These processes fed off and fueled one another, leading to a fatal cycle of error, criticism, overcorrection, distraction and politicization.

The term "culture of failure" refers not to alleged CIA incompetence, which, though it occurs, is often overstated by the agency's critics. Rather it refers to an atmosphere of declining confidence in the abilities of U.S. intelligence to do its job. This diminished faith in the abilities of the CIA existed at the CIA itself, as well as in the executive branch and Congress.

At the beginning of the 1990s, national security debate focused on managing the "peace dividend" that would come from trimming spending on defense and intelligence that had been overwhelmingly geared toward countering the now-nonexistent Soviet threat. The CIA had indeed failed to foresee the Soviet breakup until it was almost upon us. To Agency critics such as Senator Daniel Patrick Moynihan, this massive failure stemmed from a Cold War mindset that led to gross overspending on weaponry and an unnecessary aggravation and prolonging of superpower tension. It was a mindset with a political history, dating back to the mid-1970s, in which hawkish critics of the policy of détente toward the Soviet Union brought forth an alternative interpretation of Soviet weaponry and policy to counter the CIA's interpretations.

Some of these same hawks, having overridden CIA circumspection about the Soviets and implemented a massive peacetime military buildup during the Reagan presidency, would return in the 1990s to push the national missile-defense program, once again beginning their campaign with an attack on CIA analysis that downplayed the missile threat. Many of these same hawks, some now carrying the "neoconservative" label, would hold key levers of power as the second Bush administration made its case for invading Iraq.

In the early 1990s, Congress waved off Moynihan's proposals to break up the CIA and distribute its component parts among various government agencies such as the Defense and State Departments. But even before the Soviet collapse, spending on intelligence was declining, and, with the exception of a single year, the trend continued through the 1990s. The cuts cast a shadow that would reach to the intelligence failures of 9/11 and the Iraq invasion. Training of new field officers ground to a virtual halt by the mid-1990s, as did the recruiting of analysts with expertise and knowledge of languages spoken in the parts of the world that were spawning the nation's emerging enemies.

ANALYSIS of Iraq in the 1980s to lay a foundation for understanding the intelligence misjudgments about Iraq at the time of the Gulf War, and the lasting impact of those misjudgments years later as the second Bush administration contemplated going to war against Iraq once again: both these episodes contributed substantially to the post–Cold War perception that the U.S. intelligence community was woefully underperforming, a perception that prevailed even within the CIA itself.

So it was that in the early 1990s U.S. intelligence was struggling to define its post–Cold War reason for being at a time when al-Qaeda, unbeknownst to the CIA, was planning and executing its first attacks on U.S. interests. To be sure, this atmosphere of doubt, this culture of failure, had an impact that was not always negative. Concern about the intelligence community led to earnest though largely unsuccessful efforts throughout the 1990s by congressional oversight committees, blue-ribbon commissions and the intelligence community itself to redefine and reform the intelligence system. On balance, however, the culture of failure afflicting U.S. intelligence was like an old injury, a weak point that kept hobbling the intelligence community as the decade unfolded toward the 9/11 attacks and war in Iraq, two of the greatest national security disasters in U.S. history.

Again and again, episodes point to a combination of real intelligence failure and persistent, sometimes exaggerated doubt about the capability of the intelligence community that fueled developments detrimental to U.S. interests.

ONE of the great ironies of the 1990s was that U.S. intelligence, in combination with U.N. inspectors and a policy of constant military vigilance, succeeded in ridding Iraq of its weapons of mass destruction. It was, as we now know all too well, a success that went entirely unnoticed by the intelligence community that helped bring it about.

The CIA did not trust its own abilities, and no one else in the national security community, whether Republican or Democrat, was prepared to argue otherwise. The perception of failure, real and exaggerated, led the CIA to a position in 2002 in which its own analysis rested on the assumption that it could not fully perceive what the Iraqi adversary was doing.

Even when adversaries revealed their intentions and capabilities, as in the al-Qaeda bombings of the U.S. embassies in Kenya and Tanzania in 1998, Serbia's suppression of Kosovar Albanians in 1999 and the suicide attack on the USS Cole in 2000, intelligence seemed inept at guiding the nation to an appropriate response.

Flawed intelligence led to President Clinton's decision to launch cruise missiles on a pharmaceutical plant in Sudan and on a suspected al-Qaeda leadership meeting in Afghanistan after the embassy bombings in Africa. The evidence that the Al-Shifa pharmaceutical plant was manufacturing nerve gas grew shakier by the day after the strike. And the al-Qaeda meeting did not take place, almost certainly because the terrorists canceled it after realizing that U.S. intelligence had detected their plans. The next year, during the Kosovo crisis, a U.S. B-2 bomber targeted 2,000-pound precision satellite-guided bombs onto the Chinese embassy in Belgrade, owing to an almost absurd series of lapses in target identification at the CIA.

Chastened by these embarrassing misfires, the Clinton White House, with the concurrence of the CIA's leadership, declined to take military action on several occasions when intelligence reports appeared to pinpoint the location of al-Qaeda leader Osama bin Laden.

The surprise accompanying the Indian and Pakistani nuclear tests and North Korea's three-stage rocket test in 1998 helped foster a belief in the CIA's incompetence. The assumption advanced by Donald Rumsfeld in an examination of the missile threat from North Korea and other "rogue" states was that the CIA could no longer be counted on to detect the threats arrayed against the United States. It was this judgment, and Rumsfeld's role in moving the missile defense program forward, that returned him to prominence and led to his being named secretary of defense in the George W. Bush administration.

IN the case of 9/11, the CIA failed to uncover the target, timing and perpetrators of an attack that it knew, in a general way, was coming. In the case of Iraq's alleged weapons of mass destruction, the CIA perceived a threat that was not there. It knew its assessments might be wrong, but decided that the "safe" course of action was to assume the worst. As a result, it helped the Bush administration justify a war that carried the nation headlong into a different disaster.

In January 2003, two months before the U.S.-led invasion of Iraq, two intelligence community reports put together by Paul Pillar, then the CIA's senior

Mideast analyst, warned the White House and Congress of the possibility of prolonged sectarian unrest in Iraq following the defeat of Saddam's regime. The reports predicted that al-Qaeda might set up terrorist operations in remote parts of Iraq. They cautioned that democracy would not take hold easily in a country accustomed to authoritarian rule. And they said that other states in the region would not abandon their weapons of mass destruction programs based on the military action against Iraq.

But the reports were issued three months after Congress had already voted to authorize war.

To the extent that the Iraq intelligence disaster resulted from a concerted effort by Donald Rumsfeld and Dick Cheney to undermine confidence in the CIA's ability to discern threats, the damage done by the debacle has hardly been repaired.

The intelligence community today is a hybrid, redesigned out of a perception of the causes of past intelligence failures. The de facto demotion of the CIA by the creation of a new Office of the Director of National Intelligence may prove a healthy development for the Agency. The CIA, to some extent, has slipped into the shadows, producing all-source intelligence and conducting operations while leaving it to its new master to respond to Congress and the public when things go wrong. In the wake of the 9/11 and Iraq intelligence disasters, the output of the intelligence community, for a time at least, appeared to be coming out into the open. Intelligence estimates raising concern about continuing violence in Iraq, al-Qaeda's gradual recovery from the post-9/11 U.S. offensive, and Iran's ambiguous nuclear ambitions were declassified in sanitized form, even though they generated negative publicity on the Bush administration.

New bureaucratic structures, however, have done nothing to ease the tension between U.S. intelligence, the architects of U.S. foreign policy, and the American people. The intelligence disasters surrounding 9/11 and the war in Iraq did two things simultaneously: they undermined public confidence in the CIA, and they underscored how much the nation depends on quality intelligence.

Despite the new bureaucratic structures, the political pressure—the tension in that gap between the White House and the intelligence community—remains. The gulf of trust between the power centers of U.S. national security and the public at large is widening. 

 Excerpted from the book 'The CIA and the Culture of Failure: U.S. Intelligence From the End of the Cold War to the Invasion of Iraq,' by John Diamond. Published by Stanford Security Series, an Imprint of Stanford University Press, Stanford, Calif.