Image credit: Flickr

By Justin Elliott and Theodoric Meyer, ProPublica

Two weeks after Edward Snowden's first revelations about sweeping government surveillance, President Obama shot back. "We know of at least 50 threats that have been averted because of this information not just in the United States, but, in some cases, threats here in Germany," Obama said during a visit to Berlin in June. "So lives have been saved."

In the months since, intelligence officials, media outlets, and members of Congress from both parties all repeated versions of the claim that NSA surveillance has stopped more than 50 terrorist attacks. The figure has become a key talking point in the debate around the spying programs.

"Fifty-four times this and the other program stopped and thwarted terrorist attacks both here and in Europe — saving real lives," Rep. Mike Rogers, a Michigan Republican who chairs the House Intelligence Committee, said on the House floor in July, referring to programs authorized by a pair of post-9/11 laws. "This isn't a game. This is real."

But there's no evidence that the oft-cited figure is accurate.

The NSA itself has been inconsistent on how many plots it has helped prevent and what role the surveillance programs played. The agency has often made hedged statements that avoid any sweeping assertions about attacks thwarted.

A chart declassified by the agency in July, for example, says that intelligence from the programs on 54 occasions "has contributed to the [U.S. government's] understanding of terrorism activities and, in many cases, has enabled the disruption of potential terrorist events at home and abroad" — a much different claim than asserting that the programs have been responsible for thwarting 54 attacks.

NSA officials have mostly repeated versions of this wording.

When NSA chief Gen. Keith Alexander spoke at a Las Vegas security conference in July, for instance, he referred to "54 different terrorist-related activities," 42 of which were plots and 12 of which were cases in which individuals provided "material support" to terrorism.

But the NSA has not always been so careful.

During Alexander's speech in Las Vegas, a slide in an accompanying slideshow read simply "54 ATTACKS THWARTED."

And in a recent letter to NSA employees, Alexander and John Inglis, the NSA's deputy director, wrote that the agency has "contributed to keeping the U.S. and its allies safe from 54 terrorist plots." (The letter was obtained by reporter Kevin Gosztola from a source with ties to the intelligence community. The NSA did not respond when asked to authenticate it.)

Asked for clarification of the surveillance programs' record, the NSA declined to comment.

Earlier this month, Sen. Patrick Leahy, D-Vt., pressed Alexander on the issue at a Senate Judiciary Committee hearing.

"Would you agree that the 54 cases that keep getting cited by the administration were not all plots, and of the 54, only 13 had some nexus to the U.S.?" Leahy said at the hearing. "Would you agree with that, yes or no?"

"Yes," Alexander replied, without elaborating.

It's impossible to assess the role NSA surveillance played in the 54 cases because, while the agency has provided a full list to Congress, it remains classified.

Officials have openly discussed only a few of the cases (see below), and the agency has identified only one — involving a San Diego man convicted of sending $8,500 to Somalia to support the militant group Al Shabab — in which NSA surveillance played a dominant role.

The surveillance programs at issue fall into two categories: The collection of metadata on all American phone calls under the Patriot Act, and the snooping of electronic communications targeted at foreigners under a 2007 surveillance law. Alexander has said that surveillance authorized by the latter law provided "the initial tip" in roughly half of the 54 cases. The NSA has not released examples of such cases.

After reading the full classified list, Leahy concluded the NSA's surveillance has some value but still questioned the agency's figures.

"We've heard over and over again the assertion that 54 terrorist plots were thwarted" by the two programs, Leahy told Alexander at the Judiciary Committee hearing this month. "That's plainly wrong, but we still get it in letters to members of Congress, we get it in statements. These weren't all plots and they weren't all thwarted. The American people are getting left with the inaccurate impression of the effectiveness of NSA programs."

The origins of the "54" figure go back to a House Intelligence Committee hearing on June 18, less than two weeks after the Guardian's publication of the first story based on documents leaked by Snowden.

At that hearing, Alexander said, "The information gathered from these programs provided the U.S. government with critical leads to help prevent over 50 potential terrorist events in more than 20 countries around the world." He didn't specify what "events" meant. Pressed by Rep. Jim Himes, D-Conn., Alexander said the NSA would send a more detailed breakdown to the committee.

Speaking in Baltimore the next week, Alexander gave an exact figure: 54 cases "in which these programs contributed to our understanding, and in many cases, helped enable the disruption of terrorist plots in the U.S. and in over 20 countries throughout the world."

But members of Congress have repeatedly ignored the distinctions and hedges.

The websites of the Republicans and Democrats on the House Intelligence Committee include pages titled, "54 Attacks in 20 Countries Thwarted By NSA Collection."

And individual congressmen have frequently cited the figure in debates around NSA surveillance.

• Rep. Lynn Westmoreland, R-Ga., who is also on the House Intelligence Committee, released a statement in July referring to "54 terrorist plots that have been foiled by the NSA programs." Asked about the figure, Westmoreland spokeswoman Leslie Shedd told ProPublica that "he was citing declassified information directly from the National Security Agency."
• Rep. Brad Wenstrup, R-Ohio, issued a statement in July saying "the programs in question have thwarted 54 specific plots, many targeting Americans on American soil."
• Rep. Joe Heck, R-Nev., issued his own statement the next day: "The Amash amendment would have eliminated Section 215 of the Patriot Act which we know has thwarted 54 terrorist plots against the US (and counting)." (The amendment, which aimed to bar collection of Americans' phone records, was narrowly defeated in the House.)
• Mike Rogers, the Intelligence Committee chairman who credited the surveillance programs with thwarting 54 attacks on the House floor, repeated the claim to Bob Schieffer on CBS' "Face the Nation" in July."You just heard what he said, senator," Schieffer said, turning to Sen. Mark Udall, D-Colo., an NSA critic. "Fifty-six terror plots here and abroad have been thwarted by the NSA program. So what's wrong with it, then, if it's managed to stop 56 terrorist attacks? That sounds like a pretty good record." Asked about Rogers' remarks, House Intelligence Committee spokeswoman Susan Phalen said in a statement: "In 54 specific cases provided by the NSA, the programs stopped actual plots or put terrorists in jail before they could effectuate further terrorist plotting.  These programs save lives by disrupting attacks. Sometimes the information is found early in the planning, and sometimes very late in the planning. But in all those cases these people intended to kill innocent men and women through the use of terror."
• Rep. James Lankford, R-Okla., went even further in a town hall meeting in August. Responding to a question about the NSA vacuuming up Americans' phone records, he said the program had "been used 54 times to be able to interrupt 54 different terrorist plots here in the United States that had originated from overseas in the past eight years. That's documented."
• The same day, Rep. Jim Langevin, D-R.I., who sits on the Intelligence Committee, defended the NSA at a town hall meeting with constituents in Cranston, R.I. "I know that these programs have been directly effective in thwarting and derailing 54 terrorist attacks," he said. Asked about Langevin's comments, spokeswoman Meg Fraser said in an email, "The committee was given information from NSA on August 1 that clearly indicated they considered the programs in question to have been used to help disrupt 54 terrorist events. That is the information the Congressman relied on when characterizing the programs at his town hall."

Wenstrup, Heck and Lankford did not respond to requests for comment.

The claims have also appeared in the media. ABC News, CNN and the New York Times have all repeated versions of the claim that more than 50 plots have been thwarted by the programs.

The NSA has publicly identified four of the 54 cases. They are:

• The case of Basaaly Moalin, the San Diego man convicted of sending $8,500 to Somalia to support Al Shabab, the terrorist group that has taken responsibility for the attack on a Kenyan mall last month. The NSA has said its collection of American phone records allowed it to determine that a U.S. phone was in contact with a Shabab figure, which in turn led them to Moalin. NSA critic Sen. Ron Wyden, D-Ore., has argued that the NSA could have gotten a court order to get the phone records in question and that the case does not justify the bulk collection of Americans' phone records.
• The case of Najibullah Zazi, who in 2009 plotted to bomb the New York subway system. The NSA has said that an email it intercepted to an account of a known Al Qaeda figure in Pakistan allowed authorities to identify and ultimately capture Zazi. But an Associated Press examination of the case concluded that, again, the NSA's account of the case did not show the need for the new warrantless powers at issue in the current debate. "Even before the surveillance laws of 2007 and 2008, the FBI had the authority to — and did, regularly — monitor email accounts linked to terrorists," the AP reported.
• A case involving David Coleman Headley, the Chicago man who helped plan the 2008 Mumbai terrorist attack. Intelligence officials have said that NSA surveillance helped thwart a subsequent plot involving Headley to attack a Danish newspaper. A ProPublica examination of that episode concluded that it was a tip from British intelligence, rather than NSA surveillance, that led authorities to Headley.
• A case involving a purported plot to attack the New York Stock Exchange. This convoluted episode involves three Americans, including Khalid Ouazzani of Kansas City, Mo., who pleaded guilty in 2010 to bank fraud, money laundering, and conspiracy to provide material support to Al Qaeda. An FBI official said in June that NSA surveillance helped in the case "to detect a nascent plotting to bomb the New York Stock Exchange." But no one has been charged with crimes related to that or any other planned attack. (Ouazzani was sentenced to 14 years last month.) The Kansas City Star reported that one of the men in the case had "pulled together a short report with the kind of public information easily available from Google Earth, tourist maps and brochures" and that his contact in Yemen "tore up the report, 'threw it in the street' and never showed it to anyone." Court records also suggest that the men in Yemen that Ouazzani sent over $20,000 to may have been scamming him and spent some of the money on personal expenses.

For more from ProPublica on the NSA, read about the agency's campaign to crack Internet security, a look at the surveillance reforms Obama supported before he was president, and a fact-check on claims about the NSA and Sept. 11.

Photo of Sen. Patrick Leahy by Win McNamee/Getty Images.