When is scholarship not scholarship? When it's not carefully reviewed for every aspect, including the integrity of the underlying data used to draw conclusions.
For several years now, we've been hearing conservatives around the world insist austerity is the road back to prosperity -- because the smart guys in the think tanks say so. Now it turns out those smart guys aren't as smart as they thought.
Republicans based their austerity claims on a study known as Reinhart-Rogoff, which drew the conclusion that debt exceeding 90 percent of GDP leads to economic slowdowns. Other researchers sought to challenge the conclusions and were given access to Reinhart-Rogoff's raw data, contained in spreadsheets.
Their conclusions:
In a new paper, "Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff," Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst successfully replicate the results. After trying to replicate the Reinhart-Rogoff results and failing, they reached out to Reinhart and Rogoff and they were willing to share their data spreadhseet. This allowed Herndon et al. to see how how Reinhart and Rogoff's data was constructed.
They find that three main issues stand out. First, Reinhart and Rogoff selectively exclude years of high debt and average growth. Second, they use a debatable method to weight the countries. Third, there also appears to be a coding error that excludes high-debt and average-growth countries. All three bias in favor of their result, and without them you don't get their controversial result.
The coding error might be forgivable, but not when combined with the other two factors, and especially not when the combination of the three yields a false result.
Imagine a polling company excluding entire swaths of the population in order to deliver favorable polling results to the candidate paying them for the poll. Oh, wait.
The excluded data is critical:
Herndon-Ash-Pollin find that they exclude Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). This has consequences, as these countries have high-debt and solid growth. Canada had debt-to-GDP over 90 percent during this period and 3 percent growth. New Zealand had a debt/GDP over 90 percent from 1946-1951. If you use the average growth rate across all those years it is 2.58 percent. If you only use the last year, as Reinhart-Rogoff does, it has a growth rate of -7.6 percent. That's a big difference, especially considering how they weigh the countries.
Basically they left out the countries that would actually give a true result and destroy forever the claim that growth slows when debt exceeds 90 percent of GDP. How Republican.
Let the pileon begin! Jared Bernstein:
As I’ve written many times, riffing off of Bivens and Irons for one, if you mush everything together they way they do, you’re likely to get the causality backwards. You’ll convince yourself that higher debt leads to slower growth when it’s more often the opposite. Certainly in the US case, the most progress we’ve made against our debt ratios have been in periods of fast growth (and the biggest increases have been in periods of recession, slow growth, or war).
In short, the central argument in support of austerity — cited by MSNBC’s Joe Scarborough, the New York Times’ David Brooks, and multiple times by House Budget Committee Chairman Rep. Paul Ryan (R-WI) — is now defunct.
If true, this is embarrassing and worse for R-R. But the really guilty parties here are all the people who seized on a disputed research result, knowing nothing about the research, because it said what they wanted to hear.
And more Krugman on Tuesday, where he questions motives:
I was going to post something sort of kind of defending Reinhart-Rogoff in the wake of the new revelations — not their results, which I never believed, nor their failure to carefully test their results for robustness, but rather their motives. But their response to the new critique is really, really bad.
If facts mattered in economic policy debates, this should be the cause for a major reassessment of the deficit reduction policies being pursued in the United States and elsewhere. It should also cause reporters to be a bit slower to accept such sweeping claims at face value.
Rogoff and Reinhart insist these errors don't change the conclusion. Michael Hiltzik of the L.A. Times:
In their response, R & R don't directly address the UMass critique that their math was erroneous. Instead, they argue that since the critique also finds lower economic growth in high-debt situations, the new findings are no big deal. The point, however, is that the corrected figures don't show anything like a break point in economic growth at the 90% ratio, which Rogoff and Reinhart cited.
R & R also point readers to a 2012 paper that they say validated their original findings. That paper, however, is more nebulous than they suggest. In it they found lower growth in high-debt-ratio conditions, but it also acknowledged that the high debt was associated with other growth-reducing conditions, including wars, banking crises and depressions. If that sounds like the developed world in 2008-2013, no kidding.
That underscores what has been one of the most frequent critiques of the original paper: Even if high debt/GDP ratios are associated with low growth, what's the evidence that the former causes the latter? It's been widely pointed out that it's just as likely, even more likely, that low growth drives higher government debt, rather than the other way around.
Facts matter? See, that's why the journalists accept these sweeping claims. Thanks to Fox News and the unending grind of propaganda spinners, facts matter less than perception, which is why I'm not optimistic the debate will change any time soon.
