A lot of liberal bloggers are understandably skeptical of this Michael O'Hanlon piece on establishing political benchmarks in Iraq.
The most intriguing area of late is the sphere of politics. To track progress, we have established “Brookings benchmarks” — a set of goals on the political front similar to the broader benchmarks set for Baghdad by Congress last year. Our 11 benchmarks include establishing provincial election laws, reaching an oil-revenue sharing accord, enacting pension and amnesty laws, passing annual federal budgets, hiring Sunni volunteers into the security forces, holding a fair referendum on the disputed northern oil city of Kirkuk, and purging extremists from government ministries and security forces.
At the moment, we give the Iraqis a score of 5 out of 11 (our system allows a score of 0, 0.5, or 1 for each category, and is dynamic, meaning we can subtract points for backsliding). It is far too soon to predict that Iraq is headed for stability or sectarian reconciliation. But it is also clear that those who assert that its politics are totally broken have not kept up with the news.
Spencer is probably the most pungent, Ezra's the soberest, and Matt wins the prize for best metaphor:
I think Brookings Benchmarks are kind of like Disney Dollars, i.e. funny money. We get no sense of where this five out of eleven comes from or what it's really supposed to signify. The general thrust of the exercise seems to be to cast "failure" as such an extreme scenario that it can never actually happen. O'Hanlon will always be wisely positioned between the over-optimists and the over-pessimists, always urging us to hang on for a couple more Friedman Units, and so the war will continue, forever and ever just as John McCain wants.
At least, I assume that it is a good metaphor. I have never actually been to Disneyworld, because my parents do not love me.
This is the core complaint: though the index numbers have a solid, factual ring about them, in fact they're quite subjective. I don't think that this actually stands very well as a general critique. We have a lot of these sort of indices--Freedom House's world freedom index, Transparency International's corruption indices, Heritage's index of economic freedom--and while they're obviously inherently somewhat subjective, they're an extremely useful tool for gauging relative performance. There may be some quibble about whether a country is a 2.4 or a 2.6 on the world freedom index, but there's a clear and substantial difference between a 2.4 and a 4.8.
But of course, those indexes depend a lot on the credibility of the source, and there's not question that Michael O'Hanlon is on the highly optimistic side of Iraq analysts. That's going to make it hard to get consensus acceptance of his figures, especially when he hasn't published what the benchmarks are, or what constitutes the difference between a 0, a 0.5, and a 1.
I follow the Iraq index pretty closely, and from what I can tell, things have unambiguously gotten better in Iraq; civilian fatalities at their lowest level ever, the police and army better staffed, and key economic indicators such as oil and electricity production slowly but fairly steadily heading upwards. Electricity is now meeting or exceeding pre-war production, and with the attacks on the infrastructure basically halted, what problems there are now center around updating Iraq's aged infrastructure, rather than rebuilding destroyed equipment. Likewise, oil is essentially at prewar levels, and without the attacks on the pipelines and installations, there is reason to believe that it may actually exceed prewar levels in the reasonably near future. There is little data on other infrastructure, education, or healthcare, which are critical numbers to have. But the numbers we do have are all pretty much moving in the right direction. So off the cuff, O'Hanlon's assertions don't seem totally crazy to me.
On the other hand, the real question is the future: can Iraq get a government that can keep things going in the right direction? That's what O'Hanlon's benchmark is supposed to measure, and a 5 isn't a particularly hopeful number. Worse, looking at the political benchmarks in the Iraq index does suggest that it may be a trifle overoptimistic.
I follow the Iraq index pretty closely, and from what I can tell, things have unambiguously gotten better in Iraq...
So Megan, what is the unemployment rate in Iraq? Things can't stabilize until Iraq has a real economy. In the Saddam era, the only three sectors that were stable were the government (technocrats), the military, and oil.
Here's a journalist who knows what's going on in Iraq because he is there. His name is Nir Rosen.
Freedom House is not useful because it measures the freedom of one house, but because it provides a comparative frame of reference for examining many countries. You don't use an 11 point scale to summarize a single, sui generis country on a series of specific benchmarks. Instead, you describe the benchmarks, and explain how the country is doing in each. With an N of 1 and a small list of benchmarks, there's no need for summary.
Let's take the oil sharing accord as an example. If there were 100 countries trying for revenue sharing laws, it might make sense to code Iraq as a 0, .5, or 1 in its attempts. Since Iraq is sui generis, we should instead write a paragraph or a paper about Iraq's process of oil sharing, including determining if it is "backsliding". After all, Iraq as a 0, .5, or 1 on oil sharing provides an absurdly little amount of information.
Next, creating an 11 point scale treats each of the items in the scale as the same. So all of these are the same:
1) establishing provincial election laws;
2) reaching an oil-revenue sharing accord,
3) enacting pension and 4) amnesty laws,
5) passing annual federal budgets,
6) hiring Sunni volunteers into the security forces,
7) holding a fair referendum on the disputed northern oil city of Kirkuk,
8) purging extremists from government ministries and 9) security forces. (I don't know why 11 aren't listed, and I've been generous in assigning 9 separate ones).
Now, a 5 on an 11 point scale doesn't tell you which ones are doing well, or in what way. Are they all 0.5? Are half 0 and the other half 1? An 11 point scale in fact indicates that it doesn't matter! This is why the scale is only useful when viewed across multiple instances of its use, which again is not possible in Iraq.
Actually, I believe that oil production is now at around 2.5 million barrels per day, about 25% better than pre-war levels of 1.9 million barrels per day.
And that's up from 2.3M a few weeks ago.
"... civilian fatalities at their lowest level ever, ..."
I find this hard to believe. What are the numbers?
I don't know why 11 aren't listed, and I've been generous in assigning 9 separate ones.
The good people at Brookings are HUGE Nigel Tufnel fans.
I can't believe nobody's made a Spinal Tap reference yet.
Dammit - ninja'd by Matt B.
the war will continue, forever and ever just as John McCain wants.
The war is wrapping up, Matt. We're in the mop-up phase now.
Since you're a journalist who writes about Iraq for a serious magazine, how about you actually go there?
"... civilian fatalities at their lowest level ever, ..."
I find this hard to believe. What are the numbers?
Posted by James B. Shearer
"I have never actually been to Disneyworld, because my parents do not love me."
You get a lot of hate from my fellow liberals, but comments like this are why I keep on coming back.
Dan -
You're arguing that a scale like this can only be useful for analysing a cross-section (i.e. many different countries), but I think that in this case, the idea is to form a time series. Presumably the main goal is to use this over time to observe the changes.
But I agree that we can't judge just this one first measurement very well without knowing more about how the index was constructed, and people can always disagree with the weights, whether they're equally-weighted or weighted based on someone's estimate of priority.
But what really seems to have some people upset is that the index isn't starting at zero, and thus that it implies that there is already some progress. Regardless of reality, some people just have too much vested in a failure in Iraq, just as Bush, McCain and others have a vested interest in seeing it succeed.
We have a lot of these sort of indices--Freedom House's world freedom index, Transparency International's corruption indices, Heritage's index of economic freedom--and while they're obviously inherently somewhat subjective, they're an extremely useful tool for gauging relative performance.
If I understand right, the Transparency International index doesn't belong on that list. It's based on surveys of hundreds of businesspeople, right? The methodology is transparent and the ratings are not determined by one individual uber-analyst; they reflect the collective judgment of the business community. The Freedom House and Heritage indexes may be vulnerable to many of the same critiques as O'Hanlon's thing; I know a big problem with the Freedom House index is that it defines "freedom" in a way that reflects its ideological prejudices, and then finds what it set out to seek. That probably reflect the problem with having indexes for terms as vague as "freedom" or, in Iraq, "progress".
But in any case, none of those indexes are as problematic as O'Hanlon's. The problem is that his index is set up by him. He's the whole show. There is simply no chance that an index established by Michael O'Hanlon, given his prior ideological stance, is going to give Iraq a 1 or 2 out of 11. If he wants to do anything serious along these lines, he needs to come together with others and hash out a joint ratings system that reflects something other than his own prejudices. The idea of an objective ratings system where the methodology and inputs are all decided on by one guy is a contradiction in terms: "decided on by one guy" is basically the meaning of "subjective".
Re "I can't believe nobody's made a Spinal Tap reference yet.":
From Matt's comments, by P O'Neill:
True Story: [the Brookings Index] originally only went up to 10, but then O'Hanlon was thinking "once you've got to 10, where else can you go?". This one goes up to 11.
The numbers are not all moving in the right direction, just one of many factual errors in this post. To take just the Transparency International Corruption Index, Iraq is now rated the third most corrupt government of the 180 surveyed.
Nor is electricity "now meeting or exceeding pre-war production". There were fewer megawatts produced in Feb. 2008 than in Feb. 2004, and both of those are below estimated pre-war levels.
You may be following the Iraq Index, without actually comprehending the info there. In any case, going to Brookings looking for the truth about Iraq is a bit like turning to Pravda for the latest estimates on the Five-Year Plan. The Iraq Index measures "improvement". That's what it's set up to do; that's how the issues are framed, that's how evidence is evaluated for inclusion or exclusion. Even the Pentagon Quarterly reports are more candid than Brookings.
What a bizarre post.
Even an honest, insightful index would have limited utility without some absolute scale attached to it.
What we want is an index where 11 means "The Iraqi people accept the legitimacy and authority of the national government and we can go home." More electricity and less violence are excellent things, and are probably prerequisites to a real solution, but they don't address the central issues; and a few apologists for the war try to use them as clubs to suppress doubts about more fundamental problems.
I follow Instapundit on Iraq, Michael Totten, whom he links, and Iraq the Model, who had an interesting blogpost either his last or a few earlier on how he saw the politics evoving toward a stable government and so I feel more optimistic. I think the problems are more likely to be, as your post suggests, with our perceptions and consequent actions.
To certain Democrats, there will not be “enough” progress in Iraq until the country is ruled by the National Organization for Women, the Health Care industry is free to all people and exceeds the quality of care in the US, all of the guns, bombs, knives and swords are melted down into a giant abstract sculpture of President Hillary, a small 20 acre patch of sand is set aside for the remaining radioactive Israelites, and the oil wells are run in reverse to pump that nasty carbon-based goop back underground where it belongs.
Then maybe, just possibly, they might admit success.
BTW I found some fantastic articles…A MUST read for EVERYONE “The Hussein Dynamic” at http://savagepolitics.com/?p=171 and “Follow the Money” at http://savagepolitics.com/?p=165
The writing is BRILLIANT and goes beyond what the MSM feeds us. It was about time!!!!! Their sections for “Humor” and “Political Analysis” are FANTASTIC!!!!
http://savagepolitics.com
My parents love me and I've been to Disney. But I've been wanting to go again (actually, I'd prefer Epcot and MGM). Why do you need your parents love? How about putting an Asymetrical meet-up trip together?