« Oil, oil, toil and trouble . . . | Main | Busting the baby bust »

What bias?

14 Sep 2007 10:50 am

William Saletan has a piece on the conservative v. liberal study in Slate, titled "Rigging a study to make conservatives look stupid". I'm not sure "rigging" is the right word. But Saletan does make an interesting point, which is that even if the study really did identify receptiveness to change, that would not be the same as identifying who was more likely to be right:

Frank Sulloway, a Berkeley professor who co-authored a damning psychological analysis of conservatism four years ago, illustrates the problem. Appearing in the Times as a researcher "not connected to the study"—despite having co-written his similar 2003 analysis with one of its authors—Sulloway endorsed the study and pointed out, "There is ample data from the history of science showing that social and political liberals indeed do tend to support major revolutions in science." That's true: When new ideas turn out to be right, liberals are vindicated. But when new ideas turn out to be wrong, they cease to be "revolutions in science," so it's hard to keep score of liberalism's net results. And that's in science, where errors, being relatively factual, are easiest to prove and correct. In culture and politics, errors can be unrecoverable.

Comments (22)

Megan, honey, if it's not "rigged" would you accept "skewed in such a blatant way as to make one question the skills of the researchers, if it were in fact accidental"? I mean, honestly, small n, heavily skewed sample, and not very rational measure....

Set aside the study's obvious methodological weaknesses for the moment. The cognitive difference between liberals and conservatives that it purported to identify maps pretty well to the old Isaiah Berlin distinction between foxes and hedgehogs. Saletan is correct to point out that receptiveness to change, which implies susceptibility to fads, is not necessarily a winning cognitive strategy. However, Philip Wheelock, in his book Expert Political Judgment, evaluated the predictions of many experts on both sides of the epistemological divide, and the foxes outperformed the hedgehogs by a wide margin. There is nothing here that hasn't been done before, and done better.

Incidentally, I am quite sure the cause of the respective Bush and Clinton Derangement Syndromes is not political (Clinton's governance was probably more "conservative" on the whole), but epistemological. Clinton is as foxy as foxes get, while Bush is the most hedgehoggish of hedgehogs. It's their cognitive styles that drive their opponents crazy.

And let's not forget the major revolution in economics that many social and political "liberals" supported.

"There is ample data from the history of science showing that social and political liberals indeed do tend to support major revolutions in science"

Like this, for example: http://www.nyu.edu/gsas/dept/philo/faculty/boghossian/papers/bog_tls.html

If you have to constantly remind people how smart you are, there's probably a reason why.

It's odd that something that one of the scientists in the study has already stated -- that the study doesn't address smarts -- is taken as a bit of trenchant criticism when it comes from someone else.

Let us not forget one of the great moments in Progressive science:

http://www.uvm.edu/~eugenics/vtsurvey.html

Isn't the entire study rendered meaningless by the fact that participants knew the point of it going in? In other words, how do we know that some of the 7 students (a ridiculously small sample, in any case) who self-identified as "conservatives" didn't do so just to make conservatives look bad? Would a participant's brain show much "conflict" if he wasn't paying much attention?

In any case, both "liberal" and "conservative" are completely arbitrary terms, with different meanings in different places and times. A "liberal" used to be someone who supported free-market economics. A "conservative" in China, if we are to take the term literally, would have to be a hard-line Maoist who deplores the free-market reforms of the past 20 years. In present-day America, the political designations "liberal" and "conservative" have little to do with respective desires for liberty or stasis. Does the desire to preserve Affirmative Action make one a conservative? To make the assumption that "conservatives" must want to "conserve" the current state of things, and then to explain this desire as the result of rigid thinking and an intolerance for ambiguity, is to imbue the political designation "conservative" with a simplistic literalness that isn't merited.

Or, as Jonah Goldberg put it:

In Russia for a very long time, “conservatives” wanted to conserve the Soviet system which made them the anti-matter universe versions of American conservatives. Surely the substance of what people want to conserve matters, if we are to take these studies remotely seriously.

...love of change is not “liberal” any more than being for the status quo is “conservative.” For example, liberals from Jonathan Chait to Nancy Pelosi have a white-knuckled grip on the welfare state created by FDR and LBJ — in other words they’re conservative about the liberal status quo. Meanwhile, so-called paleoconservatives are quite radical in their agendas. Think about it: There are cranky, risk-averse people at any number of liberal institutions and risk-loving rightwingers at conservative institutions. Have you never met an inflexible leftwing bureaucrat? Have you never hung out with a partying, devil-may-care Republican?

...there are Marxists who haven’t embraced a new idea in 40 years, and there are conservatives who want to tear down the welfare state, imposing radical and sweeping changes on society and allowing the creative destruction of the market place to run amok (at least that’s what liberals keep telling me). Which are conservatives and which are liberals? The idea that liberals-qua-liberals are comfortable with ambiguity begs the question: Ambiguity about what? Walk around Madison, Berkeley, or Burlington some time, explaining very rationally that the evidence on global warming is ambiguous. As you’re wiping the spittle spewed from their outbursts off your cheek, take a moment to savor the liberals’ comfort with ambiguity.

- http://tinyurl.com/2u9b82

Sorry, not Wheelock, I meant Philip Tetlock. At any rate, I commend his book to anyone interested in the efficacy of different cognitive styles.

Frank Sulloway's statement "there is ample data from the history of science showing that social and political liberals indeed do tend to support major revolutions in science" makes me want to see the ample data. Presumably he is thinking of Galileo and Darwin, but what of Lysenko or Heisenberg? On the other hand, Einstein was a social liberal, but his views on quantum mechanics were dead wrong.

I interpreted this study somewhat differently since the adaptation involved changing driving routes. So when I'm stuck in awful traffic I look around and figure all those other gd drivers out there blocking my path are gd conservatives, because the gd limousine liberals are taking some alternate route.

Libs were way into eugenics back before Hitler soured them on it. That was a scientific revolution until it was exposed to be bunk.

Liberals liked communism. Hell you can still find a few loons who still do.

So yeah, receptiveness to change can be a bad thing, a very bad thing. Depends on what the change is. And I see a whole hell of a lot of resistance to change when its liberal sacred cows that are subject to obviously needed change, like public education and entitlement programs.

The whole thing is crap anyways, self-congratulatory back patting.

This test was to press or not press a button depending on which letter flashed up on the screen, in no more then 0.5 seconds. There's no thinking involved. With a sample of only nine self-ientified conservatives I don't see how statistically valid results would be possible, but if liberals actually do perform it better, it only proves that they're "better" at unthinking knee-jerk reactions than conservatives.

And here I was thinking that there are just as many knee-jerk conservatives...

Liberals are very much people that encourage and engage in revolutions. However, the 'liberal' of 235 years ago is now called a conservative.

Don't faint there, hostess, because being a Progressive doesn't make a liberal less Liberal. If one wants to really read about a 'classical liberal,' just go to http://www.ilanamercer.com/ and you'll read some incredibly good, remarkable thoughs.

Rob Leder quoted Jonah Goldberg (never a smart move):

Surely the substance of what people want to conserve matters, if we are to take these studies remotely seriously.

Actually, Rob, that's what's so interesting. It rarely does. Soviet conservatives were a lot more like American conservatives in most ways than they were like American liberals. This has become abundantly clear in post-Soviet Russian politics, where ex-Communist hardliners are now on the nationalist right wing of the spectrum, while Gorbachev and early Yeltsin-era "free market" reformers are on the left.

Presumably he is thinking of Galileo and Darwin, but what of Lysenko or Heisenberg? - pct

Lysenkoism wasn't a "scientific revolution"; it was a crackpot theory that never gathered any scientific support and became dogma in the USSR because of cronyism. To say it was a "scientific revolution" would be like saying intelligent design theory is a "scientific revolution". It's not. It is a rhetorical line, poorly supported by evidence (if at all), which flatters many people's preconceptions. It would be logical for Sulloway to speak of "scientific revolutions" as ones where new ideas were proposed that fit the data better, even though they upset widely held preconceptions (rather than flattering preconceptions, as Lysenko's ideas did for Marxists). And while I'd like to see the data too, I would really be surprised if it turned out that liberals had not been more widely inclined than conservatives to support every scientific revolution of the past 400 years -- especially Heisenberg's uncertainty principle, and its popularization as the principle that there's no such thing as objectivity. Who is it, after all, that has been inveighing against 'relativism' for the past 40-odd years?

Ok. The Liberal philosophies put into effect after the Leftists who were in power in the CCCP were tossed out with the garbage in 1989, included forms of capitalism, more of an open economy approach, but still weighted down with the almost 80 years of thinking (and leftist education in their public schools) that created fatal flaws in what they tried to create.

The Russian economy was not working whatsoever, and those who lived through it and were able to escape the nation at that time, have absolutely awful stories to tell. A number of those who were in the mid-level power structure in the New Russia, were those who were in the same levels when the communists ruled, and their socialist policies kept the USSR from maintaining a strong economic base that the average Russian could use to favorably compare with the capitalist (considered Conservative, neh?) West.

Having to sell your body to make ends meet, the rise of the mega-rich entrepreneur class, the Russian Mafia, all were manifestations of being cast loose when a society lost faith in socialism. Gorbachev was one of those who is castigated by the Russian people, as one who was a traitor to the country. Paradoxically, he has great standing in the West by those the West's Left, and so, because of his connection with Mother Russia, has some grudging respect there now. He isn't looked upon with the same iconic celebrity he is by the media/hollywood group in the West.

Now, with Putin, a former high-level KGB Boss, they have a leader who is respected by the average 'Joe' in Russia, but because he has guts, not because of his leftist approach. Indeed, there has been muted criticism of him by the People because of his totalitarian/dictatorial approach, including his clamping down on the elected system there.

So, if he actually does reinstall the Socialistic government that was in place for decades before The Fall, is he now a reactionary conservative wanting this return to Leftist ways of one-payer health insurance, the government in every part of your life? Or is he a revolutionary Leftist who is trying to change the current system in place in Russia?

In either approach, the 'Joes' lose out, but, because the West built his gas and oil infrastructure the last 18 years until it became the main source of revenue in the nation, it will be a little harder for the West's Left to honestly praise the New Stalin and the return to his Socialistic ways.

It would be logical for Sulloway to speak of "scientific revolutions" as ones where new ideas were proposed that fit the data better, even though they upset widely held preconceptions (rather than flattering preconceptions, as Lysenko's ideas did for Marxists).

Yes, it would be interesting to look at all the winners of great scientific debates in the modern era and map them to modern American political philosophies. Newton's views on Social Security, Lavoisier's opinions on gay marriage, Maxwell's thoughts about our Middle East policy, etc.

And while I'd like to see the data too, I would really be surprised if it turned out that liberals had not been more widely inclined than conservatives to support every scientific revolution of the past 400 years --

That is, unless you're defining "liberal" scientists as those who supported scientific revolutions, and then saying that liberals were more included to support scientific revolutions. But you wouldn't do such a thing.


especially Heisenberg's uncertainty principle, and its popularization as the principle that there's no such thing as objectivity. Who is it, after all, that has been inveighing against 'relativism' for the past 40-odd years?

Funny that, I was just reading an article in Nature about how the deterministic Many Worlds interpretation of quantum mechanics has slowly been supplanting Heisenberg's Copenhagen interpretation. On the other end of things, relativity has always been based on invariance, so who knows what the physicists really stand for.

I would really be surprised if it turned out that liberals had not been more widely inclined than conservatives to support every scientific revolution of the past 400 years

- brooksfoe

"Liberal" - like "conservative" - being a completely malleable term, you could make this true by definition. In which case, an awful lot of academics who no doubt think of themselves as liberals, and who would surely have been identified as such by the questionaire used in the NYU/UCLA study, revealed themselves to be hard-line "conservatives" by their reflexive, highly emotional, politically-motivated, unscientific rejection of Edward O. Wilson and his theory of sociobiology:

In one of the most remarkable displays of wounded Marxist chauvanism in American academic history (and there have been many), two of Wilson's well-known colleagues at Harvard's Museum of Comparative Zoology, paleontologist Stephen Jay Gould and geneticist Richard Lewontin, joined a group of radical activists called Science for the People to form what can only be called an "antiseptic squad." The goal, judging by their public statements, was to demonize Wilson as a reactionary eugenecist, a Nazi in embryo, and exterminate sociobiology as an approach to the study of human behavior. After three months of organizing, the cadre opened it's campaign with a letter, signed by fifteen faculty members and students in the Boston area, to the leading American organ of intellectual etiquette and deviation sniffing, The New York Review of Books. Theories like Wilson's, they charged, "tend to provide a genetic justification of the status quo and of existing privileges for certain groups according to class, race, or sex." In the past, vile Wilson-like intellectual poisons had "provided an important basis for the enactment of sterilization laws...and also for the eugenics policies which led to the establishment of gas chambers in Nazi Germany." The campaign went on for years. Protesters picketed Wilson's sociobiology class at Harvard (and the university and the faculty kept mum and did nothing). Members of INCAR, the International Committee Against Racism, a group known for its violent confrontations, stormed the annual meeting of the American Association for the Advancement of Science in Washington and commmandeered the podium just before Wilson was supposed to speak. One goony seized the microphone and delivered a diatribe against Wilson while the others jeered and held up signs with swastikas - whereupon a woman positioned behind Wilson poured a carafe of ice water, cubes and all, over his head, and the entire antiseptic squad joined in the chorus: "You're all wet! You're all wet! You're all wet!"

- Tom Wolfe, "Digibabble, Fairy Dust, and the Human Anthill", Forbes ASAP 10/4/99 (reprinted in Hooking Up)

Of course, "revolution" is just as malleable a term as political designations like "liberal" and "conservative", and you may quibble over whether Wilson's research qualifies; that isn't the point. Nor is the point whether he was right or not. Is street theater, a public smear campaign, and sinister references to Hitler the stuff of reasoned scientific debate? The academics and assorted activist riff-raff in this tale - all who would CERTAINLY be identified as liberals in the NYU/UCLA study - rejected Wilson's work dogmatically, because their ideology demands a "blank slate" view of human nature.

That's odd. According to the broad interpretation of the NYU/UCLA study promulgated by commenters like Frank Sulloway - and to a lesser extent, the researchers themselves - those who hold conservative views are dogmatic, while liberals are open-minded and prepared to change their views based on new data and fresh reasoning. If a large group of prominent liberal intellectuals was prepared to wage a sort of jihad against a biologist and his research - something which by nature is far more empirical than just about any political issue - why should we take it on faith that the views these people hold on taxation, gun control, welfare, or any other political issues are the result of careful observation and unbiased reasoning?

Political views are based as much (or more) on values and priorities as they are on anything that's quantifiable or verifiable. The NYU/UCLA study, aside from being irredeemably flawed in it's methodology, pretends that this is not so. What the Frank Sulloways of the world are really implying is: "We were right about the M's and W's, therefore we're right about everything else". Sorry, it doesn't work that way.

Surely the substance of what people want to conserve matters, if we are to take these studies remotely seriously. - j goldberg

Actually, Rob, that's what's so interesting. It rarely does. Soviet conservatives were a lot more like American conservatives in most ways than they were like American liberals. This has become abundantly clear in post-Soviet Russian politics, where ex-Communist hardliners are now on the nationalist right wing of the spectrum, while Gorbachev and early Yeltsin-era "free market" reformers are on the left.

- brooksfoe

As I've said before, "conservative", "liberal", "right wing", "left wing"...these are all malleable terms. The left in this country has done a superb job of stacking the definitions in it's favor - any totalitarian thug who has complete disregard for democracy, the rule of law, or human rights automatically gets branded a "conservative" or "right-winger". Hitler and the Nazis (National Socialist Party) have been succesfully (mis)remembered as extreme right-wingers, which conveniently sets up the implication that people like Reagan and Thatcher were watered-down Hitlers.

I deplore this one-dimensional political spectrum that we've all become accustomed to thinking in terms of.

Maybe we should give the hardline former Soviets the same questionaire that was given to the NYU & UCLA undergrads in the recent study? Who knows, maybe they share more opinions with these American liberals than you think? In which case, so what? Such a result would speak nothing as to the "rightness" or "wrongness" of holding these opinions.

In my opinion, there are just as many indoctrinated, reflexive liberals in this country as there are indoctrinated, reflexive conservatives (and libertarians).

Let's keep in mind a few facts that should be obvious, but apparently are not so to Rob Leder:
(i) The study, like any study like it, is about demographic tendencies. Anyone who interprets the study as saying "all liberals are [blah], and all conservatives are [blah-prime]," simply don't know what they are talking about.
(ii) Because of (i), even if one interpreted the study as indicating that liberals tend to be, in aggregate, more open-minded than conservatives, one should still expect to find plenty of conservatives who are more open-minded than plenty of liberals.
(iii) Therefore, the existence at some times of some groups on the left who act in a dogmatic way has ZERO bearing on the validity of the study. Did some people on the left treat Wilson and his research shoddily? Yes. Does this have any bearing on the discussion here? No.
(iv) They discuss in the paper (albeit briefly, like everything else in the paper, since it is meant to be, like, a short experimental report, and not a definitive treatise on the subject) the construct validity of their survey question on liberal/conservative. There is apparently already a preponderance of empirical evidence that the question they asked is a pretty darn good stand in for people's political behavior. The authors -- like any other sane & moderately well-educated folks-- would of course acknowledge that there are a great many nuances to these categories. I'm sure they would note further that this is very consistent with the kind of research they are doing; see (i) above. Thus, accusations that they are oversimplifying these categories, etc. are really completely besides the point.
(v) There is zero evidence that the scientists involved in the study, or any other scientist, are even vaguely behind the radical over-interpretation of their results so idiotically characterized as "We were right about the M's and W's, therefore we're right about everything else". To try to hang that on them is precisely to try to let one's politics obscure one's view of the science. Which is, y'know, a pretty closed-minded thing to do.

Btw, when Sulloway says he's got a lot of data on the historical relations between different kinds of liberalism and conservatism, etc., he's not just blowing smoke. (Though I can't vouch for the particular claim he made in the quote.) Anyone interested in these questions should check out his _Born to Rebel_, which is a pretty accessible read for a social science tract.

Let's keep in mind a few facts that should be obvious, but apparently are not so to Rob Leder: (i) The study, like any study like it, is about demographic tendencies. Anyone who interprets the study as saying "all liberals are [blah], and all conservatives are [blah-prime]," simply don't know what they are talking about.

- philosopher

Of course it's obvious that the inferences drawn pertained to demographic tendencies. Where did I indicate that I believed anyone had issued a blanket statement about ALL liberals or ALL conservatives?

(ii) Because of (i), even if one interpreted the study as indicating that liberals tend to be, in aggregate, more open-minded than conservatives, one should still expect to find plenty of conservatives who are more open-minded than plenty of liberals.

- philosopher

Even though twins are somewhat uncommon, one should still expect to find some of them. But if you were forced to guess whether or not a random person has a twin sibling, obviously the best bet is "no".

Sulloway and others are implying, in essence, that if a person is conservative, the best bet is that he clings to his political beliefs in a relatively closed-minded fashion; if a person is liberal, the best bet is that he is relatively open-minded about having his political beliefs challenged by new facts and arguments. There are no overt claims of superiority or inferiority - this isn't 19th-century anthropology - so it's completely up to the reader to conclude which viewpoints tend to be more "correct" or "well-reasoned": those that tend to be held by the group which is more closed-minded on average, or those that tend to arrived at by the group which is more open-minded on average.

Aside from the general shoddiness of the study that many (including several neuroscientists) have pointed to, it is this broad extrapolation of the test data to "open-mindedness" that I object to. Again, sorry that you misunderstood my comments as indicating that I believed someone had postulated an ironclad law concerning all conservatives and all liberals.

(iii) Therefore, the existence at some times of some groups on the left who act in a dogmatic way has ZERO bearing on the validity of the study. Did some people on the left treat Wilson and his research shoddily? Yes. Does this have any bearing on the discussion here? No.

- philosopher

brooksfoe stated "I would really be surprised if it turned out that liberals had not been more widely inclined than conservatives to support every scientific revolution of the past 400 years", so in response I provided an example where liberals (and not just fringe wackos, but prominent academics at the top of their fields) were widely DISinclined to support a perfectly valid line of scientific inquiry. When someone makes an absolute statement (something which, ironically, you accused me of doing), I think a counter-example has plenty of bearing.

Does the Wilson example invalidate the study? Of course not. It has no bearing on whether or not 36 liberal students did or didn't outperform 7 conservative students at a videogame.

(iv) They discuss in the paper (albeit briefly, like everything else in the paper, since it is meant to be, like, a short experimental report, and not a definitive treatise on the subject) the construct validity of their survey question on liberal/conservative. There is apparently already a preponderance of empirical evidence that the question they asked is a pretty darn good stand in for people's political behavior. The authors -- like any other sane & moderately well-educated folks-- would of course acknowledge that there are a great many nuances to these categories. I'm sure they would note further that this is very consistent with the kind of research they are doing; see (i) above. Thus, accusations that they are oversimplifying these categories, etc. are really completely besides the point.

-philosopher

I don't think it was a single question, because there were gradations of political conviction. Incidentally, or maybe not, nobody in the study was more than moderately conservative (3 on a scale of 5).

I accused the world of oversimplifying these categories, not the researchers. I never assailed the "accuracy" of their questionaire, because I've never seen it. I'm sure it's quite in line with most people's qualitative assesment of what it means to be "politically conservative" or "politically liberal" in 2007 America.

My point was that "conservative" and "liberal" can have vastly different meanings in different times and places (even in America, it seems a lot of these positions were cobbled together out of "big tent" political alliances). For example, in present-day America, enthusiasm for free markets is generally considered to be a "conservative" viewpoint. According to the broadest interpretation of this study, conservatives have a tendency to be dogmatic and unreflective about their political viewpoints. Someone who believes this interpretation might conclude that liberals, who are considerably less enthusiastic about free-market economics, are probably closer to having the "correct" or "reasonable" viewpoint, since after all most of them arrive at their views through an open-minded process of observation and reasoning. But in China, a "liberal" would be a someone who supports free markets, while a "conservative" would be a hard-line Maoist. Do we want to call the latter more open-minded than the former? If not, then which relatively open-minded group holds the "correct" view on economics, the American liberal or the Chinese liberal? If it sounds like a silly question, perhaps it's because drawing a broad connection between the simple unambiguous letter recognition test used in the study and the process of arriving at a political viewpoint is silly.

(v) There is zero evidence that the scientists involved in the study, or any other scientist, are even vaguely behind the radical over-interpretation of their results so idiotically characterized as "We were right about the M's and W's, therefore we're right about everything else". To try to hang that on them is precisely to try to let one's politics obscure one's view of the science. Which is, y'know, a pretty closed-minded thing to do.

- philosopher

I get the feeling that if I said something like "dogs like people" to you, you'd call me an idiot and accuse me of claiming that every dog in the world likes people, or even that every dog in the world likes every person in the world.

Obviously, I don't think Sulloway and others are LITERALLY intimating "we were right about the M's and W's, therefore we're right about everything else". How could I LITERALLY believe that, when the liberal group didn't score perfectly on the test.

The message I think they (the researchers themselves, and to a greater extent the outsiders who lauded their paper) would probably like people to walk away with, which of course nobody is going to come right out and articulate, is this: Which group do you think has the right policy prescriptions, the one in which most members tend to dogmatically cling to their opinions in the face of conflicting evidence, or the one in which most members are open-minded rational people willing and eager to reason their way to new conclusions in the face of conflicting evidence?

You are perfectly entitled to believe that there is absolutely no bias, no self-congratulation, no smug "well doesn't that explain a lot?" thoughts among these people, it's not a point that I care to argue.

Score a 'hit' for Herr Leder, and a Close Miss for Comrade philosopher. Hard to add any to what either has said (especially after the wiring diagrams I put together, trying to construct the logic), so I won't directly address their expositions.

Only one comment I would make, is that history matters. The historical context matters even more, when trying to provide instructive discourse on matters like this. You could equate it with which is similar to trying to make a free-standing sculpture of the Sears Tower, when you're using a clay mixture that is more in line with greenware molds than hand shaping.

To mix a couple of analogies, without the perfect consistency from a proper historical perspective, it's as difficult to tie down these definitions as it is to try and get the Sears Tower to stand up straight without that perfect clay mixture - no leaning to Left or Right.

Post a comment

By using this service you agree not to post material that is obscene, harassing, defamatory, or otherwise objectionable. Although The Atlantic does not monitor comments posted to this site (and has no obligation to), it reserves the right to delete, edit, or move any material that it deems to be in violation of this rule.


Copyright © 2008 by The Atlantic Monthly Group. All rights reserved.