Correlation. Is. Not. Causation. Write that on your forehead 1,000 times. I give you Mr. Kling:
I received an email from a reader who was very excited to find that over the past 70 years the correlation between excess health care inflation (the price of health care relative to the overall CPI) and the proportion of health care spending paid for by third parties was 0.92 (out of a maximum of 1.00)I wrote back saying that correlation does not imply causation. He replied that he understood that, but still, with a correlation that high there must be something.
I'm sorry, but the inability to infer causation from correlation has nothing to do with the size of the correlation coefficient. It reflects the process generating the data. In a controlled experiment, you often can say something about causation. When you just observe some data, you cannot.
In addition, time series data (data that cover long time periods) are very subject to spurious correlation. Over time, data tend to follow trends. Any two trends are automatically correlated, whether there is a causal relationship or not.
When you look at data over time, it is important to ask yourself how many data points you really have. With a strong trend, you probably should just think of yourself as having two data points--the beginning and the end point. If there are a few sharp swings in the data, then you might have three or four effective data points. The fewer the number of effective data points, the harder it is to distinguish among alternative sources of causality.
That is why most macro-econometrics is junk science. That is one reason I would tend to suspect that Larry Bartels' work on Presidential party and income inequality is junk science.
Correlations are, at best, suggestive. They are not by themselves evidence--nay, not even if you cross your arms, scowl at your opponent, and say "Well, then give me another explanation for this astonishing correlation!" Until you've got something better than a simple correlation, the burden of proof remains upon you.
The universe is expanding.
So is my waistline.
Damn astrophysics for making me fat! I'm going to go find some chocolate...
Richard, I think you have that entirely backwards. Clearly, we need to get you on a diet to stop the universe from expanding...
I remember Arnold Kling arguing, there was no reason to believe any manipulation was taking place in the California Energy market. Economics proved it. I wonder if he ever read the Enron transcripts. Man that is a pretty big miss for an economist.
Of course, he is right in this case that correlation doesn't imply causation. It is just a coincidence that he is correct, your better off not listening to him. Instead put your trust in random blog comments.
Actually, no. If your preferred world view predicts no correlation, then it's up to you to revise your opinion in the face of contrarary evidence.
If correlation is not evidence of causation, then what is?
Blackadder:
If your question was rhetorical, then I apologize and you can ignore this comment.
But correlation and causation are two very, very different things. A particularly strong correlation can suggest that there is a strong relationship between the two items - however, a strong relationship says exactly nothing about the source of that relationship, and often the source of that relationship is something very different from either of the two sets of data points. The cause of the correlation may even be something as simple as what Kling points out, which is that data, over time, tend to follow trends, virtually ensuring that you can find a correlation between just about anything if you choose an appropriate time period.
My personal favorite example of this is a study from a long while back that showed an almost perfect correlation between crime in Central Park and ice cream sales in Central Park. If you believed that correlation and causation were the same thing, then you would believe that high ice cream sales were driving people to commit crimes (or that being a crime victim led to the purchase of more ice cream if you were even more insane). As a result, the logical conclusion would have been to ban ice cream sales in Central Park.
Of course, the real cause of this relationship was just that people buy more ice cream in warmer months on days when the weather is nicest, during which they also go outside more often to places like the park. More people means more potential crime victims, which means more crime.
Correlations can be useful if you start out with a plausible a priori theory that A will cause B - but even then a correlation between A and B doesn't prove your theory. It just makes your theory somewhat more likely than it was before you found the correlation. You need to be able to repeat your correlation under circumstances where numerous possible variables are either present or not present before you can begin to conclusively demonstrate that A, in fact, causes B. In essence, until you have controlled for as many conceivable variables as possible, you will not be able to definitively say that the correlation is evidence of causation, much less proof of causation.
What a strong correlation (positive or negative) does for you is suggest where it might be productive to look for possible causes. You may not find any, or you may find something. Or you may find that both sides simply have a common cause.
On the other hand, if you find no correlation, you probably do not need to spend time looking for causation.
Don't tell that to Steve Sailer, who relies on r-squareds of 0.62 to push his theories.
This is a usual and boring point, so I will make some usual and boring responses.
1) In a deterministic universe if two events have a correlation of 1 (actually) then it is a physical law and is, usually, what we mean by causation (ultimately). For example, if I ask "why did the object fall" and you answer "because of the force of gravity", your answer is pure tautology. The force of gravity is merely a statement of a particular perfect correlation (or law-like relationship). Of course, there are other ways to think about this and distinctions to drawn between correlation and causation, but this is a standard point too. I beg you, for my health, please don't say something about Einstein vs Newton, or falsifiability, without thinking about it.
2) Practically apeaking (aka: ignoring point 1), the reason "correlation doesn't imply causation" (as Kling said but McArdle didn't exactly) tends to be an important point for, oh, say economists, is because economics is just barely a science (I'm being deliberatly offensive). In better sciences, high correlation implies causation rather strongly. Indeed, the better the science, the stronger the implication. What is true (and is what the expression means in that context) is that correlation does not imply direct causation. That is, that events A and B are correlated does not mean A causes B. The expression does not usually mean that there is no implication of some causal relationship lurking between A and B, as I gather Kling's letter writer hoped.
3) Kling's point about time series data exhibiting broad trends is well taken. The point, though, is that the distribution of correlations for data like that will have incredibly heavy tales - indeed, there will be more points scattered near 1 and -1 than near 0 (think of a pair of delta functions at the outskirts that have been Gaussian smoothed). This means our intuition regarding correlation (which is true for, oh, data with more than 5 or 6 degrees of freedom) will not work (but that's just intuition about what correlation is "enough" to imply something interesting/causal, not about whether it does).
4)As a fair example, consider:
"The universe is expanding.
So is my waistline."
These are probably not surprisingly well correlated.
Apologies.
Things I should've said:
"the better the science" -> the better controlled the science
This was a very stupid thing I said, I hope someone noticed. "delta functions" -> almost delta functions (that is, not getting too near the limit).
Science is determined by method, not by the nature of the data that's available. Astronomy, evolutionary genetics, climatology and economics all have to make do with non-experimental data.
They are not by themselves evidence--nay, not even if you cross your arms, scowl at your opponent, and say "Well, then give me another explanation for this astonishing correlation!"
I disagree. A highly statistically significant correlation is, by definition, unlikely to be spurious, and therefore strongly suggestive of some kind of causal connection. It's not conclusive evidence--dredge through enough data and you're bound to turn up some spurious collations with very small p-values--but it's evidence nevertheless.
Of course, you don't get to pick the causal relationship you like best and claim that the correlation proves it. But you also don't get to say "correlation is not causation" and then ignore the correlation completely. A highly significant correlation almost always means that there's something interesting going on, and a model that can't explain it is likely flawed, or at best incomplete.
If I had to guess, I would
They are not by themselves evidence--nay, not even if you cross your arms, scowl at your opponent, and say "Well, then give me another explanation for this astonishing correlation!"
I disagree. A highly statistically significant correlation is, by definition, unlikely to be spurious, and therefore strongly suggestive of some kind of causal connection. It's not conclusive evidence--dredge through enough data and you're bound to turn up some spurious collations with very small p-values--but it's evidence nevertheless.
Of course, you don't get to pick the causal relationship you like best and claim that the correlation proves it--there are always alternative explanations. But neither can you say "correlation is not causation" and then sweep an inconvenient correlation under the rug. A highly significant correlation almost always means that there's something interesting going on, and a model that can't explain it is likely flawed, or at best incomplete.
dredge through enough data and you're bound to turn up some spurious correlations with very small p-values
If you're doing your statistics correctly, you will, in fact, some up with spurious correlations of a given p-value a fraction of the time equal to your p-value. That's what a p-value means.
One thing to keep in mind when looking at correlations is that the events you see a correlation between are constructs.
You have to choose what to measure - that is a choice, and "forces" structure and organization onto what your are measuring. Once you have begun to place the events you measure into a construct that you have created, you are well on the road to seeing a correlation where there may be deeper relationship.
I would say that correlation is a necessary but not sufficient condition of causation. What is missing is the mechanism.
A highly statistically significant correlation is, by definition, unlikely to be spurious, and therefore strongly suggestive of some kind of causal connection.
Well, it depends on the variables you're including, yes? In the example Megan cites, the person is comparing one variable to another. I'm no expert on statistics, but, regardless of the p-values, I'd be thinking 'omitted variable bias' (or 'underspecified theoretical model that should really have more than one independent variable') if I saw that in a paper.
In better sciences, high correlation implies causation rather strongly. Indeed, the better the science, the stronger the implication. What is true (and is what the expression means in that context) is that correlation does not imply direct causation.
Which exactly are these better sciences where correlation implies causation rather strongly?
In my experience in engineering, physics and chemistry, what counts for implying causation is the controlled experiment, not correlations. I also understand in medicine the gold-standard is the double-blind trial, again aiming for causation not correlation. I'm quite curious therefore as to which sciences you are referring to.
On thinking more about it, I'm more and more curious about which sciences Random thinks are the ones where a stronger correlation implies causation. Which are the "better sciences" in which there is no chance that someone trawling through reams of data series and picking out the ones that by pure change happen to be correlated?
A highly statistically significant correlation is, by definition, unlikely to be spurious, and therefore strongly suggestive of some kind of causal connection.
However, if your definition of "highly statistical significant" is 1% chance of being due to random chance, then for every 100 correlations you examine of purely random statistics, you should get one that is highly statistically significant. How easy is it nowaday to examine 100 correlations? Before you can say that a highly statistically significant correlation is unlikely to be spurious you have to know about the process by which it was generated. If I know nothing at all about how it was generated, I think that a highly statistically significant correlation doesn't suggest anything really about any kind of causal connection.
On the off chance that Random doesn't reply, I think it's safe to assume that he's referring to Maths and Physics, and assumes that everything else is inferior.
"In my experience in engineering, physics and chemistry, what counts for implying causation is the controlled experiment, not correlations. I also understand in medicine the gold-standard is the double-blind trial, again aiming for causation not correlation."
Question: What do the results of a controlled experiment show? Is it not a correlation (or lack thereof)?
Compare: People who use product x have a higher rate of getting disease y (proves nothing, because correlation does not prove causation); so we do a controlled experiment and find that people who use product x have a higher rate of getting disease y (this does prove causation, because, you know, when did this controlled experiment, and it found a correlation).
Also, if the only way to show causation is through a controlled experiment, how did people manage to know anything at all about causation prior to the advent of modern science? Presumably people knew of the connection between, say, sex and babies or getting mauled by a tiger and dying long before there were controlled experiments on the subject. Were they all just making foolish inferences?
Actually, have there been controlled experiments on the proving that sex causes pregnancy, or that being mauled by tigers causes death? If someone could point these studies out to me, I would be really appreciative. I wouldn't want to be making any hasty inferences.
Well in maths, what counts is proofs, not correlations.
Blackadder - yes, you are right, in a controlled experiment, what is measured is a correlation. The controlled experiment, if properly designed, adds to the mere correlation the ability to draw conclusions about causation - ie the only thing we changed was x, and y happened, while as when we didn't change x, y didn't happen, so x causes y (when you can't control all the sources of variation, eg in medical tests you can't control all the variation in people's immune systems, you have to use statistics).
However, in those sciences it's quite possible to get a straight correlation that doesn't imply causation. For example, collect time series data on EM radiation hitting earth. If that's purely random, and you set out trying to find a correlation between sets of that time series, and, say, sunspot activity, you will find them in proportion to whatever you define as a statistically significant correlation. In medicine, it's even easier. Just because you have a correlation doesn't mean you have causation.
Stephan: Kling's opinions on topics such as energy and the housing market have also been suspect. From what I can tell, he has an ideological axe to grind and then rummages around the internet looking for particulars against which he can take a few whacks.
Klug: Bravo. And do note that Kling makes statements such as "But the reality is that the intelligences that feed into IQ are what drive economic success," statements that draw on correlation data which he so questions when he chooses to attack macro-econometrics. Note he says "are what drive(s)" economic success" as if this is a certainty.
I agree with Hume when he says that we cannot directly observe causation--only correlation. Then it's just a matter of looking at the variety of prior states and the network or complex of correlations. Since Kling expresses an interest in philosophy, I imagine he already knows this.
It is very easy to calculate p values incorrectly. There are many ways to do this (incorrectly). Tracy seems to be thinking about, or be confused by, the problem of multiple tests. Kling focussed on time-series. There are other things one could worry about. All of these things mean that calculating your "p value" overly naively will lead you astray. As I tried to explain (clearly badly) in my point 3, that correlations are high or low is sometimes not surprising/significant at all. That is not the same as saying correlation does not imply causation. If you want that expression to mean "high correlation" does not imply causation, no human can disagree with you, but most people use it mean "significant correlation" (even properly calculated) does not imply causation - and it is this latter point which I think is too reflexively made. Likewise, of course and as I said, it doesn't mean A causes B. Anyway, I'm making very ordinary points (or trying to!), so I'll stop now.
I notice I contradicted myself (or implied I'm not human).
I shifted meanings slightly - earlier in writing "high", I meant "significantly high" whereas later I meant "above some fixed threshold"
So, my meaning earlier was that (significantly) high correlation implies causation rather strongly and later that (some arbitrary theshold value) of high correlation can not be taken to imply causation.
In other words, I am repeating my point that the distribution of correlations may not be simple.
Also, if the only way to show causation is through a controlled experiment, how did people manage to know anything at all about causation prior to the advent of modern science?
Quite a bit that they knew turned out to be wrong. For example, an object in motion doesn't have some impetus that eventually runs out, instead an object in motion keeps moving until another force acts on it. It just so happens in real life that when you stop pushing something, friction slows it down, so it looks like if you stop pushing on something it stops moving. There was a correlation between stopping pushing something and it stopping moving, but it wasn't causation.
Knowing that if you stop pushing something it stops moving is perfectly adequate information for most earthly purposes. But ascribing causation to it was wrong.
Actually, have there been controlled experiments on the proving that sex causes pregnancy, or that being mauled by tigers causes death?
Okay, sex is correlated with pregancy, and being mauled with tigers is correlated with death, but people have had sex without getting pregnant and have been mauled by tigers without dying, so understanding causation is more complex than you present it here. If you want to understand the process of causation of pregnancy, I'd look at the experiments that established the fertility cycle in women and on the mauling topic I'd look at research on blood loss as a result of experiments on animals. I also note that it is possible to get pregnant without having sex - through artifical insemination or through implanting a fertlised cell in a womb. And it's possible to die without being mauled by tigers. We know, from direct observation, that a sperm entering an egg of the same species *can* result in a pregnancy, but our full understanding of the causation chain is still developing.
What is really going on here is different forms of certainty. Shall we rank them?:
A correlation between A and B in a world with ample large datasets doesn't tell us much at all.
A series of independent observations under different circumstances that event B tends to ocurr after event A and event B doesn't tend to ocurr in the absence of event A gives us some information. (This is what medical science mostly consists of, as it's generally impossible to control for the variation in our immune systems.)
A series of independent observations that event B only occurs after event A and never occurs in the absence of event B gives us more information.
A controlled experiment showing that event A occurs after event B has happened and event A doesn't occur if everything else is the same but event B didn't happen gives us quite a lot of information.
Absolute certainty is limited to mathematical proofs.
Random I know you want to stop now, but could you please do me a favour and explain why you think I am confused about multiple tests? I didn't realise I was confused, and if I am I would like to be able to move to being unconfused.
I am so bored by myself in this conversation that I feel my participation as a fault and it is affecting my tone, Tracy (true from the outset). You'll be sorry you asked because I'm sure you know what I'm saying (since you asked - I didn't know you knew before).
Anyway, all I am saying is that "correlation implies causation" has a necessary implication of some sort of qualifier on the "correlation". That is "high" "significantly high", maybe "zero" sometmies etc. I think people are typically overly happy to thoughtlessly say "correlation doesn't imply causation" to wave away "interesting" correlation. Notice that Kling doesn't actually do this inside his own head (he waves it away because it's time series data) just out loud. Likewise in the case you described, the qualifiers on what constitutes an "interesting" correlation, such that we might say "correlation implies causation" must incorporate all the facts of the test. I only skimmed your example, and it just sounded like the standard problem of multiple tests - which one has mtehods to deal with (methods that are itnended to get one to the point of correlation better implying causation). So, yes, a given correlation doesn't imply a given chance of causation. Nor does a thoughtlessly/deceptively obtained p value mean the same. And there are statistical methods which exist to help us figure out the correlation that implies causation (again, not necessarily direct causation). We've been happily vague about what we mean by correlation, too. I'm outta here.
Tracy,
What is the percentage of controlled experiments that turn out to be, if not simply wrong, then at least unrepeatable? It's pretty high, no?
I'd agree that absolute certain is limited to mathematical proofs, if it's even possible there (one could always have messed up the math somewhere). For most purposes, ordinary everyday practical level of certainty will do. What I don't like about the slogan "correlation does not prove causation" is that it doesn't seem to recognize this fact.
Blackadder - I think "correlation does not prove causation" doesn't imply or assume that "this fact" (that for most purposes ordinary every practical level of certainty will do) is false. Correlation often doesn't prove causation even to "ordinary everyday practical levels of certainty.
A very high correlation, combined with many examples of different circumstances where the correlation works, combined with a solid explanation of how the causation works, does suggest causation to the ordinary every day level you are talking about. But the combination of those things is a lot more than just "correlation".
Correlation all by itself might suggest the possibility of causation to those every day levels , but "suggest" is a long way from proof, or even something like "gives us solid reason to believe that", which might be a better term for your "every day practical level of certainty" than "proof" or "prove".
I think it's pretty clear that Random doesn't know what he's talking about. Also, since I've been saving these up for a while, here are a couple of fairly recent links: Factor Analysis. Check out the bit on exploratory anaylsis and causal inference. And here is a fairly good piece on what significance testing really is, and some of the perils and pitfalls for the unwary who think that they aren't (btw, physics has the same sort of problem, but given the type of experiments they run and the extremely low p-values, typically less than 10^-4 the ignorance of anything but the mechanics isn't as much of a problem.)
Finally, in a purely logical sense, while it is true that correlation does not imply causality, it is also true that causality _does_ imply correlation. It follows then that lack of correlation most definitely implies lack of causality. But of course, just what, precisely does 'lack of correlation' mean? I'll leave that as a meta-question :-)
Causation cannot be proved statistially. It can only be established either experimentally or theoretically. Granger causality for time series data is an effective tool, but even that shows spurios causality sometimes.
In short one cannot be sure about causality only by looking at the data. Unless the data is generated in a controlled environment.
Or it can be established logically/ theoretically even before you begin looking at the data.
"If you want to understand the process of causation of pregnancy, I'd look at the experiments that established the fertility cycle in women"
Beside the point Tracy. The point is people understood how one gets pregnant fairly well before any of these experiments were made. "established the fertility cycle in women?" Come on, the fertility cycle has been known to humans for thousands of years.