Monday, May 18, 2009

Correlation vs. Causality

This USA Today article was brought to my attention recently, digested into this one sentence summary:

Education matters, a new website will calculate how education influences important statistical indicators such as income, health, voting rates and even the likelihood that a person will stay out of prison.


Two things are naturally true of this brief story: it probably seems innocuous and uncontroversial, and it made me angry to read it. The problem is that the work being presented under this summary is not a series of conclusions, but rather an unsubstantiated claim being offered for uncritical acceptance. And indeed it is accepted uncritically, even in absence of a website specifically devoted to furthering the presumptuous claim. This is directly on a par with the comments from Rahm Emmanuel that I had criticized earlier, about people “earning what they learn.” He had no doubt based that conclusion upon the observation that he and his colleagues had obtained a respectable formal education and had subsequently become successful and wealthy. Now Education Matters is broadening that statistical analysis and including other indicators to make the instrumental good of obtaining institutionalized education seem more fully confirmed. But underlying all of this is what seems to me a fairly obvious problem.

You see, I would think that the people making these assertions or putting out these studies, who must thoroughly value education, would be aware of, and resistant to, common logical fallacies, such as confusing correlation with causality. I think that mistake is particularly common in part because it allows people to justify their own presuppositions. And on the basis of that mistake, you can do so in a way that seems scientific, that seems statistically significant. If you believe that college attendance increases income, and civic participation, and social morality, all you need to do is gather statistical data indicating that graduates demonstrate these characteristics, and then assert that it was their going to college that caused all of these things. This sort of misguided reasoning is applied to all sorts of things, but at least in the company I’ve kept I’ve seen it undercut much more readily only with respect to more controversial issues. Educated people often don’t buy the claim that violent video games make children more violent merely on the basis that children who demonstrate violent tendencies often play violent video games. It is recognized, in that case, that children with such tendencies may have a particular proficiency for playing such games, while non-violent children can both play them and retain psychological health. Privileging one interpretation over the other tends to be based on a preexisting ideological commitment to one conclusion.
But the summary quoted above is based on just such a prejudgment of the reasons for the statistical observations. However, that presumption is not so often called into question, perhaps because it is more convenient for people in a position to recognize and refute the error of reasoning. Go back and read that quotation again. Now, what if I were to say instead:

A new website will calculate how statistical indicators such the likelihood of a person staying out of prison, their income potential, health, and voting rates all influence the level of their education.

Does that sound implicitly worse? It’s little more that a change of syntax, but it practically reverses the conclusions suggested by the same analysis. On the surface, there is no sound reason to identify the correlation between these indicators as working in one causal direction.

And I can do the same with the content of the article, which claims that increasing rates of education would consequently have a positive influence on the other things mentioned, and which takes New Mexico as an explicit example. They say:

[M]ove all adults in the state of New Mexico up just one level of schooling — those without high school would graduate, those with a high school degree would get some college (or an associate's degree), and those with some college would earn a four-year diploma — and the tool predicts, among other things, that life expectancy in the state would increase by nearly two years.

New Mexico's murder rate would drop by more than half, from 8.6 murders per 100,000 people to 3.8.

Meanwhile, median personal income in the state would rise, from, $27,927 to $35,253, and the percentage of people who vote would jump substantially, from about 55% to 65%.

Also, the incarceration rate would drop more than 60%, from 640 people per 100,000 to 246.


But I put forth, by contrast, that if, in New Mexico, the incarceration rate were to drop about 60%, the murder rate were to be cut in half, the median income were to rise from twenty-eight to thirty-five thousand, the voting rate were substantially increased, and the life expectancy raised two years, then all adults in that state would move up one level of schooling as a consequence of the improved socio-economic circumstances.

I think the two hypotheticals rather well encapsulate my point of contention with these generally uncontended claims about the value of formal education as a means of upward social mobility. Perhaps it’s not that those who have the interest and commitment to obtain that education become better, more successful people, with more money, better health care, more virtuous friends, and so forth. Maybe it’s that those who are more financially comfortable early in life, and who aren’t exposed to many negative social influences, simply have a greater interest in, and more resources, such as time and money, to devote to obtaining a formal education.

But it seems to me that the authors of this new website, and Americans in general, don’t consider the distinction between these two interpretations. The presuppositions of the given article seem obvious, but the CEO of the United Way is quoted in its conclusion as saying simply "Intuitively, people know these things are connected." Well, of course they are connected. But connected in a unidirectional, causal way? I’m not convinced. And for those that are, that’s where the burden of proof lies – with them. I have yet to see anyone try to lift it.