Tuesday, January 09, 2007

Debunking Depression Research

Scott Burgess does an excellent job of finding the fatal error in some "research" by psychologist Oliver James. (You should read his article as it is better than mine)

The research blames capitalism for higher rates of depression and other mental disorders. And while I wont disagree that capitalism can be depressing (Especially when you get your credit card bill) the research is flawed.

The main claim that Mr. Burgess debunks is the high correlation between wealth disparity and high levels of mental illness. Using a whopping 8 datapoints (The US plus seven other nations) they find a correlation of .73. But when the US is removed from this analysis the correlation is only .24.* The US was a strong outlier, and it accounted for a large portion of the positive relationship. (This could be because the US probably has more psychologists with which to diagnose more mental illnesses, seems like everyone has a shrink these days.)

This throws into sharp focus just how easy it is to mislead with statistics. One data point separates this study from proof that capitalism is the root of all mental illnes and a meaningless excersize in graphing. Whether or not Mr. James is simply ignorant of the proper detection of outliers, or whether he deliberately ignored the obvious flaw in his research is unknown, but the fact remains that this study has been published and it will be believed by people who do not hear anyone speak to the contrary.

HT Chez

* For non stats types, a correlation is the measure of a relationship between two measures. It can range from -1 to 1, 1 being a perfect positive relationship and -1 being a perfect negative relationship. .74 is an extremely strong correlation for real world research. This study illustrates just what a weak measurement correlation is, it can easily be affected by just a few strong outliers. Also, while it shows a strong relationship it cannot prove causation. As I wrote on many a students paper back when I was a TA "Correlation DOES NOT equal Causation."

3 comments:

DED said...

Yeah, he pretty much ripped him to shreds.

submandave said...

I place much more faith in the correlation that capitalistic societies produce the level of wealth under which a thriving mental health profession seeking affirmation can find enough affluent subjects with enough leasure time and disposable income to engage in long-term therapy.

Shinobi said...

No Kidding Dave, like people in ethiopia have time for a therapist.