Friday, 29 June 2012

'The Ocean Dilemma' - a scientific tool

The Ocean Dilemma: C, C, C or C?

The Ocean dilemma is an element of the scientific method, so-named by me, which should be used in order to avoid drawing false conclusions.
Whenever presented with a Correlation, we should consider: does it indicate a Causation, a Consequence, or is it just a Coincidence?
(It is so-called because of all the ‘c’s, you c sea see) ;-)

The Ocean Dilemma can be used to avoid confirmation bias and coherence bias, which occur when one hypothesis is considered, and others are not. Bearing in mind that an observed correlation might be coincidence or consequence, rather than causation, wards the thinker away from making such an elementary error in their internal processes of appraisal.
The purpose of randomized, controlled trials (RCTs) is to exclude coincidence from the possibilities, but sometimes even they are not good enough to distinguish causation from consequence – a difference that must be identified by further, mechanistic analysis.

Case Study 1:
It was contemporarily presumed that hard knocks to the head caused brain conditions, which is why the journalist wrote this sentence:
“They revealed that Boogaard had a degenerative disease thought to be caused by repeated blows to the head”
“Usually symptoms of CTE appear when people are in their 30s or 40s, but Boogaard's case tells us the physiological changes may begin to take place long before that, Stern says”
This information provides us with something to work with, in order to distinguish between the ‘seas’ – if the damage precedes the behaviour, then it’s probably consequence rather than causation, but if the data’s not got good statistical significance, then we won’t be able to rule out the possibility of it being a coincidence.
"In his last two years of life, Boogaard did show some changes that might be consistent with what we see in CTE," says Stern. "But they also could just be from narcotic addiction - there's no way to decipher which came first or what caused what."
Ah - complications. These are typical of analyses - especially medical analyses. It might be imfeasible to know what caused what in Boogaard’s case, but aggregate data across populations can tell us how CTE works, by isolating it as a variable (controlling for all the others).
In this case study, the Ocean Dilemma, if considered by the earlier researchers, would have prevented the contemporary prejudice that knocks caused the brain conditions.  Further analysis showed that this assumption was not justified by the evidence.

Case Study 2:
The Ashanti peoples of West Africa hold a belief that the day of the week they’re born on will be a determinant of personality. This manifests to the extent that there are day-of-the-week-specific names, which reinforce people’s identities as a ‘Monday child’.
European descendants are likely to be familiar with this rhyme:
Monday's child is fair of face,
Tuesday's child is full of grace;
Wednesday's child is full of woe,
Thursday's child has far to go;
Friday's child is loving and giving,
Saturday's child works hard for a living;
But the child that is born on the Sabbath day,
Is bonny, and blithe, and good, and gay.
...except the Ashanti think theirs is true.
How can they tell whether it really is? Well, first they should consider the Ocean Dilemma.
If it’s causation, then other populations will show the same differences, independent of belief in the rhyme.
If it’s consequence, then there should be a better correlation amongst children named post-birth than pre-birth (because the parents would have had a chance to assess their personality – even though personality assessment in a baby is implausible anyway)
If it’s a coincidence, then there will be no correlation in populations that do not hold the beliefs about day of the week and personality.
If the research showed that personality-type varied according to specific beliefs about day-personality-determination, then that would be evidence that beliefs were correlated with personality-type rather than the days of the week themselves.
Assessment has since shown that this last case is the true one. Ashanti names do not cause behaviour – Ashanti beliefs do - we know this because we get negative predictive results in people who are not aware of name meanings e.g. Europeans who have Greek names.

Case Study 3:
Does dinner make a strong family, or does a strong family make dinner, or neither?
"We find that most of the association between family meals and teen well-being is due to other aspects of the family environment. Analyses that follow children over time lend even weaker evidence for causal effects of family meals on adolescent and young adult well-being"
“...ability to manage a regular family dinner is in part facilitated by family resources such as time and money, and in part a proxy for other family characteristics, including time together, closeness, and communication.”
This study appears to show that causation is not the case, and in fact consequence is more likely – ‘strong’ families have dinner together as a consequence of their being ‘strong’ – it wasn’t dining together that made them so.

If you’d like to read some more examples of cases in which the Ocean Dilemma helps, then read the ‘Box’ examples (in green) in this report, written partly by Ben Goldacre of the ‘Bad Science’ books fame:

No comments:

Post a Comment