The Ocean Dilemma: C, C, C or C?
The Ocean dilemma is an element of the scientific method,
so-named by me, which should be used in order to avoid drawing false
conclusions.
Whenever presented with a Correlation, we should consider: does it indicate a Causation, a Consequence, or is it just a Coincidence?
(It is so-called because of all the ‘c’s, you c sea
see) ;-)
The Ocean Dilemma can be used to avoid confirmation bias and
coherence bias, which occur when one hypothesis is considered, and others are
not. Bearing in mind that an observed correlation might be coincidence or
consequence, rather than causation, wards the thinker away from making such an
elementary error in their internal processes of appraisal.
The purpose of randomized, controlled trials (RCTs) is to
exclude coincidence from the possibilities, but sometimes even they are not
good enough to distinguish causation from consequence – a difference that must
be identified by further, mechanistic analysis.
Case Study 1:
It was contemporarily presumed that hard knocks to the head
caused brain conditions, which is why the journalist wrote this sentence:
“They revealed that Boogaard had a degenerative disease thought to be caused by repeated blows
to the head”
“Usually symptoms of CTE appear when people are in their 30s
or 40s, but Boogaard's case tells us the physiological changes may begin to
take place long before that, Stern says”
This information provides us with something to work with, in
order to distinguish between the ‘seas’ – if the damage precedes the behaviour,
then it’s probably consequence rather than causation, but if the data’s not got
good statistical significance, then we won’t be able to rule out the
possibility of it being a coincidence.
"In his last two years of life, Boogaard did show some
changes that might be consistent with what we see in CTE," says Stern.
"But they also could just be from narcotic addiction - there's no way to
decipher which came first or what caused what."
Ah - complications. These are typical of analyses -
especially medical analyses. It might be imfeasible to know what caused what in
Boogaard’s case, but aggregate data across populations can tell us how CTE works, by isolating it as a variable
(controlling for all the others).
In this case study, the Ocean Dilemma, if considered by the
earlier researchers, would have prevented the contemporary prejudice that
knocks caused the brain conditions.
Further analysis showed that this assumption was not justified by the
evidence.
Case Study 2:
The Ashanti peoples of West Africa hold a belief that the
day of the week they’re born on will be a determinant of personality. This
manifests to the extent that there are day-of-the-week-specific names, which
reinforce people’s identities as a ‘Monday child’.
European descendants are likely to be familiar with this
rhyme:
Monday's child is fair of face,
Tuesday's child is full of grace;
Wednesday's child is full of woe,
Thursday's child has far to go;
Friday's child is loving and giving,
Saturday's child works hard for a living;
But the child that is born on the Sabbath day,
Is bonny, and blithe, and good, and gay.
Tuesday's child is full of grace;
Wednesday's child is full of woe,
Thursday's child has far to go;
Friday's child is loving and giving,
Saturday's child works hard for a living;
But the child that is born on the Sabbath day,
Is bonny, and blithe, and good, and gay.
...except the Ashanti think theirs is true.
How can they tell whether it really is? Well, first they
should consider the Ocean Dilemma.
If it’s causation, then other populations will show the same
differences, independent of belief in the rhyme.
If it’s consequence, then there should be a better
correlation amongst children named post-birth than pre-birth (because the
parents would have had a chance to assess their personality – even though
personality assessment in a baby is implausible anyway)
If it’s a coincidence, then there will be no correlation in
populations that do not hold the beliefs about day of the week and personality.
If the research showed that personality-type varied
according to specific beliefs about day-personality-determination, then that
would be evidence that beliefs were correlated with personality-type rather
than the days of the week themselves.
Assessment has since shown that this last case is the true
one. Ashanti names do not cause behaviour – Ashanti beliefs do - we know this
because we get negative predictive results in people who are not aware of name
meanings e.g. Europeans who have Greek names.
Case Study 3:
Does dinner make a strong family, or does a strong family
make dinner, or neither?
"We find that most of the association between family
meals and teen well-being is due to other aspects of the family environment.
Analyses that follow children over time lend even weaker evidence for causal
effects of family meals on adolescent and young adult well-being"
“...ability to manage a regular family dinner is in part
facilitated by family resources such as time and money, and in part a proxy for
other family characteristics, including time together, closeness, and
communication.”
This study appears to show that causation is not the case,
and in fact consequence is more likely – ‘strong’ families have dinner together
as a consequence of their being ‘strong’ – it wasn’t dining together that made
them so.
If you’d like to read some more examples of cases in which
the Ocean Dilemma helps, then read the ‘Box’ examples (in green) in this
report, written partly by Ben Goldacre of the ‘Bad Science’ books fame:
No comments:
Post a Comment