Sunday, 25 November 2012

The Epidemiological Hazards Of Medical Screening

(title links to YouTube video version)

Date Started: 20/11/12
Date Completed: 24/11/12
Date First Published: 24/11/12

ref: ‘No Benefit from Routine Checkups’ – Dr Steven Novella, MD


Medical screening – where a population (usually large) is tested for a condition/disease, without prior knowledge of its presence
Positive result – when a test says the patient does have condition/disease X
Negative result – when a test says the patient does not have condition/disease X
Sensitivity – the chance a test will return a true result, when that result should be positive (they do have X)
Specificity – the chance a test will return a true result, when that result should be negative (they do not have X)

(A failure in sensitivity returns false negatives; a failure in specificity returns false positives)

Let’s say a condition called ‘X’ exists in 1 in 1000 people, and we test 1000 people for it.

The test we use has sensitivity of 99%, and specificity of 99% (that’s a very good test, by modern standards, by the way)

A 99% specificity means 99 in 100 will be true “negative” results, and 1 in 100 will be false “positive” results – the test will say a patient has X, when they don’t, 1 in 100 times.

With 1000 people tested, this means 1 in 100 false positives returns 10 false positive results (9.99 = 999/100, statistically).

Add this to the one true positive, and we get a total of 11 positive results, 91% (2sf) of which are false!

A 99% sensitivity means 99 in 100 will be true “positive” results, and 1 in 100 will be false “negative” results – the test will say a patient does not have X, when they do, 1 in 100 times.

In every 100th person who has X, it will not be detected. So with 100,000 people tested, and 100 of them having X, this means 1 in 100 false negatives returns 1 false negative result.

In our example, of 1000 people, there is a 1% chance of the only person who has it, being missed.

This happens because X is rare – too rare for the test to detect usefully.
Our test has a... well, let’s call it ‘precision’... has a precision of 99% - a 99 in 100 precision.
But X has a rarity of 0.1% - a 1 in 1000 prevalence.

To test for some condition and not have our data swamped with false results, we must either be in a case where X is far more abundant (e.g. a chronic, common condition, like high blood pressure) or advance our test – increase its precision, a few decimal places.

If, for example, its precision were 99.99% (doctors would be ‘over the moon’ if they could get their hands on a test like this), then the stats would play out thusly...

We still test 1000 people, looking for one case.

This time, sensitivity and specificity are 99.99%

This means 9999 in 10,000 true “negative” results, and 1 in 10,000 false “positive” results.

With 1000 people tested, this means a 10% chance of getting one erroneously positive result.

It also means 9999 in 10,000 true “positive” results, and 1 in 10,000 false “negative” results.

With 1000 people tested, this means a 0.01% chance of the only true positive being missed.

Unfortunately, this scenario is a pipe-dream. We have to work with what we’ve got.

There is definitely a role for medical screening, but it should only be done when it is worthwhile.

Plus, there are costs to screenings, which sometimes go beyond poorly-utilised man-hours, which could be applied more constructively.

Some examples, exploring the various costs of screening...

New Scientist: ‘X-ray hazard for people with obesity’
Because obese people have more body tissue, they require higher doses of radiation, to effectively ‘see’ into their bodies. This raises their risk of cancer development.
But scans can be used to detect cancer, too. And other conditions also. The costs have to be weighed against the benefits.

Margaret McCartney: ‘Self checking via Embarrassing Bodies’
Encouraging people to do their own self-checks increases fear in the audience, but not the incidence of the problem (obviously). This creates a negative-heavy scenario, like in our example, where medical providers are flooded with fearful patients, exhorting with erroneous results.

Margaret McCartney: ‘Streptococcus B in pregnancy: to screen or not to screen?’
A £30 test for Strep B in foetuses? Sounds like a good thing, but isn’t. The reason – like in our example, the tests are poor, producing a sea of erroneous results  - meaning mostly false positives - and hence even more worry than you had before. Plus, when you know your foetus has Strep B, all you can treat it with is antibiotics, which are already massively overused. This overuse is responsible for the development of superbugs, such as MRSA, which have the potential to kill many, many people. First-hand risks of these antibiotics include asthma, allergies, and death. £30 a time for a test like that? Keep your money.

‘Inside Health and screening’
Private clinics are increasingly advertising low-precision screenings to people who are unaware that they produce seas of useless, false results. And they sometimes charge hundreds, for the pleasure! Again – keep your money.

‘Question: Should I be screened for prostate cancer? Answer: No.’
Tests for prostate cancer orient around detection of PSA (Prostate-Specific Antigen). These tests are a highly inaccurate indicator of the current presence, or impending development of, prostate cancer. This renders it useless, and a million kilometres from cost-effective.

NHS: ‘Breast Cancer Screening’
Because breast cancer predominantly develops in later life, it’s much rarer in under-40s women, and even rarer still in men. This means it’s pointless routinely testing for tumours in men and young women, which would usurp resources, and time, and expose millions of people to radiation unnecessarily. As said before, the costs have to be balanced with the benefits.

It’s not effective, nor cost-effective, to have our GPs nurse us through life, checking our bodywork over for dents and scratches every morning, like we do with our cars/bikes/boats/helicopters (if you’re really rich).

An 'old wife' once said: "As long as it saves one life, then it’s worth the effort"

Sorry, but it isn't – not if more lives could be saved by doing something else.

In the Information Age, people are increasingly utilising the Web, to do their own, self-diagnosis.

This can save the Doc effort, but it does mean we run the risk of bumping into quackery, and shelling out for a packet of grated alien earlobe, to be drunk every morning, in boiled water!

Or maybe shelling out for just the water [coughs: “homeopathy”]

And then the Doc has to pick up the tab when we get worse.

So to make the most of medical screenings, we must:

1)      Have a good idea of what we’re looking for
2)      Have a good test for it
3)      Know what we can do when we find it

“All of this is a powerful reminder that we need evidence-based standards in medicine more than ever. Health care systems are straining under rising costs. We need to use the best evidence available to figure out what not to do, because it wastes resources or even does net harm. Science remains our best tool for improving the health care system.” - Dr Steven Novella, MD

No comments:

Post a Comment