Monday, 11 February 2013

Comment #20: -- Lumosity And The Brain-Training Bunk

Date Started: 6/2/13
Date Completed: 10/2/13
Date First Published: 10/2/13

Gaaaah.... Lumosity. Who can bear that dreaded ‘ting’?

It plagues YouTube with its ads, and has driven me to write a much-too-un-mini mini-essay about it.

If this seems rambly, then the reason is that Lumosity has shredded my nerves!

Pseudo-neuroscience has become much more abundant, in recent years, because no-one wants to be stupid, and no-one wants to be thought stupid. The products we see are largely targeted at people who will live a long time, and so, increasingly, also fear dementia as they age.

‘Molly Crockett: Beware neuro-bunk’

From fruit drinks, to brain-training games, entrepreneurs are producing pseudoscience, to exploit the growing market of people willing to shell out for brain-enhancements.

But there’s no evidence that they work.

The drinks certainly don’t work – not now, at least – maybe sometime in the future, they will; but the claims of brain-training games currently have greater verisimilitude.

Exercise makes people fitter, brain exercise makes brains fitter – that’s how it works, right?

Not quite.

Exercising specific parts of your body makes those parts fitter, but not the rest. This is why health and fitness depends on cardio-vascular exercise, which does have a body-wide effect. (Your cardiovascular system does go pretty-much everywhere, after all). Big biceps don’t equate with good fitness.

All the little puzzles and things, from the crosswords in ‘papers and magazines, to Sudoku books, to the quizzles of doctor hiro-whateverhisnameis, to the computer games that the media and zealous parents bemoan alike, are active in training specific areas of the brain. Which areas? The regions involved in doing these activities. Now, there’s a surprise!

“Think of the brain as a muscle” – every neuroscientist since time immemorial

If you practice ‘til you’re a god at Sudoku, then that doesn’t mean you’re going to get any better at anything else – all you’ve done is to practice Sudoku and get good at that.

Baroness Greenfield (ex-head of the Royal Institution – the one that does the Christmas Lectures) was met with adulation from the Press, and grief from the scientists, when she warned that computer games ‘change teenagers’ brains’.

Well, of course they bloody do!

If you learn something, your brain changes, in order to incorporate that information. All she was saying, in actual fact, was that when teenagers learn how to play computer games, they learn how to play computer games. Gah.....

But this doesn’t mean they’re turning into zombies, who are going to start eating each other and eventually come for you. {Yes – i know that’s an exaggeration}

Ben Goldacre on the barmy claims of Greenfield and her apparent-cohort:

There are potential dangers to such activities, however – they tend to be highly habit-forming (read: “addictive”).

Having grown up in an electronic age, and having been a slightly-OCD and more-so Generalised Anxiety-y child, i am well aware of how easy it is to grow emotionally attached to one game in particular, and experience weird withdrawal symptoms when it’s unavailable i.e. i ran out of batteries. Darn! In that case, i got a chance to recover self-control; later, with PC games, no such opportunity. Pure mental muscle was necessary.

I make myself sound a real ‘case’, don’t i – except i know i’m not.

People become behaviourally dependent on all kinds of things. I remember seeing a programme on Channel 4, about a woman who’d become dependent on coffee enemas! If you’re familiar with the UK’s TV, i expect you’ll have worked out that that programme’s ‘Embarrassing Bodies’. (Enemas are much-touted by medical pseudoscientists – please don’t do them, unless under proper medical advice/supervision). The woman in question developed a several-times-a-day habit, which risked her health, let alone her social life.

I know this case is extreme, but it is the intention of the marketers to get you ‘hooked’, because you are the source of their revenue stream. Over the decades, computer games of all kinds have advanced in their ability to capture our attention, and keep it under lock and key for extended periods of time. We, as the user, would say “they’re so much fun – so absorbing”. Well, we would if we spoke in adspeak :-P

But yes – absorbing – too absorbing, if you ask me.

Then again, i am probably the kind of person who should avoid them, LOL.

Brain-training companies are probably the most nefarious when it comes to engendering destructive habits. They don’t just feed you fun stuff that you can’t put down – they actively advise you to pick it up and pick it up again, and again, and again, for the rest of your life, potentially. You don’t want to get unfit, now, do you? Then keep playing!

And with that, i think it’s time to get stuck into Lumosity. I’ve been waiting for months, to do this [wrings hands and wipes sweat from brow]

Lumosity plagues my viewing, on YouTube – and i do a lot of it - so their ads really piss me off!

What is it?

A monthly subscription web-site, which you get a free trial for.This means you get hooked, and then you pay out for the rest of your life, or until you recover your self-control.

And doesn’t it make them money!

“Lumosity Raises $31.5M From Discovery Communications For Brain Fitness Games... This brings Lumosity’s total funding to over $70 million to date.”

“Nearly every page of the site contains health "tips" that encourage users to train their brains and train them often. "Did you know?" the site asks rhetorically before each tip.  "Did you know?  The ACTIVE study, funded by the NIH and involving 2832 adults, found that some benefits of cognitive training can last over five years after the initial training." The implication here is clear: train with Lumosity for life-long health benefits.”

“Players' user and performance data is rigorously tracked by Lumosity.  This data is then utilized for the company's own aggressive targeted advertising, as well as sold to various undisclosed third parties.  It's right there in the Privacy Policy.”

“Thanks to the Children's Online Privacy Protect Act, websites are strictly limited in what kinds of data they can collect from children, and the FTC has become more involved recently in fining children's app developers for violating these privacy laws.  Because of this, Lumosity specifically notes that "the Site and the Software are not designed for or directed at children; the subject matter of the Site is not designed for or directed at children; and the content, including any video or audio, on the Site is not designed for or directed at children." But there is a fair amount of doublespeak involved on this point.  At the same time that the Privacy Policy makes explicit the fact that children should not use the site, the site has "Scholar" training programs that are designed for use by "students."  And in the sparse scientific data presented in their "Science Behind Lumosity" the Lumosity shows to substantiate it's claims to efficacy, middle-school aged children were the demographic that their studies tested.  So the unwary parent should take note that despite any appearances to the contrary, Lumosity's all-encompassing data collection practices that make this educational gaming site off-limits to the under-18 crowd.”

“If you're serious about fitness, you are probably better off running on a treadmill instead of hula hooping on WiiFit, and if you're serious about keeping your mind sharp, you'll benefit just as much, if not more, from reading a book, learning or practicing a foreign language, doing a few math problems each day, or playing your favorite casual puzzle game, be it a crossword puzzle or Tetris.”

More, about their dodgy private-data policies; this one dating back to 2009, when the company was only 2 years old:

“I’m not sure what to make of your response. I would like you to delete my user account. Can you please do that?...”
“I’m sorry, but it’s not pos­sible for us to com­pletely purge your infor­ma­tion from our sys­tem...”
“This got me ner­vous. Why would anyone con­struct a web­site where the staff can’t purge user accounts? After all, the signup terms state that user accounts can be deleted if they are imper­son­a­tors or minors. Cle­arly, Lumos Labs were lying to me.”

Such a seemingly-malignant organisation is bound to incur plenty of complaints... Oh, it does:
This one’s my favourite:
“I signed up for Lumosity just to see how much it costs etc... I got an email 3 days later showing how much I've improved after my first lesson, showing my BPI was in the 92nd percentile, with bars graphs for Speed, Memory, Attention, Flexibility, and Problem Solving. I never went through ANY lesson. This information is all made up and purely fictional.”

Businesses like Lumosity are in the vein of classic pseudoscience – the spam mail Viagra for insecure/unhealthy men, or the extreme-diet fads for insecure/unhealthy women – they offer a simple, easy get-out-clause from complex, difficult problems.
“Lose weight with this one tip”... “Your penis could be bigger”... “Beauticians hate this woman”... “Architects hate this amphibian”.... all that bullshit.
In reality, difficult problems have difficult solutions. Tempting people that are in difficult situations, with easy ways out, is abusive, in my book!

Personal gripes:

1) Graphs with no labels on them!!!

2) Irrelevant equations and diagrams that vaguely allude to intelligence
[full image]
3) Specs drawn on, because specs 'make people clever' [grrrrrrr]
[full image]
4) Pictures of brains all over the place
[full image]

5) Pictures of light-bulbs - so superlatively cheesy
[full image]

6) The 'ting' at the beginning, which heralds the agony of the ad
7) Infuriatingly anti-scientific statements: “ is based on neuroscience”; “it just seems like games, but it’s serious brain-training”; “improving your performance with the science of neuroplasticity”; “Regardless of your age, Lumosity can make you smarter and more mentally fit.”; “I can tell a big difference – decisions come quicker, i’m more productive – it’s serious brain-training, it just feels like games”. Outright lies!
8)  An economics-style graph of ‘performance’, deliberately formulated to deceive by exaggeration – look at the axes!
[full image]

9)  The playlist of real science videos, to make their non-science seem legitimate
[full image]
10)  They quote Indre Viskontas, who is a real neuroscientist... and Skeptic! I wonder what else she said, that they cut out. Or maybe she sold out – i hope not!

All the versions of their videos, on YouTube – with ratings and comments disabled on many:
Ratings disabled, you say, Tap? Why ever would they do that...
Obscene comments? ....Actually, i’m not surprised!

Oh, and you know pseudoscientists always have a list of anecdotes as long as their arm, right? Well, here’s theirs:

Does anybody else think it's dangerous to suggest that critical professions can be enhanced by wasting time on ‘brain-training’ regimes?

The best way to get better at being a fire-fighter, is by paying attention in training sessions, and thinking a lot about how to do your job. Not by thinking about racing penguins!

But these guys are trying to be real, proper scientists, so they’ve cobbled together some articles that look like good science.

In fact, they’re farcical. Remember this: all of these are from Lumosity’s ‘Completed Research Behind Lumosity’ section, on their web-site: They purport all of this as validating their claims...

‘Study shows Lumosity training increases frontal lobe function’
 “the analysis did not include a control group”
You can’t make valid claims about an intervention’s effect, if you can’t compare it to a dummy scenario! Cancer patients get better. Does Lumosity accelerate that recovery? I’m guessing not, or they would have done this study properly.

‘Your ageing brain’
This ‘study’ shows us that practice makes people get better at the tasks they’re doing! [shocker]
It also shows us that people who start with higher scores can’t improve those scores as much as people who started with lower scores e.g. some older people. Ergo, brain training gives older people the brains of 20-somethings! Either that, or they just got good at the task, without any benefit to any other activity in their life...

‘Executive Function and Emotional Regulation: A Love Story’
“while it’s too early to say that cognitive and emotional processing training can help you be better at relationships and personal interactions,  the link between emotional well-being and certain skills that Lumosity targets—including attention, executive function, and working memory—makes this a fascinating topic for further study”
So... something to say, but nothing that specifically supports Lumosity’s expensive software...

‘Lumosity improves sustained attention in study of mild cognitive impairment’
This study (no inverted commas – apparently, this is a proper one) has a sample size of 16, and a control group! (of unknown number). Both groups were subject to treatment; only one to Lumosity.
The results, however, are puzzling. The control group, which received treatment, got worse.
And they got as much worse as the intervention group got better. This is such a small study, that we must put this result down to statistical fuzz – it’s not a real result.
“Mild cognitive impairment, which is associated with an increased risk of dementia...”
Dementia... dementia... i’m sure i’ve mentioned that, at some point, in this mini-essay...

‘Lumosity Training Can Enhance Brain Function and Math Skills, According to Stanford Study’
“Dr. Kesler and colleagues found that the course can improve cognition and math skills in girls with Turner’s syndrome... Participants exercised with Lumosity Math Tutor for 20 minutes a day, 5 days a week, over the course of 6 weeks.”
NEWSFLASH, NEWSFLASH!! Children get smarter as they grow up!! NEWSFLASH, NEWSFLASH!!
This reminds me of a non-study that Ben Goldacre wrote about, in which an entire school was given fish-oil pills, and the upward incline in test scores was attributed; not to the fact that they’re kids at school who are learning things; but to the fish-oil. Facepalm time!
P.S. if you skipped it, check back to my excerpts of Rachel Ponce’s article, in which she notes that Lumosity is forbidden from marketing to children due to their sloppy privacy behaviours. But clearly, Lumosity intends them as a market - why else would they test it on kids?

‘LEAP Results Are Out’
Yet another useless study, demonstrating that when you practice something, you get better at it. So why waste time with Lumosity when you could be practicing something that actually happens, in the real world?

‘Lumosity Cognitive Enhancement Research Published in Mensa’
“This study goes above and beyond others of its kind in building a persuasive case for cognitive training for the general population.”
Does it?
“The implications of this study are clear and compelling: Lumosity training can improve core underlying mental abilities, abilities that transfer to myriad aspects of our everyday lives”
A claim of an easy, simple solution, for a complex, difficult problem, writ right there.

From the actual publication, in Mensa Research Journal (which is not a scientific journal – at least, it’s not listed on
“All patients were mentally and physically healthy as determined by a short e-mail questionnaire”.
Erm... yeah. That’ll do. No GP’s report necessary. Geez...

14 were Lumosited; 9 were left to rot. This is, again, a tiny sample size, massively prone to false positive results. Considering that this ‘study’ (yep – the inverted commas are back) was done over the web, i would expect more people to be involved.

The motivation for pseudoscientists to do tiny studies is that they are cheap, easy, and repeatable – if you do one big study that gets a negative result, then tough – if you do ten small studies and one gets a positive result, ignore the other nine, and harp on about the one positive result ‘til raptor Jesus returns on his pink unicorn.
The methodology goes into massive technical detail, where a simple diagram would easily explain what happened. {Who do these people think they are – engineers?}. The general gist is that they played games involving orientation (spatial, not sexual) and short-term memory.
There was no attempt to distinguish between enhanced memory and spatial ability, and just enhanced concentration in an environment they had become familiarised with i.e. the Lumosity software. A valid control group would have been given benign games to play, rather than nothing at all. Being given nothing would have led to them feeling dejected, resulting in arrested enthusiasm for the study, and consequent poor results, regardless of ability. The only participant who dropped out was in, you guessed it, the boring old control group who didn’t have anything to do. Of course, there’s no blinding – they know exactly which group they’re in, and hence that they shouldn’t bother, if they’re not getting the intervention.
And now onto the nitty gritty....

They blow up the data into nice-looking graphs, for the ‘forward spatial working memory’ and the ‘reverse spatial working memory’, but not for the ‘divided visual attention’ and the ‘letter memory’, which they also tested. I wonder why...

Actually, i don’t have to. The trained group improved more in the ‘forward SWM’ and ‘reverse SWM’, but the control group (who had absolutely no practice, remember) improved more at ‘letter memory’, and were the only people to improve at ‘divided visual attention’ (the trained group actually got worse!)

Conclusion: this was a shit study, which they attempted to make the most of

Then there’re posters and cartoons, that don’t even present the semblance of research, e.g.:

So there.... i’ve got that of my chest.

And now to prepare my nerves for the next time i hear that bloody ‘ting!’ before a YouTube video....

Good luck everyone!

No comments:

Post a Comment