Take the 2-minute tour ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

Background

There is a test called Lumosity's Brain Performance Index. A sample profile of scores might be:

Speed           680
Memory          707
Attention       277
Flexibility     304
Problem Solving 900
Overall BPI     573

I'm interested in how to interpret a profile of scores.

Questions

  • What validity information is available for the BPI?
  • What normative information is available for BPI? In other words, how could the sample profile of scores be interpreted both absolutely and relative to other groups of people?
share|improve this question
3  
Lumosity's Brain Perfomance Index is not a scientifically-valid measure of mental performance. If you want a valid test, find a testing center and perform an IQ test: en.wikipedia.org/wiki/Intelligence_quotient. From the Wikipedia article on Mental Retardation: "Mental Retardation has historically been defined as an Intelligence Quotient of under 70." en.wikipedia.org/wiki/Mental_retardation –  BenCole Oct 1 '12 at 20:38
1  
So Lumosity.com isn't a valid service? They state "Lumosity.com is advised by leading experts in neuroscience and cognitive psychology from Stanford, Berkeley, UCSF and other prestigious institutions" –  40Plot Oct 1 '12 at 20:51
1  
Ah, my apologies, I didn't intend to imply that (I should have said it isn't a valid measure of general mental performance). Lumosity is valid in that they score your progress in such a way that improvement can be tracked. However, from my knowledge, they track performance for specific mental abilities/tasks, rather than specifically testing for general intelligence. Either way, their scores do not equate with IQ scores, although I suspect they may measure similar qualities. –  BenCole Oct 1 '12 at 21:10
1  
I'm going to be fairly short in my reply to "How valid is Luminosity's Brain Performance Index and what normative information is available?" (although it is lumosity...i only know this because i searched for luminosity.com as well) and just say although it can assist with sharpening ones skills and ability to use a computer, it's not necessarily going to truly make one smarter...which after stated i realized they never really say it does that. Regardless, if someone is inept in using a pc yet continues to use lumosity.com of course their scores are going to increase....they're getting more use –  user2536 Dec 20 '12 at 5:00
2  
I would posit the only assurance you have when participating in this type of commercial partnership; you have exchanged your coin for the opportunity to have your data mined by yet another ad agency masquerading as pseudo science. –  user2641 Jan 24 '13 at 21:37

2 Answers 2

up vote 25 down vote accepted

General thoughts on brain training: Lumosity is a commercial tool that aims to improve brain functioning. In general, I am sceptical of the potential for "brain training programs" to improve cognitive functioning in a generalised way (e.g., see this Nature discussion). Practice is powerful, but tends to be domain specific. So if you want to become skilled in brain training tasks, then do brain training exercises, but if you want to become skilled in a particular domain of life, then study and get experience in that domain. While I can see the value in researching the topic, I can also see the potential for sellers of "brain training" programs to exploit people's concerns about their mental shortcomings or about fears of mental deterioration with ageing.

Normative data: From what I can gather from this review, Lumosity has lots of normative data based on existing users which you can use to compare your performance. However, I imagine this is proprietary information that they would be reluctant to publish in a useful form.

There was an unpublished report I found by Cruz et al based on a small sample of young adults. The following summarises Table 2:

Number of days        4.75 ± 3.18
Number of sessions    10.79 ± 9.31
Median (IQ R )        9.00 (2.25‐17.75)
Minimum‐Maximum       0 ‐ 34
Overall BPI           676.93 ± 323.94
Lumosity points     173.43 ± 158.89

Validity of the BPI: The website writes:

Your Overall BPI is your average BPI across each of the four cognitive areas: attention, memory, processing speed and cognitive control.

It also states

"We then evaluate your game scores and use a proprietary algorithm to derive your BPI"

Performance on almost all cognitive tasks are intercorrelated to some extent, and if you take the average, or preferably the first principal component, of a battery of cognitive tasks, it will tend to have a general factor, which will have some loading on g. Of course, if the tests are of a particular type, this might lead to different outcomes.

I could not find any validity data on the website whereby BPI scores are correlated with other measures. Thus, it is unclear what relationship BPI has with other ability measures, or IQ for example.

In general, practice effects on cognitive tests are seen as a source of error variance. What a test measures after people have taken the test many times may be very different. In particular, the degree to which the test correlates with a domain general quality of interest may be reduced. Instead, the test may start to reflect a domain specific adaptation. Also, if people differ in the amount of practice, this would compound the measure of a domain general ability.

Overall thoughts: In general, the target market for Lumosity's product seems to be consumers. In contrast, tests like the WAIS or the CANTAB have a target market which is researching and applied psychologists. In the consumer market, it is typical for companies to use "proprietary algorithms" and for there not to be a test manual with extensive validity data. While this might make commercial sense, it limits the scientific value of such instruments.

References

  • Cruz, A., Canelas, A., Machado, F., Pedreira, J., Barroso, J., Beir\~ao, L., Freitas, L., Belo, L., Seabra, M., Portero, M. & others (). Brain Fitness--Clinical Trial to Measure the Effect of Cognitive Training in Brain’s Capabilities. PDF
share|improve this answer

Because it has been a few years since Jeromy's original answer, and because I just read a very apt article, I will venture an update on the state of the field with respect to the BPI's validity. Overall, despite more research into brain training and Lumosity, there is little to no peer-reviewed evidence supporting the Lumosity BPI's validity, nor evidence that an average consumer should expect practical cognitive improvements from Lumosity games.

Testing the BPI

A very recent test of the BPI's validity was published by Shute, Ventura and Fe (2015), who compared participants practicing Portal 2 with participants practicing Lumosity games; the Lumosity players served as active controls. Portal 2 measures included (1) total number of levels completed, (2) average number of portals shot, and (3) average time to complete levels; Lumosity was measured by BPI. They reported no evidence of transfer effects for measures of either problem solving or spatial abilities in the Lumosity condition, and partial support for transfer in the Portal 2 condition only on spatial abilities.

Hypotheses 2 and 3 test pretest to posttest gains on specific problem solving and spatial test scores within each condition. We computed paired t-tests for each measure in each condition. For hypothesis 2 (problem solving gains), we found no significant gains for the Portal 2 condition, from pretest to posttest, for any specific problem solving measure. We also did not find any significant pretest-to-posttest gains for the Lumosity condition across any of the problem solving measures. Results provide partial support for hypothesis 3 (spatial gains). Participants in the Portal 2 condition showed pretest to posttest improvement on the MRT, t(1, 39) = −1.80, p < .05; Cohen's d = .30 under a one-tailed test. Additionally, Portal 2 players' VSNA scores significantly differed from pretest to posttest, t(1, 40) = 2.42, p < .05; Cohen's d = .44. There was no significant improvement for Portal 2 players on the SOT test. For participants in the Lumosity condition, there were no significant pretest-to-posttest improvements on any of the three spatial tests.

Brain training

A rigorous study by Redick et al. (2013) suggested that the initial research on brain training generally, and thus the basis for Lumosity (mentioned by name in the paper), may have been compromised by design limitations and lack of theoretical grounding, and criticizes those studies for struggling to adequately explain often mixed results. They reported three relevant findings from their own validation study of dual n-back working memory training:

Our study yielded three main findings. First, subjects improved with practice on both the dual n-back and visual search tasks. Second, training group subjects showed no transfer to any of the ability measures, in keeping with the prediction outlined in Figure 4D. Third, dual n-back trained subjects reported subjective improvements in various aspects of cognition in the absence of any objective evidence for change.

Concluding remarks

The jury is still out on brain training more generally and on when, where and whether it may be useful (e.g., Deveau et al., 2014; Mayas, Parmentier, Andrés and Ballesteros, 2014). However, the Lumosity BPI appears to have little to no scientific validity as a cognitive training method for average consumers.

References

share|improve this answer

protected by Community Jan 25 '13 at 0:16

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.