Take the 2-minute tour ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

I consider reproducible analysis to be really important. In particular, I'd like to see more researchers in psychology:

  • sharing their data
  • producing their journal articles using literate programming technologies like Sweave and knitr that combine data analysis code and text into an integrated documenst.

I consider this particularly important in light of concerns over research fraud (e.g., data fabrication) and various forms of data fishing (e.g., testing many hypotheses and only reporting the significant one).

Certainly in statistical fields, there is a lot of discussion about the importance of reproducible analyses (e.g., see here, and here). However, I sometimes feel that in psychology this is only starting to filter through.

Thus, my question is:

  • Are there any journal articles in psychology that have advocated reproducible analysis?

There are many angles that such articles might take: reward structures that discourage open science and reproducible analysis; technical challenges which make it difficult to do such analyses; how-to articles that outline how to perform reproducible analyses in psychology. I'd be interested in any articles that address these topics.

share|improve this question
I don't own a copy of the APA manual myself, so I couldn't look this up, but this is a quote from a paper by Bakker & Wicherts (2011), The (mis)reporting of statistical results in psychology journals: "...since standard 8.14 of the ethical standards of the American Psychological Association (2010) states that data should be shared after research results are published...". I thought this might interested you. –  Mien Jun 18 '12 at 21:31

1 Answer 1

up vote 9 down vote accepted

It's not a journal article, but it's a project, called Reproducibility Project. They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.). Here's the main site and here's the link to their spreadsheet.

This is a short article promoting the sharing of data in psychology. Another related study (by the same author) is also mentioned.

share|improve this answer
It is PLoS (Public Library of Science) and it is not a journal, but it publishes several journals that have names that start with PLoS. None of these journals are psychology specific. Further, these journals don't "claim to be 'open-access'". They basically defined what it means to be open-access. However, I have not heard of them trying to promote reproducible data analysis, especially in psychology (they concentrate on bio and medicine mostly, although PLoS One does publish some other things); I don't see how your last paragraph has relevance. –  Artem Kaznatcheev May 17 '12 at 21:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.