10
$\begingroup$

I am interested in how open science could be done in psychology. Are there good web-based tools that could house and share a study? When I say "study" I mean:

  • Lab notes from collecting data
  • Raw data
  • Code for analyses
  • Results
  • Discussion of research process and results (ideally open to the community)
  • Posters, Presentations, Publications

Right now all I have seen done (and my best idea) would be a combination of a Wiki for the lab notebook, cloud storage (e.g., Google drive) for the data, github (or similar) for the code, blogs for discussion, and a personal website for archived versions of journal articles, posters, and presentations.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Joshua, welcome to the site! You've raised some great questions, but as you suspected, this is a bit of a detour from the Q+A format on the Exchange. Fear not, you can edit your question as much as need be. Try to focus this down to 1-2 very specific issues that can be answered by someone knowledgeable in the field. "Discussions" as you might have in a traditional forum, where everyone shares his/her personal anecdotes, are largely discouraged. As you reformulate it, we can then retag as necessary. $\endgroup$
    – Chuck Sherrington
    Jul 8, 2012 at 22:38
  • $\begingroup$ @ChuckSherrington Thanks for the advice. Hopefully this is a bit better (although I strongly suspect the answer to the question formulated now is, "No"). Maybe I do not have a good question for the Exchange. $\endgroup$
    – Joshua
    Jul 8, 2012 at 23:08
  • 1
    $\begingroup$ This is great, actually. Hang on to some of those other ideas to ask in other questions, too. $\endgroup$
    – Chuck Sherrington
    Jul 8, 2012 at 23:10
  • 3
    $\begingroup$ Welcome to the site, @Joshua! Some of these questions/answers might give you starting directions: collab tools, data organization, & lab wikis. $\endgroup$
    – Artem Kaznatcheev
    Jul 9, 2012 at 0:55
  • $\begingroup$ Another related question from Cross Validated. $\endgroup$
    – chl
    Jul 9, 2012 at 11:19

5 Answers 5

Reset to default
7
$\begingroup$

The Open Science Framework will do some of what you ask for. Additionally, it will allow you to preregister your hypotheses to properly distinguish confirmatory and exploratory research.

Features (quoted from the homepage).

  • Document and archive studies
  • Share and find materials, scripts, data
  • Detail individual contributions
  • Increase transparency
  • Time-stamp materials
Improve this answer
$\endgroup$
1
  • $\begingroup$ The preregistered hypotheses is pretty cool. Actually, I really like the whole idea. This is pretty much exactly what I was hoping for (if not quite the scale I hoped for). $\endgroup$
    – Joshua
    Jul 13, 2012 at 17:00
5
$\begingroup$

I like your question. But I have to point out one thing - if you were to use documenting tools for all those tasks that you mentioned, you wouldn't have that much time left for your research. Sharing is great, but it takes time to do it properly, even with great tools.

I tried quite a few tools for documenting research, with an idea to boost my productivity, increase amount of recorded information and share the information for peers criticism. At the end of the day, only a tiny handful of those tools survived. The ones that survived were the ones that smoothly integrated with my daily routine, mainly in terms of time efficiency and interface simplicity.

For the lab notes, quick thoughts, brutal sketches, small brainstorms and rapid conference scribbles, the winner is a good quality paper notebook. I am personally a fan of Paperchase A5 brown recycled paper jorunal with wiro bound, which I always use with thin black ink Pilot pen. I know, OCD, but in a good way. Brown paper gives this vintage quality to my notes, which is somehow stimulating for me, and I like to come back to it and store it. I tried alternatives, but they could never beat brown paper wiro bound. Livescribe pen was too big, with ugly ink, and lacking smoothness of software exporting and it ended up being only a voice recorder, rarely tho. iPad note taking apps like Noteshelf or Penultimate with Pogo stylus really slowed me down, ugly writing again, and the battery always died in the middle of important seminar.

I frequently make a snapshots of my paper notebook with Evernote using iPhone. I actually make a snapshots of many other things using Evernote and this is definitely very simple and great storage/sharing platform, that works smoothly between desktop and mobile.

For the raw data - Dropbox. Now, also Google Drive, as you pointed, but Dropbox was first, with cloud-based, super-simple set-up storage solution. Dropbox was a big thing in my documenting toolbox as it removed the need for USB dongles. I could set up my experiment to save all the participant's raw data automatically in Dropbox after the experiment. I came back home - it was there waiting for me. When we collaborated, all the data was nicely shared in collective Dropbox folders without the need to shuffle e-mails or set up transfers with raw data.

Mendeley is a great tool for both bibliography management and sharing. Fantastic idea that turned out really big and beats the shit out of EndNote. Major thing I love in Mendeley is the ability to automate creation of bibliographies for LaTeX. This is an ultimate time saver.

Regarding posters there is this new poster repository F1000 which I haven't tried, but it looks promising.

For analysis code I found it tricky. Yes, you can put code in Github, but you can't compile it there. So there is this extra step when you have to organize your code on Github, describe it, etc. Never tried it because I prefer to spend time to learn more coding, but I don't say I won't try it in the future, when I will feel I have more valuable code to share with community.

For presentation I played with Prezi for some time which has a cool option to share your presentations easily. But ultimately I abandoned it, after people complained about motion sickness and after I felt I lost some key simplicity factor. I am having my eye on Beamer but there is no time now, so Keynote has to suffice.

Regarding discussion, chat, intelligent monologue - again it depends on what you want to achieve with it. Share a coding trick in analyzing some data? Discuss some exciting data? Or write a page about your documenting toolbox approaches? In all cases it requires time, so it's good to make sure that you don't do it to avoid doing something more important. Tools for online discussion and blogging are numerous. I like it here in Stack Exchange, because it's purely community driven and it's very easy to put together an overlong piece of rant on many many topics ;-)

Improve this answer
$\endgroup$
3
  • $\begingroup$ does Evernote do optical character recognition? I.e. can I do ctrl-fs on the accumulated pictures to find some of the text inside them? Or do you have to tag every snapshot of your notebook manually? $\endgroup$
    – Artem Kaznatcheev
    Jul 10, 2012 at 15:55
  • $\begingroup$ Good thoughts. I think for larger studies, there often has to be documentation anyway (to standardize interventions, etc. across different staff working on the project or multiple sites). Part of where I envisioned this ultimately going would be projects like that, not per se a little study I run by myself. Beamer is very nice, but you might also check out HTML5 slides. I have seen some very cool stuff down with markdown (which is relatively easy) for both HTML and creating PDFs. $\endgroup$
    – Joshua
    Jul 10, 2012 at 18:04
  • $\begingroup$ @ArtemKaznatcheev: Yes, you can do it, but you need premium account for this feature. $\endgroup$
    – Geek On Acid
    Jul 10, 2012 at 19:10
4
$\begingroup$

Figshare is one option for archiving assorted research artefacts. To quote the "about page":

figshare allows researchers to publish all of their research outputs in seconds in an easily citable, sharable and discoverable manner. All file formats can be published, including videos and datasets that are often demoted to the supplemental materials section in current publishing models. By opening up the peer review process, researchers can easily publish null results, avoiding the file drawer effect and helping to make scientific research more efficient. figshare uses creative commons licensing to allow frictionless sharing of research data whilst allowing users to maintain their ownership. figshare gives users unlimited public space and 1GB of private storage space for free.

A few things that I like about figshare:

  • Creative commons licence
  • They have some form of archive policy which says that if they go out of business, the content will be backed up by another provider. However, it's not quite as open as stack exchange in that they don't make public backups of the entire site available.
  • Objects get a citable doi
  • View and share statistics for shared objects
Improve this answer
$\endgroup$
0
2
$\begingroup$

The site http://psychfiledrawer.org/ also addresses some aspects of what you are asking. It is mainly geared toward maintaining a public repository of replication attempts of psych experiments.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I have seen that. It is a good idea that hopefully catches on more. $\endgroup$
    – Joshua
    Aug 2, 2012 at 1:42
1
$\begingroup$

A lot of labs I know use Basecamp. It does cost money, but provides a centralized place to store all the information you listed and allows for comments on all of it, as well as a calendar to coordinate your team.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.