Tell me more ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.
up vote 5 down vote favorite
1

For those who have never heard of a Children's Museum before, there is a national association in the US with some information. The basic idea stems from Vygotsky-like paradigms of learning through play.

My first impulse was: write a GUI that presents a series of standard cognitive tests in a fun way, put the software on a computer in a kiosk during the museum events. Finally, compare child performance on the test to the number of visits to children's museum.

Of course, there's some considerations: if somebody takes the test multiple times, they're probably going to get better at it, so this might give a false-positive (rising with the number of visits). So, for instance, only the first test taken by each person would be considered for analysis.

But that is a rather inexperienced shot in the dark. Any other ideas?

share|improve this question
I don't have any psychological insights into this, so I'll leave it as a comment, but you'd probably be better off letting the kids do the test at home (the parents will probably end up trying to help the kids answer whether in the museum of not, so you have that confound). Provide some sort of a "badge"/"passport" that has a unique identifier on it which the children can take to each exhibit and have scanned and stamped for each visit, and then have them provide that identifier online to take the test. Once they have a complete card, they can turn it in for a t-shirt or other token gift. – Chuck Sherrington Jul 4 '12 at 22:47
Interesting question, would you be able to give us a little more context? What is the museum intended to change (i.e., presumably knowledge of museum related content, and not general cognitive functioning)? How serious is the project? What kind of budget do you have? What is the purpose of the evaluation? – Jeromy Anglim♦ Jul 5 '12 at 4:22
Children's Museums have little to do with traditional museums. They are designed for cognitive development through non-intellectual delivery (it's targeted at children that are too young to hold focus o lecture-based learning). The major subjects for each exhibit are based on child development: roleplaying, motor skills, sensory learning. A little googling might help you understand better. It's a movement based on neuro/cog development that's been rising up recently (in the US at least). It is serious, the purpose is to bring the child museum funding. Budget unknown. – Xurtio Jul 5 '12 at 6:24
@Chuck - an electronic badge was my first thought, but I think that's over budget for now. A kiosk with a comptuer and a UI that I design myself isn't so far-fetched (especially if I say.. make it a PhD project) – Xurtio Jul 5 '12 at 6:30
@Xurtio I see, I didn't realize you had the constraints. Well, any incentives, however minor, would still keep parents coming back for repeated trials. – Chuck Sherrington Jul 5 '12 at 6:36

1 Answer

up vote 4 down vote accepted

General literature on training and educational program evaluation

In general, you would want to read up about program evaluation in general, and educational program evaluation, and training evaluation in particular. With regards to the educational context I found this article online. With regards to training evaluation, you could check out Kraiger et al (1993) or Sackett and Mullen (1993) or any number of other textbook treatments of training evaluation.

Specific comments on evaluating a "children's museum"

  • What design will you use?: is it going to be a pre-post design where you compare knowledge before and after exposure to the program? Are you going to have a control group who does not experience the program or experiences an alternative program? How will you allocate participants to conditions? random allocation is better.
  • Decide on the constructs of interest?: What is the program meant to improve? You speak about quite general learning goals, but training programs are much more likely to have more specific effects. E.g., if you have a program that teaching addition, then students are likely to get better at addition, but their general intelligence is likely to be unchanged. You may also want to think about the time frame that benefits are meant to persist after exposure to the program. You can also look at Kirkpatrick's framework of training evaluation which talks about different kinds of outcome variables.
  • What will measure your constructs of interest?: Are you going to use existing measures or develop your own measures? If the content is highly specific to your program, then you will probably need to develop your own test items.
  • Determine your sample: How are you going to recruit your sample? How many participants will you need to ensure reasonable statistical power?
  • When and how will measures be administered? I think you would want to do this in a controlled setting. A kiosk would probably be problematic from a measurement perspective.

Rough starting point

Of course there are a whole range of issues related to program evaluation, and dealing with children introduces many additional issues. The following might provide a rough starting point:

  • two groups: control and intervention; about 100 participants per group
  • three time points: before intervention, immediately after intervention, a few months after intervention
  • Use some general measures of ability related to the program as well as some highly specific measures that tie in closely to program content.
  • Also measure other factors like child and perhaps parent reactions to the program including both quantitative and qualitative evaluations.
  • Perhaps link in with some surrounding schools or day care facilities to recruit participants or use some form of waiting list system for admission to the program. Administer tests in a controlled environment (e.g., in a supervised room).

References

  • Kraiger, K., Ford, J.K. & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation.. Journal of applied Psychology, 78, 311. PDF

  • Sackett, P.R. & Mullen, E.J. (1993). Beyond formal experimental design: Towards an expanded view of the training evaluation process. Personnel Psychology, 46, 613-627. PDF

share|improve this answer
1  
Good point. I didn't consider external community effects, so I took the first time visitors to be the control group. I'll have to make this web-based so I can sample parents "blindly" from the local population (if I can get them to log in)... Your post deserves more time though, so once I have specific questions, I'll open a more focused thread. Thanks for the in-depth reply. – Xurtio Jul 5 '12 at 7:10
1  
@others I will change accepted answer given somebody with direct experience or knowledge in the particular setting (Children's Museums or child education) has a more focused answer, but Jeromy has provided all the basic tools I need to research and develop the idea. – Xurtio Jul 5 '12 at 7:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.