Hot answers tagged statistics
12
I'd suggest checking out the Linear Ballistic Accumulator (Donkin et al., 2011) model for a scenario like this. While LBA can be used to model any number of alternatives in a speeded choice task, to model signal detection you'd want to model just two accumulators, one for the "signal" response and one for the "no signal" response. With this scenario, ...
10
MANOVA is definitely a bad idea given that one dv is continuous and the other is binomial. After exploring a number of different approaches to combining RT and accuracy data, I've come to conclude that the best current approach is to use linear ballistic accumulator model (e.g., see Donkin et al 2011).
The LBA is a simple (structurally and computationally) ...
9
It's not a journal article, but it's a project, called Reproducibility Project.
They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.).
Here's the main site and here's the link to their spreadsheet.
This ...
8
In addition to Mike's suggestion, see the Ratcliff diffusion model and variants thereof. E.g.:
Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two–choice decisions. Psychological Science, 9, 347–356.
Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction ...
7
I'm guessing you are relating the Big 5 factors of personality to Meyer and Allen's three factor model of commitment (normative, continuance, affective).
You certainly could adopt the simple approach of creating observed composite scores for each variable and reporting the correlation matrix and a set of multiple regressions.
However, the language of your ...
7
A paper comparing the performance of Inverse Efficiency Scores and diffusion models for the quantification of RT and accuracy can be found here.
Rach et al. (2011) "On quantifying multisensory interaction effects in reaction time and detection rate " Psychological Research
Volume 75, Number 2, 77-94, DOI , PDF
7
Programs/packages for EEG analysis
There are decent MatLab toolboxes with good tutorials for for the analysis of EEG data. The EEGLAB toolbox (tutorial) can be operated by both GUI and command-line (and script). The fieldtrip toolbox (tutorial) is mainly operated by command line / script.
Of course there are also (commercial) software packages for EEG ...
7
Here's my basic overview of the psychology - statistics textbook market.
In general, I think the choice of textbook (where you have a choice) depends on a few factors:
Applied versus statistical theory (e.g., do you have immediate needs to analyse data)
Whether and which software package you want to use (e.g., SPSS, R, etc.)
Which techniques you want learn ...
7
There are a variety of models solving accuracy and RT that have been pretty well tested and LBA is probably fine (I haven't used it). If you don't want to go that far there is a rather simple way to analyze data controlling for SAT that has much better mathematical properties than IE scores (which, as Mike said were named by me, but offhandedly proposed by ...
6
You should probably also check out:
Pleskac & Busemeyer (2010). Two-stage dynamic signal detection: A theory of choice, decision time, and confidence. Psychological Review.
Also, I believe Busemeyer has a dynamic signal detection theory paper but I don't know that it has been published. The Pleskac & Busemeyer paper probably draws on this ...
6
I use the FieldTrip toolbox in Matlab to analyze my own modified auditory MMN experiment :) But I use MEG, so I don't have that many software options. The toolbox is very powerful but it has a steep learning curve and I would recommend it only if you already have both Matlab and EEG data analysis experience.
I don't analyze my data in the classical MMN way ...
6
I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural ...
6
Using Parameters Estimated from an Individual in a Group Analysis
In a way this is exactly what usually happens when we calculate the mean reaction time across all conditions for a group of participants. When we normally calculate mean reaction times we assume that some process (P) takes t milliseconds to complete plus some Gaussian distributed noise. We ...
5
To my knowledge, there is no adjusted RMSD.
RMSD, unlike $R^2$, isn't typically used to compare models across the literature. $R^2$ represents the proportion of variance explained by the model, a construct which translates well across different experimental designs. Adjusted $R^2$ distorts this by accounting for the number of parameters in your model, but ...
5
I think your intuitions are correct, and the reason is that Likert scales often suffer from poor construct validity.
One example that comes to mind is Alter et al. (2010), who use a mixed design, and note that participants in both conditions gravitate toward the middle of the scale on their first rating. They suggest that participants do not know how to use ...
5
I recently had similar problem and I used inverse efficiency (IE) scores. These scores
were derived by dividing the response times by correct response rates separately for each condition, carried out in such a way that the higher the score was, the worse was the performance. So you get something like "corrected reaction time" scores. Here is example of paper ...
5
Baayen's book, Analyzing Linguistic Data: A practical introduction to Statistics using R is a little more niche than possibly you'd want, but it does offer some coverage of mixed effects modelling, without which I'd be reticent to call any psych stats text "cutting edge".
5
Another possible approach is using EZ-diffusion model suggested by Wagenmaker, van der Mass and Grasman (2007). Quoting Brown & Heathcote (2008; p. 4):
This model is extremely simple, with just one source of variability in
evidence accumulation—within-trial randomness—and simple linear
accumulation (although evidence for one response does count ...
5
If you're trying to work out a standardized effect size in order to calculate power for your study then it doesn't matter whether any studies like yours have been done. It's important to find studies using the same dependent measure so that you have an estimate of variability but evidence of the size of prior effects is less important. If you have the ...
5
Answer based on your original depression example
Note that this answer was originally written based on your initial example, where you asked:
Assume, I have developed a new intervention for people with light
depression. I want to compare the effectiveness of this intervention
(E) with an existing intervention (C). For this, I recruit test
subjects ...
4
Reading list
As @Jeff has mentioned Tom Griffiths has several useful resources. In particular Tom Griffiths has an extensive reading list that you might find relevant. To quote the summary of the content:
This list is intended to introduce some of the tools of Bayesian
statistics and machine learning that can be useful to computational
research in ...
4
+1 to Speldosa's suggestion. Griffiths and colleagues have written several primers on the use of Bayesian models in cogsci. Many of them can be found on Griffiths' website under 'Foundations':
http://cocosci.berkeley.edu/publications.php?topic=Foundations
e.g.
Perfors, A., Tenenbaum, J.B., Griffiths, T. L., & Xu, F. (2011). A
tutorial ...
4
The following reviews some of the articles that I found discussing and implementing SEM meta-analysis to examine mediation.
Cheung and Chan (2005)
The authors distinguish three approaches of meta-analytic structural equation modelling (MASEM).
Univariate two-stage MASEM: This includes a collection of two step approaches (note that Cheung and Chan call ...
4
I don't know if you can completely remove the selection bias from such a sample.
You seem to be referring (in some sense, at least) to regression towards the mean. This is one of the biggest (if not the biggest) problem with much research on clinical samples.
What I would do (assuming resources) would be to take the same size (or larger) sample from the ...
4
The purpose of reviewing the literature for effect sizes is to form an estimate of what effect size you might expect in your present study.
Existing meta-analysis: The principles and techniques of meta-analysis provide a good starting point for generating a predicted effect size.
If a meta-analysis has already been conducted, then the estimated population ...
3
Have you read this:
Fishbein, M., Middlestadt, S. (1995) Noncognitive Effects on Attitude Formation and Change: Fact or Artifact? Journal of Consumer Psychology, 4(2),181-202. [DOI]
Direct quote from page 187:
Note that the psychology of the double negative is an essential part of an expectancy-value formulation (Ajzen & Fishbein, 1980; Fishbein, ...
3
There are at least 3 ways to discount SSE (or RMSE) by the number of free params:
$$
\text{adjusted RMSE} = \sqrt{\frac{SSE}{n - k}}
$$
$$
AIC = n \times ln\left(\frac{SSE}{n}\right) - k \times ln(n)
$$
$$
BIC = n \times ln\left(\frac{SSE}{n}\right) - 2 \times k
$$
or in computer code style:
k = number of free params
n = number of DV's
SSE = sum of ...
3
Let's start with signal detection methods. The big one in memory literature is D-Prime analysis (hits versus false alarms). If the subject chooses X more times than Y, you can compare the number of times choice X is made correctly versus chosen incorrectly (as a false alarm). If that was your paradigm, then choice Y would be the correct rejection (if ...
3
In the fairly recent book "The Cambridge Handbook of Computational Psychology", chapter three is devoted to bayesian modeling. It's written by Thomas Griffiths, Charles Kemp, and Joshua Tenenbaum.
I haven't read this chapter yet myself but will update this answer when I have.
3
A rating scale is not necessarily a Likert scale. It sounds like you have a single rating for each picture across each dimension but I could be wrong. More details would be needed for better advice and Ana's comment should be answered in your question.
You would be best off doing a multi-level cumulative link model (multi-level ordinal regression). You ...
Only top voted, non community-wiki answers of a minimum length are eligible
