# Tag Info

12

I'd suggest checking out the Linear Ballistic Accumulator (Donkin et al., 2011) model for a scenario like this. While LBA can be used to model any number of alternatives in a speeded choice task, to model signal detection you'd want to model just two accumulators, one for the "signal" response and one for the "no signal" response. With this scenario, ...

11

MANOVA is definitely a bad idea given that one dv is continuous and the other is binomial. After exploring a number of different approaches to combining RT and accuracy data, I've come to conclude that the best current approach is to use linear ballistic accumulator model (e.g., see Donkin et al 2011). The LBA is a simple (structurally and computationally) ...

10

A paper comparing the performance of Inverse Efficiency Scores and diffusion models for the quantification of RT and accuracy can be found here. Rach et al. (2011) "On quantifying multisensory interaction effects in reaction time and detection rate " Psychological Research Volume 75, Number 2, 77-94, DOI , PDF

9

It's not a journal article, but it's a project, called Reproducibility Project. They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.). Here's the main site and here's the link to their spreadsheet. This ...

9

There are a variety of models solving accuracy and RT that have been pretty well tested and LBA is probably fine (I haven't used it). If you don't want to go that far there is a rather simple way to analyze data controlling for SAT that has much better mathematical properties than IE scores (which, as Mike said were named by me, but offhandedly proposed by ...

8

In addition to Mike's suggestion, see the Ratcliff diffusion model and variants thereof. E.g.: Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two–choice decisions. Psychological Science, 9, 347–356. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction ...

8

Here's my basic overview of the psychology - statistics textbook market. In general, I think the choice of textbook (where you have a choice) depends on a few factors: Applied versus statistical theory (e.g., do you have immediate needs to analyse data) Whether and which software package you want to use (e.g., SPSS, R, etc.) Which techniques you want learn ...

7

I'm guessing you are relating the Big 5 factors of personality to Meyer and Allen's three factor model of commitment (normative, continuance, affective). You certainly could adopt the simple approach of creating observed composite scores for each variable and reporting the correlation matrix and a set of multiple regressions. However, the language of your ...

7

Programs/packages for EEG analysis There are decent MatLab toolboxes with good tutorials for for the analysis of EEG data. The EEGLAB toolbox (tutorial) can be operated by both GUI and command-line (and script). The fieldtrip toolbox (tutorial) is mainly operated by command line / script. Of course there are also (commercial) software packages for EEG ...

7

I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural ...

7

Answer based on your original depression example Note that this answer was originally written based on your initial example, where you asked: Assume, I have developed a new intervention for people with light depression. I want to compare the effectiveness of this intervention (E) with an existing intervention (C). For this, I recruit test subjects ...

7

You can find an accessible overview of some of the issues in Whelan (2008) which contains further references discussing the issue. Note that from a statistical perspective, the sample mean and median are unbiased estimators of their population equivalents. That said, with outliers, skew and the like, the standard error of the sample mean can be quite a bit ...

6

Using Parameters Estimated from an Individual in a Group Analysis In a way this is exactly what usually happens when we calculate the mean reaction time across all conditions for a group of participants. When we normally calculate mean reaction times we assume that some process (P) takes t milliseconds to complete plus some Gaussian distributed noise. We ...

6

I use the FieldTrip toolbox in Matlab to analyze my own modified auditory MMN experiment :) But I use MEG, so I don't have that many software options. The toolbox is very powerful but it has a steep learning curve and I would recommend it only if you already have both Matlab and EEG data analysis experience. I don't analyze my data in the classical MMN way ...

6

You should probably also check out: Pleskac & Busemeyer (2010). Two-stage dynamic signal detection: A theory of choice, decision time, and confidence. Psychological Review. Also, I believe Busemeyer has a dynamic signal detection theory paper but I don't know that it has been published. The Pleskac & Busemeyer paper probably draws on this ...

6

Baayen's book, Analyzing Linguistic Data: A practical introduction to Statistics using R is a little more niche than possibly you'd want, but it does offer some coverage of mixed effects modelling, without which I'd be reticent to call any psych stats text "cutting edge".

6

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with ...

6

Some extremely large personality data sets can be downloaded at http://personality-testing.info/_rawdata/ in CSV format. They are not packaged in R, but I post the link because you indicate in your reply to USER:what that you are having trouble finding any data sets what so ever.

5

Another possible approach is using EZ-diffusion model suggested by Wagenmaker, van der Mass and Grasman (2007). Quoting Brown & Heathcote (2008; p. 4): This model is extremely simple, with just one source of variability in evidence accumulation—within-trial randomness—and simple linear accumulation (although evidence for one response does count ...

5

I recently had similar problem and I used inverse efficiency (IE) scores. These scores were derived by dividing the response times by correct response rates separately for each condition, carried out in such a way that the higher the score was, the worse was the performance. So you get something like "corrected reaction time" scores. Here is example of paper ...

5

Reading list As @Jeff has mentioned Tom Griffiths has several useful resources. In particular Tom Griffiths has an extensive reading list that you might find relevant. To quote the summary of the content: This list is intended to introduce some of the tools of Bayesian statistics and machine learning that can be useful to computational research in ...

5

+1 to Speldosa's suggestion. Griffiths and colleagues have written several primers on the use of Bayesian models in cogsci. Many of them can be found on Griffiths' website under 'Foundations': http://cocosci.berkeley.edu/publications.php?topic=Foundations e.g. Perfors, A., Tenenbaum, J.B., Griffiths, T. L., & Xu, F. (2011). A tutorial ...

5

To my knowledge, there is no adjusted RMSD. RMSD, unlike $R^2$, isn't typically used to compare models across the literature. $R^2$ represents the proportion of variance explained by the model, a construct which translates well across different experimental designs. Adjusted $R^2$ distorts this by accounting for the number of parameters in your model, but ...

5

The following reviews some of the articles that I found discussing and implementing SEM meta-analysis to examine mediation. Cheung and Chan (2005) The authors distinguish three approaches of meta-analytic structural equation modelling (MASEM). Univariate two-stage MASEM: This includes a collection of two step approaches (note that Cheung and Chan call ...

5

I think your intuitions are correct, and the reason is that Likert scales often suffer from poor construct validity. One example that comes to mind is Alter et al. (2010), who use a mixed design, and note that participants in both conditions gravitate toward the middle of the scale on their first rating. They suggest that participants do not know how to use ...

5

If you're trying to work out a standardized effect size in order to calculate power for your study then it doesn't matter whether any studies like yours have been done. It's important to find studies using the same dependent measure so that you have an estimate of variability but evidence of the size of prior effects is less important. If you have the ...

5

Determining sample size for an animal experiment is no different than in research with human subjects. What you need to know is the effect size, the significance level and the power (which is the probability that the test detects a significant effect assuming that there is one). The tricky part is getting the effect size (for an interesting discussion have a ...

5

It's an interesting question. Here are a bunch of thoughts that came to my mind for why researchers might focus on observed variables. Many researchers report reliability and observed relationships between variables. By adopting a few assumptions, the reader can estimate what the latent relationships would be (see for example, the formula for correction of ...

5

Murphy & Cleveland (1995) mention, that a good way to reduce rater errors in general is to inform raters of the existence and nature of these errors and then to simply urge to avoid them. While this reduces rater errors, it also decreases the accuracy of ratings, though. These findings come from the literature on performance assessment, where halo is ...

5

IQ scores in general: An IQ score is a normative score. The norm group is typically defined as the general population, and where the respondent is a child, the norm group is defined in terms of the general population of children of that same age. IQ scores typically have a mean of 100 and a standard deviation of 15. In order to get an estimate of ...

Only top voted, non community-wiki answers of a minimum length are eligible