Take the 2-minute tour ×
Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

We have a lot of data regarding a 5 point ordinal scale with descriptors for each point like 'very little' to 'very much'. I am worried that this scale will show insufficient variance in a further study I am designing, and would like to extend it in some way which a) captures meaningful information but b) maintains maximum comparability to the previous data.

Options under consideration: 1 - add blank items in between each of the old items on the horizontal row, rely on individuals to interpolate. 2 - invent new labels which somehow fit in between the old. This may be linguistically challenging. 3 - convert to a numeric scale with comparably labeled beginning, mid, and endpoints 4- convert to a numeric scale using the old labels at every appropriate point, and blanks otherwise.

Adding additional questions to capture the variance is not an option, but I'm interested in any others I haven't thought of, and for and against arguments.

share|improve this question
    
You might want to paraphrase this question on Cross Validated as well. I wouldn't recommend migrating this question there, because there's enough of a non-statistical issue to think about for us here too, but I also wouldn't recommend just cross-posting it verbatim, because people frown on that. Maybe you could post an elaborated version over there. For instance, you could list all your anchors (descriptors), explain how your current scale isn't numeric (re: option 3 & 4), explain why you can't add additional questions, and mention the 9-point option again... –  Nick Stauner Jan 17 at 0:29

2 Answers 2

See this question on comparing scales with different response scales. In short, you have to do it with care. A good option for ensuring comparability is to get a sample of participants to provide responses on both response scales in order to see how they relate. This could then be used to create a conversion scale.

More generally, there is a large debate about the optimal number of response options on traditional survey items. In particular, if you are more interested in means of groups rather than individual-level measurement or if you have multiple items that are aggregated, then the move from a 5 to a 7 or 9-point scale is not going result in much more differentiation. It can be good to think about how well and how consistently your participants interpret the meaning of the scale. Adding points would increase the variance, but a question remains about whether it reflects variance in the true score of interest, or whether it reflects rating bias or something else.

Personally, if comparison with previous research is important, I'd generally leave the rating options as they were.

share|improve this answer
1  
Thanks. We're very interested in individual fluctuations over time, and will model measurement error etc, hence the switch to a finer grained scale. We have no scope for giving only some participants the old scale, sadly, though perhaps one group of items using the scale can use the old and another group of items the new, if we separate them within the survey. Comparison with previous research is only interesting in the case that interesting findings come up - so a little catch 22, yes. –  Charlie Jan 17 at 9:18

I just wrote three big comments on @JeromyAnglim's answer, but I'm opting to move them all to this answer instead. This isn't an attempt to answer the OP completely, but hopefully they'll be of some use for part of the question.

From a mainly statistical and psychometric standpoint, I've heard (from an expert whose opinion I could mention confidently in a manuscript I'd intend to publish) that adding point options to extend a Likert scale tends to increase the size of the general factor (in principal components / factor analysis). This could be due in part to bias of the sort Jeromy mentions. Thought that might be useful to add.

Another consideration is how one anchors those extra points. Being ordinal, a Likert scale can deliberately deviate from symmetry and even spacing between options in potentially useful ways. E.g., ratings from -1 to 5, or adding more extreme ratings like, "0 - Absolutely not at all whatsoever," or "6/max - As much as I could ever imagine for anyone ever in the history of humanity and within the laws of physics as we know them." These are slightly absurd wordings that I wouldn't recommend verbatim; slightly terser and more meaningful wordings might be more useful. In general, extreme wordings may serve to reduce ceiling/floor effects somewhat. On that topic, see also: Reducing ceiling effect with a Likert scale measure.

However, including anchors that are worded even as absurdly extreme as those I mentioned above might actually help for discriminating between useful, meaningful responses, and inattentive or biased responding, as where a person pays more attention to the number than the meaning you intend to assign to it with the anchor. This is one way of addressing the methodological problem of interpreting Likert scale responses and identifying response bias in general (though it's an untested idea of my own invention as far as I know).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.