Context: When reading research articles in psychology, you often encounter new non-aptitude self-report psychological tests (e.g., measures of personality, well-being, psychopathology, learning style, stress, goal orientation, etc.). Most of these tests involve obtaining the sum or the mean of a set of items for one or more scales or subscales. Each item typically involves item text and a consistent set of response options (e.g., Likert-type response scales, yes-no response scales, etc.). There are thousands of such scales, some which are frequently used; others are used only once.
A lot has been written about how to assess test validity. However, here I wanted to focus particularly on how to assess what a particular individual or group score means. Specifically, I'm thinking of the situation where you are reading a journal article and you encounter a scale that you have not seen before, such as:
- A 10-item measure of self-esteem on 1 to 5 scale; group mean is 3.2. Is the sample generally high, low, or moderate in self-esteem?
- A 30-item check-list of good eating habits with a mean of 20.7. Does the sample have good eating habits?
For well developed tests, there is a test manual. Other times there is a website or key journal article with information. Finally, in some cases, very little information is readily available. In some cases useful information is given in the methods section, but typically, on its own, this is inadequate. Thus, the process of trying to locate information such as item content and normative comparisons begin.
A lot has been said about normative and criterion referenced test interpretation. If you are using a test in clinical practice or you are using a test in your own research, you should invest time in understanding what different scores on the test actually mean. However, when casually reading a journal article you can encounter many tests. In such cases, the article is one of many, and you might just want to get a quick sense of what a given score means.
- What is a good way to get both a normative and absolute understanding of the meaning of a group or individual non-aptitude test score?
- What is an efficient way of obtaining the information required to make such a judgement?
Note the use of the term "non-aptitude". I'd like to exclude discussion of intelligence testing, ability testing, and knowledge testing, and so on.
Update: To clarify, I didn't mean to imply in the question that item information was not available anywhere. I guess the question is referring to the general situation where there probably are previous uses of the measure in the literature, and there might be a key article that describes the measure somewhere. I guess I'm thinking of a good general strategy for securing relevant information and then using it to form an understanding of the scale properties.