I need a measure of the degree to which each of several features biases participants to respond "yes" in a category present / absent task for each of several categories.
I have stimuli defined along 3 binary feature dimensions, let's call them a, b, and c. Each feature is perfectly diagnostic for 1 of 3 categories, let's call them x, y, and z, and perfectly non-diagnostic for the other 2. In other words, for example, a stimulus belongs to category x if it has feature a and not if it doesn't, and it is equally likely to belong to category y or not, and to category z or not, both when feature a is present and when it is absent. Participants are shown stimuli and asked whether or not they belong to a given category. In a given block of the experiment, they are only asked about 1 of the 3 categories.
So going back to my question, if the feature is the one diagnostic for the category, then correct response would be "yes" 100% of the time the feature is present and "no" 100% of the time the feature is absent. If the feature is one of the two irrelevant ones, then "yes" will be the correct response 50% of the time regardless of feature presence. In fact it turns out that people respond "yes" more than 50% of the time when the irrelevant features are present, and less than 50% of the time when they are absent, indicating that the irrelevant features bias them to respond "yes".
What I need is a way to quantify that effect. Things I've considered. (1) Subtract the positive response rate for feature absent from the positive response rate for feature present. 0% would indicate no bias for irrelevant features. (2) Divide the positive response rate for feature present by the positive response rate for feature absent. 1.0 would indicate no bias for irrelevant features. (3) Combining feature present and absent, divide the positive response rate by the correct response rate (i.e. the response rate of the perfect responder). 1.0 would indicate no bias for irrelevant features.
I am wondering whether there is a good "standard" measure for this kind of situation or, failing that, whether anyone can think of a good reason to prefer any one of the options above, or another one I haven't thought of.
I'm not sure whether or not signal detection theory is relevant here. My sense is that it's not, because I'm not looking for general response bias but rather the bias induced by specific features, and also because I have binary rather than continuous features, but I'm willing to be persuaded otherwise - maybe I just don't understand SDT well enough.
Oh, one other thing in case it matters - the stimuli in question are graphs and tables of data from 2x2 experimental designs, while the features / categories are presence / absence of main effects of each variable and the interaction.
