# How do humans optimize noisy multi-variable functions in experimental settings?

Imagine an experiment like this:

A participant is asked to optimize an unknown function (let's say minimize) . On each trial the participant provides several input values, and receives an output value. Now also imagine that the output is noisy, in that the same inputs lead to an output plus a random component.

To think of but one of many possible specific examples imagine the following function

$$Y = (X -3)^2 + (Z-2)^2 + (W-4)^2 + e,$$

where $e$ is normally distributed, mean = 0, sd = 3.

On each trial, the participant would provide a value for $X$, $Z$, and $W$. And they would obtain a $Y$ value based on this underlying function. Their aim would be to minimise the value of $Y$. They have not been told the underlying functional form. They only know that there is a global minimum and that there is a random component.

I'm interested in reading about the strategies used by humans to do this task in experimental settings. Note I'm not directly interested in how computers do the task or how programmers and mathematicians might complete this task.

### Questions

• What are some good references for learning about the literature on how humans learn to optimize noisy multi-variable functions?
• What are some of the key findings on how humans optimize noisy multi-variable functions?
-
I thought the standard rule-of-thumb was that people assume everything is a straight-line. – Artem Kaznatcheev Feb 15 '12 at 6:37
Can you offer a hypothetical about how you would expect them to perform this operation? I guess I'm confused as to whether they'd just be using raw values u = f(w,x,y,z) +e, where w,x,y,z were given or would they be aware of the actual function equation (since your function is in R4, they wouldn't really be able to plot it effectively)? I think you're on to something here, but I'm just not sure if you're testing what you think you are. – Chuck Sherrington Feb 15 '12 at 7:08
@ArtemKaznatcheev I'm assuming that there is an actual set of input values that results in a global minimum; you could imagine an underlying quadratic function if you like, but I'm interested in more general problems. – Jeromy Anglim Feb 15 '12 at 9:40
@jonsca Example: On a given trial, the participant provides raw values for X, Z and W, and the program returns a Y based on the unknown function. The participant does not know the underlying function other than perhaps the information that there is a global minimum Y value that they are trying to find. Thus, on the next trial, they might try a different set of X, Z and W values, and they would get a new Y value. And thus, over time, they would tweak the values to try to find the minimum. Their performance might be measured as reverse of the sum of their Y obtained values. – Jeromy Anglim Feb 15 '12 at 9:46
You might already know this, but just in case, you can also use $$\small Y = (X -3)^2 + (Z-2)^2 + (W-4)^2 + e,$$ (or any of the specifiers here)‌​if you find the equations are too big. – Chuck Sherrington Jun 29 '12 at 6:39

This is a bit of a tangential answer, but hopefully still useful.

When we give humans noisy data, we can basically think of them as some sort of Bayesian inference machines that try to figure out what the function that data came from looks like. The important thing we then need to know, is how strong of a bias (prior) humans have towards expecting certain relationships.

Unfortunately, it seems that humans are extremely biased towards positive linear relationships. I think this will make it very hard for them to optimize data presented as in your question, because they will constantly assume it comes from a straight line. This is really well captured by the following figure from Kalish et al. (2007):

The experiment that generated the above picture is rather different from the one you describe, but we can think of it is a very particular type of noise. A person at stage $n$ is given 25 $(x,y)$ pairs from the function at stage $n - 1$. The person is then tested by being given an x value and asked for a y, 25 times. The results of this are passed on to the person at trial $n + 1$ as the training data. Thus, we could think of the errors of person at stage n as noise/errors (although systematic errors) for the person at stage $n + 1$. As you can see, it doesn't take much of this noise to lose all structure of the function you started with and revert to the natural bias of a positive linear relationship. In fact, in condition 1 participants are already completely confused about the U-shaped function after strage $n = 1$ (so the first participant, with no error already has a hard time understanding the function from $(x,y)$ pairs).

### References

Kalish, M. L., Griffiths, T. L., & Lewandowsky, S. (2007). Iterated learning: Intergenerational knowledge transmission reveals inductive biases. Psychonomic Bulletin & Review, 14(M), 288-294. [pdf]

-
Thanks for the interesting thoughts. Now that you mention it, I had had a look at the Kalish et al study a few years back. However, I think there are two big differences: the study is iterative and therefore assumptions about the relationship carry over into subsequent trials; Also, from a quick look, the paper seems to be about describing the functional form rather than finding the optimum. Surely almost anyone can estimate an unkonwn numeric quantity (e.g., 271) when the feedback they are given is "higher" or "lower" (e.g., 200, "higher", 300, "lower", 250, "higher", 280, "lower", etc.) – Jeromy Anglim Feb 18 '12 at 1:10
Of course, adding noise, including multiple input variables, and making the feedback direction-less would make the task harder. – Jeromy Anglim Feb 18 '12 at 1:11
@JeromyAnglim yeah, the model is different, in that they are given x and asked for y, instead of asked for an x that minimizes the function. So this answer is conditional on the assumption that people actually end up forming some sort of mental representation of the function. If they do not form such a representation (which is often the case in different modalities, like pole-balancing) then my answer is completely irrelevant, haha. However, if the participants give X,Z,W and get a Y as feedback, then I suspect the participants will try to form SOME mental representation of the function. – Artem Kaznatcheev Feb 18 '12 at 1:15
the biggest point of my answer/long-comment, was that we shouldn't expect participants to be very good at this. – Artem Kaznatcheev Feb 18 '12 at 1:16

This seems related to the literature on multiple-cue probability learning (MCPL). In this paradigm, a typical task presents subjects a list of cues and values, and asks them to predict the probability of certain outcomes. This paradigm has a decent amount of literature both in the JDM (judgment and decision making) community as well as the human factors community. To see the relevance, consider a doctor who has to diagnose a patient (provide treatment) based on a finite set of cues (symptoms).

Empirically, human judgment of this type has been modeled using Egon Brunswik's probabilistic functionalism, perhaps more commonly known as the lens model or social judgment theory. This is a useful methodology for comparing human judgment to true ecological correlations.

The image of above depicts the lens model. To give an example, consider the task of a college admissions board who must decide who to admit. The environment/criterion might be their final college GPA, and cues might be high school GPA, SAT scores, writing sample, etc. You can use multiple regression to find the 'true' ecological weights of these cues on the environment criterion, and similarly you can do the same for the admission board's estimate of a student's success (if they were to estimate GPA).

An admission board will (hopefully) observe the effect of different cues on success, and revise their cue weights with experience. Unfortunately, people are typically not so good at this task.

Some common findings:

• People tend to use no more than 3 cues, even if they claim that they use more.
• People are typically outperformed by a bootstrap model of themselves
• People are often outperformed by a unit weight model of themselves: In other words, if you simply set the highest observed cues weights (on the right hand side) to 1, and all others to 0, you may get a better predictor of outcome.

What I take from this is that people will probably have little chance of success at estimating cue weights from a complex equation such as the one you present. However, you could measure these cue weights iteratively to observe learning rates and do other fun stuff-- even if we are all better off being judged by computer algorithms.

-