9
$\begingroup$

In reinforcement learning, there is a stark distinction between model-based and model-free learning algorithms, where model-free methods don't make use any explicit information about the dynamics of the environment.

It seems like this distinction would have some analog in human learning, but I'm having a very hard time finding any mention of it. Perhaps it would conditioning versus more cognitive forms of learning? I'd be overjoyed if someone could find an article using the term 'model-free' to refer to some aspect of human learning, or just reassure me of the term's correct human analog.

Does research on human learning featuring a model-free/model-based distinction exist?

$\endgroup$
7
  • 4
    $\begingroup$ Are you using google? Google scholar search for "model free reinforcement learning" brings up, as the first hit - before anything else - a cognitive neuroscience study with over 103 citations. There is indeed a developing literature on this topic, and the appropriate term is indeed "model free." $\endgroup$
    – CHCH
    Oct 1, 2012 at 20:50
  • $\begingroup$ @CHCH is referring to this article which for me is also first result. As this is not your first question, I am disappointed by the lack of initial research. It is also not clear to me what you are trying to ask here. Although you make some fun points, I am not sure if this is a question and am voting to close as NARQ. $\endgroup$ Oct 2, 2012 at 0:17
  • $\begingroup$ Sorry about the google miss -- this was question I rediscovered from a few years ago. I should've regoogled before posting, but I didn't realize that something would have changed in a couple years. Sorry for the mishap. However, I don't understand how this isn't a question. What part could use rewording? $\endgroup$
    – zergylord
    Oct 2, 2012 at 4:48
  • 1
    $\begingroup$ It is a question, and at its root is a very interesting one, but at this point it is very broad. I don't want to speak for anyone, but I think the others are trying to say that now that you know the terminology, we have "answered" this particular question, so if you use that information to make the question more specific to what you want to know, it will be stronger. FWIW, I'm glad to see you back again as I think you do ask great/interesting questions, this one just needs a bit of tuning and specificity to it. $\endgroup$ Oct 2, 2012 at 8:25
  • 3
    $\begingroup$ Ideally, answer it yourself and go a bit more in depth in your answer while reading through the google hits you now found. It's a genuine question, I prefer not closing it. Just make sure you google a bit better next time. ;p $\endgroup$
    – Steven Jeuris
    Oct 3, 2012 at 7:42

1 Answer 1

6
$\begingroup$

As per the comments to the question, human research observing this distinction does exist. CHCH possibly alludes to an article by Gläscher, Daw, Dayan and O'Doherty (2010) which concisely defines the difference between model-free learning and model-based learning:

Reinforcement learning (RL) uses sequential experience with situations (“states”) and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model.

Gläscher et al. (2010) report fMRI evidence for neural activity consistent with model-based learning in the human intraparietal sulcus and lateral prefrontal cortex, and for model-free learning in the ventral striatum. They conclude:

This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

References

  • Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585-595.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.