Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

When I first saw this video of Spaun and the tasks it can complete (solving the Towers of Hanoi problem, completing the Raven matrices), I was really impressed, but then I realized I didn't really know what other tasks were being completed by other neural architectures. Consequently, I almost asked a question here that was basically "How does Spaun compare in terms of task complexity to other neural architectures?". However, I realized that it would be better to if I could learn how to do this evaluation myself.

So, what metrics or heuristics can be used to evaluate the complexity of a task completed by a neural architecture? How can one evaluate the complexity of a problem/puzzle such at the Tower of Hanoi, to another problem, such as modelling how people solve algebraic equations, like ACT-R has accomplished?

share|improve this question
    
This is a nice question. One approach is to go as you are proposing from the direction of task difficulty, a dual idea would be to look at the complexity of the neural networks that solve the tasks. For that you might find this CogSci question and this cstheory question useful. – Artem Kaznatcheev Apr 14 '14 at 7:12

More than two years later, I now have a heuristic (which is a close as I can get to an objective metric) that I rely on. As I write about in one of my blog posts, there are a couple of tasks that humans are capable of that are still really hard for machines. They require the following things for a machine (or a neural model):

  • Balancing Planning and Exploration: Modelling the environment's future state and the possible future states via exploration efficiently.
  • Scaling Skills: Transfering skills between tasks and building more complex skills atop of basic skills.
  • Knowledge Representation: Keeping relations between skills, environmental variables, past knowledge and priorities all usefully organised.

Any neural model that begins to leverage these things is evaluated as impressive in my eyes.

However, these requirements are mostly for Artificial Intelligence. What about neural models specifically? All neural models that complete a task are, by definition, cognitive models, so in addition to the above criteria they must satisfy various Cognitive Criteria to prove the system is achieving it's goals in a human-like manner.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.