Neuroscience

Inference

Central nervous system

Midbrain

Biology

Politics

Reinforcement learning

Reward system

Machine learning

Political science

Artificial intelligence

Management

Psychology

Economics

Set (abstract data type)

Programming language

Representation (politics)

Computer science

Task (project management)

Dopamine

Reinforcement

Social psychology

Nature Communications

Learning to predict future outcomes is critical for driving appropriate behaviors. Reinforcement learning (RL) models have successfully accounted for such learning, relying on reward prediction errors (RPEs) signaled by midbrain dopamine neurons. It has been proposed that when sensory data provide only ambiguous information about which state an animal is in, it can predict reward based on a set of probabilities assigned to hypothetical states (called the belief state). Here we examine how dopamine RPEs and subsequent learning are regulated under state uncertainty. Mice are first trained in a task with two potential states defined by different reward amounts. During testing, intermediate-sized rewards are given in rare trials. Dopamine activity is a non-monotonic function of reward size, consistent with RL models operating on belief states. Furthermore, the magnitude of dopamine responses quantitatively predicts changes in behavior. These results establish the critical role of state inference in RL.

Belief state representation in the dopamine system