Paper Info
Title
Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum.
Abstract
Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the "win-stay, lose-switch" strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.
Year
DOI
Venue
2015
10.1371/journal.pcbi.1004540
PLOS COMPUTATIONAL BIOLOGY
Field
DocType
Volume
Ventral striatum,Dorsum,Biology,Markov model,Parallel voting,Markov chain,Striatum,Neural activity,Finite state,Artificial intelligence
Journal
11
Issue
ISSN
Citations 
11
1553-7358
0
PageRank 
References 
Authors
0.34
3
2
Name
Order
Citations
PageRank
Makoto Ito1132.06
Kenji Doya21330164.07