Title
Evaluating generalizability and parameter consistency in learning models
Abstract
A new evaluation method is proposed for comparing learning models used for predicting decisions based on experience. The method is based on the generalization of models' predictions at the individual level. First, it evaluates the ability to make a priori predictions for decisions in new tasks using parameters from different tasks performed by an individual decision-maker. Second, it evaluates the consistency of parameters estimated in different tasks performed by the same person. We use this method to examine two rules for updating past experience with payoff feedback: The Delta rule, where only the chosen option is updated; and a Decay-Reinforcement rule, where additionally, non-chosen options are discounted. The results reveal that although the Decay-Reinforcement rule fits the data better, it has poor generality and parameter consistency at the individual level. The current method thus improves the ability to select models based on their correspondence to consistent characteristics within individual decision-makers.
Year
DOI
Venue
2008
10.1016/j.geb.2007.08.011
Games and Economic Behavior
Keywords
Field
DocType
C52,C63,D81,D83
Economics,Delta rule,A priori and a posteriori,Artificial intelligence,Learning models,Generality,Reinforcement learning,Generalizability theory,Mathematical economics,Model selection,Statistics,Machine learning,Stochastic game
Journal
Volume
Issue
ISSN
63
1
0899-8256
Citations 
PageRank 
References 
6
1.73
3
Authors
2
Name
Order
Citations
PageRank
Eldad Yechiam1689.23
Jerome R. Busemeyer210425.82