Name
Abstract | ||
|---|---|---|
Human gaze behavior prediction is important for behavioral vision and for computer vision applications. Most models mainly focus on predicting free-viewing behavior using saliency maps, but do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. We modeled the viewer's internal belief states as dynamic contextual belief maps of object locations. These maps were learned and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goal-directed fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive target-dependent patterns of object prioritization, which we interpret as a learned object context. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/CVPR42600.2020.00027 | 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
Keywords | DocType | Volume |
target-object categories,goal-directed fixations,goal-directed human attention prediction,high-quality search fixations,multiple target categories,behavioral scanpaths,object locations,dynamic contextual belief maps,visual search,internal reward function,inverse reinforcement learning model,visual target object,goal-directed behavior,saliency maps,free-viewing behavior,computer vision applications,behavioral vision,human gaze behavior prediction,learned object context,distinctive target-dependent patterns,reward maps,human search behavior,search fixation scanpaths,IRL model,COCO-Search18 | Conference | 2020 |
ISSN | ISBN | Citations |
1063-6919 | 978-1-7281-7169-2 | 0 |
PageRank | References | Authors |
0.34 | 26 | 8 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| zhibo yang | 1 | 8 | 4.31 |
| Lihan Huang | 2 | 1 | 1.03 |
| Yupei Chen | 3 | 1 | 0.69 |
| Zijun Wei | 4 | 5 | 3.45 |
| Seoyoung Ahn | 5 | 2 | 2.07 |
| Gregory J. Zelinsky | 6 | 183 | 20.64 |
| Dimitris Samaras | 7 | 1740 | 101.49 |
| Minh Hoai | 8 | 290 | 35.69 |