Paper Info
Title
Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games
Abstract
Recognizing Activities of Daily Living (ADL) is a vital process for intelligent assistive robots, but collecting large annotated datasets requires time-consuming temporal labeling and raises privacy concerns, e.g., if the data is collected in a real household. In this work, we explore the concept of constructing training examples for ADL recognition by playing life simulation video games and introduce the SIMS4ACTION dataset created with the popular commercial game THE SIMS 4. We build SIMS4ACTION by specifically executing actions-of-interest in a "top-down" manner, while the gaming circumstances allow us to freely switch between environments, camera angles and subject appearances. While ADL recognition on gaming data is interesting from the theoretical perspective, the key challenge arises from transferring it to the real-world applications, such as smart-homes or assistive robotics. To meet this requirement, SIMS4ACTION is accompanied with a GAMING -> REAL benchmark, where the models are evaluated on real videos derived from an existing ADL dataset. We integrate two modern algorithms for video-based activity recognition in our framework, revealing the value of life simulation video games as an inexpensive and far less intrusive source of training data. However, our results also indicate that tasks involving a mixture of gaming and real data are challenging, opening a new research direction. We will make our dataset publicly available at https://github.com/aroitberg/sims4action.
Year
DOI
Venue
2021
10.1109/IROS51168.2021.9636381
2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)
DocType
ISSN
Citations 
Conference
2153-0858
0
PageRank 
References 
Authors
0.34
0
6
Name
Order
Citations
PageRank
Alina Roitberg166.20
David Schneider201.01
Aulia Djamal300.34
Constantin Seibold401.35
Simon Reiss521.76
Rainer Stiefelhagen63512274.86