Name
Abstract | ||
|---|---|---|
AbstractWe present a system for real-time hand-tracking to drive virtual and augmented reality (VR/AR) experiences. Using four fisheye monochrome cameras, our system generates accurate and low-jitter 3D hand motion across a large working volume for a diverse set of users. We achieve this by proposing neural network architectures for detecting hands and estimating hand keypoint locations. Our hand detection network robustly handles a variety of real world environments. The keypoint estimation network leverages tracking history to produce spatially and temporally consistent poses. We design scalable, semi-automated mechanisms to collect a large and diverse set of ground truth data using a combination of manual annotation and automated tracking. Additionally, we introduce a detection-by-tracking method that increases smoothness while reducing the computational cost; the optimized system runs at 60Hz on PC and 30Hz on a mobile processor. Together, these contributions yield a practical system for capturing a user's hands and is the default feature on the Oculus Quest VR headset powering input and social presence. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1145/3386569.3392452 | ACM Transactions on Graphics |
Keywords | DocType | Volume |
motion capture, hand tracking, virtual reality | Journal | 39 |
Issue | ISSN | Citations |
4 | 0730-0301 | 2 |
PageRank | References | Authors |
0.37 | 0 | 16 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Shangchen Han | 1 | 4 | 1.08 |
| Beibei Liu | 2 | 11 | 1.25 |
| Randi Cabezas | 3 | 2 | 1.04 |
| Christopher D. Twigg | 4 | 2 | 0.37 |
| Peizhao Zhang | 5 | 2 | 0.37 |
| Jeff Petkau | 6 | 2 | 0.37 |
| Tsz-Ho Yu | 7 | 2 | 0.37 |
| Chun-Jung Tai | 8 | 2 | 1.04 |
| Muzaffer Akbay | 9 | 2 | 0.37 |
| Zheng Wang | 10 | 2 | 0.37 |
| Asaf Nitzan | 11 | 2 | 0.37 |
| Dong Gang | 12 | 45 | 5.07 |
| Yuting Ye | 13 | 179 | 10.18 |
| Lingling Tao | 14 | 2 | 0.71 |
| Chengde Wan | 15 | 2 | 0.37 |
| Robert Y. Wang | 16 | 544 | 26.88 |