Name
Abstract | ||
|---|---|---|
We present a framework for training and synthesis of an audio-driven dancing avatar. The avatar is trained for a given musical
genre using the multicamera video recordings of a dance performance. The video is analyzed to capture the time-varying posture
of the dancer’s body whereas the musical audio signal is processed to extract the beat information. We consider two different
marker-based schemes for the motion capture problem. The first scheme uses 3D joint positions to represent the body motion
whereas the second uses joint angles. Body movements of the dancer are characterized by a set of recurring semantic motion
patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model)
structures and the associated beat frequency. In the synthesis phase, an audio signal of unknown musical type is first classified,
within a time interval, into one of the genres that have been learnt in the analysis phase, based on mel frequency cepstral
coefficients (MFCC). The motion parameters of the corresponding dance figures are then synthesized via the trained HMM structures
in synchrony with the audio signal based on the estimated tempo information. Finally, the generated motion parameters, either
the joint angles or the 3D joint positions of the body, are animated along with the musical audio using two different animation
tools that we have developed. Experimental results demonstrate the effectiveness of the proposed framework. |
Year | DOI | Venue |
|---|---|---|
2008 | 10.1007/s12193-008-0009-x | J. Multimodal User Interfaces |
Keywords | DocType | Volume |
Multicamera motion capture,Audio-driven body motion synthesis,Dancing avatar animation | Journal | 2 |
Issue | ISSN | Citations |
2 | 1783-7677 | 3 |
PageRank | References | Authors |
0.49 | 15 | 12 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ferda Ofli | 1 | 426 | 27.96 |
| Yasemin Demir | 2 | 3 | 0.49 |
| Yücel Yemez | 3 | 300 | 21.37 |
| Engin Erzin | 4 | 256 | 32.53 |
| A. Murat Tekalp | 5 | 2603 | 278.04 |
| Koray Balci | 6 | 119 | 9.37 |
| İdil Kızoğlu | 7 | 3 | 0.49 |
| lale akarun | 8 | 1201 | 70.68 |
| Canton-Ferrer, C. | 9 | 237 | 13.78 |
| Joëlle Tilmanne | 10 | 107 | 12.24 |
| Elif Bozkurt | 11 | 63 | 6.65 |
| A. Tanju Erdem | 12 | 253 | 31.72 |