Paper Info
Title
Music Driven Human Motion Manipulation for Characters in a Video
Abstract
Multimedia content creation and manipulation have garnered attention in recent days due to the desires of personalization. As a content producing application, we propose a novel idea that requires the fusion of video and audio intelligence. The system is composed of at least three core techniques: 1) the capability to process the video sequence to have access to the geometric and appearance information pertaining to meaningful and representative targets, 2) a systematic way to reliably classify and identify important emotions from the music, 3) effective approaches to manipulate the video targets according to the extracted music emotions. In this paper, we report preliminary results of the proposed system. Specifically, we introduce the employed framework to manipulate the magnitude and speed of music conducting gestures of a video sequence of human skeleton according to the emotion intensity and tempo of an arbitrary music excerpt, using state-of-the-art inverse kinematics and music information retrieval techniques. We present the details of the prototype system and validate its effectiveness with a video demonstrating how we can manipulate the music conducting gestures according to the proposed manipulation rules.
Year
DOI
Venue
2014
10.1109/ISM.2014.31
ISM
Keywords
Field
DocType
content producing application,video signal processing,multimedia computing,video targets,music emotion,multimedia content manipulation,music driven human motion manipulation,music,human skeleton,video characters,robotics,motion manipulation,multimedia content creation,emotion recognition,image sequences,video fusion,audio intelligence,music emotion extraction,music information retrieval techniques,state-of-the-art inverse kinematics,music conducting,video sequence,image motion analysis,kinematics,trajectory
Computer vision,Music information retrieval,Kinematics,Inverse kinematics,Computer science,Gesture,Content creation,Artificial intelligence,Robotics,Trajectory,Personalization
Conference
Citations 
PageRank 
References 
0
0.34
8
Authors
4
Name
Order
Citations
PageRank
Che-Hua Yeh11077.59
Yi-Hsuan Yang2102284.71
Ming-Hsu Chang300.34
h y m liao42353198.72