Name
Abstract | ||
|---|---|---|
Research on the interaction between movement and music often involves analysis of multi-track audio, video streams and sensor data. To facilitate such research a framework is presented here that allows synchronization of multimodal data. A low cost approach is proposed to synchronize streams by embedding ambient audio into each data-stream. This effectively reduces the synchronization problem to audio-to-audio alignment. As a part of the framework a robust, computationally efficient audio-to-audio alignment algorithm is presented for reliable synchronization of embedded audio streams of varying quality. The algorithm uses audio fingerprinting techniques to measure offsets. It also identifies drift and dropped samples, which makes it possible to find a synchronization solution under such circumstances as well. The framework is evaluated with synthetic signals and a case study, showing millisecond accurate synchronization. |
Year | DOI | Venue |
|---|---|---|
2015 | 10.1007/s12193-015-0196-1 | J. Multimodal User Interfaces |
Keywords | Field | DocType |
Multimodal data synchronization,Audio fingerprinting,Audio-to-audio-alignment,Music performance research,Digital signal processing | Digital signal processing,Synchronization,Embedding,Computer science,Synchronizing,Data synchronization,Real-time computing,Millisecond,Frame synchronization (video),Multimedia | Journal |
Volume | Issue | ISSN |
9 | 3 | 1783-7677 |
Citations | PageRank | References |
1 | 0.34 | 13 |
Authors | ||
2 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Joren Six | 1 | 19 | 4.72 |
| Marc Leman | 2 | 67 | 11.96 |