Paper Info
Title
Mic2Mic: using cycle-consistent generative adversarial networks to overcome microphone variability in speech systems
Abstract
Mobile and embedded devices are increasingly using microphones and audio-based computational models to infer user context. A major challenge in building systems that combine audio models with commodity microphones is to guarantee their accuracy and robustness in the real-world. Besides many environmental dynamics, a primary factor that impacts the robustness of audio models is microphone variability. In this work, we propose Mic2Mic - a machine-learned system component - which resides in the inference pipeline of audio models and at real-time reduces the variability in audio data caused by microphone-specific factors. Two key considerations for the design of Mic2Mic were: a) to decouple the problem of microphone variability from the audio task, and b) put minimal burden on end-users to provide training data. With these in mind, we apply the principles of cycle-consistent generative adversarial networks (CycleGANs) to learn Mic2Mic using unlabeled and unpaired data collected from different microphones. Our experiments show that Mic2Mic can recover between 66% to 89% of the accuracy lost due to microphone variability for two common audio tasks.
Year
DOI
Venue
2019
10.1145/3302506.3310398
Proceedings of the 18th International Conference on Information Processing in Sensor Networks
Keywords
Field
DocType
GAN, microphone variability, robustness, speech models
Unpaired Data,Training set,Inference,Computer science,Robustness (computer science),Real-time computing,Speech recognition,Computational model,Generative grammar,Microphone,Adversarial system
Conference
ISBN
Citations 
PageRank 
978-1-4503-6284-9
1
0.39
References 
Authors
18
5
Name
Order
Citations
PageRank
Akhil Mathur114515.21
Anton Isopoussu222.08
Fahim Kawsar390980.24
Nadia Berthouze412314.38
Nicholas D. Lane54247248.15