Paper Info
Title
Web-based monitoring, logging and reporting tools for multi-service multi-modal systems.
Abstract
This paper describes MILER (Multi-modal data Logger for Evaluation and Report), a web-based multi-service monitoring, logging and reporting tool for advanced multi-modal dialog systems. MILER has been designed to directly arrange and synchronize logging data collected from live services and to provide real-time reports about service usage and system performance. Special attention has been given to the architecture design in order to achieve service and access-device independence and reliable synchronization of data from distributed logs. MILER allows researchers to analyze multi- modal interactions, analyze the call flow, reconstruct the system/user dialogue turns, play the recorded user utterances, and provide a preliminary dialogue performance evaluation. It also supports labeling and annotation of the dialogue turns for further offline analysis. Once the user inputs (i.e. speech and other input modalities) are manually transcribed and labeled, along with detailed log events from each dialog, MILER derives a set of objective measures, which includes word and concept accuracy, number of attempts per concept, dialog turn counts and duration, and task completion rates. Subjective measures extracted from user's surveys, including perceived task success and ease of use measures, can be combined with the objective measures and the results used later for accuracy computation.
Year
Venue
Field
2000
INTERSPEECH
Computer science,Speech recognition,Human–computer interaction,Web application,Modal,Logging
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
5
3
Name
Order
Citations
PageRank
Giuseppe Di Fabbrizio133044.45
Giuseppe Di Fabbrizio233044.45
Shrikanth Narayanan300.34