Title
Developing an evaluation methodology for spoken language systems
Abstract
There has been a long-standing methodology for evaluating work in speech recognition (SR), but until recently no community-wide methodology existed for either natural language (NL) researchers or speech understanding (SU) researchers for evaluating the systems they developed.Recently considerable progress has been made by a number of groups involved in the DARPA Spoken Language Systems (SLS) program to agree on a methodology for comparative evaluation of SLS systems, and that methodology is being used in practice for the first time.This paper gives an overview of the process that was followed in creating a meaningful evaluation mechanism, describes the current mechanism, and presents some directions for future development.
Year
DOI
Venue
1990
10.3115/116580.116614
HLT
Keywords
Field
DocType
considerable progress,long-standing methodology,speech recognition,evaluation methodology,darpa spoken language systems,meaningful evaluation mechanism,sls system,community-wide methodology,language system,comparative evaluation,speech understanding,current mechanism,natural language
Computer science,Speech recognition,Natural language,Artificial intelligence,Natural language processing,Linguistics,Spoken language
Conference
Citations 
PageRank 
References 
14
10.51
1
Authors
3
Name
Order
Citations
PageRank
Madeleine Bates1188136.85
Sean Boisen2165196.52
John Makhoul363189.91