Title
An Automatic Evaluation Framework for Improving a Configurable Text Summarizer
Abstract
CALLISTO is a text summarization system that depends on machine learning techniques and is therefore sensitive to pre-established biases that may not be wholly appropriate. We set out to test whether other biases, modifying the space that CALLISTO explores, lead to improvements in the overall quality of the summaries produced. We present an automatic evaluation framework that relies on a summary quality measure proposed by Lin and Hovy. It appears to be the first evaluation of a text summarization system conducted automatically on a large corpus of news stories. We show the practicality of our methodology on a few experiments with the Machine Learning module of CALLISTO. We conclude that this framework gives reliable hints on the adequacy of a bias and could be useful in developing automatic text summarization systems that work with Machine Learning techniques.
Year
DOI
Venue
2004
10.1007/978-3-540-24840-8_49
ADVANCES IN ARTIFICIAL INTELLIGENCE
Keywords
Field
DocType
artificial intelligence,quality control,bias
Automatic summarization,Content word,Information retrieval,Computer science,Machine translation,Human judgment
Conference
Volume
ISSN
Citations 
3060.0
0302-9743
0
PageRank 
References 
Authors
0.34
3
4
Name
Order
Citations
PageRank
Loïs Rigouste1262.00
Stan Szpakowicz21200114.50
Nathalie Japkowicz32581182.43
Terry Copeck4469.08