Name
Abstract | ||
|---|---|---|
An update summary should provide a fluent summarization of new information on a time-evolving topic, assuming that the reader has already reviewed older documents or summaries. In 2007 and 2008, an annual summarization evaluation included an update summarization task. Several participating systems produced update summaries indistinguishable from human-generated summaries when measured using ROUGE. However, no machine system performed near human-level performance in manual evaluations such as pyramid and overall responsiveness scoring. We present a metric called Nouveau-ROUGE that improves correlation with manual evaluation metrics and can be used to predict both the pyramid score and overall responsiveness for update summaries. Nouveau-ROUGE can serve as a less expensive surrogate for manual evaluations when comparing existing systems and when developing new ones. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1162/coli_a_00033 | Computational Linguistics |
Keywords | DocType | Volume |
existing system,manual evaluation,update summary,new information,update summarization task,pyramid score,manual evaluation metrics,fluent summarization,overall responsiveness,annual summarization evaluation,system performance | Journal | 37 |
Issue | ISSN | Citations |
1 | 0891-2017 | 15 |
PageRank | References | Authors |
1.03 | 6 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| John M. Conroy | 1 | 48 | 3.85 |
| Judith D. Schlesinger | 2 | 201 | 12.43 |
| O'Leary, Dianne P. | 3 | 1064 | 222.93 |