Name
Abstract | ||
|---|---|---|
To date, few attempts have been made to develop and validate methods for automatic evaluation of linguistic quality in text summarization. We present the first systematic assessment of several diverse classes of metrics designed to capture various aspects of well-written text. We train and test linguistic quality models on consecutive years of NIST evaluation data in order to show the generality of results. For grammaticality, the best results come from a set of syntactic features. Focus, coherence and referential clarity are best evaluated by a class of features measuring local coherence on the basis of cosine similarity between sentences, coreference information, and summarization specific features. Our best results are 90% accuracy for pairwise comparisons of competing systems over a test set of several inputs and 70% for ranking summaries of a specific input. |
Year | Venue | Keywords |
|---|---|---|
2010 | ACL | local coherence,linguistic quality,summarization specific feature,automatic evaluation,nist evaluation data,multi-document summarization,best result,specific input,text summarization,well-written text,test linguistic quality model,multi document summarization |
Field | DocType | Volume |
Cosine similarity,Computer science,Artificial intelligence,Natural language processing,Automatic summarization,Pairwise comparison,Multi-document summarization,Coreference,Information retrieval,Ranking,Grammaticality,Linguistics,Test set | Conference | P10-1 |
Citations | PageRank | References |
45 | 1.70 | 24 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Emily Pitler | 1 | 573 | 27.65 |
| Annie Louis | 2 | 443 | 24.78 |
| Ani Nenkova | 3 | 1831 | 109.14 |