Title
Neural Text Summarization: A Critical Evaluation
Abstract
Text summarization aims at compressing long documents into a shorter form that conveys the most important parts of the original document. Despite increased interest in the community and notable research effort, progress on benchmark datasets has stagnated. We critically evaluate key ingredients of the current research setup: datasets, evaluation metrics, and models, and highlight three primary shortcomings: 1) automatically collected datasets leave the task underconstrained and may contain noise detrimental to training and evaluation, 2) current evaluation protocol is weakly correlated with human judgment and does not account for important characteristics such as factual correctness, 3) models overfit to layout biases of current datasets and offer limited diversity in their outputs.
Year
DOI
Venue
2019
10.18653/v1/D19-1051
EMNLP/IJCNLP (1)
DocType
Volume
Citations 
Conference
D19-1
3
PageRank 
References 
Authors
0.39
0
5
Name
Order
Citations
PageRank
Wojciech Kryściński181.80
nitish shirish keskar232516.71
McCann, Bryan31057.04
Caiming Xiong496969.56
Richard Socher56770230.61