Paper Info
Title
ASHuR: Evaluation of the Relation Summary-Content Without Human Reference Using ROUGE.
Abstract
In written documents, the summary is a brief description of important aspects of a text. The degree of similarity between the summary and the content of a document provides reliability about the summary. Some efforts have been done in order to automate the evaluation of a summary. ROUGE metrics can automatically evaluate a summary, but it needs a model summary built by humans. The goal of this study is to find a quantitative relation between an article content and its summary using ROUGE tests without a model summary built by humans. This work proposes a method for automatic text summarization to evaluate a summary (ASHuR) based on extraction of sentences. ASHuR extracts the best sentences of an article based on the frequency of concepts, cue-words, title words, and sentence length. Extracted sentences constitute the essence of the article; these sentences construct the model summary. We performed two experiments to assess the reliability of ASHuR. The first experiment compared ASHuR against similar approaches based on sentences extraction; the experiment placed ASHuR in the first place in each applied test. The second experiment compared ASHuR against human-made summaries, which yielded a Pearson correlation value of 0.86. Assessments made to ASHuR show reliability to evaluate summaries written by users in collaborative sites (e.g. Wikipedia) or to review texts generated by students in online learning systems (e.g. Moodle).
Year
DOI
Venue
2018
10.4149/cai_2018_2_509
COMPUTING AND INFORMATICS
Keywords
Field
DocType
Text summarization,summary evaluation,ROUGE,sentences extraction
Computer science,ROUGE,Theoretical computer science,Mathematics education
Journal
Volume
Issue
ISSN
37
2
1335-9150
Citations 
PageRank 
References 
0
0.34
0
Authors
5
Name
Order
Citations
PageRank
Alan Ramírez-Noriega100.68
Reyes Juárez-Ramírez25617.83
Samantha Jimenez3188.59
Sergio Inzunza4184.58
Yobani Martínez-Ramírez511.03