Paper Info
Title
Human-level Ordinal Maintainability Prediction Based on Static Code Metrics
Abstract
One of the greatest challenges in software quality control is the efficient and effective measurement of maintainability. Thorough expert assessments are precise yet slow and expensive, whereas automated static analysis yields imprecise yet rapid feedback. Several machine learning approaches aim to integrate the advantages of both concepts. However, most prior studies did not adhere to expert judgment and predicted the number of changed lines as a proxy for maintainability, or were biased towards a small group of experts. In contrast, the present study builds on a manually labeled and validated dataset. Prediction is done using static code metrics where we found simple structural metrics such as the size of a class and its methods to yield the highest predictive power towards maintainability. Using just a small set of these metrics, our models can distinguish easy from hard to maintain code with an F-score of 91.3% and AUC of 82.3%. In addition, we perform a more detailed ordinal classification and compare the quality of the classification with the performance of experts. Here, we use the deviations between the individual expert's ratings and the eventually determined consensus of all experts. In sum, our models achieve the same level of performance as an average human expert. In fact, the obtained accuracy and mean squared error outperform human performance. We hence argue that our models provide an automated and trustworthy prediction of software maintainability.
Year
DOI
Venue
2021
10.1145/3463274.3463315
PROCEEDINGS OF EVALUATION AND ASSESSMENT IN SOFTWARE ENGINEERING (EASE 2021)
Keywords
DocType
Citations 
Software Maintainability, Maintainability Prediction, Ordinal Classification, Expert Judgment, Machine Learning
Conference
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Markus Schnappinger132.41
Arnaud Fietzke231.40
Alexander Pretschner3269.69