Paper Info
Title
An Industrial Evaluation of Unit Test Generation: Finding Real Faults in a Financial Application.
Abstract
Automated unit test generation has been extensively studied in the literature in recent years. Previous studies on open source systems have shown that test generation tools are quite effective at detecting faults, but how effective and applicable are they in an industrial application? In this paper, we investigate this question using a life insurance and pension products calculator engine owned by SEB Life & Pension Holding AB Riga Branch. To study fault-finding effectiveness, we extracted 25 real faults from the version history of this software project, and applied two up-to-date unit test generation tools for Java, EvoSuite and Randoop, which implement search-based and feedback-directed random test generation, respectively. Automatically generated test suites detected up to 56.40% (Evosuite) and 38.00% (Randoop) of these faults. The analysis of our results demonstrates challenges that need to be addressed in order to improve fault detection in test generation tools. In particular, classification of the undetected faults shows that 97.62% of them depend on either \"specific primitive values\" (50.00%) or the construction of \"complex state configuration of objects\" (47.62%). To study applicability, we surveyed the developers of the application under test on their experience and opinions about the test generation tools and the generated test cases. This leads to insights on requirements for academic prototypes for successful technology transfer from academic research to industrial practice, such as a need to integrate with popular build tools, and to improve the readability of the generated tests.
Year
DOI
Venue
2017
10.1109/ICSE-SEIP.2017.27
ICSE-SEIP
Keywords
Field
DocType
Automated Tests Generation, Empirical Software Engineering, Search-based Testing, Random Testing
Test harness,Test suite,Calculator,Random testing,Test Management Approach,Software engineering,Computer science,Unit testing,Test case,Test data generation
Conference
ISBN
Citations 
PageRank 
978-1-5386-2718-1
20
0.61
References 
Authors
21
5
Name
Order
Citations
PageRank
Mohammad Moein Almasi1201.28
Hadi Hemmati262227.54
Gordon Fraser32625116.22
Andrea Arcuri4263092.48
Janis Benefelds5200.94