Paper Info
Title
Automation and consistency analysis of test cases written in natural language: An industrial context
Abstract
We present here a novel test automation strategy that receives as input a freestyle natural language (NL) test case (consisting of a sequence of test steps) and produces executable test scripts. This strategy relies on a database of previously automated seed test steps, available for reuse. New steps are automated via a capturing process by a tester, without requiring any programming knowledge. Automated tests can be executed by a replay facility. We discuss the reuse improvement, implementation effort, and user feedback regarding the industrial applicability and usability of our capture & replay tool. We then show that restricting the input textual description to obey a proposed Controlled NL (CNL) brings significant advantages: (1) reuse improvement; (2) the possibility of integration with a test generation framework; and (3) definition of consistency notions for test actions and test action sequences, that ensure, respectively, well-formedness of each action and a proper configuration to safely execute a sequence of actions. We formalize these consistency notions in Alloy and use the Alloy Analyzer to carry out the consistency check; the scalability of the analysis is assessed via an evaluation considering a repository with real test cases; the practical context of our work is mobile device testing, involving a partnership with Motorola Mobility, a Lenovo company.
Year
DOI
Venue
2020
10.1016/j.scico.2019.102377
Science of Computer Programming
Keywords
Field
DocType
Test automation,Controlled natural language,Alloy,Test case consistency
Programming language,Mobile-device testing,Computer science,Usability,Alloy Analyzer,Test script,Automation,Test case,Executable,Scalability
Journal
Volume
ISSN
Citations 
189
0167-6423
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Filipe Arruda100.34
Flávia Barros200.34
Augusto Sampaio350143.38