Paper Info
Title
GBGallery : A benchmark and framework for game testing
Abstract
Software bug database and benchmark are the wheels of advancing automated software testing. In practice, real bugs often occur sparsely relative to the amount of software code, the extraction and curation of which are quite labor-intensive but can be essential to facilitate the innovation of testing techniques. Over the past decade, several milestones have been made to construct bug databases, pushing the progress of automated software testing research. However, up to the present, it still lacks a real bug database and benchmark for game software, making current game testing research mostly stagnant. The missing of bug database and framework greatly limits the development of automated game testing techniques. To bridge this gap, we first perform large-scale real bug collection and manual analysis from 5 large commercial games, with a total of more than 250,000 lines of code. Based on this, we propose GBGallery, a game bug database and an extensible framework, to enable automated game testing research. In its initial version, GBGallery contains 76 real bugs from 5 games and incorporates 5 state-of-the-art testing techniques for comparative study as a baseline for further research. With GBGallery, we perform large-scale empirical studies and find that the current automated game testing is still at an early stage, where new testing techniques for game software should be extensively investigated. We make GBGallery publicly available, hoping to facilitate the game testing research.
Year
DOI
Venue
2022
10.1007/s10664-022-10158-x
Empirical Software Engineering
Keywords
DocType
Volume
Game testing, Bug database, Automated testing, Deep reinforcement learning
Journal
27
Issue
ISSN
Citations 
6
1382-3256
0
PageRank 
References 
Authors
0.34
6
6
Name
Order
Citations
PageRank
Zhuo Li100.34
Yuechen Wu200.34
Lei Ma335734.63
Xiaofei Xie420827.13
yingfeng chen56913.64
Changjie Fan65721.37