Title
Creating real value in software engineering experiments
Abstract
Undertaking empirical work to understand the software engineering process is an increasingly common activity. However, due to the nature of most software engineering studies, drawing reliable conclusions from a single study is inherently dangerous. A single study undertaken by a small group of practitioners, in a single location, with a single group of subjects, will possess a large number of parameters, some controlled, some completely unconstrained. Any one of which can potentially cause a significant change in the result of the study; and hence their effect must be controlled, eliminated or understood, if reliable results are to be derived. The normal method of achieving this objective is to repeat the study, often while changing some of the parameters, to see if the original result is stable with regard to repetition and alteration of some of its components. Without this confirming power, empirically based claims in software engineering should be regarded as having at best limited value and at worst with suspicion and mistrust. Hence, despite their now significant volume, software engineering experiments impact has been rather limited upon industrial software processes. The potential problems with single software engineering empirical studies are three-fold: low statistical power; large number of potential covariates with the treatment variable; and the verification of the process and products of the study.
Year
DOI
Venue
2006
10.1007/978-3-540-71301-2_11
Empirical Software Engineering Issues
Keywords
Field
DocType
real value,software engineering experiment,software process,empirical study,software engineering,statistical power
Software Engineering Process Group,Personal software process,Software engineering,Computer science,Software construction,Software measurement,Software sizing,Software verification,Social software engineering,Search-based software engineering
Conference
Volume
ISSN
Citations 
4336
0302-9743
0
PageRank 
References 
Authors
0.34
1
1
Name
Order
Citations
PageRank
James Miller114116.61