Title
Active Learning in Performance Analysis
Abstract
Active Learning (AL) is a methodology from machine learning in which the learner interacts with the data source. In this paper, we investigate application of AL techniques to a new domain: regression problems in performance analysis. For computational systems with many factors, each of which can take on many levels, fixed experiment designs can require many experiments, and can explore the problem space inefficiently. We address these problems with a dynamic, adaptive experiment design, using AL in conjunction with Gaussian Process Regression (GPR). The performance analysis process is "seeded" with a small number of initial experiments, then GPR provides estimates of regression confidence across the full input space. AL is used to suggest follow-up experiments to run, in general, it will suggest experiments in areas where the GRP model indicates low confidence, and through repeated experiments, the process eventually achieves high confidence throughout the input space. We apply this approach to the problem of estimating performance and energy usage of HPGMG-FE, and create good-quality predictive models for the quantities of interest, with low error and reduced cost, using only a modest number of experiments. Our analysis shows that the error reduction achieved from replacing the basic AL algorithm with a cost-aware algorithm can be significant, reaching up to 38% for the same computational cost of experiments.
Year
DOI
Venue
2016
10.1109/CLUSTER.2016.63
2016 IEEE International Conference on Cluster Computing (CLUSTER)
Keywords
Field
DocType
Active Learning,Performance Analysis,Gaussian Process Regression,Prediction Confidence
Kriging,Low Confidence,Data modeling,Reduced cost,Active learning,Algorithm design,Regression,Computer science,Artificial intelligence,Machine learning,Design of experiments
Conference
ISSN
ISBN
Citations 
1552-5244
978-1-5090-3654-7
3
PageRank 
References 
Authors
0.39
2
3
Name
Order
Citations
PageRank
Dmitry Duplyakin1112.93
Jed Brown2447.33
Robert Ricci327617.48