Paper Info
Title
Enhanced Benchmark Datasets for a Comprehensive Evaluation of Process Model Matching Techniques.
Abstract
Process Model Matching (PMM) refers to the automatic identification of corresponding activities between a pair of process models. Recognizing the pivotal role of PMM in numerous application areas a plethora of matching techniques have been developed. To evaluate the effectiveness of these techniques, researchers typically use PMMC'15 datasets and three well-established performance measures, precision, recall and F1 score. The performance scores of these measures are useful for a surface level evaluation of a matching technique. However, these overall scores do not provide essential insights about the capabilities of a matching technique. To that end, we enhance the PMMC'15 datasets by classifying corresponding pairs into three types and compute performance scores of each type, separately. We contend that the performance scores for each type of corresponding pairs, together with the surface level performance scores, provide valuable insights about the capabilities of a matching technique. As a second contribution, we use the enhanced datasets for a comprehensive evaluation of three prominent semantic similarity measures. Thirdly, we use the enhanced datasets for a comprehensive evaluation of the results of twelve matching systems from the PMM Contest 2015. From the results, we conclude that there is a need for developing the next generation of matching techniques that are equally effective for the three types of pairs.
Year
DOI
Venue
2018
10.1007/978-3-030-00787-4_8
Lecture Notes in Business Information Processing
Keywords
Field
DocType
Business process management,Process Model Matching,PMMC' 15 datasets,Enhanced datasets,Comprehensive evaluation
Semantic similarity,Business process management,Model matching,F1 score,Computer science,Process modeling,Artificial intelligence,Recall,Machine learning
Conference
Volume
ISSN
Citations 
332
1865-1348
0
PageRank 
References 
Authors
0.34
15
2
Name
Order
Citations
PageRank
Muhammad Ali101.01
Khurram Shahzad216525.77