Title
Benchmark construction and experimental evaluations for incoherent ontologies
Abstract
As the core building blocks of the Semantic Web, ontologies provide shared vocabularies and conceptual knowledge for specific application fields. At the same time, ontologies can restrict their individuals and relationships through their semantic schema. However, logical conflicts of an ontology are often inevitably unavoidable in actual application scenarios when the ontology contains any form of disjointness or negation. Generally, logical conflicts can be divided into incoherence and inconsistency. Reasoning with incoherent ontologies may obtain many redundant relationships, and incoherence is a potential cause of inconsistency which seriously affects the correctness of semantic reasoning Therefore, handling incoherence is imperative, which mainly involves the fields of conflicts detection, ontology repair and justification computation. Various incoherent ontologies are indispensable to evaluate the proposed methods for handling incoherence. We provide a survey of relevant research works to study the proposed construction methods of incoherent ontologies. It is observed that incoherent ontologies in existing works still have the following limitations: (1) Lots of web pages used to download ontologies were temporarily constructed are now inaccessible; (2) Most of the existing incoherent ontologies were constructed in a simple way such as randomly adding disjointness axioms or merging ontologies through their alignments without considering all possible incoherence cases. To address the limitations, we propose a general framework to construct incoherent ontologies and design a hybrid algorithm to instantiate this framework. With the implemented construction methods, a comprehensive benchmark containing 116 ontologies is constructed. In our evaluations, incoherent ontologies in the benchmark are measured with 11 classic metrics. We then compare 5 representative ontology debugging systems and 3 repair methods. The evaluation results reveal that these ontologies could reflect different characteristics of each ontology system and repair method. All observations could guide researchers to select incoherent ontologies. The availability of our benchmark makes a contribution to the community of ontology debugging and repair fields.
Year
DOI
Venue
2022
10.1016/j.knosys.2021.108090
Knowledge-Based Systems
Keywords
DocType
Volume
Incoherent ontologies,Ontology debugging,Ontology repair,Justification computation,Ontology matching
Journal
239
ISSN
Citations 
PageRank 
0950-7051
0
0.34
References 
Authors
0
5
Name
Order
Citations
PageRank
Qiu Ji100.34
Weizhuo Li200.34
Shiqi Zhou300.34
Guilin Qi400.68
Yuan-Fang Li524539.15