Paper Info
Title
Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study.
Abstract
Validation is one of the most important aspects of clustering, particularly when the user is designing a trustworthy or explainable system. However, most clustering validation approaches require batch calculation. This is an important gap because of the value of clustering in real-time data streaming and other online learning applications. Therefore, interest has grown in providing online alternatives for validation. This paper extends the incremental cluster validity index (iCVI) family by presenting incremental versions of Calinski-Harabasz (iCH), Pakhira-Bandyopadhyay-Maulik (iPBM), WB index (iWB), Silhouette (iSIL), Negentropy Increment (iNI), Representative Cross Information Potential (irCIP), Representative Cross Entropy (irH), and Conn&_Index (iConn&_Index). This paper also provides a thorough comparative study of correct, under- and over-partitioning on the behavior of these iCVIs, the Partition Separation (PS) index as well as four recently introduced iCVIs: incremental Xie-Beni (iXB), incremental Davies-Bouldin (iDB), and incremental generalized Dunn's indices 43 and 53 (iGD43 and iGD53). Experiments were carried out using a framework that was designed to be as agnostic as possible to the clustering algorithms. The results on synthetic benchmark data sets showed that while evidence of most under-partitioning cases could be inferred from the behaviors of the majority of these iCVIs, over-partitioning was found to be a more challenging problem, detected by fewer of them. Interestingly, over-partitioning, rather then under-partitioning, was more prominently detected on the real-world data experiments within this study. The expansion of iCVIs provides significant novel opportunities for assessing and interpreting the results of unsupervised lifelong learning in real-time, wherein samples cannot be reprocessed due to memory and/or application constraints.
Year
DOI
Venue
2020
10.1109/ACCESS.2020.2969849
IEEE ACCESS
Keywords
DocType
Volume
Clustering,validation,incremental cluster validity index (iCVI),adaptive resonance theory (ART),incremental (online) clustering algorithms,data streams
Journal
8
ISSN
Citations 
PageRank 
2169-3536
1
0.35
References 
Authors
0
3
Name
Order
Citations
PageRank
Leonardo Enzo Brito da Silva193.31
Niklas Melton210.35
Wunsch II Donald C.3135491.73