Name
Abstract | ||
|---|---|---|
Scaling up data mining algorithms for data of both high dimensionality and cardinality has been lately recognized as one of the most challenging problems in data mining research. The reason is that typical data mining tasks, such as clustering, cannot produce high quality results when applied on high-dimensional and/or large (in terms of cardinality) datasets. Data preprocessing and in particular dimensionality reduction constitute promising tools to deal with this problem. However, most of the existing dimensionality reduction algorithms share also the same disadvantages with data mining algorithms, when applied on large datasets of high dimensionality. In this article, we propose a fast and efficient dimensionality reduction algorithm (FEDRA), which is particularly scalable and therefore suitable for challenging datasets. FEDRA follows the landmark-based paradigm for embedding data objects in a low-dimensional projection space. By means of a theoretical analysis, we prove that FEDRA is efficient, while we demonstrate the achieved quality of results through experiments on datasets of higher cardinality and dimensionality than those employed in the evaluation of competitive algorithms. The obtained results prove that FEDRA manages to retain or ameliorate clustering quality while projecting in less than 10% of the initial dimensionality. Moreover, our algorithm produces embeddings that enable the faster convergence of clustering algorithms. Therefore, FEDRA emerges as a powerful and generic tool for data pre-processing, which can be integrated in other data mining algorithms, thus enhancing their performance. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1145/1921632.1921637 | TKDD |
Keywords | Field | DocType |
particular dimensionality reduction,large datasets,landmark-based dimensionality reduction,data mining algorithm,initial dimensionality,clustering quality,existing dimensionality reduction algorithm,enhancing clustering quality,data mining research,landmarks,efficient dimensionality reduction algorithm,typical data mining task,embedding data object,high dimensionality,dimensionality reduction,data preprocessing,data mining,projective space | Convergence (routing),Data mining,Dimensionality reduction,Embedding,Computer science,Cardinality,Data pre-processing,Curse of dimensionality,Artificial intelligence,Cluster analysis,Machine learning,Scalability | Journal |
Volume | Issue | ISSN |
5 | 2 | 1556-4681 |
Citations | PageRank | References |
3 | 0.37 | 31 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Panagis Magdalinos | 1 | 35 | 5.55 |
| Christos Doulkeridis | 2 | 899 | 55.91 |
| Michalis Vazirgiannis | 3 | 3942 | 268.00 |