Name
Abstract | ||
|---|---|---|
This paper considers the clustering problem for large data sets. We propose an approach based on distributed optimization. The clustering problem is formulated as an optimization problem of maximizing the classification gain. We show that the optimization problem can be reformulated and decomposed into small-scale sub optimization problems by using the Dantzig-Wolfe decomposition method. Generally speaking, the Dantzig-Wolfe method can only be used for convex optimization problems, where the duality gaps are zero. Although, the considered optimization problem in this paper is non-convex, we prove that the duality gap goes to zero, as the problem size goes to infinity. Therefore, the Dantzig-Wolfe method can be applied here. In the proposed approach, the clustering problem is iteratively solved by a group of computers coordinated by one center processor, where each computer solves one independent small-scale sub optimization problem during each iteration, and only a small amount of data communication is needed between the computers and center processor. Numerical results show that the proposed approach is effective and efficient. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1109/ITNG.2011.98 | Clinical Orthopaedics and Related Research |
Keywords | DocType | Volume |
large-scale clustering,dantzig-wolfe decomposition method,duality gap,pattern clustering,convex optimization problem,data analysis,data compression,center processor,linear programming,small-scale sub optimization problem,optimization problem,dantzig-wolfe method,problem size,concave programming,distributed optimization,data mining,machine learning,classification gain maximization,clustering problem,convex optimization problems,unsupervised learning,independent small-scale sub optimization,clustering,distributed databases,signal processing,clustering algorithms,covariance matrix,distributed database,optimization | Conference | abs/1010.4253 |
ISSN | ISBN | Citations |
Proceeding of the 8th International Conference on Information
Technology : New Generations, April 11-13, 2011, Las Vegas, Nevada, USA | 978-0-7695-4367-3 | 0 |
PageRank | References | Authors |
0.34 | 9 | 1 |