Name
Abstract | ||
|---|---|---|
Due to the increasing availability of large data sets, the need for general-purpose massively-parallel analysis tools become ever greater. In unsupervised learning, Bayesian nonparametric mixture models, exemplified by the Dirichlet-Process Mixture Model (DPMM), provide a principled Bayesian approach to adapt model complexity to the data. Despite their potential, however, DPMMs have yet to become a popular tool. This is partly due to the lack of friendly software tools that can handle large datasets efficiently. Here we show how, using Julia, one can achieve efficient and easily-modifiable implementation of distributed inference in DPMMs. Particularly, we show how a recent parallel MCMC inference algorithm - originally implemented in C++ for a single multi-core machine - can be distributed efficiently across multiple multi-core machines using a distributed-memory model. This leads to speedups, alleviates memory and storage limitations, and lets us learn DPMMs from significantly larger datasets and of higher dimensionality. It also turned out that even on a single machine the proposed Julia implementation handles higher dimensions more gracefully (at least for Gaussians) than the original C++ implementation. Finally, we use the proposed implementation to learn a model of image patches and apply the learned model for image denoising. While we speculate that a highly-optimized distributed implementation in, say, C++ could have been faster than the proposed implementation in Julia, from our perspective as machine-learning researchers (as opposed to HPC researchers), the latter also offers a practical and monetary value due to the ease of development and abstraction level. Our code is publicly available at https://github.com/dinarior/dpmm subclusters.jl |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/CCGRID.2019.00066 | 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) |
Keywords | Field | DocType |
sufficient statistic,sub cluster,mixture model,single machine,image denoising,sub cluster parameter,dirichlet process,data point,multiple multi core machine,bayesian nonparametric mixture model,image patch,process mixture model,dirichlet process mixture,bayesian nonparametric,cluster parameter,multi core,distributed memory model,sub cluster weight,sub cluster assignment,sample sub cluster,cluster weight,master machine,parallel and distributed computing,julia implementation,synthetic data,image patch model,restricted gibbs sampling,shared memory model,entire training,recent parallel mcmc inference | Dirichlet process,Inference,Computer science,Unsupervised learning,Synthetic data,Artificial intelligence,Abstraction layer,Multi-core processor,Mixture model,Machine learning,Distributed computing,Bayesian probability | Conference |
ISSN | ISBN | Citations |
2376-4414 | 978-1-7281-0913-8 | 1 |
PageRank | References | Authors |
0.35 | 10 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Or Dinari | 1 | 1 | 0.35 |
| Angel Yu | 2 | 1 | 0.35 |
| Freifeld, Oren | 3 | 181 | 14.99 |
| John W. Fisher III | 4 | 878 | 74.44 |