Name
Title | ||
|---|---|---|
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring |
Abstract | ||
|---|---|---|
We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.
- Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels.
- Augmentation anchoring} feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input.
To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained.
Our new algorithm, dubbed ReMixMatch, is significantly more data-efficient than prior work, requiring between 5 times and 16 times less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach 93.73% accuracy (compared to MixMatch's accuracy of 93.58% with 4000 examples) and a median accuracy of 84.92% with just four labels per class.
|
Year | Venue | Keywords |
|---|---|---|
2020 | ICLR | semi-supervised learning |
DocType | Citations | PageRank |
Conference | 1 | 0.35 |
References | Authors | |
17 | 7 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| David Berthelot | 1 | 209 | 10.40 |
| Nicholas Carlini | 2 | 1599 | 63.23 |
| Ekin D. Cubuk | 3 | 164 | 11.09 |
| Alexey Kurakin | 4 | 723 | 24.45 |
| Kihyuk Sohn | 5 | 629 | 32.95 |
| Han Zhang | 6 | 243 | 15.29 |
| Colin Raffel | 7 | 190 | 21.50 |