Title
Hyperbolic Vision Transformers: Combining Improvements in Metric Learning
Abstract
Metric learning aims to learn a highly discriminative model encouraging the embeddings of similar classes to be close in the chosen metrics and pushed apart for dissimilar ones. The common recipe is to use an encoder to extract embeddings and a distance-based loss function to match the representations - usually, the Euclidean distance is utilized. An emerging interest in learning hyperbolic data embeddings suggests that hyperbolic geometry can be beneficial for natural data. Following this line of work, we propose a new hyperbolic-based model for metric learning. At the core of our method is a vision transformer with output embeddings mapped to hyperbolic space. These embeddings are directly optimized using modified pairwise cross-entropy loss. We evaluate the proposed model with six different formulations on four datasets achieving the new state-of-the-art performance. The source code is available at https://github.com/htdt/hyp_metric.
Year
DOI
Venue
2022
10.1109/CVPR52688.2022.00726
IEEE Conference on Computer Vision and Pattern Recognition
Keywords
DocType
Volume
Recognition: detection,categorization,retrieval, Representation learning
Conference
2022
Issue
Citations 
PageRank 
1
0
0.34
References 
Authors
0
5
Name
Order
Citations
PageRank
Aleksandr Ermolov101.01
Leyla Mirvakhabova262.13
Valentin Khrulkov300.34
Nicu Sebe47013403.03
Ivan V. Oseledets530641.96