이 논문은 컴퓨터 비전 작업, 예를 들면 이미지 분류, 검색 및 몇 번의 학습과 같은 작업에서의 하이퍼볼릭 임베딩의 사용에 대해 논의합니다. 저자들은 이미지 간의 계층적 관계를 임베딩하는 데 하이퍼볼릭 공간이 더 적합하다고 주장하며, 이러한 관계는 컴퓨터 비전 작업에서 흔히 볼 수 있습니다. 그들은 데이터셋의 초계성을 평가하는 방법을 제안하고, 하이퍼볼릭 임베딩이 이미지 분류와 몇 번의 학습을 위해 사용되는 표준 아키텍처의 성능을 향상시킬 수 있다고 보여줍니다. 또한, 이 논문은 하이퍼 볼릭 공간과 하이퍼볼릭 추정에 대한 기억을 상기시켜 줍니다.
7. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
8. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
9. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
10. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
11. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
Area of Hyperbolic plane grows rapidly than Euclidean plane!
12. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
13. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
14. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
15. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
16. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
17. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
18. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
19. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
20. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
21. Prerequisites-Comparing Geometries
- All geometries shown are 2D!
- If we cover geometries with squares:
Ref: https://www.youtube.com/watch?v=zQo_S3yNa2w
Euclidean Geometry Hyperbolic Geometry
Basic euclidean operations not defined in hyperbolic geometry!
22. Introduction
How can images have hierarchies?
Example 1) Image Retrieval
Whole
Fragment
Example 2) Recognition tasks
Low resolution
High resolution
23. Introduction
Why is hyperbolic space useful for embedding hierarchy?
Example 1) Image Retrieval
Whole
Fragment
Example 2) Recognition tasks
Low
resolution
High
resolution
The exponentially expanding
hyperbolic space will be able to
capture the underlying hierarchy of
visual data
(all edges have equal length)
Volume
Expansion
(=Hyperbolic space)
(=Euclidean space)
Given h depth, number of total Nodes in binary
tree = 2^h+1
= the number of tree nodes grows exponentially
with tree depth
r
25. Introduction
Volume
Expansion
(=Hyperbolic space)
(=Euclidean space)
The exponentially expanding
hyperbolic space will be able to
capture the underlying hierarchy of
visual data
(all edges have equal length)
Ref: https://nurendra.me/hyperbolic-networks-tutorial/
Points get clustered
Better capture hierarchical dependencies!
26. Introduction
Hierarchy in this paper
Uncertain
Certain
Network: “I have 0.1 confident score that this is 7”
Network: “I have 0.9 confident score that this is 7”
∝ P(y=c|x)
28. Related Works
Few-shot Learning
Natural Language
Preprocessing
Person re-
identification
WordNet Hierarchy
The overall ability of the model to generalize to
unseen data during training
Similarity score
Similarity score
To match pedestrian images captured by possibly
non-overlapping surveillance cameras
While the previous methods employ either Euclidean or
spherical geometries, there was no extension to
hyperbolic spaces.
Our work is inspired by the recent body of works that
demonstrate the advantage of learning hyperbolic
embeddings for language entities such as taxonomy
entries, common words, phrases, and for other NLP
tasks, such as neural machine translation
Papers adopt the pairwise models that accept pairs of
images and output their similarity scores. The resulting
similarity scores are used to classify the input pairs as
being matching or non-matching.
sentence entailment classification task
Ref: Hyperbolic Neural Networks (Advances in Neural Information Processing Systems,2018)
34. Hyperbolic Spaces & Hyperbolicity Estimation
Ref: https://www.youtube.com/watch?v=nu4mkwBEB0Y
- Most layers use Euclidean operators, such as standard
generalized convolutions, while only the final layers operate
within the hyperbolic geometry framework
- More ambiguous objects closer to the origin while moving more
specific objects towards the boundary. The distance to the
origin in our models, therefore, provides a natural estimate
of uncertainty
35. Gromov 𝛿-Hyperbolicity
: a property that measures how "curved" or "bendy" a space is
𝜹-Hyperbolicity
A metric space (X,d) is 𝛿 hyperbolic if there exists a 𝛿>0 such that four
points a,b,c,v in X:
Ref: Hyperbolic Deep Neural Networks: A Survey (IEEE
TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE
INTELLIGENCE, VOL. 44, NO. 12, DECEMBER 2022)
d: distance function
Minimal value given the
constraints!
36. 𝜹-Hyperbolicity
Ref: Hyperbolic Deep Neural Networks: A Survey (IEEE
TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE
INTELLIGENCE, VOL. 44, NO. 12, DECEMBER 2022)
Gromov 𝛿-Hyperbolicity
: a property that measures how "curved" or "bendy" a space is
the longest distance between any two points within that set
= specifies how close is a dataset X to a hyperbolic space
scale-invariant metric
Strong
hyperbolicity
Non
hyperbolicity
37. 𝜹-Hyperbolicity
Ref: Hyperbolic Deep Neural Networks: A Survey (IEEE
TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE
INTELLIGENCE, VOL. 44, NO. 12, DECEMBER 2022)
Gromov 𝛿-Hyperbolicity
: a property that measures how "curved" or "bendy" a space is
the longest distance between any two points within that set
= specifies how close is a dataset X to a hyperbolic space
scale-invariant metric
Strong
hyperbolicity
Non
hyperbolicity
The degree of hyperbolicity in image datasets are quite high
38. Hyperbolic Operations
We need to define well-known operations in new space!
-> Utilizing the formalism of Mӧbius gyrovector spaces to generalize many standard operations to
hyperbolic spaces
● Distance
● Exponential and logarithmic maps
Euclidean vectors
Hyperbolic space
Euclidean vectors Hyperbolic space
Euclidean vectors Hyperbolic space
Hyperbolic averaging
Ref:
- https://nurendra.me/hyperbolic-networks-
tutorial/
- https://arxiv.org/pdf/1805.09112.pdf
40. Experimental Setup
Hypothesis
: the distance to the center in Poincaré ball indicates a model’s uncertainty
1. Train a classifier in hyperbolic space on the MNIST dataset
2. Evaluate it on the Omniglot dataset
3. Investigate and compare the obtained distributions of distances
to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
+) Few-shot Learning
+) Person re-identification
41. Distance to the origin as the measure of uncertainty
Validate Hypothesis
(=the distance to the center in Poincaré ball indicates a model’s uncertainty)
the distance to the center in Poincaré ball will correspond to visual ambiguity of images
The output of the last hidden layer was mapped to the Poincaré ball using the
exponential map
42. Distance to the origin as the measure of uncertainty
Validate Hypothesis
(=the distance to the center in Poincaré ball indicates a model’s uncertainty)
the distance to the center in Poincaré ball will correspond to visual ambiguity of images
followed by the hyperbolic multi-linear regression (MLR) layer
43. Distance to the origin as the measure of uncertainty
Validate Hypothesis
(=the distance to the center in Poincaré ball indicates a model’s uncertainty)
the distance to the center in Poincaré ball will correspond to visual ambiguity of images
Train the model to ~ 99% test accuracy
Uncertain
Certain
Network: “I have 0.1 confident score that this is 7”
Network: “I have 0.9 confident score that this is 7”
∝ P(y=c|x)
44. Distance to the origin as the measure of uncertainty
Validate Hypothesis
(=the distance to the center in Poincaré ball indicates a model’s uncertainty)
Train Evaluate
.
Omniglot Omniglot
Omniglot
Omniglot
45. Distance to the origin as the measure of uncertainty
Validate Hypothesis
(=the distance to the center in Poincaré ball indicates a model’s uncertainty)
observe that more “unclear” images
are located near the center, while
the images that are easy to classify
are located closer to the boundary
Certain
Uncertain
46. Few-shot classification
Validate Hypothesis
(=The few-shot classification task can benefit from hyperbolic embeddings, due to the ability of hyperbolic space to
accurately reflect even very complex hierarchical relations between data points)
47. Few-shot classification
Validate Hypothesis
(=The few-shot classification task can benefit from hyperbolic embeddings, due to the ability of hyperbolic space to
accurately reflect even very complex hierarchical relations between data points)
48. Few-shot classification
Validate Hypothesis
(=The few-shot classification task can benefit from hyperbolic embeddings, due to the ability of hyperbolic space to
accurately reflect even very complex hierarchical relations between data points)
Prototypical networks (ProtoNets, NIPS 2017)
Euclidean mean operation
Average s
replace
(3 way 5 shot)
49. Few-shot classification
Validate Hypothesis
(=The few-shot classification task can benefit from hyperbolic embeddings, due to the ability of hyperbolic space to
accurately reflect even very complex hierarchical relations between data points)
Prototypical networks (ProtoNets, NIPS 2017)
Standard
ProtoNet
50. Few-shot classification
Validate Hypothesis
(=The few-shot classification task can benefit from hyperbolic embeddings, due to the ability of hyperbolic space to
accurately reflect even very complex hierarchical relations between data points)
Prototypical networks (ProtoNets, NIPS 2017)
Standard
ProtoNet
53. Discussion and Conclusion
● Shown that across a number of tasks, in particular in the few-shot image classification,
learning hyperbolic embeddings can result in a substantial boost in accuracy.
● Speculate that the negative curvature of the hyperbolic spaces allows for embeddings that are
better conforming to the intrinsic geometry of at least some image manifolds with their
hierarchical structure.
● A better understanding of when and why the use of hyperbolic geometry is warranted is
therefore needed.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets
듣는 입장에서 이해 쉽게
Introduction 설명 충분하게
Ambiguity를 Hierarchy로 mapping한 것을 강조
ArcCosh is the inverse hyperbolic cosine function.
Relative delta
Evaluate it on the Omniglot dataset
Investigate and compare the obtained distributions of distances to the origin of hyperbolic embeddings of the MNIST and Omniglot test sets