This document describes the ORB (Oriented FAST and Rotated BRIEF) image feature detection and description algorithm. It first discusses image features in general, including corner detection methods like FAST, BRIEF and SIFT. It then provides an overview of ORB, which combines oFAST for keypoint detection with rBRIEF for descriptor extraction. The document proceeds to explain the details of the ORB algorithm, including using an image pyramid for scale invariance, computing keypoint orientations, and generating rotation-invariant BRIEF descriptors.

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ORB 特征
高洪臣
cggos@outlook.com
2020 年 7 月 10 日
高洪臣 ORB 特征 2020 年 7 月 10 日 1 / 12

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Image Features
feature types
feature extraction(detection) & matching
• feature detector
• feature descriptor
高洪臣 ORB 特征 2020 年 7 月 10 日 2 / 12

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Corner / Keypoint / Interest Point
• keypoint detector
• Harris (1988)
• Shi-Tomas (1994)
• FAST1
(Features from Accelerated and Segments Test) (2006)
• AGAST (2010)
• keypoint descriptor
• BRIEF2
(Binary robust independent elementary feature) (2010)
• keypoint detector & descriptor
• SIFT (1999, 2004)
• SURF (2006)
• BRISK (Binary Robust Invariant Scalable Keypoints) (2011)
• ORB3
(2011)
• FREAK (2012)
• KAZE (2012)
1Edward Rosten and Tom Drummond. “Machine Learning for High-Speed Corner Detection.”. In: ECCV (1).
Ed. by Ales Leonardis, Horst Bischof, and Axel Pinz. Vol. 3951. Lecture Notes in Computer Science. Springer,
2006, pp. 430–443. isbn: 3-540-33832-2. url: http://dblp.uni-trier.de/db/conf/eccv/eccv2006-1.html#RostenD06.
2Michael Calonder et al. “BRIEF: Binary Robust Independent Elementary Features”. In:
Proceedings of the 11th European Conference on Computer Vision: Part IV. ECCV’10. Heraklion, Crete,
Greece: Springer-Verlag, 2010, pp. 778–792. isbn: 364215560X.
3Ethan Rublee et al. “ORB: An efficient alternative to SIFT or SURF”. In:
2011 International conference on computer vision. Ieee. 2011, pp. 2564–2571.
高洪臣 ORB 特征 2020 年 7 月 10 日 3 / 12

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ORB Overview
ORB = Oriented FAST + Rotated BRIEF
• oFAST (ORB Detector)
• keypoint position - FAST
• keypoint orientation - image patch moment
• keypoint response/score - NMS(Non-Maximal Supression)
• scale-invariant - image pyramid
• rBRIEF (ORB Descriptor)
• keypoint descriptor - BRIEF
• rotation-invariant - keypoint orientation
高洪臣 ORB 特征 2020 年 7 月 10 日 4 / 12

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Multiscale Image Pyramid
Parameters
• level: 8
• scale: 1.2
• downsample:
bilinear interpolation
produce FAST features and compute descriptors at each level in the pyramid
高洪臣 ORB 特征 2020 年 7 月 10 日 5 / 12

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
FAST
fast-9
• 9 contiguous pixels in a circle (of 16 pixels) brighter than Ip + t or darker than Ip − t
• Rapid rejection by testing 1, 9, 5 then 13
NMS (Non-Maximal Suppression) 1
remove corners which have an adjacent corner with higher score
Uniform Distribution
DistributeOctTree() 2
1all called non-maximum suppression
2use Quad-Tree to iteratively segment image regions in ORB-SLAM2
高洪臣 ORB 特征 2020 年 7 月 10 日 6 / 12

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Orientation by Intensity Centroid (IC)
the moments of a patch:
mpq =
∑
x,y
xp
yq
I(x, y)
the first order moment of a patch 1
I (radius = 15):
m10 =
15∑
x=−15
15∑
y=−15
xI(x, y) =
15∑
y=0
15∑
x=−15
x [I(x, y) − I(x, −y)]
m01 =
15∑
x=−15
15∑
y=−15
yI(x, y) =
15∑
y=1
15∑
x=−15
y [I(x, y) − I(x, −y)]
the intensity centroid:
C =
(
m10
m00
,
m01
m00
)
the orientation (from the corner’s center to the centroid):
θ = atan 2 (m01, m10)
1a circular patch
高洪臣 ORB 特征 2020 年 7 月 10 日 7 / 12

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Image Gaussian Filtering/Blurring/Smoothing 1
why filtering
start by smoothing image using a Gaussian kernel at each level in the pyramid in order to
prevent the descriptor from being sensitive to high-frequency noise
Gaussian Kernel
G(u, v) =
1
2πσ2
e
− u2+v2
2σ2
Gn(u, v) =
1
s
· e
− u2+v2
2σ2 , s =
w∑
u=−w
w∑
v=−w
e
− u2+v2
2σ2
(a) (u,v) (b) kernel (σ=1.5) (c) normalized
Image Gaussian Filtering: I′(i, j) =
∑w
u=−w
∑w
v=−w
I(i + u, j + v)Gn(u, v)
1low-pass filter
高洪臣 ORB 特征 2020 年 7 月 10 日 8 / 12

9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Separability of the Gaussian filter
s =
w∑
u=−w
w∑
v=−w
g(u) · g(v) =
( w∑
u=−w
g(u)
)
·
( w∑
v=−w
g(v)
)
= s′
· s′
Gn(u, v) =
1
s
· e− u2
2σ2
· e− v2
2σ2
=
1
s
· g(u) · g(v) =
g(u)
s′
·
g(v)
s′
Separable Kernel Matrix:
G(2w+1)×(2w+1) =
1
s








g(−w)g(−w) . . . g(−w)g(0) . . . g(−w)g(w)
...
...
...
g(0)g(−w) . . . g(0)g(0) . . . g(0)g(w)
...
...
...
g(w)g(−w) . . . g(w)g(0) . . . g(w)g(w)








=
1
s′








g(−w)
...
g(0)
...
g(w)








·
1
s′
[
g(−w) . . . g(0) . . . g(w)
]
高洪臣 ORB 特征 2020 年 7 月 10 日 9 / 12

10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Separability of the Gaussian filter
Image Gaussian Filtering:
I′
(i, j) =
w∑
u=−w
w∑
v=−w
I(i + u, j + v)Gn(u, v)
=
w∑
u=−w
w∑
v=−w
I(i + u, j + v)
1
s
g(u)g(v)
=
w∑
u=−w
w∑
v=−w
I(i + u, j + v)
1
s′
g(u)
1
s′
g(v)
=
w∑
u=−w
[ w∑
v=−w
I(i + u, j + v)
g(v)
s′
]
g(u)
s′
=
w∑
u=−w
S(i + u)
g(u)
s′
• kernel size: (2w + 1) × (2w + 1), w = 3
1 int gaussKernel[4] = { 224, 192, 136, 72 };
高洪臣 ORB 特征 2020 年 7 月 10 日 10 / 12

11. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BRIEF
• vector dim: 256 bits (32 bytes)
• each vector ←→ each keypoint
for each bit, select a pair of points in a patch I which centered a corner p and compare
their intensity
S =
( p1, . . . , pn
q1, . . . , qn
)
∈ R(2×2)×256
τ(I; pi, qi) :=
{
1 : I(pi) I(qi)
0 : I(pi) ≥ I(qi)
the descriptor (each bit ←→ each pair of points (pi, qi)):
f(n) =
n
∑
i=1
2i−1
τ(I; pi, qi), (n = 256)
高洪臣 ORB 特征 2020 年 7 月 10 日 11 / 12

12. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
rBRIEF
• construct a lookup table of precomputed BRIEF patterns
1 static int ORB_pattern[256*4] = {
2 8, -3, 9, 5,
3 4, 2, 7,-12,
4 -11, 9, -8, 2,
5 7,-12, 12,-13, // ...
6 }
• steered BRIEF, for each bit of the descriptor
p′
i = p + Rθ(pi − p)
q′
i = p + Rθ(qi − p)
, Rθ =
[
cos θ − sin θ
sin θ cos θ
]
τ(I; p′
i, q′
i) :=
{
1 : I(p′
i) I(q′
i)
0 : I(p′
i) ≥ I(q′
i)
1 float angle = (float)kpt.angle*factorPI;
2 float a = (float)cos(angle), b = (float)sin(angle);
3 #define GET_VALUE(idx)
4 center[cvRound(pattern[idx].x*b + pattern[idx].y*a)*step +
5 cvRound(pattern[idx].x*a - pattern[idx].y*b)]
高洪臣 ORB 特征 2020 年 7 月 10 日 12 / 12