Invariant test

Invariant models of vision between
phenomenology, image statistics and neurosciences

Gonzalo Sanguinetti

Universidad de la Rep´blica, Montevideo, Uruguay
u

Thesis Directors:
Prof. Giovanna Citti
Prof. Alessandro Sarti

March 28th, 2011

Gonzalo Sanguinetti (Udelar) Phd thesis March 28th, 2011 1 / 34

Outline

1 Background

2 Natural Image Statistics
Computation of the histograms
Relation with the Cortical Model
Stochastic Model

3 Scale
The symplectic model
Image Statistics

4 Ladders


Association Fields

Psychophysical experiment

[Field, Hayes,Hess, 1993]

The visual pathway and the 3 main cortical structures

L
R
L
R I
L
II
R
III
L
R IV
V
VI


The visual Cortex is a Fiber Bundle: R2 ×S 1
V1 Simple cells
[DeAngelis et al., 1995]

Fitting with a DoG wavelet

To each retinal point (x, y ) is associated a copy of the set S 1 .
Each point g = (x, y , θ) ∈ R2 ×S 1 represents a set of cells with the same OP
θ and RP centered at (x, y )
The space is identiﬁed with the SE (2) if it is considered with the appropriate
group operation [Citti and Sarti, 2006].

Lifting of Curves into the Left Invariant Structure

Non-maximal suppression

Left-invariant basis:

X1 = (cos θ, sin θ, 0)
X2 = (0, 0, 1)
X3 = (− sin θ, cos θ, 0)

associated to the diﬀ. operators
(Xi (f ) =< X , f >):
Sub-Riemannian structure
X1 = cos θ∂x + sin θ∂y
Only the vector ﬁelds X1 and X2 are
X2 = ∂θ
considered, a subset of the tangent space.
X3 = − sin θ∂x + cos θ∂y


Integral curves and horizontal connections
The lateral connectivity is modeled as
integral curves with constant coeﬃcients:
y

γ(t) = X1 (t) + k X2 (t)
˙

The solution is (for x0 = y0 = θ0 = 0)
sin(kt)

 x= k
1−cos(kt)
 y= k Θ
θ = kt

x

[Field, Hayes, Hess, 1993]


Outline

1 Background

Stochastic Model

3 Scale
Image Statistics

4 Ladders


Natural Image database

Image Database: 4000 images, 1536×1024 pixels

From http://hlab.phys.rug.nl/imlib/index.html

Processing of the images
Bank of oriented wavelets (steerable).


Processing of the images
Bank of oriented wavelets (steerable).

(x0 , y0 , θ0 )
(x1 , y1 , θ1 )

(xi , yi , θi )

(xN , yN , θN )


Cross-correlation assuming translation invariance
Construction of a 4D histogram

H(∆x, ∆y , θ0 , θ1 )
x1 ,y1 ,Θ1

y

xo ,yo ,Θo x

|∆x|, |∆y | < 32 px, S 1 discretized in 32 diﬀerent values,
then H is large 65×65×32×32


Co-occurrence Histogram

H(∆x, ∆y , 0, 0) (two horizontal edges)


Co-occurrence Histogram

H(∆x, ∆y , 0, 0) (two horizontal edges)

1.5 107

1.0 107

5.0 106

x
30 20 10 10 20 30


4D histogram


3D histogram


Comparison with the Association Fields

H(x, y , θm (x, y )) = maxθ∈S 1 H(x, y , θ) Mean error from co-circularity condition:
V (x, y ) = (cos(θm (x, y )), sin(θm (x, y )))
Eθ = n
1
x,y
θm (x, y ) − 2 arctan x
y
2
≈ 0.2rad ≈ 8◦

3Π
8

Π
4

Π
8

Θm x,y
0

Π
8

Π
4

3Π
8

3Π Π Π Π Π 3Π
4 8
0 8 4
8 8
2atan y x


Probabilistic framework

Mumford’s direction process
Langevin equation (SDE):
s
x(s) = 0 cos θ(t)dt + x(0)
s
y (s) = 0 sin θ(t)dt + y (0)
θ(s) = σW (s)

W (s) is a Brownian motion


Time dependent Fokker-Planck equation

Fokker-Planck equation associated to Mumford’s stochastic process:

σ2
∂t p = − cos(θ)∂x p − sin(θ)∂y p + ∂θθ p
2
where p(x, y , θ, t) is the transition probability from an initial state:
 
x ≤ x(t) ≤ x + ∆x x(0) = 0
p(x, y , θ, t)∆x∆y ∆θ = P  y ≤ y (t) ≤ y + ∆y y (0) = 0 
θ ≤ θ(t) ≤ θ + ∆θ θ(0) = 0


Time dependent Fokker-Planck equation

Fokker-Planck equation associated to Mumford’s stochastic process:

σ2
∂t p = − cos(θ)∂x p − sin(θ)∂y p + ∂θθ p
2
where p(x, y , θ, t) is the transition probability from an initial state:
 
x ≤ x(t) ≤ x + ∆x x(0) = 0
p(x, y , θ, t)∆x∆y ∆θ = P  y ≤ y (t) ≤ y + ∆y y (0) = 0 
θ ≤ θ(t) ≤ θ + ∆θ θ(0) = 0

It can be written in terms of left invariant vector ﬁelds of SE(2):

σ2
∂t p = −X1 p + X22 p, X22 = X2 (X2 ) = ∂θθ
2
advection in the direction X1 = cos θ∂x + sin θ∂y and;
diﬀusion in the direction of X2 = ∂θ .


Time independent Fokker-Planck equation
forward - backward

We propose to use the fundamental solution of the time independent
equation, i.e. the solution of

σ2 1
−X1 p(x, y , θ) + X22 p(x, y , θ) = δ(x, y , θ)
2 2
plus the fundamental solution of its backward equation:

σ2 1
X1 p(x, y , θ) +
X22 p(x, y , θ) = δ(x, y , θ)
2 2
Only one free parameter: σ, the variance of the underlying stochastic process.
The solution was numerically computed using COMSOL Multiphysics, a
commercial FEM solver.


Fitting with the statistics.

σ = 1.7px minimizes the mean square error E . At the minimum, E ≈ 2%

Fokker-Planck fundamental solution. Histogram of co-occurrences.


Comparison with the probabilistic model
Level curves of the Fokker-Planck fundamental solution

Fokker−Plank fundamental solution level sets: θ = 0 x 10
−4 Fokker−Plank fundamental solution level sets: θ = 11.25 x 10
−4

30 30 5 30
8
2.5
4.5
20 7 20 20
4

6 2
10 10 3.5 10

5
3
∆y

∆y

∆y
0 0 0 1.5
4 2.5

−10 −10 2 −10
3
1
1.5
2
−20 −20 −20
1

1 0.5
−30 −30 0.5 −30

−30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30
∆x ∆x ∆x
Fokker−Plank fundamental solution level sets: θ = 45 x 10
−5

9 6.5
30 30 30
12

11 6
8
20 20 20

10
5.5
7
10 10 10
9
5
8
6
∆y

∆y

∆y
0 0 0
4.5
7

−10 −10 5 −10
6 4

5
−20 −20 4 −20 3.5

4
3
−30 −30 3 −30
3
−30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30
∆x ∆x ∆x


Comparison with the probabilistic model
Level curves of the histogram

Co−occurrences histogram level sets: θ = 0 x 10
−4 Co−occurrences histogram level sets: θ = 11.25 x 10
−4

30 30 5 30 2.5
8

4.5
20 7 20 20
4 2

6
10 10 3.5 10

5 3
1.5
∆y

∆y

∆y
0 0 0
2.5
4

−10 −10 2 −10
1
3
1.5
−20 −20 −20
2 1
0.5

−30 −30 0.5 −30
1
−30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30
∆x ∆x ∆x
Co−occurrences histogram level sets: θ = 45 x 10
−5

30 30 30 6.5
12
8
11 6
20 20 20

10
7 5.5

10 10 10
9
5
6
8
∆y

∆y

∆y
0 0 0
4.5
7
5
−10 −10 −10 4
6

5
4 3.5
−20 −20 −20

4
3
−30 −30 3 −30
3
−30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30 −30 −20 −10 0 10 20 30
∆x ∆x ∆x


Outline

1 Background

Stochastic Model

3 Scale
Image Statistics

4 Ladders


Extension to the aﬃne group [Sarti, Citti, Petitot, 2008]
Invariant under translations, rotations and scaling transformations

2
+η 2 )
ϕ0 (ξ, η) = e −(ξ cos(2η)

ϕx,y ,θ,σ (ξ, η) = ϕ0 A−1 (ξ, η)

Ax,y ,θ,σ (ξ, η) =

x cos θ − sin θ ξ
+ eσ
y sin θ cos θ η


Lifting into R2 × S 1 × R+
Geometric interpretation

The scale parameter σ may be interpreted as the distance to boundary.

Θ

1
eΣ
2

x,y

Oθm ,σm (x, y ) = max Oθ,σ (x, y )
(θ,σ)

The function O represents the output of the cells, i.e. the cortical activity.

Space diﬀerential structure and connectivity
Symplectic structure, 2 sub-Riemannian metrics.
y y

Left Invariant Vector Fields:

Θ Σ

Xσ,1 = e σ (cos θ∂x + sin θ∂y )
Xσ,2 = ∂θ
Xσ,3 = e σ (− sin θ∂x + cos θ∂y )
Xσ,4 = ∂σ x x

2 types of Integral curves:

γ(t) = Xσ,1 (γ(t)) + kXσ,2 (γ(t))
˙
γ(0) = (x0 , y0 , θ0 , σ0 ) y

γ(t) = Xσ,3 (γ(t)) + kXσ,4 (γ(t))
˙
γ(0) = (x0 , y0 , θ0 , σ0 )
x


Image Statistics

Same methodology
Even symmetric filters, implemented with the steerable architecture:

Each detected edge is a 4d point (x, y , θ, σ)
Histogram of co-occurrences (translation invariance assumption) is 6D
H(∆x, ∆y , θc , θp , σc , σp )
|∆x|, |∆y | ≤ 16px, 8 different orientation, 10 different scales.
Dimensions of H, 33 × 33 × 8 × 8 × 10 × 10


Results
Visualization of 5 dimensions

σc = 3px ﬁxed at the lowest scale


Plane spanned by {X3,σ , X4,σ }

θc , θp are ﬁxed


The model of the connectivity.

Integral curves of X3,σ + αX4,σ :
y

Pure advection in the directions:
Xσ,3 − Xσ,4 and Xσ,3 + Xσ,4 .

Σ

x


Test on the synthetic cartoon image database

100 images randomly generated


Outline

1 Background

Stochastic Model

3 Scale
Image Statistics

4 Ladders


Snakes vs Ladders
Psychophysical experiment

[May-Hess, 2008]

“increasing the separation between the
elements had a disruptive eﬀect on the
detection of snakes but had no eﬀect on
ladders, so that as separation increased,
performance on the two types
converged”


Reinterpretation of the results

s s

Α Α


Natural Image Statistics
Snake vs Ladder
1.0

0.9

0.8

0.7

0.6

0.5

1.09 1.54 2.18 3.08 0.4
1.09 1.54 2.18 3.08

1.0 0.30

0.25
0.8

0.20
0.6
0.15
0.4
0.10

0.2
0.05

0.0 0.00
40 20 0 20 40 10 15 20 25 30 35 40 45


Invariant test

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Invariant test

Similar to Invariant test (20)

Recently uploaded

Recently uploaded (20)

Invariant test