Create lecture part 1 contours and texture

APPLICATIONS OF VISION SCIENCE
PART I:
CONTOURS AND TEXTURES

James
Elder

Centre for Vision Research, York University

2

Applications of Contour and Texture
Processing
Applications of Vision Science Part I: Contours and Textures


Contours
 Background:

Contour extraction
 Application 1. Interactive contour editing (ICE)
 Application 2. Demarcating water features in
satellite imagery


Texture: Enhanced / Synthetic Vision Systems
(ESVS)
 Application

1. Optimal Texture Maps for ESVS
 Application 2. Understanding Texture Fusion

J.
Elder

Applications of Contour Processing
3







Background: Contour extraction
Application 1. Interactive contour editing (ICE)
Application 2. Demarcating water features in
satellite imagery

J.
Elder

4







satellite imagery

J.
Elder

Edge Detectors and Scale Space
5


• Edge detection algorithms based on Gaussian scale space
(Koenderink 84, Young 87, Parent & Zucker 89, Elder &
Zucker 98, Lindeberg et al 98, etc.):
f ( x, y)  

x
2  y
3
x

e

1
 [( x /  x )2  ( y /  y )2 ]
2

J.
Elder

Information Loss
6




What about all of the information we are
throwing away?

J.
Elder

Brightness Filling-In
7


Cornsweet (1970)
J.
Elder

Reconstruction from Contours
8


Elder, IJCV 1999

J.
Elder

9

Blurring of Contours in the Natural
World

J.
Elder

10

Incorporating blur in image
reconstruction


Elder, IJCV 1999

J.
Elder

Reconstruction
11


J.
Elder

12


J.
Elder

But now what?
13


J.
Elder

First-Order Model
14


Contour Grouping: 1st-Order Cues

b

d

Proximity + Good Continuation
(Wertheimer 1923)

a

J.
Elder

Markov Chain Model
15



Model contours as Markov chains: assume long-range statistics
completely determined by local statistics.

n 1

L   Li i ,
i 1

1

p (di i | {ti ,ti }  C )
where L 
ij
p (di i | {ti ,ti }  C )
1

1

1

1

J.
Elder

Tracing Contours using ICE (Elder & Goldberg, PAMI 01)

17

Gestalt Cues: Natural Image
Statistics


Elder & Goldberg, 2002

J.
Elder

18







Application 1. Interactive contour editing
(ICE)
satellite imagery

J.
Elder

User Interface
19


Elder & Goldberg, PAMI 2001
J.
Elder

Rapid Interactive Contour Grouping with ICE

21

Interactive Correction of Grouping
Errors

Grouping Error

Corrected Contour

J.
Elder

Examples
22


J.
Elder

Examples
23


J.
Elder

Examples
24


J.
Elder

25







Application 2. Demarcating water features
in satellite imagery

J.
Elder

The Challenge: Enhanced/Synthetic Vision for
Aviation
26


J.
Elder

3D Database Consistency
27


J.
Elder

Using lakes to index prior models for DEM refinement
28


CDED

Prior SPOT

Posterior SPOT

J.
Elder

Fusing GIS & IKONOS data to Compute Accurate Lake
Boundaries
29


Elder et al, PAMI 2003
J.
Elder

Cues
30




Grouping Cues






Proximity
Good Continuation
Luminance Similarity

Object cues:








Distance between
tangent and model
Angle between tangent
and model
Distance between
tangent and nearest
neighbouring tangent on
dark side
Intensity on dark side of
tangent

J.
Elder

Results
31




Tested on 7 new lakes from
IKONOS data



Average 44% improvement in
accuracy over NTDB vector
data

Algorithm

Human

J.
Elder

Feedback: Prior Knowledge (Elder et al, PAMI 2003)
32


J.
Elder

Related Application: Skin Region Detection
33


J.
Elder

34

Application: Finding Major Skin
Boundaries


J.
Elder

35







satellite imagery

J.
Elder

36

Processing


(ESVS)
 Application


J.
Elder

Aviation
37


J.
Elder

Motivation
38


Enhanced Synthetic Vision System (E/SVS)
(Ricciardi, 2000)

J.
Elder

39

Processing


(ESVS)
 Application


Velisavljevic & Elder 2006 Vision Research.
J.
Elder

Surface Attitude from Texture
40


Slant: 45°, Tilt: 45°

Slant: 45°,Tilt: 135°

J.
Elder

Objectives
41


Objective 1: Determine the optimal texture for
a range of viewing distances.
Objective 2: Determine if texture anisotropy
biases tilt judgements.

J.
Elder

Experiment 1 - Textures
42


Multi-scale
random (2D 1/f)

Multi-scale
random disks

Single-scale
random
(2D bandpass)

Single-scale
random disks

Multi-scale random
rectilinear (1D 1/f)

Single-scale
random rectilinear
(1D bandpass)

Multi-scale
regular rectilinear

Single-scale
regular rectilinear
J.
Elder

Experiment 1
43


Multi-scale random disks
Simulated distance: 26 m

Multi-scale random disks
Simulated distance: 228 m

J.
Elder

General Procedure
44




Observers indicated the perceived surface attitude using a mousecontrolled gauge figure superimposed on a textured plane.



Rotation and tilt were randomly selected between -180 deg and 180 deg.
Slant was randomly distributed between 40 deg and 60 deg in the first
and third experiment but fixed at 60 deg in the second experiment.



There were 20 random slant and tilt pairs for each condition.



Textures were created from 2048x2048 pixel tiles unless otherwise
noted.

J.
Elder

Experiment 1 Results
45

20
Multi-scale
Single-scale

Mean Slant Error (Degrees)

10
0
-10
-20
Range: multi-scale

-30

Range: single-scale

-40
-50

10 0

10 2
Simulated Distance (Meters)

10 4

Multi-scale random
rectilinear (1D 1/f)
J.
Elder

Experiment 2 - Procedure
46


To test the effects of anisotropy, we used three
textures:

Multi-scale disks Multi-scale random rectilinearSingle-scale triangles
Isotropic
Anisotropic (90°)
Anisotropic (45°)

Texture rotation relative to tilt was randomly selected to
be between [-45°, -30°, -15°, 0°, 15°, or 30°].
J.
Elder

Experiment 2 - Procedure
47


Multi-scale random rectilinear
Slant: 60°, Tilt: 0°, Rotation: 0°
Tilt (0°) - Rotation (0°) = 0°

Multi-scale random rectilinear
Slant: 60°, Tilt: 0°, Rotation: 30°
Tilt (0°) - Rotation (30°) = -30°
J.
Elder

48

Bias and Precision in Tilt
Judgements




Despite bias induced by rectilinear structure,
rectilinear textures yield more accurate tilt
judgements.

J.
Elder

Conclusions
49




Multi-scale textures support accurate surface
attitude judgements over a greater range of
viewing distances than single-scale textures.



Attitude judgements are best with structured
textures. However, textures with anisotropic
structure induce a bias in tilt judgements.

J.
Elder

50

Processing


(ESVS)
 Application


J.
Elder

Data fusion in the natural world
51


Natural images may contain mixtures of independent textures. These
textures may project from the same surface, or they may project from
distinct transparent surfaces.

J.
Elder

Aviation
52


J.
Elder

Example
54


Slant = 35°
Tilt A = 150°
Tilt B2 = 60°
Tilt Difference = 90°

J.
Elder

When do observers perceive a single
surface?
55


Texture A

P(1 Surface Perceived)

Perception of a single surface
1.2
Texture B1

1
0.8
0.6

Texture B2

0.4
0.2
0
0

30

60

90

120

150

180

Tilt difference (deg)

Texture B1

Texture B2

Texture B3

Texture B3
J.
Elder

Perception of 2 Distinct Surfaces
56




When 2 surfaces are perceived, what are their
perceived attitudes?
Mean Relative Perceived Tilt
Relative Tilts for Textures

120

120

90

90

90

60

60

30
0
0

30

60

90

120

150

180

-60
-90

30
0
-30

0

30

60

90

120

150

180

-60

Relative tilt (deg)

60
Relative tilt (deg)

Relative tilt (deg)

A &B3

A & B2

120

-30



A & B1

30
0
-30

0

30

60

90

120

150

180

-60
-90

-90

-120

-120

True Tilt of A
Perceived Tilt of A

Perceived Tilt of B1

-120

Tilt Difference (Deg)

True Tilt of B1

Texture B1

True Tilt of B2

Perceived Tilt of A

Texture A

True Tilt of A

True Tilt of A

True Tilt of B3

Perceived Tilt of A



Texture A

Texture B2

Texture A

Texture B3

J.
Elder


Relative tilt (deg)

57

Mean Perceived Tilt of Unitary
Surface
60

30
0

30

60

90

-30
-60
True Tilt A

Tilt Difference (Deg)
True Tilt B1 Perceived Surface pt. Prob. Model
O

J.
Elder

Modeling Fusion
58




The percept of a unitary surface may arise for 1 of 2
reasons:


Selection: each judgement based EITHER on
information from Texture A OR Texture B (may change
from trial to trial, subject to subject)



Fusion: each judgement based on a fusion of information
from both textures

J.
Elder

Selection or Fusion
59


Selection

0.02

Fusion

denotes
true tilt of
surface

0.03

0.02

Data

p

Selection A
Selection B

0.01

Data
p

0.03

Fusion
0.01

0

0

Relative tilt (deg)
Distribution of perceived tilts
modeled as a mixture of 2
Gaussians with means μA, μB equal
to the true relative tilts of the 2
surfaces.

Relative tilt (deg)
Distribution of perceived tilts by a
single Gaussian of unknown
mean μ.

J.
Elder

Bayesian Model Selection
60


Log(p(fusion)/p(selection))

* log
30

¹ 0, p<0.05

30

*

20
10

p(fusion)
p(selection)

20

*

*

20

10

10

0

*

0

30

0

-10

-10

-20

* *

-20

-30

50

100

150


Texture A

Texture B1

-30

100

150


Texture A

Texture B2

*

*

-20
-30

Selection

*

-10

*
50

Fusion

50

100

150

*


Texture A

Texture B3

J.
Elder

Methods
62



Stimuli







Two planar surfaces (A, B) were rendered in full perspective
within a 24 deg window.
Tilts of the 2 surfaces were identical and were uniformly
distributed over [-180 180] deg.
Mean slant of the 2 surfaces was random and uniformly
distributed over [30 40] deg.
Slant difference between the 2 surfaces varied in a block
design over
{-40, -26.67, -13.34, 0, 13.34, 26.67, 40} deg.

Procedure





Overlaid textured surfaces were visible for an unlimited amount
of time.
Participants indicated:
a) whether they perceived 1 or 2 distinct 3-D surfaces;
b) the perceived attitude of the surface(s) using a
superimposed
mouse-controlled gauge figure.
J.
Elder

Perception of 2 Distinct
Surfaces

Results
64




2 distinct surfaces are perceived more often
when the strong texture is less slanted.
A

B

p(2 Surfaces Perceived)

1
0.8
0.6
0.4
0.2
0
-40

A-B

-26.67

-13.34

0

13.34

26.67

40

Slant Difference (degrees)

J.
Elder

Slant Precision
65




The precision of slant estimation is unaffected
by the other texture (replicates Rosenholtz &
Malik, 1995).
SD Perceived Slant (deg)

Perception of Surface A
SD Perceived Slant
(deg)

25
20
15
10
5
0

alone

+ B1

A

+ B2

+ B3

Perception of Surface B
20
18
16
14
12
10
8
6
4
2
0

alone

B1

+A

alone

B2

+A

alone

+A

B3
J.
Elder

Slant Accuracy
66




However, the accuracy of slant estimation is
strongly affected by the other texture. Slants
are grossly underestimated.
* p(alone=paired)<.05
-20

*

-8
-6
-4
-2
0
2

+ B1

A

+ B2

*

-16

+ B3

*

*

alone + A

alone + A

-14
-12
-10
-8
-6
-4
-2
0

alone

Perception of Surface B

-18

Slant Error (deg)

Slant Error (deg)

-10

Perception of Surface A

B1

B2

alone + A

B3
J.
Elder

Modeling Slant Capture
68


Perception of fused surface is biased toward surface with greater s
If sources of error are independent and normally-distributed,
optimal fusion estimator has the form
P   A A  BB
where
P is the perceived slant of the fused surface
A , B are the perceived slants of each surface alone
 A , B are the weights assigned to the 2 surfaces

J.
Elder

Modeling Slant Capture
69


For optimal estimation, A ,B reflect the inverse variance of the sources.
Geometry suggests that variance  1/ sin2  , thus

k A sin2 A
A 
k A sin2 A  kB sin2 B
kB sin2 B
B 
k A sin2 A  kB sin2 B
Here, k A ,kB are 'mixing constants' (k A  kB =1), independent
of geometry, that reflect the relative strength of the textures

J.
Elder

70

Modeling Slant
Capture


20
15
10
5
0

Slant relative to mean slant (deg)

-40

-26.67

-13.34

0

13.34

-40

-26.67

-13.34

0

13.34

26.67

40

-5

kA = 0.51
kB = 0.49

-10
-15
-20

20
15
10
5
0
26.67

40

-5

kA = 0.40
kB = 0.60

-10
-15
-20

20

Perceived slant of
surface B alone

15
10
5
0
-40
-5

-26.67

-13.34

0

13.34

26.67

40

kA = 0.11
kB = 0.89

Perceived slant of
surface A alone

-10
-15
-20

Optimal fusion
estimator

J.
Elder

71

Processing


(ESVS)
 Application


J.
Elder

Create lecture part 1 contours and texture

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Create lecture part 1 contours and texture

Similar to Create lecture part 1 contours and texture (20)

Recently uploaded

Recently uploaded (20)

Create lecture part 1 contours and texture