Biological inspired system applied to Computer Vision

Motivation
HMAX Model
Improvements
Summary

Biological Inspired Systems
applied to Computer Vision

Federico Raue Rodriguez
(raue@iupr.com)

IUPR

July 2, 2012

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Federico Raue Rodriguez (raue@iupr.com) Biological Inspired Systems applied to Computer Vision

Motivation
HMAX Model
Improvements
Summary

Contents

1 Motivation

2 HMAX Model

3 Improvements
Sparsity
Pooling Mechanism
Input

4 Summary


Motivation
HMAX Model
Improvements
Summary

(slide from Fundamentals of AI – Prof. De Schreye (KULeuven))


Motivation
HMAX Model
Improvements
Summary


Motivation
HMAX Model
Improvements
Summary

Neuroscience may begin to provide new ideas and approaches
to machine learning, AI and computer vision (Tomaso Poggio)
Interesting properties for visual recognition
a Invariance
b Speciﬁcity
Visual processing in cortex is classically modeled as a hierarchy


Motivation
HMAX Model
Improvements
Summary

(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))

Motivation
HMAX Model
Improvements
Summary

(Perception Strategies in Hierarchical Vision Systems. (Wolf et al))


Motivation
HMAX Model
Improvements
Summary

(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))


Motivation
HMAX Model
Improvements
Summary

Goal: Object categorization based on human visual system
Assumptions:
a Invariance to position and scale
b Feature speciﬁcity must be built up through separate
mechanisms
c Extending the model of simple and complex cells of Hubel and
Wiesel
d Hierarchical feedforward architecture
e Pooling mechanism


Motivation
HMAX Model
Improvements
Summary

(Hierarchical models of Object recognition in cortex (Riesenhuber et al.))


Motivation
HMAX Model
Improvements
Summary

General Description of HMAX model
The standard model consists of four layers of computational units
where simple S units, which combine their inputs with Gaussian-like
tuning to increase object selectivity, alternate with complex C
units, which pool their inputs through maximum operation, thereby
introducing gradual invariance to scale and translation


Motivation
HMAX Model
Improvements
Summary

Simple Cells(S1) is a battery of Gabor filters

X 2 + γ2Y 2 2π
G (x, y ) = exp − × cos X
2σ 2 λ

Complex Cells(C1) show some tolerance to shift and size
a Larger receptive fields
b Shape Invariance: respond to oriented bars or edges anywhere
within their receptive field
c Scale Invariance: more broadly tuned to spatial frequency than
simple cells


Motivation
HMAX Model
Improvements
Summary

Pooling operation from S1 to C1

S1 units: 16 scales arranged in 8 bands
For each orientation, it contains two S1 maps, two ﬁlter size
C1 responses: these maps are sub-sampled using a grid cell of
size N Σ × N Σ (8x8)
From each grid cell we obtain one measurement by taking the
maximum of all 64 elements
As a last stage we take a max over the two scales, by
considering for each cell the maximum value from the two
maps


Motivation
HMAX Model
Improvements
Summary

(Object Recognition with Features Inspired by Visual Cortex (Serre et al.))


Motivation
HMAX Model
Improvements
Summary

Learning Process

Large pool of K patches of various sizes at random positions
are extracted from a target set of images at the C1 level for
all orientations
The patch size is n x n x 4 (The value 4 is due to 4
orientations)
The training process ends by setting each of those patches as
prototypes or centers of the S2 units, which behave as radial
basis function (RBF) units during recognition


Motivation
HMAX Model
Improvements
Summary

Visual words in C2

When a new input is presented, each stored S2 unit is
convolved with the new (C 1)Σ input image at all scales (this
leads to K x 8 (S2)Σ images), where the K factor corresponds
i
to the K patches extracted during learning and the 8 factor,
to the 8 scale bands
After taking a final max for each (S2)i map across all scales
and positions, we get the final set of K shift- and
scale-invariant C2 units
The size of our final C2 feature vector thus depends only on
the number of patches extracted during learning and not no
the input image size


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Contents

1 Motivation

2 HMAX Model

3 Improvements
Sparsity
Pooling Mechanism
Input

4 Summary


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

1 Extend the model using more biological information
Saliency Models
New Pooling mechanism
Redeﬁne the input image
2 Reduce the computational cost


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Biological Motivation
Increase sparsity is to use a lateral inhibition model that
eliminates weaker responses that disagree with the locally
dominant ones
Our attention will be attracted to some locations mostly
because their saliency, deﬁned by contrasts in color, intensity
or orientation
(Treisman) presented a theory about feature integration in
human brain, which has two stages, the simple pre-attention
processing and complex attention processing. Some low level
features will pop up automatically and generate the attention
area in pre-attention processing


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Computational Motivation
Simpliﬁes structures and reduces computational costs
Feature or variable selection
Enhance the generalization ability of learning machines


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

(Multiclass Object Recognition with Sparse, Localized Features (Mutch and
Lowe)


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

n
α
|Fx(i) | + |Fy (i) | ≥ (|Fx(k) | + |Fy (k) |)
n
k=1

(Enhanced Biologically Inspired Model (Huang et al.))


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary



Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

1 Find the maximal response and its neighbors
2 Weak responses are removed due to inhibition eﬀect
3 New pooling Mechanism
a sum the energy of all responses remained by using diﬀerent
weights for S1 units
1
C= [wi S 2 (xi , yi )]
NI 0
xi ,yi ∈I0

(Human age estimation using bio-inspired features (Guo et al.))
b the STD operation is performed on the maximum map using a
cell grid of size Ns x Ns

Ns ×Ns
1 ¯ 2
std = Fi − F
Ns × Ns
i=1


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

1
C= [wi S 2 (xi , yi )]
NI 0
xi ,yi ∈I0


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Relevant Component Analysis (RCA): ﬁnds a linear embedding
transformation that minimizes the distances between points


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Complement using dorsal stream (where)
Cells respond to colored stimuli more strongly than colorless
one in the Inferior Temporal (IT) and the visual areas V4 of
the visual cortex
Analogous to the ’center-on surround-oﬀ’ center surround
processing that occurs in the retina and in the lateral
geniculate nucleus (LGN)
Some region (in the brain) are more active for face images
when compared to images of other objects


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Hierarchical: gradually increase both the selectivity of neurons
along with their invariance to 2D transformations
Hypothesis: neurons in intermediate visual areas of the dorsal
stream such as MT, MST and higher polysensory areas are
tuned to spatio-temporal features of intermediate complexity,
which pool over aﬀerant input units tuned to diﬀerent
directions of motion


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

S1 Units
Gray-value video sequence at all position
Three different types of S1
a Space-time gradient-based: Space and time gradients
It It
| | | |
Ix + 1 Iy + 1
b Optical flow based S1 units: Optical flow of the input using
Lucas & Kanade’s alg.
1
b(θ, θp ) = { [1 + cos(θ − θp )]}q × exp(−|v − vp |)
2
4 directions and two speeds
c Space-time oriented S1 units:
Add a temporal dimension to their receptive fields
3rd derivatives fo Gaussians
8 space-time filters tuned to 4 directions and 2 speeds
Size of receptive fields was 9(pixels)x9(pixels)x9(frames)

Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Proposed by Plebe et al
Extract visual attribute for object recognition based on
infant’s brain
Children between 8 and 10 months old, their object
categorization model is stable and ﬂexible
10-month-old infants ’are sensitive to social cues but cannot
recruit them for word learning’
Early vocabulary is made up of the objects infants most
frequency see
Connectionist model with backpropagation developed a
general model based on similarities without taking into
account physiological and cognitive constraints


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Implementation based on Laterally Interconnected
Synergetically Self Organizing Map architecture (LISSOM)
Hebbian Law: explains the adaptation of neurons in the brain
during the learning process
“. . . , that any two cells or systems of cells that are
repeatedly active at the same time will tend to become
associated, so that activity in one facilitates activity in the
other.”
Two paths: one for visual and the other for auditory channel


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary


Motivation
Sparsity
HMAX Model
Pooling Mechanism
Improvements
Input
Summary

Exposure to stimuli

Visual path in the model develops in two stages.
1 Random blobs, simulating pre-natal waves of spontaneous
activity, known to be essential in the early development of the
visual system
2 Natural images are used (After eye opening)
Auditory path there are diﬀerent stages
1 Random patches in frequency-time domain, with shorter
duration for HPC and longer for LPC
2 7200 most common English words (lengths between 3 and 10
characters)
Last stage: an object is viewed and a word corresponding to
its basic category is heard simultaneouly


Motivation
HMAX Model
Improvements
Summary

Biological models are suitable features for visual recognition
Robust
Invariance
Two Pathways
1 Ventral stream (What?)
2 Dorsal stream (Where?)
Depending on the task HMAX model changes
Parameters (Aging Detection)
Pooling function (Energy model, Standard Deviation)


Biological inspired system applied to Computer Vision

More Related Content

Viewers also liked

Similar to Biological inspired system applied to Computer Vision

Recently uploaded

Biological inspired system applied to Computer Vision