1. Efficient Coding Hypothesis
Idea: “Because not all signals are equally likely, it is natural
to assume that perceptual systems should be able to best
process those signals that occur most frequently. Thus, it
is the statistical properties of the environment that are
relevant for sensory processing.”
Goal: “the establishment of a precise quantitative
relationship between environmental statistics and neural
processing...” (Simoncelli & Olshausen, 2001)

2. Standard Linear ICA
N
I  x , y =∑ ai  x , y  s i
image model:
●
i=1
si= 〈 w i , I 〉
extraction of independent components:
●
linear ICA yields linear filters similar to simple cell RFs, e.g.
●
Olshausen & Field (1997), Bell & Sejnowski (1997)
cannot model invariances; estimated components are usually
●
not independent.

3. Topographic ICA model: Hyvärinen & Hoyer
(2001)
Main idea: invariant feature subspaces + “making the
●
dependency structure of the estimated components visible”
N
2
c it =∑ hi , j 〈 w j , I t 〉
j=1
Log-likelihood:
●
T N
log L I 1 ,... , I T ; w 1 , ... , w N =∑ ∑ G cit 
t =1 i=1

4. Topographic ICA: results
L in e a r F ilt e rs

5. Optimal stimuli
x-
x+
Optimal
Optimal
inhibitory
excitatory
stimuli
stimuli

6. Invariances learned by TICA
translation invariance:
phase invariance: ●
●
optimal optimal
-π/2 shifted stimulus π/2 shifted -π/6 shifted stimulus π/6 shifted
For the analysis and interpretation of the quadratic forms:
http://www.gatsby.ucl.ac.uk/~berkes/software/qforms-tk/index.html

7. Comparison with SFA
SFA is a more intuitive and principled model for complex
●
cells than TICA
Most variants of ICA don't take time structure into account
●
(but, Blaschke et al., 2006)
SFA yields a richer repertoire of complex cell responses
●
(Berkes & Wiskott, 2005):
direction selectivity,
–
side- and end-inhibition,
–
active inhibition,
–
near-optimal response to corners and T-shaped stimuli
–

8. Discussion
Lindgren & Hyvärinen (2006): “Emergence of conjuctive
●
visual features by quadratic independent component
analysis” ?
Enter: pros & cons of semi-analytic derivation of the
●
properties of sensory processing.