Presentation at SSVM07 (as a poster).

1. Spatio-Temporal
Scale-Spaces
Daniel Fagerström
CVAP/CSC/KTH
danielf@nada.kth.se

2. Motivation
• Allow for differential geometric methods on
moving images
• Temporal causality
• Recursive formulation based on current
signal and earlier measurement rather
than on previous signal
– More realistic from a conceptual view point
– Much more efficient from a computational
view point

3. Earlier Approaches
• Koenderink (88), Florack (97), Salden et al
(98) – Requires convolution with past
signal, non recursive
• Lindeberg (02)
– Recursive, but discrete in the spatio-temporal
dimensions

4. Current Approach
• ”Standard” axioms + Galelian similarity
• Infinitesimal formulation
• No time causal solution possible
• But scale-space on space and memory
exists

5. Galilean Similarity
• Spatial and temporal translation, spatial
rotation and spatio-temporal shear
• Allows for measurement of relative motion
µ ¶ µ ¶µ ¶
t0 ¿ 0 t
= +a
x0 v ¾R x
x; v 2 Rn ; t 2 R; R 2 SO(n); ¾; ¿ 2 R+ and a 2 Rn+1

6. Point Measurement
De¯nition © : L1 (Rn+1 ) ! C 1 (Rn+1 ), for any u; v 2 L1 (Rn+1 ) and ®; ¯ 2
R, is a point measurement operator if it ful¯lls:
linearity ©(®u + ¯v) = ®©(u) + ¯©(v)
gray level invariance k©ukL = kukL
positivity u ¸ 0 ) ©u ¸ 0
1 1
point lims!0 ©s u = u

7. G Covariance
De¯nition Given a Lie group g 2 G that acts on the base space as g 7!
g ¢ x = Tg x, where Tg : (Rn+1 ). A family H 3 h 7! ©h of measurement
operators ful¯lling,
Tg ©h Tg ¡1 = ©g ¢h ;
is called aG-covariant point measurement space. Where,
G £ H 3 (g; h) 7! g ¢ h = ¾(g; h) = ¾g (h) 2 H;
is a Lie group action on the set H.

8. G Scale-Space
De¯nition A G scale-space is a minimal family of G-covariant point measure-
ments that also is a semigroup of operators. I.e. that ful¯lls,
©h ©h = ©h ¢h :
1 2 1 2

9. Infinitesimal Criteria
An infinitesimal condition for G scale-
spaces:
• Simplifies derivations as it liniarizes the
problem
• Allows for more general boundary
conditions (which later will be seen to be
necessary)

10. Infinitesimal Generators
• The infinitesimal object that corresponds
to a Lie groupLGis a g algebra
G Lie
=
• The infinitesimal object that corresponds
LH h
to a Lie semigroup H is a Lie wedge
=
• Infinitesimal generators v 7! A
A dT (e) : g ! L(§; §), g 3
= v = A(v),
B = d©(e) : h ! L(§; §), h 3 w 7! Bw = B(w)

11. Pseudo Differential Operators
The in¯ntesimal operators of transformation group and the semigroup of
operators can be expressed in terms of pseudodi®erential operators, ªDO.
ªDO's are de¯ned by
Z
Au(x) = (2¼)¡n eix¢» a(x; »)~(»)d»;
u
R P
where u(») = e¡ix¢» u(x)dx and a(x; ») = j j· a® (x)» ® , and is called the
~
® m
symbol of A. The corresponding operator is denoted a(x; D).
Translation invariant symbols are position independent, b(x; ») = b(»).

12. Infinitesimal Covariance
The in¯nitesinal form of the covariance equation is,
[Av ; Bw ] = BC(v;w) ;
where g £ h 3 (v; w) 7! C(v; w) 2 h is the di®erential of ¾ with respect to both
arguments. And an operator Bw full¯lling such a condition is called a
covariant tensor operator. The operator C is denoted the covariance tensor
and is a Lie algebra representation in the ¯rst argument and a Lie wedge
representation in the second.

13. Positivity and Gray Level
Invariance
• A positive translation invariant linear
operator semigroup is generated by a
negative definite symbol
• For a gray level invariant semigroup that is
generated by the generators B, the
genrators B are conservative, i.e. B1=0

14. Infinitesimal G Scale-Space
De¯nition A g scale-space wedge, is a minimal Lie wedge of negative de¯nite
conservative operators h 3 w 7! Bw : L(§; §), that is a covariant tensor oper-
ator, with respect of the Lie algebra action g 3 v 7! Av : L(§; §) and the Lie
algebra and Lie wedge representation (v; w) 7! C(v; w).
Theorem A G scale-space is generated by its corresponding g scale-space
wedge, ½
@ s u = Bw u
u(e; x) = f (x);
where f 2 §.

15. Causal Affine Line
Theorem A gl(1)(= f@x ; x@x g) temporally causal scale-space wedge is gener-
ated by left sided Riemann Liouville fractional derivatives D ® with 0 < ® < 1,
+
where
D§ (») = (¨i»)® = j» j® e¨i(»)®¼=2 :
®

16. Euclidean Similarity
Theorem A Euclidean similarity es(2) = t(2) [ s(2) [ so(2) scale-space wedge
is generated for any 0 < ® · 2 by the Riesz fractional derivative ¡(¡¢)®=2 (»
j» j® ).
t(2) = f@1 ; @2 g is the translation generator, s(2) = fx1 @1 + x2 @2 g is the
scaling generator and so(2) = fx2 @1 ¡ x1 @2 g is the rotation generator.

17. No Time Causal Galilean Similarity
Theorem A °s(2) scale-space wedge is generated by f@0 ; @0 @1 ; @1 g.
2 2
°s(2) = t(2) [ s(1) © s(1) [ °(1);
where °(1) = fx0 @1 g is the Galilean boost that skewquot; space-time and s(1)©s(1)
is a direct sum of the scaling generator in space and time respectively.
Corollary It should be noted that the generated scale-space is symmetric
booth in time and space and thus no time causal scale-spaces are possible with
this axiomatization. And as °s(n); n ¸ 2 have °s(2) as a sub algebra, no time
causal scale-spaces are possible for them either.

18. What to do?
• A realistic temporal measurement system
should not have access to the past signal,
only the memory of the past signal
• Let the memory have the same propeties
as the past signal, i.e. it should be an
affine half space
• Define the time causal Galilean scale-
space as a evolution equation on space
and memory instead of on space and time

19. Galilean Similarity
De¯nition Let °s(d + 1) = es(d) © gl(1) [ °(d). The d + 1-dimensional
time causal Galilean scale-space R+ £ Rd £ R+ £ R £ Rd 3 (¾; v; ¿; t; x) 7!
u(¾; v; ¿; t; x) 2 R, where ¾ is spatial scale, v is velocity, ¿ is memory (and
temporal scale) is a °s(d + 1)-covariant, point measurement space in space-time
(t; x) and a gl(1)-wedge in memory ¿ .

20. General Evolution Equation
Theorem A d + 1-dimensional time causal Galilean scale-space (for d = 1; 2)
is generated by the evolution equation,
8
< @tu = ¡v ¢ rxu + D¿ ¡u ®0
@¾ u = ¡(¡¢x)®=2u
:
u(0; 0; 0; t; x) = f(t; x);
where 1 < ®0 · 2, 0 < ® · 2, ¾ is the spatial scale direction, v 2 Rd the velocity
vector, @s = @t + v ¢ rx is the spatio-temporal direction, rx = (@1; : : : ; @d) is
the spatial gradient and ¢x is the spatial Laplacian.

21. Heat Equation on Half-Space
Theorem The equation,
8
< @tu = ¡v ¢ rxu + @2u
¿
@¾ u = ¢xu
:
u(0; 0; 0; t; x) = f(t; x);
is unique in the family of evolution equations as it is the only one that has local
generators.

22. Closed Form
u(¾; v; ¿; t; x) = Á(¾; v; ¿; ¢; ¢) ¤ f (t; x);
¿ exp(¡ ¿ 2 ¡ (x¡tv)¢(x¡tv) )
p 4t
Á(¾; v; ¿; t; x) = 4¾
4¼t3=2 (4¼¾)d=2

23. Mixed Derivatives