This document discusses approximation and compression using orthogonal decompositions. It covers approximation error decay rates for different types of functions using transforms like Fourier and wavelets. Fourier bases provide good approximation for smooth functions, while wavelets work well for piecewise smooth functions with singularities. The document explores compression techniques like thresholding and quantization of transform coefficients. It analyzes the rate of approximation error decay and number of bits needed for compression under different assumptions about the function regularity.
Interactive Visualization in Human Time -StampedeCon 2015StampedeCon
At the StampedeCon 2015 Big Data Conference: Visualizing large amounts of data interactively can stress the limits of computer resources and human patience. Shaping data and the way it is viewed can allow exploration of large data sets interactively. Here we will look at how to generate a large amount of data and to organize it so that it can be explored interactively. We will use financial engineering as a platform to show approaches to making the large amount of data viewable.
Many techniques in financial engineering utilize a co-variance matrix. A co-variance matrix contains the square of the number of individual data series. Interacting with this data might require generating the matrix for thousands to millions of different starting and ending time combinations. We explore aggregation techniques to visualize this data interactively without spending more time than is available nor using more storage than can be found.
Interactive Visualization in Human Time -StampedeCon 2015StampedeCon
At the StampedeCon 2015 Big Data Conference: Visualizing large amounts of data interactively can stress the limits of computer resources and human patience. Shaping data and the way it is viewed can allow exploration of large data sets interactively. Here we will look at how to generate a large amount of data and to organize it so that it can be explored interactively. We will use financial engineering as a platform to show approaches to making the large amount of data viewable.
Many techniques in financial engineering utilize a co-variance matrix. A co-variance matrix contains the square of the number of individual data series. Interacting with this data might require generating the matrix for thousands to millions of different starting and ending time combinations. We explore aggregation techniques to visualize this data interactively without spending more time than is available nor using more storage than can be found.
Overview of Stochastic Calculus FoundationsAshwin Rao
This is a quick refresher/overview of Stochastic Calculus Foundations. This assumes you have done a Stochastic Calculus course previously and now want to review/revise the material to prepare for a course that lists Stochastic Calculus as a pre-req. In these 11 slides, I list the key content you must be familiar with within Stochastic Calculus.
What is Fourier Transform
Spatial to Frequency Domain
Fourier Transform
Forward Fourier and Inverse Fourier transforms
Properties of Fourier Transforms
Fourier Transformation in Image processing
Overview of Stochastic Calculus FoundationsAshwin Rao
This is a quick refresher/overview of Stochastic Calculus Foundations. This assumes you have done a Stochastic Calculus course previously and now want to review/revise the material to prepare for a course that lists Stochastic Calculus as a pre-req. In these 11 slides, I list the key content you must be familiar with within Stochastic Calculus.
What is Fourier Transform
Spatial to Frequency Domain
Fourier Transform
Forward Fourier and Inverse Fourier transforms
Properties of Fourier Transforms
Fourier Transformation in Image processing
Model Selection with Piecewise Regular GaugesGabriel Peyré
Talk given at Sampta 2013.
The corresponding paper is :
Model Selection with Piecewise Regular Gauges (S. Vaiter, M. Golbabaee, J. Fadili, G. Peyré), Technical report, Preprint hal-00842603, 2013.
http://hal.archives-ouvertes.fr/hal-00842603/
1. Approximation and Coding
with Orthogonal
Decompositions
Gabriel Peyré
http://www.ceremade.dauphine.fr/~peyre/numerical-tour/
2. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons
10. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons
12. Compression by Transform-coding
forward
f
transform
a[m] = ⇥f, m⇤ R q[m] Z
bin T
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Image f Zoom on f Quantized q[m]
13. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Image f Zoom on f Quantized q[m]
14. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
ing
decod
dequantization
a[m]
˜ q[m] Z
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Dequantization:
Image f Zoom on f Quantized q[m]
15. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
ing
decod
backward dequantization
fR = a[m]
˜ m a[m]
˜ q[m] Z
transform
m IT
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Dequantization:
Image f Zoom on f Quantized q[m] fR , R =0.2 bit/pixel
18. Non-linear Approximation and Compression
2T T T 2T a[m]
a[m]
˜
⇥
|a[m]|
Quantization: q[m] = sign(a[m]) Z T
T =⇥ |a[m] a[m]|
˜
⇥
1 2
Dequantization: a[m] = sign(q[m]) |q[m]| +
˜ T
2
19. Non-linear Approximation and Compression
2T T T 2T a[m]
a[m]
˜
⇥
|a[m]|
Quantization: q[m] = sign(a[m]) Z T
T =⇥ |a[m] a[m]|
˜
⇥
1 2
Dequantization: a[m] = sign(q[m]) |q[m]| +
˜ T
2
⇤ ⇤ ⇤ ⇥2
T
||f fR || =
2
(a[m] a[m])
˜ 2
|a[m]| + 2
m
2
|a[m]|<T |a[m]| T
||f fM ||2 #=M
Theorem: ||f fR ||2 ||f fM ||2 + M T 2 /4
where M = # {m a[m] = 0}.
˜
20. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
21. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
22. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M
23. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M
= Rval = O(M | log2 (T )|) = O(M log2 (M ))
24. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M
= Rval = O(M | log2 (T )|) = O(M log2 (M ))
Theorem: Under hypotheses (H1 ) and (H2 ), ||f fR ||2 = O(R log (R)).
35. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons
42. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons
47. 1D Wavelet Coefficient Behavior
1
0.8
0.6
0.4
0.2
0
0.2
0.1
0
−0.1
−0.2
0.2
If f is C in supp( j,n ), then 0.1
0
2 j( +1/2) −0.1
| f, j,n ⇥| ||f ||C || ||1 −0.2
0.5
0
−0.5
0.5
0
−0.5
48. 1D Wavelet Coefficient Behavior
1
0.8
0.6
0.4
0.2
0
0.2
0.1
0
−0.1
−0.2
0.2
If f is C in supp( j,n ), then 0.1
0
2 j( +1/2) −0.1
| f, j,n ⇥| ||f ||C || ||1 −0.2
0.5
If f is bounded (e.g. around 0
a singularity), then −0.5
0.5
| f, j,n ⇥| 2 j/2
||f || || ||1
0
−0.5
49. Piecewise Regular Functions in 1D
Theorem: If f is C outside a finite set of discontinuities:
O(M 1
) (Fourier),
n
[M ] = ||f fM ||2
n
=
O(M 2
) (wavelets).
For Fourier, linear non-linear, sub-optimal.
For wavelets, linear non-linear, optimal.
51. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
52. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2
53. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2
1
Singular: 2j1
= (T /C) +1/2
Cut-o scales (depends on T ):
Regular: 2j2 = (T /C)2
54. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2
1
Singular: 2j1
= (T /C) +1/2
Cut-o scales (depends on T ):
Regular: 2j2 = (T /C)2
Hand-made approximate: ˜
fM = f, j,n ⇥ j,n + f, j,n ⇥ j,n
j j2 n Cj c
j j1 n Cj
55. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2j2 2 j1
) = O(T + T
2 2
+1/2 ) = O(T
2
+1/2 ) regular part
56. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T2 2
+1/2 ) = O(T
2
+1/2 ) regular part
M |Cj | + c
|Cj | K|S| + 2 j
j j2 j j1 j j2 j j1
1 1
= O(| log(T )| + T +1/2 ) = O(T +1/2 )
57. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T2 2
+1/2 ) = O(T
2
+1/2 ) regular part
M |Cj | + c
|Cj | K|S| + 2 j
j j2 j j1 j j2 j j1
1 1
= O(| log(T )| + T +1/2 ) = O(T +1/2 ) ||f fM ||2 = O(M 2
)
58. 2D Wavelet Approximation
If f is C in supp( j,n ), then
| f, j,n ⇥| 2j( +1)
||f ||C || ||1
If f is bounded (e.g. around a singularity), then
| f, j,n ⇥| 2j ||f || || ||1
59. Piecewise Regular Functions in 2D
Theorem: If f is C outside a set of finite length edge curves,
O(M 1/2
) (Fourier),
n
[M ] = ||f fM ||2
n
=
O(M 1
) (wavelets).
Fourier Wavelet, both sub-optimal.
Wavelets: same result for BV functions (optimal).
62. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j
63. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j
1
Singular: 2j1
= (T /C) +1
Cut-o scales (depends on T ):
Regular: 2j2 = T /C
64. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j
1
Singular: 2 j1
= (T /C) +1
Cut-o scales (depends on T ):
Regular: 2j2 = T /C
Hand-made approximate: ˜
fM = f, j,n ⇥ j,n + f, j,n ⇥ j,n
j j2 n Cj c
j j1 n Cj
65. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2
j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part
66. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part
M |Cj | + c
|Cj | LK2 j
+ 2 2j
j j2 j j1 j j2 j j1
1
= O(T 1
+T +1 ) = O(T 1
)
67. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part
M |Cj | + c
|Cj | LK2 j
+ 2 2j
j j2 j j1 j j2 j j1
1
= O(T 1
+T +1 ) = O(T 1
) ||f fM ||2 = O(M 1
)
68. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons
69. Geometrically Regular Images
Geometric image model: f is C outside a set of C edge curves.
BV image: level sets have finite lengths.
Geometric image: level sets are regular.
| f|
Geometry = cartoon image Sharp edges Smoothed edges
70. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges.
˜
Piecewise linear approximation on M triangles: fM .
71. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M
72. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M
Singular areas:
M/2 anisotropic triangles.
73. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M
Singular areas:
M/2 anisotropic triangles.
Theorem: If f is C2 outside a set of C2 contours, then one has for an adapted
˜
triangulation ||f fM ||2 = O(M 2 ).
Di culties to build e cient approximations.
No optimal strategies (greedy solutions).
77. Curvelet Approximation
M -term curvelet approximation:
Theorem: If f is C2 outside a set of C2 edges, ||f fM ||2 = O(M 2
(log M )3 ).
Discrete curvelets: O(N log(N )) algorithm.
Redundancy 5 =⇥ not e cient for compression.