Emanuele Schiavi-'La visión computacional se encuentra con la medicina'

Collaborators Magnetic Resonance Imaging (MRI) Modelling Total Generalized Variation Model Proposed Numerical Resolu
Advanced Mathematical Methods for MR Images
Processing and Analysis
Emanuele Schiavi
Computer Vision Image Processing Group (CVIP)
1Dpto de Matemática Aplicada, Ciencias de los Materiales y Tecnolog´ıa
Electrónica.
Universidad Rey Juan Carlos, Móstoles, Madrdid, Spain
In collaboration with the Research Laboratory of Electronics, Massachusetts
Institute of Technology and the A. A. Martinos Center for Biomedical Imaging,
Massachusetts General Hospital
Computer Vision meets Medicine:
Present and future of imaging modalities and biomarkers
Madrid, November 14th, 2016
Emanuele Schiavi URJC Advanced Mathematical Methods

Collaborators

Supporting Institutions
TEC2012-39095-C03-02 Biomarkers Based on Mathematical Models

Projects and Contracts
2015 -TEC2012-39095-C03-02 Ref. Int. M1010 T´ıtulo del
proyecto: Biomarcadores Basados en Modelos Matemáticos.
Proyecto de I+D. IP: E. Schiavi. (URJC). Equipo: 12
miembros Entidad Financiadora: Ministerio de Econom´ıa y
Competitividad. Desde 01/01/2013 hasta el 31/12/2015.
Programa Nacional. 3 años
2014 - Ref. Int. M1294 T´ıtulo del Proyecto: Reconstrucción de
Imágenes de MRI. MRI Reconstruction. IP: E. Schiavi. (URJC)
Art´ıculo 83. Contrato de I+D. Equipo: E. Schiavi Entidad
Financiadora: Massachussets Institute of Technology.
Duración: 6 meses. Desde 01/01/2015 hasta el 30/06/2015.

Projects and Contracts
2014 - Ref. Int. M1088 T´ıtulo del Proyecto: Improving the
Safety and Efficiency of High-Fields MR Imaging. Subaward
no. 5710003517 IP: E. Schiavi. (URJC) Art´ıculo 83. Contrato
de I+D. Equipo: E. Schiavi Entidad Financiadora:
Massachussets Institute of Technology. Duración: 1 año y 6
meses. Desde 1/7/2013 hasta el 30/06/2014.
2013 - Ref. Int. M1019 T´ıtulo del Proyecto: Improving the
Safety and Efficiency of High-Fields MR Imaging. IP: E.
Schiavi. (URJC) Art´ıculo 83. Contrato de I+D. Equipo: E.
Schiavi Entidad Financiadora :Massachussets Institute of
Technology. Duración: 1 año y 6 meses. Desde 1/1/2012
hasta el 30/06/2013.

Papers
Chatnuntawech, I., Martin, A., Bilgic, B., Setsompop, K.,
Adalsteinsson, E., Schiavi, E. Vectorial Total Generalized
Variation for Accelerated Multi-Channel Multi-Contrast MRI.
Magnetic Resonance Imaging, June 2016. Impact Factor:
1.980
Martin, A., Schiavi, E., Eryaman, Y., Herraiz, J. L., Gagoski,
B., Adalsteinsson, E.,Wald, L. L., Guerin, B. Parallel
Transmission Pulse Design with Explicit Control for the Specic
Absorption Rate in the Presence of Radiofrequency Errors.
Magnetic Resonance in Medicine, 73(5):1896-1903. Impact
Factor: 3.571

Papers
Eryaman, Y., Guerin, B., Akgun, C., Herraiz, J. L., Martin, A.,
Torrado- Carvajal, A., Malpica, N., Hernandez-Tamames, J.
A., Schiavi, E., Adalsteinsson, E., Wald, L. L.: Parallel
transmit pulse design for patients with Deep brain stimulation
implants. Magnetic Resonance in Medicine, July 3.
doi:10.1002/mrm.25820 Impact Factor: 3.571
2015 Eryaman, Y., Guerin, B., Keil, B., Mareyam, A., Herraiz,
J. L., Kosior, R. K., Martin, A., Torrado-Carvajal, A., Malpica,
N., Hernandez-Tamames, J. A., Schiavi, E., Adalsteinsson, E.,
Wald, L. L.: SAR reduction in 7T C-spine imaging using a
”dark modes” transmit array strategy.
Magnetic Resonance in Medicine, 73(4):1533-1539. Impact
Factor: 3.571

PhD
Iván Ram´ırez. Deep Variational Models for HAR. PhD student.
Directores: J. Pantrigo, E. Schiavi. URJC, Madrid.
Eduardo Alca´ın. GPU accelerated Variational Models. PhD
student. Directores: A. Sanz, E. Schiavi. URJC, Madrid.
Adrián Mart´ın, adrian.martin@uni-graz.at, Institute for
Mathematics and Scientific Computing University of Graz,
Austria. Nonlinear optimization Methods for Accelerating
Magnetic Resonance Imaging. Director: E. Schiavi. URJC,
Madrid, 2016
Juan Francisco Garamendi. A Unified Variational Framework
for Image Segmentation and Denoising. Directores: E. Schiavi,
N. Malpica. URJC, Madrid, 2011.

References
Gonzalo Galiano, Emanuele Schiavi, Julián Velasco
Well-posedness of a nonlinear integro-differential problem and
its rearranged formulation Nonlinear Analysis Pages 74-90.
2016
Adrián Mart´ın, Emanuele Schiavi, Sergio Segura de León. On
1-Laplacian Elliptic Equations Modeling Magnetic Resonance
Image Rician Denoising Journal of Mathematical Imaging and
Vision pp 1-23. 2016
A Martin, I Chatnuntawech, B Bilgic, K Setsompop, E
Adalsteinsson, E. Schiavi Total Generalized Variation Based
Joint Multi-Contrast, Parallel Imaging Reconstruction of
Undersampled k-space Data Proc. Intl. Soc. Mag. Reson. Med
23, 0080. 2015

References
Adri´an Mart´ın, Emanuele Schiavi Noise Modelling in Parallel
Magnetic Resonance Imaging: A Variational Approach Image
Analysis and Recognition Volume 8814. Lecture Notes in
Computer Science pp 121-128. 2014
Adri´an Mart´ın, Emanuele Schiavi Automatic Total Generalized
Variation-Based DTI Rician Denoising Image Analysis and
Recognition Volume 7950. Lecture Notes in Computer Science
pp 581-588. 2013
A Martin, A Marquina, JA Hernandez-Tamames, P
Garcia-Polo, E Schiavi MRI TGV based super-resolution
ISMRM 21st Annual Meeting, Salt Lake City, 20-26. 2013

Magnetic Resonance Imaging
Figure: Fundaci´on CIEN-Hospital Fundaci´on Reina Sof´ıa, Lab. de
Neuroimagen, Madrid, Spain

Motivation
MRI is one of the most widely used medical imaging techniques.
Powerful diagnosis properties without using ionizing radiation.
Slow compared with Ultrasonography or X-ray Computer
Tomography (CT).
Patient discomfort, motion during the acquisition...
Very expensive technique.

Outline
MRI data acquisition .
Frequency Gaussian Noise and Rician Spatial Noise.
The Rician probability density function
Bayesian modelling for the Energy Functional
TGV based regularization and edge preserving
Super-resolution for MRI Rician Denoising.
VTGV Based Multi-Coils Multi-Contrast for pMRI
Reconstruction
Non-Smooth Non-Local Non-Convex Saliency detection for
Brain Tumor Segmentation

MRI data acquisition
1 Acquisition in K-space (frequency domain) (C)
2 Inverse Fourier transform from K-space (C) to the spatial
domain (C)
3 Compute the magnitude image

The Rudin, Osher and Fatemi (ROF) Model
TV based Gaussian Denoising
J(u) =
Ω
| u|dx
Regularity of u
+
1
2λ Ω
|f − u|2
dx
Fidelity to f
L. Rudin, S. Osher and E. Fatemi. Nonlinear total variation
based noise removal algorithms. Physica D: Nonlinear
Phenomena. 60 (1992). pp 259-268.
A. Chambolle. An algorithm for total variation regularization
and denoising. Journal Math. Imaging. 20. (2004). pp 89-97.
F. Andreu-Vaillo, V. Caselles, J. M. Maz´on Parabolic
Quasilinear Equations Minimizing Linear Growth Functionals
(Progress in Mathematics) 2004. Birkh¨auser Basel

From Gaussian noise to Rician noise
1 K-space (C): MK = ( ˆDR + ˆN1(0, σ))
Real part
+i ( ˆDI + ˆN2(0, σ))
Imaginary part
2 Spatial domain (C):
MS = (DR + N1(0, σ)) + i(DI + N2(0, σ))
3 Magnitude image:
MR = ((DR + N1(0, σ))2 + (DI + N2(0, σ))2
H. Gudbjartsson, S. Patz. The Rician Distribution of Noisy
MRI Data. Magn. Reson. Med. ,34 (1995), pp. 910-914.

Bayesian Modelling
Rician probability distribution function
p(u|f )p(f ) = p(f |u)p(u)
p(u|f ) ∝ p(f |u)p(u)
Maximizing p(u|f ) amounts to minimizing the (minus)
log-likelihood
arg min
u
{− log p(f |u) − log p(u)}
p(f |u) = exp(−H(u, f )) = exp −
Ω
h(u, f )dx ,

Modelling Rician noise
Rician probability density function
p(f |u) =
f
σ2
exp −
u2 + f 2
2σ2
I0
uf
σ2
u : true image intensity
f : noisy image data
I0 : the modified zeroth-order Bessel function of the first kind
σ2: variance of the original noise
Basu, S., Fletcher, T., and Whitaker, R. Rician noise removal
in diffusion tensor MRI. (2006). MICCAI 2006 (pp. 117-125).
A. Mart´ın, J.F. Garamendi, E. Schiavi. Iterated Rician
Denoising. IPCV’11. Vol. I pp 959-963, 2011

Bayesian Modelling
Rician likelihood
H(u, f ) =
Ω
h(u, f )dx =
Ω
u2
2σ2
− log I0
uf
σ2
dx
Energy minimization functional
arg min
u∈X
E(u) = arg min
u∈X
λJ(u) + H(u, f ) = (1)
= arg min
u∈X
λ
Ω
j(u)dx +
Ω
h(u, f )dx =
= arg min
u∈X
λ
Ω
j(u)dx +
Ω
u2
2σ2
− log I0
uf
σ2
dx

Total Variation (TV)
Deﬁnition
TV [u] = |Du|(Ω) =
Ω
|Du| = sup
¯p∈P Ω
u · ¯pdx
with
¯p = (px1
, .., pxD
) ∈ C1
0 (Ω)D
, |¯p|L∞(Ω) ≤ 1
Notice that for u ∈ W 1,1(Ω):
|Du|(Ω) =
Ω
|Du| =
Ω
| u|dx
Rudin, L. I., Osher, S., Fatemi, E. Nonlinear total variation
based noise removal algorithms Physica D: Nonlinear
Phenomena, 60(1), 259-268.

Total Generalized Variation (TGV)
Deﬁnition
Let Ω ⊂ Rd , u ∈ L1
loc(Ω, R), α = (α0, ..., αk−1) > 0 weights, the
TGV functional of order k ∈ N is deﬁned as,
TGVk
α(u) = sup
Ω
u (divk
v) dx |v ∈ Ck
c (Ω, Symk
(Rd
)),
divl
v ∞ ≤ αl , l = 0, ..., k − 1
Bredies, K., Kunisch, K., Pock, T. Total generalized variation.
(2010). SIAM Journal on Imaging Sciences, 3(3), 492-526
TGVk
α is proper, convex, lower semi-continuous.
TGVk
α is translation and rotation invariant.
ker(TGVk
α) = Pk−1(Ω) polynomials of degree less than k.

TGV vs TV

Noisy low-resolution MR modalities: fMRI, ASL, DTI...
20 40 60 80
20
40
60
80
100
Figure: Arterial Spin Labelling (ASL) and Phantom degraded MRI
synthetic Image

Noisy low-resolution MR modalities: FA-DTI...
Figure: Diﬀusion Weighted Images (DWI) and Fractional Anisotropy (FA)

Edge-preserving Up (Down)-sampling operator
The Super-resolution model
p(f |Du) =
f
σ2
exp −
(Du)2 + f 2
2σ2
I0
(Du)f
σ2
Low-Resolution data f and High-Resolution image intensity u
Low-Resolution image intensity Du being D a down-sampling
operator and S its Up-sampling operator. D ◦ S = Id ,
S ◦ D = Id .
Edge preserving piecewise linear approximation
Sjk(x, y) = ujk + a(x − xj ) + b(y − yk)
Joshi, S. H., Marquina, A., Osher, S. MRI Resolution
Enhancement Using Total Variation Regularization. ISBI 2009,
pp. 161–164.

Bayesian Modelling
Rician p.d.f.
p(u|f )p(f ) = p(f |Du)p(u)
p(u|f ) ∝ p(f |Du)p(u)
Maximizing p(u|f ) amounts to minimizing the (minus)
log-likelihood
arg min
u
{− log p(f |Du) − log p(u)}
p(f |Du) = exp(−λH(Du, f )) = exp −λ
Ω
h(Du, f )dx ,

Bayesian Modelling
Rician likelihood
H(Du, f ) =
Ω
h(Du, f )dx =
Ω
(Du)2
2σ2
− log I0
(Du)f
σ2
dx
The TGV prior for Super-resolution
p(u) = exp(−J(u)) = exp −
Ω
j(u)dx ,
J(u) = − log p(u) = TGV2
α(u) = sup
Ω
u div2
v dx, v ∈ K
K = v ∈ C2
c (Ω, Sym2
(Rn
)), v ∞ ≤ α0, div v ∞ ≤ α1
BGV 2
α(Ω) = {u ∈ L1
(Ω) / TGV2
α(u) < ∞}

The Total Generalized Variation for MR-DTI
Super-resolution
Energy minimization functional
min
u∈X
E(u) = min
u∈X
J(u) + λH(Du, f ) = (2)
= min
u∈X Ω
j(u)dx + λ
Ω
h(Du, f )dx =
= min
u∈X Ω
j(u)dx + λ
Ω
(Du)2
2σ2
− log I0
(Du)f
σ2
dx
A Mart´ın, A Marquina, JA Hernandez-Tamames, P
Garcia-Polo, E Schiavi MRI TGV based super-resolution
ISMRM 21st Annual Meeting, Salt Lake City, 20-26. 2013

References for TGV, 2010
Bredies, K., Kunisch, K., Pock, T. Total generalized variation.
(2010). SIAM Journal on Imaging Sciences, 3(3), 492-526
Knoll, F., Bredies, K., Pock, T., Stollberger, R. Second order
total generalized variation (TGV) for MRI (2011). Magnetic
resonance in medicine 65(2), 480-491
Bredies, K., and Valkonen, T. Inverse problems with
second-order total generalized variation constraints. (2011).
Proceedings of SampTA, 1-4.

Energy minimization
TGV prior: equivalent formulation
TGV2
α(u) = min
v∈BD(Ω)
α1|| u − v||M + α0||E(v)||M
BD(Ω) = {v ∈ L1
(Ω) / ||E(v)||M < ∞}
Energy Minimization Problem
min
v∈BD(Ω)
α1|| u − v||M + α0||E(v)||M+
+
Ω
(Ku)2
2σ2
− log I0
(Ku)f
σ2
dx

Numerical Results with Fractional Anisotropy images
Figure: High resolution FA images: NN interpolation Vs. Model Proposed

Numerical Results with Fractional Anisotropy images

Numerical Results with Diﬀusion Tensor images

Multi Contrast Imaging
Clinical MRI protocols typically include multiple acquisition of the
same ROI with diﬀerent contrast settings.

Fully-Sampled Acquistion
Fu = g

Parallel Imaging: Using multiple receiver coils
Acquisition time can be reduced because of having multiple
samples of the same data.
FScu = gc c = 1, ..., Nc
Figure: S1u, S2u, ..., S8uEmanuele Schiavi URJC Advanced Mathematical Methods

FScu = gc c = 1, ..., Nc
Figure: S1, S2, ..., S8

FScu = gc c = 1, ..., Nc
Figure: g1, g2, ..., g8

SENSE Reconstruction, 1999
u = min
u
Nc
c=1
FScu − gc
2
2
K.P. Pruessmann, M. Weiger, M. B. Scheidegger, P. Boesiger
SENSE: Sensitivity encoding for fast MRI Magn Reson Med.
1999;42(5):952–962
TV-SENSE Reconstruction, 2007
u = min
u
|Du|(Ω) +
Nc
c=1
FScu − gc
2
2
Liu, B., Ying, L., Steckner, M., Xie, J., Sheng, J. Regularized
SENSE reconstruction using iteratively reﬁned total variation
method. 2007. 4th IEEE International Symposium on Biomedical
Imaging: From Nano to Macro (pp. 121-124). IEEE.Emanuele Schiavi URJC Advanced Mathematical Methods

Sparse MRI: Compressed Sensing, 2007
Donoho DL. Compressed Sensing. IEEE Trans Inf Theory.
2006;52:1289–1306.
Candes E, Romberg J, Tao T. Robust uncertainty principles: Exact
signal reconstruction from highly incomplete frequency information.
IEEE Trans Inf Theory. 2006;52(2):489–509
Lustig M, Donoho D, Pauly JM. Sparse MRI: The application of
compressed sensing for rapid MR imaging. Magn Reson Med.
2007;58(6):1182–1195.

Sparse MRI: Undersampling below Nyquist rate
Undersampled MRI Reconstruction
u = min
u
Ψu 1 +
λ
2
MFu − g 2
2

Multi-contrast MRI: Exploit structural similarities, 2011
Based in CS-concepts or/and PI, several methods have used shared
features across Multi-Contrast MRI to reduce the data acquired
while preserving the reconstruction quality
Bilgic B, Goyal, V, Adalsteinsson, E. Multi-contrast reconstruction
with Bayesian compressed sensing Magn Reson Med.
2011;66(6):1601–1615.
Huang, J., Chen, C., Axel, L. (2014). Fast multi-contrast MRI
reconstruction. Magnetic resonance imaging, 32(10), 1344-1352.
Gong E, Huang F, Ying K, Wu W, Wang S, Yuan C. PROMISE:
parallel-imaging and compressed-sensing reconstruction of
multicontrast imaging using SharablE information. Magn Reson
Med. 2015;73(2):523–35.

Vectorial Total Generalized Variation (VTGV)- 2014
The TGV can be naturally extended for vector valued images
Deﬁnition
Let Ω ⊂ Rd , u ∈ L1
loc(Ω, RL), α = (α0, ..., αk−1) > 0 weights, the
VTGV functional of order k ∈ N is deﬁned as,
VTGVk
α(u) = sup
Ω
L
l=1
ul (divk
vl ) dx |
v ∈ Ck
c (Ω, Symk
(Rd
)L
), divl
v l,∞ ≤ αl , l = 0, ..., k − 1
Bredies, K Recovering piecewise smooth multichannel images by
minimization of convex Functionals with Total generalized variation
penalty.. Glob Optim Methods LNCS. 2014;8293:44?77.

MCMC-TGV-SENSE, 2016
Multi-contrast SENSE Reconstruction using VTGV
min
u∈U
VTGV2
α(u) +
λ
2
Nc
c=1
L
l=1
Ml FScul − gc,l
2
2 (3)
equivalent formulation
VTGV2
α(u) = min
v∈V
α1|| u − v||1V + α0||Ev||1W
A Martin, I Chatnuntawech, B Bilgic, K Setsompop, E
Adalsteinsson, E. Schiavi Total Generalized Variation Based
Joint Multi-Contrast, Parallel Imaging Reconstruction of
Undersampled k-space Data Proc. Intl. Soc. Mag. Reson. Med
23, 0080. 2015

MCMC-TGV-SENSE Numerical Resolution
Primal-dual algorithm for MCMC-TGV-SENSE
Initialization: choose τd , τp = 1√
12
, ¯u0 = u0 = K∗g,
v0 = p0 = 0, q0 = 0, r0 = 0.
Iterate for k ≥ 0 until convergence as follows:
(˜p, ˜q, ˜r)T
= (pk
, qk
, rk
)T
+ τd ( h ¯uk
− ¯vk
, Eh ¯vk
, K ¯u)T
(pk+1
, qk+1
, rk+1
)T
= (PP (˜p), PQ(˜q),
˜r
1 + τd /λ
)T
uk+1
= uk
− τp( ∗
hpk+1
+ K∗
rk+1
)
vk+1
= vk
− τp(E∗
h qk+1
− pk+1
)
(¯uk+1
, ¯vk+1
)T
= 2(uk+1
, vk+1
) − (uk
, vk
)

MC-TGV-SENSE compared to TV-SENSE: R=5, 8 coils

MC-TGV-SENSE compared to TV-SENSE (zoom in)

Non-Smooth Non-Local Non-Convex Model for joint
Saliency Detection and Image Restoration
Non-local Energy Functional
E ,p(u) = αJNL
,p (u) +
1
α
H(u) + λF(u)
JNL
,p (u) =
1
4 Ω×Ω
w(x − y)φ ,p(u(y) − u(x))dxdy
φ ,p(s) =
2
p
s2
+ 2 p/2
−
2
p
p
H(u) = −
Ω
|1 − δu|2
dx, F(u) =
Ω
|u − f |2
dx
I. Ram´ırez, G. Galiano, N. Malpica, E. Schiavi. A Non-local Diﬀusion
saliency Model in Magnetic Resonance Imaging. Bioimaging, Oporto,
2017.

Figure: NL diﬀusion model for p = {0.1, 0.5, 1, 2, 3}.

Figure: FLAIR subjects from BRATS2015 dataset. From left to right
(columns): original image, output image, perimeter of the segmentation
of the output image, output binary mask, perimeter of the ground truth,
ground truth segmentation, ground truth and output binary mask overlap
.

Muchas gracias

Emanuele Schiavi-'La visión computacional se encuentra con la medicina'

Recommended

Recommended

More Related Content

Similar to Emanuele Schiavi-'La visión computacional se encuentra con la medicina'

Similar to Emanuele Schiavi-'La visión computacional se encuentra con la medicina' (20)

More from Fundación Ramón Areces

More from Fundación Ramón Areces (20)

Recently uploaded

Recently uploaded (20)

Emanuele Schiavi-'La visión computacional se encuentra con la medicina'