Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Inverse problems in medical
imaging
Nikita Moriakov
Diagnostic Image Analysis Group
Department of Radiology and Nuclear Me...
Plan
• Inverse problems
• Deep Learning in Inverse Problems: Fully
learned inversion vs. Learned iterative
schemes
• Resul...
Inverse problems
What is inverse problem?
• 𝑓𝑡𝑟𝑢𝑒 ∈ 𝑋 is an unknown true model parameter, 𝑔 ∈ 𝑌 is data, 𝑒 is sample
from measurement noise...
Examples: Radon Transform
• Let 𝑓 ∈ 𝐿1 ℝ2 be a function such
that 𝑓(𝑥, 𝑦) is the attenuation
coefficient of scanned object...
Examples: Radon Transform
• Parametrize all lines by the angle 𝜃 and
the distance 𝑠 to the origin
• The Radon transform of...
Radon, 1907
Back projection (key for inversion)
Examples: Radon Transform
• This direct analytic inversion is called
Filtered Back Projection today.
• Radon transform is ...
Examples: DBT
• Available since late 2000s.
• Rapidly replacing Digital
Mammography
• Can have resolution as high as DM
st...
Examples: DBT
• The x-ray tube moves in an arc
over the compressed breast
capturing multiple images of each
breast from di...
Examples: MR
• Fourier transform ℱ maps images
to frequency domain.
• Some measurements in this domain
are taken, giving s...
Problems with inverse problems?
• 𝐴 can have non-trivial kernel. E.g., for CT with limited number of view
angles the follo...
Bayesian view and regularization,
MAP
• Finding most likely reconstruction given measurements amounts to
finding 𝑎𝑟𝑔𝑚𝑎𝑥𝑖𝑚𝑎...
Deep Learning for Inverse
Problems
Inverse problems with DL
Fully learned data driven
reconstruction
• Generic parametrization by a neural
network to find an...
Inverse problems with DL
Learned regularizers
• Train a regularizer 𝑆 𝜃: 𝑋 → ℝ
separately, which is parametrized
by a neur...
Fully learned inference
• Zhu, Liu, Rosen, Rosen “Image reconstruction by domain transform
manifold learning” 2018
Learned Iterative Schemes: LPD
• Learned Primal-Dual (LPD) is an example of learned iterative schemes.
• Architecture moti...
Learned Primal-Dual
• A deep neural network
• Iterative procedure inspired by
Primal-Dual Hybrid Gradient
algorithm
• Cons...
Network architecture
Results (Adler & Öktem)
Learned Iterative Schemes: RIM
• The goal in MAP estimate is finding
• This is often done in via an iterative scheme
• To ...
Recurrent Inference Machines for MR
Lonning, Putzky, Caan, Welling Recurrent Inference Machines for Accelerated MRI Recons...
RIM for MRI Reconstruction
Lonning, Putzky, Caan, Welling Recurrent Inference Machines for Accelerated MRI Reconstruction ...
Results and future work
SPIE 2019 Medical Imaging
Data for the experiment
• Synthetic breast images generated by realistic digital breast
phantom (from I. Sechopoulos et al...
Results (LPD on DBT)
Network architecture (LPD)
LPD for DBT architecture (DBToR)
𝑚
𝑇(𝑚)
Results (beginning of training)
Results (end of training)
Results
Future work
Compressed breast phantom
Classification
Image Segmentation:
Skin – Adipose – Glandular
Caballo M. et al . 2018 “An Unsupe...
Finite Element Compression
Breast Density Map
• Create a mesh of the breast
• Simulate the compression of soft tissue (adi...
Compression
Compression
The breast support is move up by 20
mm (to make the bottom flat);
the compression is performed by
...
Results
Sources
• Arridge, Maass, Öktem, Schönlieb “Solving Inverse Problems Using Data
Driven Models”, Acta Numerica 2019.
• Adle...
Thank you!
Upcoming SlideShare
Loading in …5
×

Inverse problems in medical imaging

159 views

Published on

Talk by Dr. Nikita Morikiakov on inverse problems in medical imaging with deep learning.

Inverse problem is the type of problems in natural sciences when one has to infer from a set of observations the causal factors that produced them. In medical imaging, important examples of inverse problems would be recontruction in CT and MRI, where the volumetric representation of an object is computed from the projection and Fourier space data respectively. In a classical approach, one relies on domain specific knowledge contained in physical-analytical models to develop a reconstruction algorithm, which is often given by a certain iterative refinement procedure. Recent research in inverse problems seeks to develop a mathematically coherent foundation for combining data driven models, based on deep learning, with the analytical knowledge contained in the classical reconstruction procedures. In this talk we will give a brief overview of these developments and then focus on particular applications in Digital Breast Tomosynthesis and MRI reconstruction.

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

Inverse problems in medical imaging

  1. 1. Inverse problems in medical imaging Nikita Moriakov Diagnostic Image Analysis Group Department of Radiology and Nuclear Medicine Radboud University Medical Center, Nijmegen
  2. 2. Plan • Inverse problems • Deep Learning in Inverse Problems: Fully learned inversion vs. Learned iterative schemes • Results and future work
  3. 3. Inverse problems
  4. 4. What is inverse problem? • 𝑓𝑡𝑟𝑢𝑒 ∈ 𝑋 is an unknown true model parameter, 𝑔 ∈ 𝑌 is data, 𝑒 is sample from measurement noise and 𝐴: 𝑋 → 𝑌 is a continuous operator mapping model parameter to data (in the absence of noise) • Examples: CT & MR reconstruction, denoising, inpainting.
  5. 5. Examples: Radon Transform • Let 𝑓 ∈ 𝐿1 ℝ2 be a function such that 𝑓(𝑥, 𝑦) is the attenuation coefficient of scanned object at point (𝑥, 𝑦) • If X-ray going along the line 𝐿 has incident intensity 𝐼0, the outcoming intensity equals 𝐼0 𝑒 − 𝑥,𝑦 ∈𝐿 𝑓 𝑥,𝑦 𝑑𝐿
  6. 6. Examples: Radon Transform • Parametrize all lines by the angle 𝜃 and the distance 𝑠 to the origin • The Radon transform of 𝑓 is a function of 𝑠 ∈ ℝ, 𝜃 ∈ [0, 2 𝜋) given by
  7. 7. Radon, 1907
  8. 8. Back projection (key for inversion)
  9. 9. Examples: Radon Transform • This direct analytic inversion is called Filtered Back Projection today. • Radon transform is invertible (when we know projections from all directions and all rays intersecting the object).
  10. 10. Examples: DBT • Available since late 2000s. • Rapidly replacing Digital Mammography • Can have resolution as high as DM standards.
  11. 11. Examples: DBT • The x-ray tube moves in an arc over the compressed breast capturing multiple images of each breast from different angles in a continuous or step-and-shoot fashion.
  12. 12. Examples: MR • Fourier transform ℱ maps images to frequency domain. • Some measurements in this domain are taken, giving subsampling mask 𝑃. • Inverse Fourier transform maps back to the image domain. • Reconstructed image can deviate from target if k-space is undersampled.
  13. 13. Problems with inverse problems? • 𝐴 can have non-trivial kernel. E.g., for CT with limited number of view angles the following holds: • 𝐴 can have discontinuous inverse, thus variations in noise can have strong effect on reconstruction.
  14. 14. Bayesian view and regularization, MAP • Finding most likely reconstruction given measurements amounts to finding 𝑎𝑟𝑔𝑚𝑎𝑥𝑖𝑚𝑎𝑔𝑒 𝑃 𝐼𝑚𝑎𝑔𝑒 𝑀𝑒𝑎𝑠). • 𝑎𝑟𝑔𝑚𝑎𝑥𝑖𝑚𝑎𝑔𝑒 𝑃 𝐼𝑚𝑎𝑔𝑒 𝑀𝑒𝑎𝑠) = 𝑎𝑟𝑔𝑚𝑎𝑥𝑖𝑚𝑎𝑔𝑒 𝑃 𝑀𝑒𝑎𝑠 𝐼𝑚𝑎𝑔𝑒)⋅𝑃 𝐼𝑚𝑎𝑔𝑒 𝑃(𝑀𝑒𝑎𝑠) = 𝑎𝑟𝑔𝑚𝑎𝑥𝑖𝑚𝑎𝑔𝑒 (log 𝑃 𝑀𝑒𝑎𝑠 𝐼𝑚𝑎𝑔𝑒 + log 𝑃(𝐼𝑚𝑎𝑔𝑒)) • log 𝑃(𝑀𝑒𝑎𝑠|𝐼𝑚𝑎𝑔𝑒) is the log-likelihood of observed data, log 𝑃(𝐼𝑚𝑎𝑔𝑒) is the prior term. • Prior term is used for regularization such as TV regularization.
  15. 15. Deep Learning for Inverse Problems
  16. 16. Inverse problems with DL Fully learned data driven reconstruction • Generic parametrization by a neural network to find an inverse mapping 𝑅 𝜃: 𝑌 → 𝑋 with 𝜃 being the neural network weights • No need to have explicit forward operator or data likelihood • Need to use fully connected layers and hence requires a lot of parameters. Learned iterative schemes • Contains explicit knowledge of forward operator built in the architecture of 𝑅 𝜃: 𝑌 → 𝑋 • Architecture motivated by existing optimization algorithms
  17. 17. Inverse problems with DL Learned regularizers • Train a regularizer 𝑆 𝜃: 𝑋 → ℝ separately, which is parametrized by a neural network, and is ideally proportional to the image prior (Bayesian view). • Can be trained adversarily.
  18. 18. Fully learned inference • Zhu, Liu, Rosen, Rosen “Image reconstruction by domain transform manifold learning” 2018
  19. 19. Learned Iterative Schemes: LPD • Learned Primal-Dual (LPD) is an example of learned iterative schemes. • Architecture motivated by Primal-Dual Hybrid Gradient Method.
  20. 20. Learned Primal-Dual • A deep neural network • Iterative procedure inspired by Primal-Dual Hybrid Gradient algorithm • Consists of a primal/dual reconstruction blocks which performs small “steps” in image and projection space respectively.
  21. 21. Network architecture
  22. 22. Results (Adler & Öktem)
  23. 23. Learned Iterative Schemes: RIM • The goal in MAP estimate is finding • This is often done in via an iterative scheme • To avoid the need to learn prior, we can “generalize” this to the form • So can optimize this as a recurrent neural network.
  24. 24. Recurrent Inference Machines for MR Lonning, Putzky, Caan, Welling Recurrent Inference Machines for Accelerated MRI Reconstruction 2018
  25. 25. RIM for MRI Reconstruction Lonning, Putzky, Caan, Welling Recurrent Inference Machines for Accelerated MRI Reconstruction 2018
  26. 26. Results and future work
  27. 27. SPIE 2019 Medical Imaging
  28. 28. Data for the experiment • Synthetic breast images generated by realistic digital breast phantom (from I. Sechopoulos et al.) • 25 view angles: [-24, -22, … , 22, 24] deg
  29. 29. Results (LPD on DBT)
  30. 30. Network architecture (LPD)
  31. 31. LPD for DBT architecture (DBToR) 𝑚 𝑇(𝑚)
  32. 32. Results (beginning of training)
  33. 33. Results (end of training)
  34. 34. Results
  35. 35. Future work
  36. 36. Compressed breast phantom Classification Image Segmentation: Skin – Adipose – Glandular Caballo M. et al . 2018 “An Unsupervised Automatic Segmentation algorithm for breast tissue” – Med. Phys. Finite Element Compression BCT images acquired from patients at Radboudumc
  37. 37. Finite Element Compression Breast Density Map • Create a mesh of the breast • Simulate the compression of soft tissue (adipose, glandular & skin) using the high-performance explicit finite element solver, developed for medical application
  38. 38. Compression Compression The breast support is move up by 20 mm (to make the bottom flat); the compression is performed by moving down the compression paddle Voxel Resolution of (0.273 mm)3
  39. 39. Results
  40. 40. Sources • Arridge, Maass, Öktem, Schönlieb “Solving Inverse Problems Using Data Driven Models”, Acta Numerica 2019. • Adler, Öktem “Learned Primal-dual Reconstruction” • Markoe, “Analytic Tomography” • Zhu, Liu, Rosen, Rosen “Image reconstruction by domain transform manifold learning”, Nature 2018 • Lønning, Putzky, Caan, Welling “Recurrent Inference Machines for Accelerated MRI Reconstruction”, Medical Image Analysis, 2018
  41. 41. Thank you!

×