Use of DL/ML in tomographic image reconstruction

1. Machine learning for
Tomographic Imaging
Dr. M Ahmad
- PhD Medical Physics/Imaging
- Clinical Imaging Experience
munirahm@gmail.com

2. Tomographic
Imaging
o Tomographic imaging is a technique to
image objects in-vivo by taking
measurements in-vitro.
o It evades the necessity of resecting
objects in order to visualize internal
anatomy or functionality.
o PET/CT is a nice example from medical
imaging of tomographic imaging.
Pre-Requisite
- PET/CT imaging physics of transmission and emission imaging.
- Numerical and statistical algorithms.
- Basic neural networking and machine learning.

3. Tomographic
Data
Acquisition
o Patient is injected with a
positron emitter.
o Positron catches an electron
and annihilates and results
in two gammas.
o These gamma rays are
detected in two oppositely
placed detectors and
registered as an event.
o These events are placed in
holder with angle of
detection with x-axis on one
axis and its radial position
from center on the other
axis.
o The registered events are summed
up and termed as a sinogram
because a single point in image
space registers a sine curve in
sinogram space.
o Sinogram represents the detector
data and is not in form of an object’s
internal image.
We need to reconstruct
images from this data
to obtain patient’s
anatomical images.

4. Acquired Data
Modeling
Mathematically tomographic data can be
represented as line integrals of function
distribution and can be represented as;
Or with unit impulse;
Or as a linear system of equations;
Reconstruction Algorithm

5. Reconstruction
Methods
Analytical Reconstruction Methods
• Analytical Reconstruction Methods try to
implement analytical inversion formula
or its approximation for the line integral
model.
Filtered
Backprojection
Novikov’s Inverse Radon Transform Formula
Direct Fourier
Reconstruction
(Fourier Slice
Theorem)
Algebraic Reconstruction Techniques (ART)
Least Squares Reconstruction Method
Maximum Likelihood Expectation Maximization (MLEM)
Iterative Reconstruction Methods
• Analytical Reconstruction Methods either
maximize some objective function based on
statistical properties of the emission
process or try to solve a linear system of
equations.

6. False Activity
More noise with iterations
Theoretical
Issues
ill-possedness
ill-conditioning
Even with the use of object
distribution priors, still
reconstructed images have
data dependencies and data
noise issues. Use of neural
network modeling can
overcome these issues.

7. Image reconstruction
from sinogram data
(pre-processing)
Image denoising after
reconstruction data
(pre-processing)
Evades dependency on system
matrix and noisy data and tries
to fit to true target.

8. FBP Mapping to Neural Networks
Several variations proposed further

9. Result
output of
FBP
Mapping
to Neural
Network
In this simple implementation, sinogram filtering has been performed by a convolutional layers
and to reduce backprojection parameters, all the parameters in this layerwere made un-adjustable
or non-trainable. [Tobias Wurfl et al. 2016, DOI: 10.1007/978-3-319-46726-9_50]
MSE
FBP 0.00531
NN 0.00392

10. My Favorite FBP Mapping to NN
This model used a very simple network with an initial backprojection step. Each sinogram was backprojected
once and 2D images were generated which were then passed through a CNN network of 15 layers to obtain final
images. [Tobias Wurfl et al. 2016, DOI: 10.1007/978-3-319-46726-9_50]

11. CT Image
Denoising
CNN based cross network residual encoder model
for low dose CT image denoiser (RED-CNN).
[Hu Chen, Yi Zang, 2017, DOI:
10.1109/TMI.2017.2715284]
Network Model for Image Denoising

12. Network Model with Denoiser - FBPNet
Filtering has been performed in frequency domain by CNN and denoiser part tries to improve noise
characteristics induced by this filtering. [Bo Wang & Huafeng Liu, 2020, DOI: 10.1088/1361-6560/abc09d]

13. Result
output of
FBP
Mapping
to Neural
Network Filtering has been performed in frequency domain by
CNN and denoiser part tries to improve noise
characteristics induced by this filtering. [Bo Wang &
Huafeng Liu, 2020, DOI: 10.1088/1361-6560/abc09d]

14. Network Model with Residual Learning
Unrolled Log-likelihood reconstruction method uses EM updates as input and then they may further be passed through
penalty and again EM update may be used with denoising model.
[Hu Chen, Yi Zang, 2017, DOI: 10.1109/TMI.2017.2715284]

15. Iterative
PET
Recon
Mapping
to
Neural
Network
[Kuang Gong, et.al., 2018,
DOI: 10.1109/TMI.2018.2869871]
Use GA for opt.
ADMM

16. [A J Reader et al., 2021, DOI: 10.1109/TRPMS.2020.3014786]

17. [A J Reader et al., 2021, DOI: 10.1109/TRPMS.2020.3014786]
Network
Architectures

18. Deep PET SPECT Positronium Imaging
Totally new diagnosis….
Unsupervised
Semi-supervised
Supervised learning
Multiple Instance Learning.

19. Positronium
Imaging
[Powel Moskel et al., 2021, DOI:
10.1126/sciadv.abh4394]
Tumour
Infection
Inflammation

20. Deep
PET
Imaging
with ToF
[Sun II Kwon et al., 2021, DOI:
10.1038/s41566-021-00871-2]
My vision is to
use this for
SPECT, if
possible.

21. Thanks
for
listening