The document presents a new deep learning method called DL-ESPIRiT that improves the robustness of MRI reconstruction to errors in sensitivity map models. DL-ESPIRiT incorporates a more robust SENSE model called ESPIRiT into a deep learning reconstruction network, allowing it to achieve higher quality reconstructions than conventional compressed sensing even with accelerated acquisitions and reduced field of view. Evaluation on real data demonstrated DL-ESPIRiT's superior performance over traditional methods for enabling fast cardiac imaging, including in pediatric patients.

1. DL-ESPIRiT: Improving robustness to
SENSE model errors in deep learning-based
reconstruction
Christopher M. Sandino1, Peng Lai2, Shreyas S. Vasanawala3, Joseph Y. Cheng3
1Department of Electrical Engineering, Stanford University, Stanford, CA. 2General Electric Healthcare, Menlo Park,
CA. 3Department of Radiology, Stanford University, Stanford, CA.

2. Speaker Name: Christopher Sandino
I have the following financial interest or relationship to disclose with regard to the subject
matter of this presentation:
Company Name: GE Healthcare
Type of Relationship: Research support
Declaration of
Financial Interests or Relationships

3. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Learning to reconstruct MR images1-5
Convolutional Neural Network
Input(s) Output
1K Hammernik, et al. MRM, 2018.
2J Schlemper, et al. IEEE TMI, 2018.
3C Qin, et al. IEEE TMI, 2019.
4M Mardani, et al. IEEE TMI, 2019.
5JY Cheng, et al. arXiv, 2019.
“Physics-informed” CNN
Advantages of physics-informed DL:
• Requires less training data
• Improves generalization to unseen
anatomies, pathology, sequence
parameters, …
MRI Signal Model:
• Coil sensitivities (SENSE)
• Discrete Fourier Transform
• Sampling pattern
ky
t
Disadvantage of physics-informed DL:
• Susceptible to model error / mismatch

4. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Zero-Filled (R=8)
l1-ESPIRiT
spatial + temporal TV 3D Unrolled3 Fully-Sampled
Example of model error: anatomy overlap1-2
1M Griswold, et al. “Field-of-view limitations in parallel imaging” MRM, 2004.
2JW Goldfarb. “The SENSE ghost: Field-of-view restrictions for SENSE imaging” JMRI, 2004.
3JY Cheng, et al. “Compressed sensing: From research to clinical practice with data-driven learning” arXiv, 2019.

5. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Objective
Improve robustness to sensitivity map-related model
errors in deep learning-based reconstruction
..and enable fast cardiac cine MRI exams in pediatric
patients with high heart rates.

6. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
ESPIRiT1: A more robust SENSE model for DL
1M Uecker, et al. ESPIRiT. MRM, 2014.
l1-ESPIRiT
1 set of sensitivity maps
l1-ESPIRiT
2 sets of sensitivity maps

7. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
DL-ESPIRiT: Where DL meets robustness
1S Diamond, et al. Unrolled optimization with deep priors. arXiv, 2017.
2K He, et al. Identity mappings in deep residual networks. arXiv, 2016.
3D Tran, et al. Spatiotemporal convolutions for action recognition. CVPR, 2018.
4CM Sandino, et al. Separable 3D convolutions for MRI recon. ISMRM ML Workshop, 2019.
• ESPIRiT channels are
stacked and jointly
regularized by CNN
• 3D Unrolled1
• 10 iterations
• 2 residual learning blocks2 /
iteration
• Separable 3D
convolutions3,4

8. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Network Training
• With IRB approval, collected 10 fully-sampled datasets from healthy adult volunteers
• Balanced SSFP
• 1.5T and 3.0T scanner data
• Mix of cardiac views (SAX, LAX, 4Ch)
• Data Augmentation:
• Retrospective undersampling (VD, R = 10->15)
• Phase FOV reduction (0-15% red. factors)
• Other details: TensorFlow, trained on 2x NVIDIA GTX 1080 Ti (1.5 weeks total)
Simulate
15% phase FOV
reduction

9. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Evaluation
• Collected another 11 fully-sampled datasets from healthy adult volunteers
• Compared two reconstruction methods:
• l1-ESPIRiT with spatial TV (λ=0.002) and temporal TV (λ=0.01) - BART1
• DL-ESPIRiT - TensorFlow
• Evaluation metrics:
• Peak signal-to-noise ratio (PSNR)
• Structural similarity index (SSIM)
• Cardiac output measurements based on automatic segmentation (E-Net2)
1M Uecker, et al. BART. Sedona, 2013.
2J Lieman-Sifry, et al. “FastVentricle” FIMH, 2017

10. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
l1-ESPIRiT
(2 set of maps)
DL-ESPIRiT
(2 set of maps)
l1-ESPIRiT
(1 set of maps)
DL-ESPIRiT
(1 set of maps)
Zero-Filled Fully-sampled
Healthy Volunteer @ 1.5T
Matrix size: 200 x 180
Channels: 8 (compressed)
Cardiac phases: 20
Retrospective case: Healthy volunteer (R=8)

11. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Retrospective case: Healthy volunteer (R=10)
Zero-Filled (R=10)
l1-ESPIRiT
spatial + temporal TV DL-ESPIRiT Fully-Sampled
Healthy Volunteer @ 3.0T
Matrix size: 200 x 180
Channels: 8 (compressed)
Cardiac phases: 20

12. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Image Quality Metric Assessment (R=12)
Peak signal-to-noise ratio (PSNR) Structural Similarity Index (SSIM)
l1-ESPIRiT
DL-ESPIRiT
l1-ESPIRiT
DL-ESPIRiT

13. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Left Ventricular Function Measurements (R=12)
End-Diastolic Volume (EDV)
y = 0.95x
R² = 0.98
y = 0.98x
R² = 0.99
y = x
70
90
110
130
150
170
190
210
70 90 110 130 150 170 190 210
EDV(Estimate)[mL]
EDV (Ground Truth) [mL]
y = 1.06x
R² = 0.95
y = 0.99x
R² = 0.99
y = x
30
35
40
45
50
55
60
65
70
75
80
30 40 50 60 70 80
ESV(Estimate)[mL]
ESV(Ground Truth) [mL]
End-Systolic Volume (ESV)
y = 0.91x
R² = 0.67
y = 0.99x
R² = 0.95
y = x
45
50
55
60
65
70
45 50 55 60 65 70
LVEF(Estimate)[%]
LVEF (Ground Truth) [%]
Ejection Fraction (EF)
l1-ESPIRiT DL-ESPIRiT

14. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Prospective case: Healthy adult volunteerMagnitudey-t
l1-ESPIRiT DL-ESPIRiT Fully-sampled
Scan #1: Single breath-hold (R=12.6)
5 slices, 10 seconds total scan time
Scan #2: Five breath-holds (R=1)
5 slices, 25 seconds per breath-hold
Un-zoomed

15. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Prospective case: Pediatric patient with muscular dystrophy (118 BPM)
Scan #1 (R=2)
Seven breath-holds (8 sec/BH)
FOV: 34 x 30.4 cm2
Number of slices: 14
Cardiac phases: 12
Scan time: 96 seconds
Scan #2 (R=12)
Three breath-holds (5 sec/BH)
FOV: 34 x 27.2 cm2
Number of slices: 14
Cardiac phases: 12
Scan time: 14 seconds
Slice 5 Slice 6 Slice 7
SENSEDL-ESPIRiTl1-ESPIRiT

16. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Prospective case: Pediatric patient with abnormal trabeculations (70 BPM)
Scan #1 (R=2)
Seven breath-holds (8 sec/BH)
Number of slices: 15
Cardiac phases: 24
Scan time: 3 minutes
Scan #2 (R=11.9)
Single breath-hold
Number of slices: 15
Cardiac phases: 24
Scan time: 25 seconds
SENSEDL-ESPIRiTl1-ESPIRiT

17. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Conclusions
• We present a novel DL reconstruction method, which:
• Improves robustness to model errors caused by anatomy overlap
• Enables higher fidelity reconstructions of vastly accelerated data than
state-of-the-art compressed sensing method
• Further validation is necessary in patients with severely impaired breath-hold
capacity

18. Acknowledgements
Stanford University
Shreyas Vasanawala
John Pauly
Brian Hargreaves
Daniel Ennis
Joseph Cheng
Marcus Alley
Morteza Mardani
Adam Bush
Frank Ong
Ukash Nakarmi
Edgar Rios
Mario Malavé
Feiyu Chen
David Zeng
Neerav Dixit
Elizabeth Cole
Lisa Lei
Lucile Packard
Children’s Hospital
Tim DelHagan
Brett Castner
GE Healthcare
Martin Janich
Anja C.S. Brau
Peng Lai
Anne Menini
Valentina Taviani
Eric Printz
Matthew Bingen
Funding Sources
NIH R01EB009690
GE Healthcare
NSF GRFP
Stanford Pediatric MRI Group

19. Sandino, et al. DL-ESPIRiT: Improving robustness to SENSE model errors in deep learning-based reconAbstract #0659 E-mail: sandino@stanford.edu
Thank You!
Name: Christopher Sandino
E-mail: sandino@stanford.edu
Website: chrsandino.github.io