This summarizes a list of MICCAI 2018 papers that catch my attention.

1. MICCAI 2018 Summary
Papers that catch my attention
S. Kevin Zhou
Professor, Institute of Computing Technology, CAS
formerly Principal Expert, Siemens Healthineers
Email: s.kevin.zhou@gmail.com; zhoushaohua@ict.ac.cn

2. Personal disclaimer
• There are over 300 papers and all of them have some merits that
worth attention.
• This is a list of papers that catch my personal attention.
• Due to my ‘near-sightedness’, I admittedly miss a lot of excellent
works.

3. Deep learning
Input
image
X
Output
variable
Y
Learning from {(Xi , Yi)}
arg minW Si Loss(Yi, f(Xi; W)) + Reg(W)
Algorithm:
Deep network
Y = f(X; W)

4. Deep learning: the manual parts
Input
image
X
Output
variable
Y
Learning from {(Xi , Yi)}
arg minW Si Loss(Yi, f(Xi; W)) + Reg(W)
Algorithm:
Deep network
Y = f(X; W)

5. Towards automating deep learning
Input
image
X
Output
variable
Y
Learning from {(Xi , Yi)}
arg minW Si Loss(Yi, f(Xi; W)) + Reg(W)
Algorithm:
Deep network
Y = f(X; W)
GAN
Self-, semi-, or
weakly supervised
Neural architecture search
or meta-learning
New
representation

6. Reinforcement learning
Q-learning:

7. RL for parameter inference from image
Action, state, reward
Action a
• Move each parameter by ±dqi
while keeping the other
parameters the same
• A: action space, |A|=2n
State s
• The observations with all actions
taken so far
• qt= qt-1+at = q0+Si ai
• <I, qt>, I[qt]: image (or image
patch) ‘centered’ at qt
Reward r
 Rewards when the target is hit or closer.
 r(st+1, st, at )=| qt - q0 |2-| qt+1 - q0 |2
MICCAI2018 Tutorial: https://www.hvnguyen.com/deepreinforcementlearning

8. Keywords
• Towards automating DL
• Adversarial
• New representations
• New network architecture &
meta-learning
• Semi-supervised, weakly
supervised
• Uncertainty and loss
• Deep reinforcement learning
• Others
• Own

9. Adversarial (it is everywhere)
• M-18 Adversarial Sparse-View
CBCT Artifact Reduction
• Addresses how to handle streaky
artifacts
• M-84 Adversarial Similarity
Network for Evaluating Image
Alignment in Deep Learning based
Registration
• Uses adversarial network as a
similarity function
• Unsupervised!
• A trick a *real + (1-a) * fake,
increasing a
A lot of papers (reconstruction,
image enhancement, segmentation,
etc.) use adversarial trick
• CVPR2018 GAN tutorial
(Alexei Efros)
• Set up arms race
• Learned loss function
• Meta-supervision
• A super data memorizer
• A new computer art

10. Adversarial Sparse-View CBCT Artifact Reduction

11. Adversarial Sparse-View CBCT Artifact Reduction

12. Adversarial Similarity Network for Evaluating Image
Alignment in Deep Learning based Registration

13. New representations
• M-107 Instance Segmentation and
Tracking with Cosine Embeddings
and Recurrent Hourglass Networks
• Introduce cosine embedding
• M-111 A Pixel-wise Distance
Regression Approach for Joint
Retinal Optical Disc and Fovea
Detection
• Bi-distance map to encode spatial
configuration
• T-12 The Deep Poincare Map: A
Novel Approach for Left Ventricle
Segmentation
• A new limit cycle representation

14. Instance Segmentation and Tracking with Cosine
Embeddings and Recurrent Hourglass Networks

15. Instance Segmentation and Tracking with Cosine
Embeddings and Recurrent Hourglass Networks

16. A Pixel-wise Distance Regression Approach for
Joint Retinal Optical Disc and Fovea Detection

17. The Deep Poincare Map: A Novel Approach for
Left Ventricle Segmentation

18. The Deep Poincare Map: A Novel Approach for
Left Ventricle Segmentation

19. New network architecture & meta-learning
• M-64 Concurrent Spatial and
Channel ‘Squeeze & Excitation’
in Fully Convolutional Networks
• As title suggests
• M-62 Training Medical Image
Analysis Systems like
Radiologists
• Uses meta-learning
• T-48 Fast CapsNet for Lung
Cancer Screening
• CapsNet!
• T-139 Autofocus Layer for
Semantic Segmentation
• A new layer that selects scale

20. Concurrent Spatial and Channel ‘Squeeze &
Excitation’ in Fully Convolutional Networks

21. Autofocus Layer for Semantic Segmentation

22. Training Medical Image Analysis Systems like
Radiologists

23. Fast CapsNet for Lung Cancer Screening

24. Fast CapsNet for Lung Cancer Screening

25. Semi-supervised
• M-131 A Probabilistic Model Combining Deep
Learning and Multi-atlas Segmentation for Semi-
automated Labelling of Histology
• T-11 Factorised Spatial Representation Learning:
Application in Semi-supervised Myocardial
Segmentation
• T-44 A Diagnostic Report Generator from CT
Volumes on Liver Tumor with Semi-supervised
Attention Mechanism
• T-138 Semi-Supervised Learning for Segmentation
under Semantic Constraint
• W-64 Semi-Automatic RECIST Labeling on CT
Scans with Cascaded Convolutional Neural
Networks
• W-47 ASDNet: Attention based Semi-supervised
Deep Networks for Medical Image Segmentation
• T-152 Cost-Sensitive Active Learning for
Intracranial Hemorrhage Detection
• Active learning
• [DLMIA] Deep semi-supervised segmentation with
weight-averaged consistency targets
• Semi-Automated Extraction of Crohns Disease MR
Imaging Markers using a 3D Residual CNN with
Distance Prior

26. Weakly supervised
• M-140 A Weakly-Supervised Learning-Based
Feature Localization in Confocal Laser
Endomicroscopy Glioma Images
• M-143 Weakly Supervised Representation
Learning for Endomicroscopy Image Analysis
• T-31 Iterative Attention Mining for Weakly
Supervised Thoracic Disease Pattern
Localization in Chest X-Rays
• T-43 Liver Lesion Detection from Weakly-
labeled Multi-phase CT Volumes with a
Grouped Single Shot MultiBox Detector
• T-56 DeepEM: Deep 3D ConvNets with EM
for Weakly Supervised Pulmonary Nodule
Detection
• T-80 Detection and Delineation of Acute Cerebral
Infarct on DWI using Weakly Supervised Machine
Learning
• W-16 Towards Automatic Report Generation in
Spine Radiology using Weakly Supervised
Framework
• W-45 Deep Learning Based Instance Segmentation
in 3D Biomedical Images Using Weak Annotation
• W-63 Accurate Weakly-Supervised Deep Lesion
Segmentation using Large-Scale Clinical
Annotations: Slice-Propagated 3D Mask
Generation from 2D RECIST
DLMIA: Deep Learning in Medical Image Analysis
• Weakly Supervised Localisation for Fetal
Ultrasound Images

27. Deep Learning Based Instance Segmentation in 3D
Biomedical Images Using Weak Annotation

28. Accurate Weakly-Supervised Deep Lesion
Segmentation

29. Cost-Sensitive Active Learning for Intracranial
Hemorrhage Detection

30. DRL
MICCAI2018
• M-33 Automatic View Planning with Multi-scale
Deep Reinforcement Learning Agents
• M-68 Group-driven Reinforcement Learning for
Personalized mHealth Intervention
• W-23 Deep Reinforcement Learning for Surgical
Gesture Segmentation and Classification
• W-69 Deep Reinforcement Learning for Vessel
Centerline Tracing in Multi-modality 3D Volumes
CBM: MICCAI Workshop on Computational
Biomechanics for Medicine XIII
• Muscle excitation estimation in biomechanical
simulation using NAF reinforcement learning
MICCAI2017
• Multimodal Registration with Deep Context
Reinforcement Learning
• Deep Reinforcement Learning for Active Breast
Lesion Detection from DCE-MRI
• Robust non-rigid registration through agent-based
action learning
• Robust Multi-Scale Anatomical Landmark
Detection in Incomplete 3D-CT Data
• Supervised action classifier: approaching landmark
detection as image partitioning
MICCA2016 An Artificial Agent for Anatomical
Landmark Detection.
MICCAI2015 Vito – A Generic Agent for Multi-physics
Model Personalization: Application to Heart Modeling.

31. Automatic View Planning with Multi-scale Deep
Reinforcement Learning Agents

32. Automatic View Planning with Multi-scale Deep
Reinforcement Learning Agents

33. Uncertainty & loss
• M-75 Exploring Uncertainty
Measures in Deep Networks for
Multiple Sclerosis Lesion
Detection and Segmentation
• Monte Carlo drop out to derive
uncertainty
• T-140 3D Segmentation with
Exponential Logarithmic Loss
for Highly Unbalanced Object
Sizes
• unbalanced

34. Exploring Uncertainty Measures in Deep Networks for
Multiple Sclerosis Lesion Detection and Segmentation

35. 3D Segmentation with Exponential Logarithmic Loss for
Highly Unbalanced Object Sizes

36. Others
• T-27 TextRay: Mining Clinical
Reports to Gain a Broad
Understanding of Chest X-rays
• A large-scale study
• W-1 X-ray-transform Invariant
Anatomical Landmark Detection
for Pelvic Trauma Surgery
• W-6 DeepDRR-A Catalyst for
Machine Learning in
Fluoroscopy-guided Procedures
• W-58 Fine-Grained
Segmentation Using
Hierarchical Dilated Neural
Networks
• W-67 Btrfly Net: Vertebrae
Labelling with Energy-based
Adversarial Learning of Local
Spine Prior

37. Mining Clinical Reports to Gain a Broad
Understanding of Chest X-rays

38. Mining Clinical Reports to Gain a Broad
Understanding of Chest X-rays

39. X-ray-transform Invariant Anatomical Landmark
Detection for Pelvic Trauma Surgery

40. X-ray-transform Invariant Anatomical Landmark
Detection for Pelvic Trauma Surgery

41. DeepDRR-A Catalyst for Machine Learning in
Fluoroscopy-guided Procedures

42. Fine-Grained Segmentation Using Hierarchical
Dilated Neural Networks

43. Btrfly Net: Vertebrae Labelling with Energy-based
Adversarial Learning of Local Spine Prior

44. Btrfly Net: Vertebrae Labelling with Energy-based
Adversarial Learning of Local Spine Prior

45. My own papers
MICCAI2018
• M-18 Adversarial Sparse-View CBCT Artifact
Reduction
• Addresses how to handle streaky artifacts
• T-24 More Knowledge is Better: Cross-Modality
Volume Completion and 3D+2D Segmentation
for Intracardiac Echocardiography Contouring
• As title suggests. A really tough job to do ICE contouring
• T-45 Less is more: Simultaneous view
classification and landmark detection for
abdominal ultrasound images
• Multitask learning
• T-60 3D Anisotropic Hybrid Network:
Transferring Convolutional Features from 2D
Images to 3D Anisotropic Volumes
• When the volume is highly anisotropic, it is a good idea
to transfer features from 2D to 3D
MICCAI2017
• Supervised action classifier: approaching landmark detection as
image partitioning
• A DRL paper for landmark with supervised path design
• Automatic liver segmentation using an adversarial image-to-
image network.
• Using adversarial as a learned shape prior.
• Deep image-to-image recurrent network with shape basis
learning for automatic vertebra labeling in large-scale 3D CT
volumes
• Imposing shape constraint in detecting a cohort of vertebra
landmarks.
MICCAI2018 rejection
• Learning to recognize Abnormalities in Chest X-Rays with
Location-Aware Dense Networks arXiv:1803.04565
• Still a solid work that was then state-of-the-art. The trick is
to use more data and the accessory location information

46. Adversarial Sparse-View CBCT Artifact Reduction

47. Adversarial Sparse-View CBCT Artifact Reduction

48. ICE auto contouring:
A knowledge-fused DL algorithm…
…
• DL-based
• Image completion + segmentation
Sparse
representation
Dense
representation
Dense
representation
Cross-modal
Appearance
3D
Geometry

49. Results
3D network without appearance
knowledge doesn’t converge!
• Liao et al. More knowledge is better: Cross-domain volume completion and 3D+2D segmentation for intracardiac echocardiography contouring, MICCAI
2018 (accepted)

50. 3D Anisotropic Hybrid Network: Transferring Convolutional
Features from 2D Images to 3D Anisotropic Volumes
Liver lesion segmentation challenge

51. Less is more: Simultaneous view classification and
landmark detection for abdominal ultrasound images

52. Simultaneous view classification and landmark
detection for abdominal ultrasound images
View classification
• MTL: 85.29%, STL: 81.22%,
Human: 78.87%
Measurement
• Xu et al., Less is More: Simultaneous View Classification and Landmark Detection for Abdominal Ultrasound Images, MICCAI 2018 (accepted)

53. Supervised action classifier: approaching landmark
detection as image partitioning
Landmark representation: spatially local vs distributed
RepresentationTrainingTesting
• Xu et al., Supervised Action Classifier: Approaching Landmark Detection as Image Partitioning, MICCAI 2017.

54. Landmark detection using DI2IN + supervised
action map [MICCAI’2017]
• Novel representation -- supervised
action map
• Deep image2image network
(DI2IN)
• Xu et al., Supervised Action Classifier: Approaching Landmark Detection as Image Partitioning, MICCAI 2017.
RepresentationTrainingTesting

55. Automatic liver segmentation using an
adversarial image-to-image network [MICCAI’2017]
• Using image2image network and
adversarial shape prior
• Liver segmentation: 34% error
reduction when using 1000 CT
data sets
• Yang et al., Automatic Liver Segmentation Using an Adversarial Image-to-Image Network, MICCAI 2017

56. Deep image-to-image recurrent network with shape
basis learning for automatic vertebra labeling in large-
scale 3D CT volumes [MICCAI’2017]