Our presentation on Deep Learning for Osteoporosis held at the European Congress of Radiologists in Vienna in March 2017

1. Why Big Image Analytics is essential for Imaging Biobanks:
An Osteoporosis Radiomics Study
K. Mader, T.J. Re,
J. Cyriac, B. Stieltjes
ECR 2017

2. Helpful Buzzwords
● Big Image Analytics - using the techniques of machine
learning and Big Data to analyze image data rapidly and
efficiently on computing clusters or in the cloud
● Imaging biobank - large (>10K cases) of ‘homogenous’
imaging studies taken in a broad population including many
healthy people
● Radiomics - field or practice of extracting a large number of
descriptive values from images or regions in images to
support quantitative rather than qualitative decision
making

3. Imaging biobanks are growing in size and volume
Bones are present in almost every image
Osteoporosis is a very expensive disease and is often visible in
radiological images that are missed
Most imaging tasks are complicated
most data sets are very heterogeneous

4. ● Isn’t segmenting bone easy
○ one click
○ just a window
○ Anything denser than 300 HU

5. ● Actually not so easy…
○ Bone is mostly dense
○ So are other things
○ Not all bone is dense
● We need more criteria
○ Where it is
○ what surrounds it
○ other scan parameters
○ contrast agents...

6. ● Concrete problems are much
easier than hypothetical
ones
● 180 images taken from chest
CTs from patients between
23 and 89 years old
● Can we find features that
distinguish young and old
patients?

7. Classical
Image
Analysis
Approach

8. ● We need more criteria
○ Where it is
○ what surrounds it
○ other scan parameters
○ contrast agents...

9. ● We need more criteria
○ Where it is
○ what surrounds it
○ other scan parameters
○ contrast agents...

10. ● We need more criteria
○ Where it is
○ what surrounds it
○ other scan parameters
○ contrast agents...
○ We are very quickly
overwhelmed with
rules and still have
poor performance in
many cases

11. Little differences
Every person looks a little bit different
Every scan looks a little bit different
Every scanner makes a slightly different image
Every physician marks a slightly different
region

12. ● Many of these manual rules
are exceptionally sensitive
and fragile
One pixel change

13. From Regions
to Radiomics

14. Old
Young

15. Radiomics seems Great!
*60 metrics with p<0.05
**55 metrics with p<0.01
***35 metrics with p<0.001

16. Radiomics seems Great!
*60 metrics with p<0.05
**55 metrics with p<0.01
***35 metrics with p<0.001

17. Flip one pixel (0.00038%)
- Changing one pixel (at random)
- 24 of these radiomics metrics change
more than the differences between
group
- Only 3 metric (of 92) are less sensitive
than the pixel change
- Maximum intensity
- Minimum Intensity
- GLCM Imc1

18. Deep Learning is different
Instead of making hundreds of manual rules and features
Learn millions of rules from the data itself

19. Deep Learning lets us efficiently learn millions of rules
Each orange box contains 9 different weights
which are adapted with each piece of training
data

20. Learning to recognize vertebrae
- The initial iterations are very
poor at identifying regions of
bone
- After a few iterations of
training the algorithm can
recognize bone
- After another few iterations it
can accurately distinguish
different types of bone

21. Learning to recognize vertebrae
- The progress of the algorithm is
tracked using the following curve
- As more patients are seen the
accuracy generally increases.

22. Robustness Against Noise
● Adding noise or slight changes
in pixels does not change
overall segmentation
● Up until nearly unrecognizable
levels of noise the
segmentation is intact

23. Rapid Execution
● Convolutional Neural Networks have been implemented for nearly every OS
/ framework
○ Linux, Mac, Windows, iOS, Android, Web-browsers, ...
● Performance has been highly optimized for GPU and TPU execution
○ A single GPU can process over 24 slices per second
○ 80M images in 925 hours per GPU

24. Conclusion
● Traditional image analysis methods are very complicated and ill-suited for
large scale studies with heterogenous patients, scans, and scanners
● Radiomics shows promise but its extreme sensitivity to small changes is a
hurdle for widespread clinical acceptance.
○ Be skeptical!
● Careful Machine Learning approaches can alleviate many of these issues

25. - Develop better radiometrics
- DeepRobustRadiometrics
- Improve Interpretability of
results
- Generate automatic interactive
reports
Next Steps

26. Joachim Hagger
Master of Science in Physics
Digital business leader
PD Dr. Tobias Heye
Oberarzt und stv. Leiter Kardiale
und Thorakale Diagnostik
Dr. Kevin Mader
Doctor of Sciences ETHZ
Machine Learning / Image
Analysis
Prof. Dr. Elmar M. Merkle
Chefarzt und Leiter Klinik für
Radiologie und Nuklearmedizin
Bram Stieltjes, MD, PhD
Forschungsgruppenleiter
Radiologie
Dr. med. Dipl. Phys. Gregor Sommer
Stv. Oberarzt Cardiale und
Thorakale Diagnostik
PD Dr. Alexander Sauter
Facharzt Radiologie,
Assistenzarzt Nuklearmedizin
Joshy Cyriac
Machine Learning / Software
Engineering
Dr. Thomas J Re, MD, MSEE
Research / Radiology
Insights
Flavio Trolese
Dipl. Ing. FH Informatik
Digital business leader

27. Thanks for your attention!
Read More
https://medium.com/@4quant
Learn More
Deep Learning at SIIM
Code More
github.com/4quant
Get in touch
info@4quant.com,
mader@biomed.ee.ethz.ch
bram.stieltjes@usb.ch

28. Constructing simple rules from metrics
T1 Tumor ≤3 cm across its greatest dimension, surrounded
pleura, without invasion, and more proximal than the lobar b
T1a Tumor ≤2 cm across its greatest dimension
T1b Tumor >2 cm and ≤3 cm across its greatest dimension
T2 Tumor >3 cm and ≤7 cm or with any of the following fea
bronchus and is more than 2 cm distal to the carina; inva
associated with atelectasis or obstructive pneumonitis that
region without involvement of the entire lung
T2a Tumor >3 cm and ≤5 cm across its greatest dimension
T2b Tumor >5 cm and ≤7 cm across its greatest dimension
T3 Tumor >7 cm or any of the following features: direct invas
(including the superior sulcus), diaphragm, phrenic nerve,
or parietal pericardium; involvement of the main bronchus
carina (without involvement of the carina); associated atele
pneumonitis of the entire lung; or a tumor nodule within th
of the primary tumor
T4 Tumor of any size with invasion of the mediastinum, h
trachea, recurrent laryngeal nerve, esophagus, vertebral b
separate tumor nodule within an ipsilateral lobe
Our Results
Clinical Guidelines

29. A standard topology is
presented as follows starting
with the input data and
resulting in the final map of
segmented bone

34. From Annotations to Textures

35. Validating Radiomics

36. From Annotations to Radiomics
3D tumor shape analysis

37. Annotation