A.I. in Radiology: Hype or Hope?

A.I. in Radiology
“Hype or Hope”?
The Radiologist’s view
Erik Ranschaert, MD, PhD, CIIP
President EuSoMII

Disclosures
• President EuSoMII
• CMO Diagnose.me
• Advisory Board MedicalPHIT
BSR annual meeting 2018

De Griekse god Janus
Data
Information
Knowledge
Wisdom

EOS, Oct.2017BSR annual meeting 2018

Recent changes
Cloud
technology
Processing
power, GPU’s
Big data A.I.

Radiologists are not unfamiliar with AI
• 1963 – 2013: first 50 years failed
• 2012 IMAGENET competition: AlexNet gave
a dramatic decrease in image classification
error rate
• 2016 Geoffry Hinton: “it's quite obvious that
we should stop training radiologists”
• Last 2–3 yrs: increased activity in
development of DL algorithms for radiology
• For narrow-based tasks the accuracy rates of
CNNs surpass those of humans (e.g. nodule
detection)

ML and DL
• Machine Learning (ML) learns computers “to
think” without being programmed: from
experience, by input of training data
• ML makes “prospections” based upon skills
learned from the “training data” it has been fed
with
• ML makes advanced statistical calculations with
algorithms
• Deep Learning (DL) is subfield of ML
• DL is the type of ML based upon multiple layers
(tens or hundreds) -> “Layered Cake” = ....
“Layered cake”
D
E
E
P

Layered Cake
• Neural Network
• “Multistage information distillation” model to
“purify” information
• Input layer = fed with information
• The “hidden layers” have artificial neurons
combining signals and calculating different
“weights” for the data in each neuron.
• The output of the layer is passed through to the
next layer.
• Last layer = output layer = “fully connected layer” =
classifier

Current trend for deep learning.
Fei Jiang et al. Stroke Vasc Neurol 2017;2:230-243

Where is it all happening?
MICCAI 2018 Neural Information Processing
Systems (NIPS)• Medical Image Computing
& Computer Assisted
Intervention - MICCAI
• Collaboration with ACR
• >1600 attendees
• >1000 submissions
• +33%
8000 attendees in 2017
3240 submissions
https://medium.com/syncedreview/a-statistical-tour-of-nips-2017-
438201fb6c8a

AI Startups

UK can lead in Radiology AI. Here’s how...Hugh Harvey, 2017, Medium - http://bit.ly/2CoKJE8
Industry partnerships
• Dominant industry vendors formed strategic
partnerships with academic institutions.
• GE, IBM Watson, Philips and many others have
created deals to allow data access in return for
funding research.
• The American College of Radiologists (ACR) has
announced it’s own Data Science Institute.
• In Europe we can see EU/industry partnerships
forming with the academy

Is the future of radiology in the hands of robots?

Fake news?
• “Algorithm can diagnose pneumonia better
than radiologists”
≠
• “Algorithm can detect pneumonia from
chest X-rays better than radiologists”

Analyse Luke Oakden-Rayner
• The training dataset has labels that don’t
really match the images, it has questionable
relevance.
• For detecting pneumonia-like image features
on chest x-rays, this system performs at least
on par with human experts.
https://lukeoakdenrayner.wordpress.com/2018/01/24/chexnet-an-in-depth-review/

Canadian Association of Radiologists Journal 2018 69, 120-135DOI: (10.1016/j.carj.2018.02.002)
Copyright © 2018 The Authors Terms and Conditions
Tasks of radiologists

A.I. Revolutionizing Workflow
• Most attention goes to automated image analysis
• Radiology workflow is another active area of A.I.
research
• A.I. also has the potential to improve patient care
and safety
• It’s essential to define relevant clinical use cases for
implementation of A.I. in the radiological workflow

Use cases for AI
• Most AI start-ups are focusing on image analysis
• AI for broader range of imaging applications
– Pre-scan
– Intra-scan
– Post-scan

What’s the greatest potential of AI in HC?
• The greatest potential of A.I. lies
in making back-end processes
more efficient.
Source: B. Kalis et al, Harvard Business Review, May 10, 2018
https://www.accenture.com/us-en/insight-artificial-intelligence-healthcare

AI
Patient and
Referring
Provider
Imaging
Appropriateness
& Utilization
Patient
Scheduling
Imaging Protocol
selection
Imaging
Modality
operations, QA,
dose reduction
Hanging
protocols,
Optimization
staffing &
worklist
Interpretation
and reporting
Communication
and billing
Source: JM Morey et al.Applications of AI Beyond Image Interpretation, Springer 2018 –
in press
A.I. Imaging
Value Chain

Reduced acquistion time
• Collaboration between NYU (CAI2R) and Facebook (FAIR) to make MRI
scans 10x faster with neural networks (AI)
• Automated calculation of missing information gaps with DL
• 10.000 clinical cases with 3 million images

Left: Standard high dose CT at 12.4mGy. Middle: Ultra-low dose CT at 1.3mGy. Right: AI-enhanced ultra-low dose CT at 1.3mGy.
Diagnostic image quality between the left and right images was rated as comparable by independent radiologists, despite a
significant dose reduction of 11.1mGy. The middle image is noisy and non-diagnostic. Images courtesy of Algomedica.
SIIM 2017 Poster

MRIguidance
MRIguidance–gettingmoreinformationout ofyour MRI data
In an ideal world, a single medical imaging exam would generate 3D images of both bone and the soft tissue without the
use of harmful radiation. However, orthopedic surgeons and radiologists now often choose between a MRI scan for soft
tissueand an X-ray based CT scan for boneinformation.To reducepatient burden and healthcarecosts,they haveto work
with suboptimal information for diagnosisand treatment planning and in some cases,even two examsare required.
MRIguidance now brings this ideal world to reality with BoneMRI.Bone MRI is a software solution that generates a CT-
like image to complement the soft tissue images derived from a MRI scan. For the clinicians Bone MRI works in the
background in aseamlessworkflow without the need of new desktop applications.
Focusoninitialanatomicalregions:
Shoulder:diagnosisof instability,arthrosis,treatment
planningfor prosthesis.
Spine:diagnosisand treatment planningof hernia,
stenosisand vertebra fusion surgery.
“Atechnological innovation likeBoneMRI
will optimizethediagnosticworkflow and
greatlybenefit our doctorsand patients.”
Prof.dr.M.vandenBosch- Radiologist and
ExecutiveBoard,OLVGhospital,Amsterdam
R&DandIPposition
MRIguidance isaspin-off company of the UMC Utrecht
and under acollaboration agreement MRIguidance has
Customers
Orthopedic surgeonsand radiologists.
Ø One-stop-shop:efficient
clinical workflow
Ø Better diagnostic
information
Ø Lower healthcarecosts
Ø Noradiation
Spin-off of UMC Utrecht, collaboration with 10 Dutch hospitals

P. Lakhani et al. JACR https://doi.org/10.1016/j.jacr.2017.09.044
Automated hanging protocol
• Radiologist opens study
• ML assistant creates optimal
hanging protocol
• Radiologist makes report and
sends it to RIS or EHR

Triage and prioritization
• Detection of Intracranial Hemorrhage (ICH)
• Algorithm processes exam and generates
binary output (negative or positive ICH).
• If results are positive, priority of study is
upgraded to “stat”.
• The reading list is updated in real-time.
• Possible to reduce TAT up to 96% for head CT
exams
npj Digital Medicine (2018) 1:9 ; doi:10.1038/s41746-017-0015-z

AI for reporting
• Semi-automisation of reporting
1. Automated inclusion of quantitative data and Radiomics
2. Automated Computer-aided Reporting (CAD)
3. Automated scoring (PIRADS, BIRADS, LIRADS, TIRADS, C-RADS)
4. Automated adaption of style and language for reader
5. Automated quality control of report, offering “missing topics”
6. Automated presentation of guidelines depending on diagnosis
7. Automated method for labeling and coding of findings, inclusion and
registration of Common Data Elements (CDE’s)

FINDINGS
T11-T12
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
T12-L1
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
L1-L2
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
L2-L3
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
L3-L4
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
L4-L5
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
L5-S1
• X1 X2 X3 X4 X5 X6
• X7 X8 X9 X10 X11 X12
(COMMON DATA ELEMENTS)
Source: K. Dreyer, Clinical Data Science Interagency Working Group on Medical Imaging National Science and Technology Council 2016, https://slideplayer.com/slide/11642143/

Google and others?
• Google and others
believe that delivering
AI through the cloud
will be a big, lucrative
trend in computing in
coming years.

Amazon
• Amazon sees a new grow market
in HC and will offer HC services
• Own health insurance – first for
Amazon employees (500.000 !)
• “We have the size for influencing
the insurance companies. The
problem of the rising costs in HC is
only becoming worse. Innovation
is necessary”.
Chief Technology
Officer Amazon
NRC, 6 april 2018

Retrospective quality assessment?

Ethical issues
1. Human bias
2. Transparency of algorithm(s)
3. Collective brain

Human bias & transparency
• Training data can be biased
– Google Photos image classifier
tagged black person as gorilla
• What is the real purpose of
the algorithm?
– Transparency of “black box”
https://medium.com/@DrHughHarvey/building-ethical-ai-in-healthcare-why-we-must-demand-it-ca60f4d28412

When implementing new decision-making tools,
the hospital will need to guard against “learned
helplessness”, where people become so reliant on
automated instructions that they abandon
common sense...

AI-bias
• Automation bias (automation
induced complacency)
– the tendency to over-rely on
automation, failure to recognise
new errors that AI can introduce
Carol Beer, Little Britain
“The exemplary receptionist”
The
computer
says no...

Adversarial examples (AE’s)
• Inputs to ML models that have been crafted to force the model to make a
classification error.
• AE’s can be crafted to be very effective... without being visible to human
eyes!

Min. visible
Disease
or not
Finlayson,S.G.,Chung,H.W.,Kohane,I.S.,&
Beam,A.L.(2018,April15).AdversarialAttacks
AgainstMedicalDeepLearningSystems.

So what should radiologists do with A.I.?
“Data Wrangler” ?
DICOM 2018, Mainz

From factory worker to consultant
• Avoid time-consuming repetitive tasks (factory worker)
• Use A.I. technology for routine tasks and time-consuming
tasks
• Add expertise where it counts
• Exploit modern technology for functional information
• Regain partnership in patient care (Consultant)

Elias Zerhouni
Radiologist
Biomedical Engineer
Johns Hopkins, Sanofi
Director of NIH

PUSH or PULL?
• PUSH:
– Radiologists will decide what applications are being used and how
– Will they push the right techniques? Do radiologists know what clinicians
want? Will they accept it? …e.g perfusion imaging
• PULL:
– Radiologists do not adapt -> clinicians will start using it first
– Referring clinicians / insurance / authorities /patients will demand radiologists
to use AI
• Standardisation, accreditation, guidelines (e.g. RECIST for immunotherapy, interstitial
lung disease)
• Economics (hospital)
• Quality (clinicians, patients,…?)
https://www.auntminnieeurope.com/index.aspx?sec=
sup&sub=pac&pag=dis&ItemID=616573

Major roadblocks for AI
Sli.do live survey, EuSoMII Annual Meeting, Rotterdam, Nov 3, 2018

Possible scenarios for integrating AI
1. A.I. on demand
2. Automated image analysis
3. Automated routing to EPR
P. Lakhani, NIBIB AI in Medical Imaging Workshop, Aug 23, 2018

Possible scenarios
1. A.I. on demand
– For a single image or series of images
– PACS radiologist  AI server  PACS, RIS,
EHR
– Radiologist would be in control of asking relevant
A.I. interpretations
– Requires manual intervention from radiologist

Possible scenarios
2. Automated A.I. image analysis
– Exams automatically sent to A.I. server (before reading)
– Analysis “pipeline” starts execution upon reception if
there is a positive match
– modality  AI server  PACS radiologist  RIS, EHR
– Helps to prioritize reading order -> reduce TAT
– Radiologist views A.I.-findings before final report is made
– Radiologist is able to ensure accuracy and validate report

Possible scenarios
3. Automated routing
– As in 2. but results are generated and automatically routed to RIS or EPR
– Requires discrepancy management
– AI -> preliminary -> RIS/EHR -> staff radiologist -> final
– Accurate AI needed (highly sens and spec), high confidence
– Fastest TAT although potential risk
– Might increase calls to radiology reading room
– Might have medicolegal consequences
Source: P. Lakhani, NIBIB AI in Medical Imaging Workshop, Aug 23, 2018

Implementation of AI algorithms for radiology
• “Ensuring that algorithms can be SEAMLESSLY integrated into radiologists’
clinical workflow is of paramount importance because if the A.I. tool is
not readily available to the end users in their workflow, adoption in clinical
practice will be less likely to occur.”
(B. Allen, K. Dreyer)
• Interoperability between all systems is prerequisite
• Radiologists have to determine the best model for implementing A.I.
– When, how for what purpose
– What to do with the results and reports
– How to validate the apps? What quality criteria?
M. Walter, Radiology Business, May 07, 2018
B. Allen, JACR, DOI: https://doi.org/10.1016/j.jacr.2018.02.032

Automated analysis
Integration of quantitative data in SR
Seamless integration of AI with
PACS and EMR, “all-in-one”
Interoperability
Facilitates training of
new algorithms &
applications
Data
Swiss-knife for radiologists: all-in-one

Types of marketplaces to expect
1. Vendor-neutral model, launched from PACS
2. Vendor-locked model, embedded in platform
of traditional vendor
3. Cloud-based SaaS model (Apple-store)

What business model?
• AppStore model, application on request
• Pay per analysis = fee for each analysis
• Subscription fee = fee per package of monthly analyses
• License = pay for whole platform with limited n
analyses
• Included with modality
• Included with PACS/RIS

Training

A.I. is roaring at the start
• AI = racing machine
• What is fuel for AI?
– Sharing of data internally & externally
– Vetted / annotated data
• Where are the roads for AI?
– IT infrastructure in the hospital (interoperability)
– Connected & communicating systems
– Optimal integration in radiological workflow
• AI-ecosystem = traffic regulation
• The responsible pilot remains the radiologist

What role should radiologists play?
1. Radiologists are in the lead with AI for HC and should stay there
2. They should pro-acitvely invest in their future now
3. They should guide the new developments instead of undergoing them
4. They should implement AI with a strategic plan and goal of increasing value
5. The should have a crucial role in provision and curation of training data
6. They should inform hospital managers and politicians about their role and
importance in the clinical decison making
7. They should create partnerships with all stakeholders, including hospital IT
managers
8. They should include imaging informatics in radiology training and motivate the
younger generation to embrace this new technology

Curtis Langlotz
“Artificial intelligence will
not replace radiologists.
Yet, those radiologists who
use AI will replace the
ones who don’t.”
Curtis Langlotz, Professor of Radiology and
Biomedical Informatics at Stanford University, GPU
Tech Conference in San Jose, May 2017

springer.com
1st ed. 2019, X, 396 p. 105 illus., 93 illus.
in color.
Printed book
Hardcover
119,99 € | £109.99 | $149.99
128,39 € (D) | 131,99 € (A) | CHF[1]
141,50
eBook
101,14 € | £87.50 | $109.00
101,14 € (D) | 101,14 € (A) | CHF[2]
113,00
Available from your library or
springer.com/shop
MyCopy [3]
Printed eBook for just
€ | $ 24.99
springer.com/ mycopy
Erik R. Ranschaert, Sergey Morozov, Paul R. Algra (Eds.)
Artificial Intelligence in
Medical Imaging
Opportunities, Applications and Risks
Provides a thorough overview of the impact of artificial intelligence (AI ) on
medical imaging
I ncludes contributions from radiologists and I T professionals, ensuring a
multidisciplinary approach
Makes practical recommendations for the use of AI technology for both
clinical and nonclinical applications
This book provides a thorough overview of the ongoing evolution in the application of artificial
intelligence (AI) within healthcare and radiology, enabling readers to gain a deeper insight into
the technological background of AI and the impacts of new and emerging technologies on
medical imaging.After an introduction on game changers in radiology, such as deep learning
technology, the technological evolution of AI in computing science and medical image
computing is described, with explanation of basic principles and the types and subtypes of AI.
Subsequent sections address the use of imaging biomarkers, the development and validation of
AI applications, and various aspects and issues relating to the growing role of big data in
radiology. Diverse real-life clinical applications of AI are then outlined for different body parts,
demonstrating their ability to add value to daily radiology practices. The concluding section
focuses on the impact of AI on radiology and the implications for radiologists, for example with
respect to training. Written by radiologists and IT professionals, the book will be of high value
for radiologists, medical/clinical physicists, IT specialists, and imaging informatics professionals.

Join EuSoMII now!
www.eusomii.org

A.I. in Radiology: Hype or Hope?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to A.I. in Radiology: Hype or Hope?

Similar to A.I. in Radiology: Hype or Hope? (20)

More from Erik R. Ranschaert, MD, PhD

More from Erik R. Ranschaert, MD, PhD (20)

Recently uploaded

Recently uploaded (20)

A.I. in Radiology: Hype or Hope?