The document discusses the potential applications of deep learning in healthcare. It begins by explaining that deep learning models can improve accuracy of diagnosis, prognosis, and risk prediction by analyzing large datasets. It then discusses how deep learning can optimize hospital processes like resource allocation and patient flow by early and accurate prediction of diseases. Finally, it mentions that deep learning can help identify patient subgroups for personalized and precision medicine approaches.

1. Deep Learning for Healthcare
DR. MEENAKSHI SOOD
ASSOCIATE PROFESSOR
NITTTR, CHANDIGARH, INDIA
MEENAKSHI@NITTTRCHD.AC.IN

2. Artificial Intelligence
The recent progress in machine learning and artificial intelligence can be
attributed to:
• Explosion of tremendous amount of data
• Cheap Computational cost due to the development of CPUs and GPUs
• Improvement in learning algorithms
Current excitement concerns a subfield called “Deep Learning”.
4/6/2022 DR MEENAKSHI S NITTTR CHD 2

3. Artificial Intelligence
4/6/2022 DR MEENAKSHI S NITTTR CHD 3

4. Transition…….
4/6/2022 M SOOD JAYPEE UNIVERSITY OF INFORMATION TECHNOLOGY 4

5. Traditional and
Deep Learning networks
4/6/2022 DR MEENAKSHI S NITTTR CHD 5

6. Why deeper?
• Deeper networks are able to use far fewer units per layer and far
fewer parameters, as well as frequently generalizing to the test set.
• But harder to optimize!
• Choosing a deep model encodes a very general belief that the
function we want to learn involves composition of several simpler
functions.
4/6/2022 DR MEENAKSHI S NITTTR CHD 6
Hidden layers (cascading tiers) of processing
“Deep” networks (3+ layers) versus “shallow” (1-2
layers)

7. Curse of dimensionality
• The core idea in deep learning is that we assume that the data was generated by
the composition factors or features, potentially at multiple levels in a hierarchy.
• This assumption allows an exponential gain in the relationship between the
number of examples and the number of regions that can be distinguished.
• The exponential advantages conferred by the use of deep, distributed
representations counter the exponential challenges posed by the curse of
dimensionality.
4/6/2022 DR MEENAKSHI S NITTTR CHD 7

8. Deep Neural Networks (DNN)
4/6/2022 DR MEENAKSHI S NITTTR CHD 8
Deep Neural Network is a deep and wide Neural Network.
More number of hidden layers Many Input/ hidden nodes
Deep
Wide

9. Continued….
4/6/2022 DR MEENAKSHI S NITTTR CHD 9
• Utilizes learning algorithms that
derive meaningful data using
hierarchy of multiple layers that
mimics the neural network of human
brain.
• If we provide the systems tons of
information, it begins to understand it
and respond in a useful way.
• Can learn increasingly complex
features and train complex networks.
• More specific and more general-
purpose than hand-engineered
features.

10. Universality Theorem
4/6/2022 DR MEENAKSHI S NITTTR CHD 10
Reference for the reason:
http://neuralnetworksandde
eplearning.com/chap4.html
Any continuous function f
M
: R
R
f N

Can be realized by a network
with one hidden layer
(given enough hidden
neurons)
Why “Deep” neural network not “Fat” neural network?
Deeper is Better?

11. Fat + Short vs Thin + Tall
4/6/2022 DR MEENAKSHI S NITTTR CHD 11
1
x 2
x …… N
x
Deep
1
x 2
x …… N
x
……
Shallow
Which one is better?
The same number
of parameters

12. Fat + Short v.s. Thin + Tall
4/6/2022 DR MEENAKSHI S NITTTR CHD 12
Seide, Frank, Gang Li, and Dong Yu. "Conversational Speech Transcription Using
Context-Dependent Deep Neural Networks." Interspeech. 2011.
Layer X Size
Word Error
Rate (%)
Layer X Size
Word Error
Rate (%)
1 X 2k 24.2
2 X 2k 20.4
3 X 2k 18.4
4 X 2k 17.8
5 X 2k 17.2 1 X 3772 22.5
7 X 2k 17.1 1 X 4634 22.6
1 X 16k 22.1

13. Why Deep?
Deep → Modularization
4/6/2022 DR MEENAKSHI S NITTTR CHD 13
Image
Long or
short?
Boy or Girl?
Sharing by the
following classifiers
as module
can be trained by little data
Girls with
long hair
Boys with
short hair
Boys with
long hair
Classifier
1
Classifier
2
Classifier
3
Girls with
short hair
Classifier
4
Little data
fine
Basic
Classifier

14. Why Deep?
Deep → Modularization
4/6/2022 DR MEENAKSHI S NITTTR CHD 14
1
x
2
x
……
N
x
……
……
……
……
……
……
The most basic
classifiers
Use 1st layer as module
to build classifiers
Use 2nd layer as
module ……
The modularization is
automatically learned from data.
→ Less training data?

15. Hand-crafted
kernel function
SVM
Source of image: http://www.gipsa-lab.grenoble-
inp.fr/transfert/seminaire/455_Kadri2013Gipsa-lab.pdf
Apply simple
classifier
Deep Learning
1
x
2
x
…
N
x
…
…
…
y1
y2
yM
…
……
……
……
simple
classifier
Learnable kernel
4/6/2022 DR MEENAKSHI S NITTTR CHD 15

16. o Manually designed features are often over-specified, incomplete
and take a long time to design and validate
o Learned Features are easy to adapt, fast to learn
o Deep learning provides a very flexible, (almost?) universal,
learnable framework for representing world, visual and linguistic
information.
o Can learn both unsupervised and supervised
Why is DL useful?
In ~2010 DL started outperforming other
ML techniques
first in speech and vision, then NLP
4/6/2022 DR MEENAKSHI S NITTTR CHD 16

17. Size of Data
Performance
Traditional ML algorithms
“Deep Learning doesn’t do different things,
it does things differently”
4/6/2022 DR MEENAKSHI S NITTTR CHD 17

18. Technology
Deep learning is a fast-growing field, and new architectures, variants appearing frequently.
1. Convolution Neural Network (CNN)
CNNs exploit spatially-local
correlation by enforcing a local
connectivity pattern between
neurons of adjacent layers.
4/6/2022 DR MEENAKSHI S NITTTR CHD 18

19. Architecture
CNNs are multilayered neural networks that include input and output layers as well
as a number of hidden layers:
 Convolution layers – Responsible for filtering the input image and extracting
specific features such as edges, curves, and colors.
 Pooling layers – Improve the detection of unusually placed objects.
 Normalization layers – Improve network performance by normalizing the inputs
of the previous layer.
 Fully connected layers – In these layers, neurons have full connections to all
activations in the previous layer (similar to regular neural networks).
4/6/2022 DR MEENAKSHI S NITTTR CHD 19

20. Sample Architecture
4/6/2022 DR MEENAKSHI S NITTTR CHD 20

21. Cont..
2. Recurrent Neural Network (RNN)
RNNs are called recurrent because they perform the
same task for every element of a sequence, with the
output being depended on the previous computations.
4/6/2022 DR MEENAKSHI S NITTTR CHD 21

22. Cont…
3. Long-Short Term Memory
LSTM can learn "Very Deep Learning" tasks that require
memories of events that happened thousands or even
millions of discrete time steps ago.
4/6/2022 DR MEENAKSHI S NITTTR CHD 22

23. The popular CNN
• LeNet, 1998
• AlexNet, 2012
• VGGNet, 2014
• ResNet, 2015
23

24. Pre-Trained Models
• URLs: https://github.com/BVLC/caffe/wiki/Model-Zoo
http://deeplearning4j.org/model-zoo
4/6/2022 24
MEENAKSHI S NITTTR CHD

25. Computational complexity
The memory bottleneck
GPU, a few GB
25

26. Research and Applications
Phyological Research
Neurological Research
Medical Research
BioInformatics Research
Educational Research and Application
Therapeutic Application
4/6/2022 JAYPEE UNIVERSITY OF INFORMATION TECHNOLOGY 26

27. DLApplications
4
Slide credit: Lior Rokach

28. Artificial intelligence in medicine : The
virtual branch
The virtual component is represented by Machine
Learning, (also called Deep Learning)-mathematical
algorithms that improve learning through experience.
Three types of machine learning algorithms:
1. Unsupervised (ability to find patterns)
2. Supervised (classification and prediction
algorithms based on previous examples)
3. Reinforcement learning (use of sequences of
rewards and punishments to form a strategy for
operation in a specific problem space)

29. Benefits of Artificial intelligence
AI can definitely assist physicians
◦ Clinical decision making - better clinical decisions
◦ Replace human judgement in certain functional areas of healthcare (eg, radiology).
◦ up-to-date medical information from journals, textbooks and clinical practices
◦ Experienced vs fresh Clinician
◦ 24x7 availability of expert
Early diagnosis
Prediction of outcome of the disease as well as treatment
Feedback on treatment
Reinforce non pharmacological management
Reduce diagnostic and therapeutic errors
Increased patient safety and Huge cost savings associated with use of AI
AI system extracts useful information from a large patient population
Assist making real-time inferences for health risk alert and health outcome
prediction
Learning and self-correcting abilities to improve its accuracy based on
feedback.

30. What makes healthcare different?
• Often very little labeled data (e.g., for clinical NLP)
– Motivates semi-supervised learning algorithms
• Sometimes small numbers of samples (e.g., a rare disease)
– Learn as much as possible from other data (e.g. healthy
patients)
– Model the problem carefully
• Lots of missing data, varying time intervals, censored
labels

31. What makes healthcare different?
• Difficulty of de-identifying data
– Need for data sharing agreements and sensitivity
• Difficulty of deploying ML
– Commercial electronic health record software is difficult
to modify
– Data is often in silos; everyone recognizes need for
interoperability, but slow progress
– Careful testing and iteration is needed

32. Large datasets
Laboratory for
Computational
Physiology
De-identified health
data from
~40K critical care
patients
Demographics, vital
signs, laboratory tests,
medications, notes, …

33. President Obama’s initiative to create a 1 million
person research cohort
[Precision Medicine Initiative (PMI) working Group Report, Sept. 17 2015]
THEPRECISIONMEDICINEINITIA
TIVE
Large datasets
Core data set:
• Baseline health exam
• Clinical data derived
from electronic health
records(EHRs)
• Healthcare claims
• Laboratorydata

34. Diversity of digital health data
genomics
imaging
lab tests
phone
vital signs
proteomics
devices
social media

35. Industry interest in AI &
healthcare
43
Slide credit: David Sontag

36. Emergency Department:
• Limited resources
• Time sensitive
• Critical decisions

37. Lab results
(Continuous valued)
MD comments
(free text)
Triage Information Specialist consults
(Free text)
Physician
documentation
Repeated vital signs
(continuous values)
Measured every 30 s
T=0
30 min
2 hrs
Disposition
Data in Emergency Department (ED)
Collaboration with
Steven Horng, MD
Electronic records for over 300,000 ED visits

38. Generic Flow Diagram

39. What can Deep
Learning do for
the healthcare?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

40. What can Deep
Learning do for
the healthcare?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

41. Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
new methods for chronic disease risk prediction and visualization
that give clinicians a comprehensive view of their patient population,
risk levels, and risk factors, along with the estimated effects of potential interventions.
Increased
risk
of
heart
attack

Prashar, N., Sood, M., Jain, S., “Design and
implementation of a robust noise removal system in
ECG signals using dual-tree complex wavelet
transform”
Biomedical Signal Processing and Control, Jan
2021, 63, 102212 DOI:10.1016/j.bspc.2020.102212
(SCI indexed, IF: 3.137)

43. Hierarchical Infinite Factor Model
Speeding up:
Stochastic
Gradient
Nose-Hoover
Thermostats
Sampling

44. What can Deep
Learning do for
the healthcare?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

45. Optimize hospital
processes such as
resource allocation
and patient flow.
By early and accurate prediction of disease ,
predict demand and allocate scarce hospital resources such as beds
and operating rooms.

46. What can Deep
Learning do for
the healthcare?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

47. Automate detection
of relevant findings
in pathology,
radiology, etc.
Key advance 1: Very efficient, accurate
search over subareas of an image.
Key advance 2: Use hierarchy to search
at multiple resolutions (coarse to fine).

48. 48
Classification of Non-Proliferative Diabetic Retinopathy from Retinal Fundus
Images Employing Hierarchical Severity Level Grading system
1. Bhardwaj, C., Jain, S. & Sood, M. Deep
Learning–Based Diabetic Retinopathy
Severity Grading System Employing
Quadrant Ensemble Model. J Digit
Imaging (2021). April
https://doi.org/10.1007/s10278-021-
00418-5 (SCI indexed, IF:
4.056)

49. Diabetic Retinopathy Severity
Grading Classification
49

50. Key advance 1: Very efficient, accurate search over subareas of
an image.
Key advance 2: Use hierarchy to search at multiple resolutions
(coarse to fine).
Detection is also valuable when
key patterns of interest are
discovered by integrating
information across many
patients, and might not be visible
from a single patient’s data.

51. What can
machine learning
do for the
healthcare
industry?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

52. Discover new
medical knowledge
(clinical guidelines,
best practices).
Claims data: ~125K
patients with diseases of
the circulatory system
APC-Scan
Most significant detected pattern:
Glucocorticoids are associated with dramatically
increased hospitalizations and length of stay in the
subpopulation of ~2K overweight, hypertensive
males with endocrine secondary diagnoses.
Regression on separate, held-out patient dataset:
51% increase in hospitalizations for this
subpopulation; vs. 11% for entire patient population.
Glucocorticoids
Yes No
Number of Patients 264 1713
Mean Number of
Hospitalizations
0.606
(0.069)
0.280
(0.016)

53. What can
machine learning
do for the
healthcare
industry?
Improve accuracy
of diagnosis,
prognosis, and
risk prediction.
Optimize hospital
processes such as
resource allocation
and patient flow.
Identify patient
subgroups for
personalized and
precision medicine.
Improve quality of care and
population health outcomes,
while reducing healthcare costs.
Reduce
medication errors
and adverse
events.
Discover new
medical knowledge
(clinical guidelines,
best practices).
Model and
prevent spread of
hospital acquired
infections.
Automate detection
of relevant findings
in pathology,
radiology, etc.

54. Identify patient
subgroups for
personalized and
precision medicine.
At the very beginning of the image analysis workflow, machine learning will be
used to triage incoming studies based on the initial AI findings, to route the study to
the appropriate specialist
Once the radiologist opens a new case, machine learning will search for any
clinically relevant prior studies, automatically register studies, select the hanging
protocol and provide the radiologist with contextually relevant tools and supporting
information, for example from the patient’s electronic medical record .
In parallel, machine learning will automatically detect, segment, visualize and
quantify any abnormalities . If an abnormality is detected, machine learning will
then provide diagnostic decision support, such as a probability score for malignancy
or a differential diagnosis.
Triage-Diagnostic Decision Support

55. AI-Enabled Connected Health
Informatics
software frameworks for deep-learning are becoming increasingly
capable of training advanced neural-network models, while on the other
hand, heterogeneous hardware components such as GPUs, FPGAs and
ASICs dedicated to deep learning are beginning to challenge the
computational limits of Moore’s law.
Together, these trends have influenced connected-health informatic
systems, which comprise various processes for sensing, data transfer,
storage and analytics to improve overall health and wellbeing.
4/6/2022 55

56. Identify a variety of cancers such as breast cancer,
prostate cancer, and lung lesions
iCAD
http://signifyresearch.net/analyst-insights/
• Automation
• Accuracy
• Consistency

57. Identify a variety of cancers such as breast cancer,
prostate cancer, and lung lesions
iCAD
automatic detection and measurements of imaging
features (biomarkers) to assist with diagnosis, such
as lung density, breast density, analysis of coronary
and peripheral vessels, etc.
4D Flow fromArterys
http://signifyresearch.net/analyst-insights/
• Automation
• Accuracy
• Consistency

58. Identify a variety of cancers such as breast cancer,
prostate cancer, and lung lesions
iCAD
detection and quantification,
alongside supporting information
extracted from an EHR,
pathology reports and other
patient records, to assist with
diagnosis
IBM Watson Health
automatic detection and
measurements of imaging
features (biomarkers) to assist
with diagnosis, such as lung
density, breast density, analysis of
coronary and peripheral vessels,
etc.
4D Flow fromArterys
http://signifyresearch.net/analyst-insights/
• Integration
• X-collaboration

59. Artificial Intelligence for Radiology

60. Use Case: Tumor Tissue Characterization using Ultrasound
Problem Definition:
GS 3+3 GS 4+4
GS 4+3
Benign GS 3+4
...
Prostate under Ultrasound
Prostate under microscope
https://www.cancer.gov/types/prostate/patient/prostate-treatment-pdq
Courtesy of Dr. Abolmaesumi
Cancer Map

61. Benign Malignant
CT Images US Images
Microscopic Images of Blood
MRI Images
DNA sequence signal
Non-invasive visualization of internal organs, tissue, etc.
Medical Imaging
Jyotsna Dogra, Shruti Jain,
Meenakshi Sood, “Glioma
Extraction from MR Images
Employing Gradient Based Kernel
Selection Graph Cut Technique”
The Visual Computer, vol 36(5), pp-
875-891 DOI: 10.1007/s00371-019-
01698-3. ISSN: 0178-2789
(Print) 1432-2315 (Online)
Springer
(SCI IF: 1.415)

62. Biosignals
4/6/2022 62

63. Use Case: Tumor Tissue Characterization using Ultrasound
Steps:
1. Understand the problem
2. Define input(s) and output(s)
3. Investigate limitations and boundary conditions
4. Collect representative data
1. [Labels]
5. [Calculate features/engineer features]
6. Define ML framework
7. Define Metric for evaluation
...

64. 1. Understand the problem
2. Define input(s) and output(s)
3. Investigate limitations and boundary conditions
4. Collect representative data
1. [Labels]
5. [Calculate features/engineer features]
6. Define ML framework
7. Define Metric for evaluation
...
Courtesy of Dr. Abolmaesumi

65. Use of robots to deliver treatment..robotic surgery
Use of robots to monitor effectiveness of treatment
Use of robots to deliver treatment - Robotic surgery

66. Successes!
▶ Mammographic mass
classification
▶ Brain Lesions
▶ Air way leakages
▶ Diabetic Retinopathy
▶ Prostrate Segmentation
▶ Breast cancer metastasis
▶ Skin Lesion Classification
▶ Bone suppression in Chest X-Rays
6
Source: arXiv:1702.05747

67. Medical Imaging Open Datasets
▶ http://www.cancerimagingarchive.net/
▶ Lung Cancer, Skin Cancer, Breast Cancer….
▶ Kaggle Open Datasets
▶ Diabetic Retinopathy, Lung Cancer
▶ Kaggle Data Science Bowl 2018
▶ https://www.kaggle.com/c/data-science-bowl-2018
▶ ISIC Skin Cance r Dataset
▶ https://challenge.kitware.com/#challenge/583f126bcad3a51cc66c8d9a
▶ Grand Challenges in Medical Image Analysis
▶ https://grand-challenges.grand-challenge.org/all_challenges/
▶ And more…
▶ https://github.com/sfikas/medical-imaging-datasets
26

68. Resources
▶ CBInsights AI in Healthcare Map: https://www.cbinsights.com/research/artificial-intelligence-startups-healthcare/
▶ DL in Medical Imaging Survey : https://arxiv.org/pdf/1702.05747.pdf
▶ Unet: https://arxiv.org/pdf/1505.04597.pdf
▶ Learning to diagnose from scratch exploiting dependencies in labels: https://arxiv.org/pdf/1710.10501.pdf
▶ TieNet Chest X-Ray Auto-reporting: https://arxiv.org/pdf/1801.04334.pdf
▶ Dermatologist level classification of Skin Cancer using DL: https://www.nature.com/articles/nature21056
▶ Tensorflow Intel CPU Optimized: https://software.intel.com/en-us/articles/tensorflow-optimizations-on-modern-intel-
architecture
▶ Tensorflow Quantization: https://www.tensorflow.org/performance/quantization
▶ Deep Compression Paper: https://arxiv.org/abs/1510.00149
▶ Microsoft’s Project Brainwave: https://www.microsoft.com/en-us/research/blog/microsoft-unveils-project-brainwave/
▶ Can FPGAs Beat GPUs?: http://jaewoong.org/pubs/fpga17-next-generation-dnns.pdf
▶ ESE on FPGA: https://arxiv.org/abs/1612.00694
▶ Intel Spark BigDL: https://software.intel.com/en-us/articles/bigdl-distributed-deep-learning-on-apache-spark
▶ Baidu’s Paddle-Paddle on Kubernetes: http://blog.kubernetes.io/2017/02/run-deep-learning-with-paddlepaddle-on-
kubernetes.html
▶ Uber’s Horovod Distributed Training framework for Tensorflow: https://github.com/uber/horovod
▶ TFX: Tensorflow based production scale ML Platform: https://dl.acm.org/citation.cfm?id=3098021
▶ Explainable AI: https://www.cc.gatech.edu/~alanwags/DLAI2016/(Gunning)%20IJCAI-16%20DLAI%20WS.pdf
28

69. Questions?