Alexandros Sigaras | @AlexSigaras
Research Associate in Computational Biomedicine
Weill Cornell Medicine
Microsoft Reactor
Grand Central Tech
1.11.2017
How HoloLens
Transforms Healthcare
NYC HoloLens Developers Meetup
Source: EMC & IDC
“The Digital Universe. Driving Data Growth in Healthcare”
https://www.emc.com/analyst-report/digital-universe-healthcare-vertical-report-ar.pdf
Healthcare & Big Data
Today
Healthcare & Big Data
2025 Predictions
Astronomy Twitter Youtube Genomics
Acquisition 25
zetta-bytes/year
0.5–15
billion
tweets/year
500–900
million
hours/year
1
zetta-
bases/year
Storage 1
EB/year
1–17
PB/year
1–2
EB/year
2–40
EB/year
Source: NIH
“Big Data: Astronomical or Genomical Zachary Stephens et al.”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4494865/
Source: NIH
https://www.genome.gov/sequencingcosts/
Cost per Genome
“My hope is that this
becomes the foundation,
the architecture,
whereby in 10 years from
now we can look back
and say that we have
revolutionized
medicine.”
Precision Medicine Initiative
Budget
$130M
F Y 1 6 E N A C T E D F Y 1 7 P R E S I D E N T ’ S
R E Q U E S T
$230M
Precision Medicine Initiative
Regional Medical Centers across USA
• Able to enroll diverse
patient populations
• Strong electronic
health record
capacity
• Geographic spread
• Capacity to enroll
35,000 a year
Northwestern	
University
Institute	for	Public	
Health	and	Medicine
Evanston,	IL
University
of	Pittsburgh
Clinical	and	Translational	
Science	Institute
Pittsburgh,	PA
University
of	Arizona
College	of	Medicine	
&	Public	Health
Tucson,	AZ
Columbia/Cornell/N
YP/Harlem	Hospital
Institute	for	Genomic	
Medicine
New	York,	NY
University
of	California
San	Diego
Department	of	
Biomedical	Informatics
La	Jolla,	CA
Massachusetts	
General	Hospital
Genetics	&	Genomics	Unit
Boston,	MA
Henry	Ford
Health	System
Public	Health	Sciences
Detroit,	MI
Geisinger
Health	System
Genomic	Medicine	Institute
Danville,	PA
Precision Medicine Initiative
NYC
Cancer Moonshot
State of the Union | January 2016
21st Century Cures Act Signed
December 13th 2016
https://www.whitehouse.gov/CancerMoonshot
Mission
Weill Cornell Medicine
Mission
Care
TeachDiscover
The New style of
tailored treatment
http://meyercancer.weill.cornell.edu/news/2015-01-22/new-style-tailored-treatment
Irene Price | 76y Female
• 2009 | Diagnosed with Bladder Cancer; initially
followed traditional treatment showing signs of
success
• Relapse | High-grade urothelial carcinoma in
bladder and several lymph nodes
• Cystectomy (Bladder removal), sample sent for
Genetic Analysis (WES)
• Underwent Radiation Therapy, Genetic Analysis
Results showed HER2 Amplification (mutation
common in breast cancer)
• Relapse | Signs of metastasis to the liver
• Chemotherapy targeted treatment | 8 cycles with
Taxol and Herceptin
• Disease free since Nov 2014
Medical History
EXaCT-1 Test Workflow
15
Orders Test Receives Report
Patient
Referring Physician
2000 Cores
2PB Storage
Institute for Computational
Biomedicine
Today
0 200 400 600 800 1000 1210 aa
0
38
#Mutations
EGFR
A289D/ I/ N/ T/ V
Rec.. Furin- like Rec.. GF_recep_IV Pkinase_Tyr
*source: ipm-cbio.med.cornell.edu
https://youtu.be/_Fly4U9eFRY
*credits: Lucas Films
Star Wars
Episode V – The Empire Strikes Back (1980)
*credits: Microsoft Research
https://www.microsoft.com/en-us/research/project/holoportation-3/
Video: https://youtu.be/7d59O6cfaM0
Microsoft Hololens
Holoportation
*credits: Case Western Reserve University
Microsoft Hololens
Healthcare
*credits: Case Western Reserve University
Microsoft Hololens
Healthcare
Reality-Virtuality
Continuum
Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum"
Physical
Reality (PR)
Virtual
Reality (VR)
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Mixed Reality (MR)
Gaze VoiceGesture
Ways to Interact
Visualizing
Protein Structures
Visualizing
Networks
Source: http://www.creativeapplications.net/linux/gephi-mac-windows-linux/
Visualizing
Cancer Drug Networks
Data: Kaitlyn Gayvert
Bringing Networks
into Reality
https://youtu.be/Yu3Vxy6lQwg
Tumor Board
Using Hololens across the world
https://youtu.be/O99CMi2ahD0
Demos
Minority Report
Release Date: June 21 2002
John Underkoffler
https://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture
Healthcare
Looking into the future
https://youtu.be/ZhAzqaV-lQk
Caryland Israel Englander
Institute for Precision Medicine
45
ipm.weill.cornell.edu

How HoloLens Transforms Healthcare