1.
Alexandros Sigaras | @AlexSigaras
Research Associate in Computational Biomedicine
Weill Cornell Medicine
Microsoft Reactor
Grand Central Tech
1.11.2017
How HoloLens
Transforms Healthcare
NYC HoloLens Developers Meetup

2.
Source: EMC & IDC
“The Digital Universe. Driving Data Growth in Healthcare”
https://www.emc.com/analyst-report/digital-universe-healthcare-vertical-report-ar.pdf
Healthcare & Big Data
Today

3.
Healthcare & Big Data
2025 Predictions
Astronomy Twitter Youtube Genomics
Acquisition 25
zetta-bytes/year
0.5–15
billion
tweets/year
500–900
million
hours/year
1
zetta-
bases/year
Storage 1
EB/year
1–17
PB/year
1–2
EB/year
2–40
EB/year
Source: NIH
“Big Data: Astronomical or Genomical Zachary Stephens et al.”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4494865/

4.
Source: NIH
https://www.genome.gov/sequencingcosts/
Cost per Genome

5.
“My hope is that this
becomes the foundation,
the architecture,
whereby in 10 years from
now we can look back
and say that we have
revolutionized
medicine.”

6.
Precision Medicine Initiative
Budget
$130M
F Y 1 6 E N A C T E D F Y 1 7 P R E S I D E N T ’ S
R E Q U E S T
$230M

7.
Precision Medicine Initiative
Regional Medical Centers across USA
• Able to enroll diverse
patient populations
• Strong electronic
health record
capacity
• Geographic spread
• Capacity to enroll
35,000 a year
Northwestern
University
Institute for Public
Health and Medicine
Evanston, IL
University
of Pittsburgh
Clinical and Translational
Science Institute
Pittsburgh, PA
University
of Arizona
College of Medicine
& Public Health
Tucson, AZ
Columbia/Cornell/N
YP/Harlem Hospital
Institute for Genomic
Medicine
New York, NY
University
of California
San Diego
Department of
Biomedical Informatics
La Jolla, CA
Massachusetts
General Hospital
Genetics & Genomics Unit
Boston, MA
Henry Ford
Health System
Public Health Sciences
Detroit, MI
Geisinger
Health System
Genomic Medicine Institute
Danville, PA

8.
Precision Medicine Initiative
NYC

9.
Cancer Moonshot
State of the Union | January 2016

10.
21st Century Cures Act Signed
December 13th 2016
https://www.whitehouse.gov/CancerMoonshot

11.
Mission

12.
Weill Cornell Medicine
Mission
Care
TeachDiscover

13.
The New style of
tailored treatment
http://meyercancer.weill.cornell.edu/news/2015-01-22/new-style-tailored-treatment
Irene Price | 76y Female
• 2009 | Diagnosed with Bladder Cancer; initially
followed traditional treatment showing signs of
success
• Relapse | High-grade urothelial carcinoma in
bladder and several lymph nodes
• Cystectomy (Bladder removal), sample sent for
Genetic Analysis (WES)
• Underwent Radiation Therapy, Genetic Analysis
Results showed HER2 Amplification (mutation
common in breast cancer)
• Relapse | Signs of metastasis to the liver
• Chemotherapy targeted treatment | 8 cycles with
Taxol and Herceptin
• Disease free since Nov 2014
Medical History

14.
EXaCT-1 Test Workflow
15
Orders Test Receives Report
Patient
Referring Physician

15.
2000 Cores
2PB Storage
Institute for Computational
Biomedicine
Today

16.
0 200 400 600 800 1000 1210 aa
0
38
#Mutations
EGFR
A289D/ I/ N/ T/ V
Rec.. Furin- like Rec.. GF_recep_IV Pkinase_Tyr
*source: ipm-cbio.med.cornell.edu

17.
https://youtu.be/_Fly4U9eFRY

18.
*credits: Lucas Films
Star Wars
Episode V – The Empire Strikes Back (1980)

19.
*credits: Microsoft Research
https://www.microsoft.com/en-us/research/project/holoportation-3/
Video: https://youtu.be/7d59O6cfaM0
Microsoft Hololens
Holoportation

20.
*credits: Case Western Reserve University
Microsoft Hololens
Healthcare

21.
*credits: Case Western Reserve University
Microsoft Hololens
Healthcare

22.
Reality-Virtuality
Continuum
Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum"
Physical
Reality (PR)
Virtual
Reality (VR)
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Mixed Reality (MR)

23.
Gaze VoiceGesture
Ways to Interact

24.
Visualizing
Protein Structures

25.
Visualizing
Networks
Source: http://www.creativeapplications.net/linux/gephi-mac-windows-linux/

26.
Visualizing
Cancer Drug Networks
Data: Kaitlyn Gayvert

27.
Bringing Networks
into Reality

28.
https://youtu.be/Yu3Vxy6lQwg

29.
Tumor Board
Using Hololens across the world

30.
https://youtu.be/O99CMi2ahD0

31.
Demos

32.
Minority Report
Release Date: June 21 2002

33.
John Underkoffler
https://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture

34.
Healthcare
Looking into the future

35.
https://youtu.be/ZhAzqaV-lQk

36.
Caryland Israel Englander
Institute for Precision Medicine
45
ipm.weill.cornell.edu