Minimally Invasive Surgery Week 
Society of Laparoendoscopic Surgeons Annual Meeting 2017 Future Technology Session San Francisco, CA September 9, 2017

1. “Quantified Surgery:
Personalizing your Surgery”
Minimally Invasive Surgery Week
Society of Laparoendoscopic Surgeons Annual Meeting 2017
Future Technology Session
San Francisco, CA
September 9, 2017
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
http://lsmarr.calit2.net
1

2. Abstract
With high resolution MRI/CAT scans of patients prior to surgery now fairly routine, it should be able to provide the
surgeon with a detailed 3D “transparent patient” to guide pre-surgical planning and perhaps even be used in the OR to
provide “situational awareness” to the surgery team. To test this hypothesis, we carried out this process for a sigmoid
colon resection surgery in the UC San Diego Jacobs Medical Center on November 29, 2016. Both volumetric
visualization and 3D organ segmentation of the patient’s abdomen were created and viewed by the surgeon prior to the
surgery. The da Vinci XI surgical robot enabled the surgical team to view both the 3D and the internal camera on large
screens in the OR. Finally, time series of biomarkers for both blood, stool, and microbiome were obtained before and
after the surgery, providing a view into the body’s reaction to the surgery.

3. Quantifying the Need for Surgery

4. I Have Been Tracking My Internal Biomarkers For A Decade
To Understand My Body’s Dynamics
My Quarterly
Blood Draw
Calit2 64 Megapixel VROOM
Calit2 64 Megapixel VROOM

5. Only One of My Blood Measurements
Was Far Out of Range--Indicating Chronic Inflammation
Normal Range <1 mg/L
27x Upper Limit
Complex Reactive Protein (CRP) is a Blood Biomarker
for Detecting Presence of Inflammation
Episodic Peaks in Inflammation
Followed by Spontaneous Drops

6. Adding Stool Tests Revealed
Oscillatory Behavior in an Immune Variable Which is Antibacterial
Normal Range
<7.3 µg/mL
124x Upper Limit for Healthy
Lactoferrin is a Protein Shed from Neutrophils -
An Antibacterial that Sequesters Iron
Typical
Lactoferrin Value for
Active Inflammatory
Bowel Disease
(IBD)

7. Descending Colon
Sigmoid Colon
Threading Iliac Arteries
Major Kink
Confirming the IBD (Colonic Crohn’s) Hypothesis:
Finding the “Smoking Gun” with MRI Imaging
I Obtained the MRI DICOM Slices
From UCSD Medical Services
and Converted to Interactive 3D
Working With Calit2 Staff
Transverse Colon
Liver
Small Intestine
Diseased Sigmoid Colon
Cross Section
MRI Jan 2012
Severe Colon
Wall Swelling

8. MRI 2d DICOM Slices Converted to 3d Volumetric Imaging
and Sigmoid Colon 3d Printed
Descending Colon
Possible Fistula
3d software: Jurgen Schulze, Calit2, UCSD
3d printing: Philip Weber, Calit2, UCSD

9. 3D Volumetric Visualization From MRI
In Calit2 Virtual Reality StarCAVE
3D Volumetric
Visualization
Created by
Calit2’s Jurgen
Schulze
from January
2012 MRI

10. Full Body CAT Scan at mm Resolution, Including Virtual Colonoscopy
June 2016 Convinced Me Time Had Come for Surgery
No Air
Source: June 2016
Dr. Harvey Eisenberg,
Body Scan Intl., Irvine, CA

11. Slice Through Sigmoid Colon
Reveals Thick Walls and Very Narrow Lumen
4.2mm
13.3mm
16.4mm
Source: Smarr MRI Oct. 25, 2016
Reading by Cynthia Santillan,
MD Radiologist, UCSD
Normal opening (bright yellow) is 40mm,
so mine is 1/10 the diameter it should be.
Normal wall thickness (dark yellow)
is 3mm, so wall thickness is 4-5 times
as thick as it should be.

12. Pre-Surgical Planning

13. From Quantified Self to Quantified Surgery:
Converting MRI Slices to 3D Organ Segmentation for Surgical Pre-Planning
MRI Slice from Dr. Cynthia Santillan 3D Organ Segmentation Made by Dr. Jurgen Schulze
from Dr. Santillan’s 150-Slice MRI
Images of Dr. Smarr’s Abdomen
To Support Sigmoid Colon Resection Surgery

14. Pre-Surgical Planning in QI Virtual Reality:
Using Virtual Reality As Input for Positioning The Two Resection Cuts
Colon visualization by Jurgen Schulze, Calit2;
Photo credit Tom DeFanti, Calit2
Surgeon Sonia Ramamoorthy, MD
in Calit2 Virtual Reality CAVE
Friday November 25, 2016

15. Diseased Sigmoid Colon
is Sitting on Top of Bladder
Bladder
Sigmoid colon
Region to Investigate
Possible Fistula
Between
Sigmoid colon
and Bladder
Volumetric Visualization by Jurgen Schulze, Calit2

16. Key Outcomes
of 3D Presurgical Planning
• Three Specific Problem Areas Discovered:
– Sigmoid Has Kinks at Both Ends
– Sigmoid Colon is Sitting on Bladder
– Splenic Flexure Colon Has Folds and is Attached to Spleen
• Led to Decisions About Surgical Procedures:
– Location of Ports Were Changed on Body
– Sigmoid Colon Resection End Points Were Determined
– Extra Time Will Be Needed Near Spleen

17. Pre-Surgery Mental/Emotional Preparation
Over Thanksgiving 2016

18. New Technology Supporting Surgery

19. The Vision for Integrating Individualized
Radiological Imaging and Robotic Surgery is 30 Years Old
1988 Slide from
Rick Satava, MD

20. QI Collaboration with Radiology, GI, and Surgery
to Support Sigmoid Colon Resection
UC San Diego Health Communications Planning Press Release Early March

21. Using QI Organ Segmentation in Jacobs OR
on Tuesday November 29, 2016
Patient Smarr
With da Vinci Robot
Arms Inside Him
OR Team Using Large Screens
To Watch Dr. Schulze’s da Vinci Images
Dr. Ramamoorthy Operating
Da Vinci Xi Robot During Surgery
Dr. Schulze Rotating 3D Organs To Match Up
With da Vinci Arms and Internal Camera

22. Dr. Ramamoorthy Using Interactive 3D Anatomy in UCSD Jacobs OR
For Resection of Sigmoid Colon, Which is Resting on Top of Bladder

23. Dr. Ramamoorthy’s Use of 3D Interactive Anatomy
To Guide Surgical Team on Site For Bowel Division

24. Dr. William Sandborn Using Colonoscope in OR
To Examine Stapled Anastomosis
Dr. William Sandborn Performing Colonoscopy
While da Vinci Observes from Inside Abdomen
Dr. Santi Horgan, Director of Minimal Invasive Surgery,
Pointing to Stapled Anastomosis

25. Dr. Kurisu Using Oculus Rift Virtual Reality
to Observe 360 Degree Time Lapse of Dr. Smarr’s Surgery
Jurgen Schulze Showing Dr. Mike Kurisu
How to Use
an Oculus Rift Virtual Reality Headset
Jurgen Schulze Used 360 Degree Camera
To Record Time Lapse Movie of Surgery

26. Quantifying Surgery Recovery

27. Colon Before and After Surgery

28. hs-CRP Provided a Biomarker
of Reduced Inflammation Following Surgery
Surgery
Normal Range <1 mg/L

29. EGG Array in UCSD Professor Todd Coleman’s Lab
Experiment with PhD Student Armen Gharibans
2 Days After Surgery 2 Weeks After Surgery1 Week Before Surgery

30. GI Activity 24 to 72 Hours After Surgery
GI Hyper-Activity Passed Gas
1st Bowel Movement
2nd Night of Sleep 3rd Night of Sleep

31. Quantified Recovery (Steps Walked Per Day) -
Recovered to Pre-Surgery Level in Two Weeks
10,000
Steps
Surgery
LeftJMC
5 Miles
Per Day
Dec 14
Nov 29

32. Sonia Ramamoorthy, MD Explains
How These Technology Injections Changed Her Surgical Practice
http://ucsdnews.ucsd.edu/feature/visualizing_the_future_of_surgery
Video Created by UC San Diego Health Marketing and Communications