3D Slicer is an open-source platform for medical image analysis and visualization developed by Isomics, with collaborations from various academic and commercial institutions. It facilitates multi-modality visualization and supports image-guided therapy through integration with zSpace technology, enhancing surgical navigation and planning. The platform is still in development and not FDA-approved, with ongoing research to assess its clinical applications.

1. NA-MIC
National Alliance for Medical Image Computing
http://na-mic.org
3D Slicer and zSpace:
Medical Visualization and Clinical Applications
Steve Pieper, PhD
3D Slicer Chief Architect
Jayender Jagadeesan, PhD
Harvard Medical School
Isomics is a small technology development company working closely with academic and commercial institutions in
medical imaging research and software development.
Monday, October 21, 13

2. Goal and Outline
• Introduction to Our
Software and
Research (Steve)
• Example Research
in Image Guided
Therapy (Jay)
• Integration of
zSpace (Steve)
Images: A. Golby, F. Talos, P. Black
© 2013, All Rights Reserved
Monday, October 21, 13

3. Domestic Network of Peers
•
Leadership:
–
BWH: Ron Kikinis, (Overall PI)
•
Algorithms
–
Utah: Ross Whitaker (Core 1 PI)
–
–
–
–
•
MIT: Eric Grimson
UNC: Guido Gerig
MGH: Bruce Fischl, Dave Kennedy
GaTech: Allen Tannenbaum
•
•
•
•
Engineering
–
–
–
–
–
Kitware: Will Schroeder (Core 2 PI)
GE: Jim Miller
Isomics: Steve Pieper
UCSD: Mark Ellisman
UCLA: Art Toga
•
DBP 2004
–
BWH: Martha Shenton
–
Dartmouth: Andy Saykin
–
UCI: Steve Potkin
–
UofT: Jim Kennedy
•
•
DBP 2011
–
MGH: G. Sharp
–
Iowa: H. Johnson
–
Utah: R. MacLeod
–
UCLA: J. Van Horn
Service
–
Kitware: Stephen Aylward
Training
–
BWH: Sonia Pujol
Core 6 Dissemination
–
Isomics: Steve Pieper
–
BWH: Tina Kapur
Management
–
BWH: Sanjay Manandhar
DBP 2007
–
UNC: H. Cody
–
BWH: M. Kubicki
–
Mind Institute: J. Bockolt
–
Queens University: G. Fichtinger
© 2013, All Rights Reserved
Monday, October 21, 13
Provided by Pieper, Kikinis

4. International Network of
Peers
© 2013, All Rights Reserved
Monday, October 21, 13

5. What is 3D Slicer?
• A platform for image analysis and
visualization
• Current Release 4.3
• A freely-downloadable program
– Source code and executables available
for Windows, Linux, and Mac OS X
– All Code Reusable
• Slicer is a research platform:
– NOT an FDA approved medical device
– NOT finished – some parts will work
better than others
© 2013, All Rights Reserved
Monday, October 21, 13

6. What: FDA or Research
© 2013, All Rights Reserved
Monday, October 21, 13
Slide by Andras Lasso

7. What: FDA or Research
© 2013, All Rights Reserved
Monday, October 21, 13
Slide by Andras Lasso

8. Multi-Modality Visualization
•
Integrated Display of What is
Known About the Subject
– Anatomical Space as Common
Coordinate System
– Segmented Anatomy and
Volume Rendering for Context
– Statistics Volumes
– Interactive Visualization (View,
Visibility, Cropping, Slicing… )
– Therapy Plans, Measurements,
Annotations
– Analysis Results
– Linked Views
© 2013, All Rights Reserved
Monday, October 21, 13
Plesniak, Aucoin et al - BWH
Jakab and Berenyi - University of Debrecen

9. Registration
Preprocedure MR
•
Intraprocedure CT
Intra-subject
– Pre-Intra-Post Procedure
– Longitudinal Tracking of Disease Progression
•
Inter-subject
– Support Group Comparison (fMRI)
– Map Anatomical Atlas to Individual
•
Degrees of Freedom (DOFs)
– Rigid (Rotation + Translation)
– Similarity (Rigid + Uniform Scale)
– Affine (Rigid + Nonuniform Scale and Shear)
– Polyaffine (Locally Affine Interpolation)
– B-Spline (Cubic Displacement)
– Vector Field
© 2013, All Rights Reserved
Monday, October 21, 13
Sota - BWH

10. Navigation & Guidance
• OpenIGTLink Standard
– Tracked Instruments
– Real-Time Images
– Scanner and Robot
Control
• BWH AMIGO
– 3T MR + PET CT in OR
22
© 2013, All Rights Reserved
Monday, October 21, 13

11. Provided by Archip, Warfield
Intra-procedural Mapping
Alignment of all pre-operative
datasets to the intra-operative
images achieved during the
neurosurgery.
Archip N, Clatz O, Whalen S, Kacher
D, Fedorov A, Kot A, Chrisochoides N,
Jolesz F, Golby A, Black PM, Warfield
SK. Non-rigid alignment of preoperative MRI, fMRI, and DT-MRI with
intra-operative MRI for enhanced
visualization and navigation in imageguided neurosurgery. Neuroimage.
2007 Apr 1;35(2):609-24
© 2013, All Rights Reserved
Monday, October 21, 13

12. Extension Catalog
© 2013, All Rights Reserved
Monday, October 21, 13

13. 3D Slicer / AMIGO
Parathyroid
Adenoma Surgery
Cue Jay...
© 2013, All Rights Reserved
Monday, October 21, 13

14. 3D Slicer and
zSpace
© 2013, All Rights Reserved
Monday, October 21, 13

15. AMIGO CaseHub Display/
Control Options
Imaging
Modalities
MR, PET/CT, Fluroro,
US
PACS
Workstation for Technical Lead
Mouse/Keyboard/Monitor
SPL Compute
Resources
CaseHub Server
Wall Mounted Monitors
Display Only
Point of Care Interface
Touch/Touchless Interaction
Medical Records,
Real-Time Monitors,
Video
Vendor Workstations:
Navigation, Planning
(Brainlab, Siemens)
© 2013, All Rights Reserved
Monday, October 21, 13
• Use-Case Needs Assessment
• Clinician Control of Imaging
• Collaborative Data Management

16. AMIGO CaseHub Display/
Control Options
Imaging
Modalities
MR, PET/CT, Fluroro,
US
PACS
Workstation for Technical Lead
Mouse/Keyboard/Monitor
SPL Compute
Resources
CaseHub Server
Wall Mounted Monitors
Display Only
Point of Care Interface
Touch/Touchless Interaction
Medical Records,
Real-Time Monitors,
Video
Vendor Workstations:
Navigation, Planning
(Brainlab, Siemens)
© 2013, All Rights Reserved
Monday, October 21, 13
• Use-Case Needs Assessment
• Clinician Control of Imaging
• Collaborative Data Management
zSpace?

17. zSpace Slicer Prototype
Module
• Developed with Scott Dolim of zSpace
– Uses zSpace Python API
– Written as Slicer Scripted Module
– Integrates with Qt Event Loop
– Controls VTK Camera
• Not Yet Publicly Available
– zSpace to Create Slicer Extension
(Planned)
© 2013, All Rights Reserved
Monday, October 21, 13

18. zSpace - Slicer Status
• Already:
– Any Slicer Data in Displayable
– Easy to Use
– Ready for TLX-Style Human
Factors Testing
• Not Yet:
– No Stylus Support
– No GPU Volume Rendering
(Quad-Buffer Issue)
– Works with Python 2.6, not 2.7
© 2013, All Rights Reserved
Monday, October 21, 13

19. zSpace in AMIGO Issues
• No Table Space
– Would Need Boom Mounted Version
• Surgeon May Not Be Able To Move
– May Minimize Value of Head Tracking
• Glasses May Not Be Compatible
with Other Eyewear
• Polarized Glasses May Disrupt
Surgeon’s Normal Vision
• Only One Person Sees Correct
View
– Rest of Surgical Team Sees Garbage
© 2013, All Rights Reserved
Monday, October 21, 13

20. zSpace/Slicer Clinical
Research
• Utility Will Be Highly ProcedureDependent
• Screen Possible Target Areas
– Interventional
– Needle Placement
– Robot Control
– In-Scanner Procedures
© 2013, All Rights Reserved
Monday, October 21, 13

21. Clinical Research Strategy
• Phase I
– Bench Testing
– Pre-Procedure Planning/Review
• Phase II
– Place in AMIGO Control Room
– Intra-Procedural Imaging Review
• Phase III
– Navigation
– Patient Trials
© 2013, All Rights Reserved
Monday, October 21, 13

22. Summary
• 3D Slicer is Free and Open
Research Platform for Medical Image
Computing
• AMIGO Environment Ideal for Novel
Hardware/Software/Clinical Projects
• zSpace Integration Well Underway
and Promising
© 2013, All Rights Reserved
Monday, October 21, 13