This document proposes using augmented reality to improve safety in laparoscopic surgeries by defining a safe zone for surgical tools to avoid critical structures. It will automatically extract surgical actions from video, train a network to predict the safe zone and flag unsafe actions based on tool proximity to critical areas and surgical workflow phase. The project aims to develop this for real-time use by capturing surgery videos, selecting areas of interest, training a network on tool distances and classifying risky actions, developing visualization and notifications, and implementing it during actual surgeries while getting user feedback.

1. Augmented Reality in Medical Practice: Surgical
Distortion Detection In Real Time
Presented by
Tushar Dinesh Mahore (21MM61R16)
1
Supervised by
Dr. Subhamoy Mandal

2. CONTENTS
1. Background and Motivation
2. Outline of the Project
3. Work Plan
4. References
2

3. Background and Motivation
AR can improve the safety of the patient and surgical outcomes for laparoscopic surgeries by defining a
safe zone for using surgical tools and avoiding critical structures.
The safe zone can be defined by utilizing the spatial and temporal registration between a patient, the
anatomical model tracked surgical instruments and the laparoscopic view of the patient.
We can automatically extract the semantic correlations of surgical actions and their impact, and train a
network for predicting the safe zone and flag unsafe actions.
i) Delineation of Safe Zone
ii) Is the tool at safe distance from critical structures?
iii) Which surgical workflow phase is being executed?
iv) Which tools perform what action through analysis of the pose data.
v) Is the action safe?
3

4. Outline of the Project
AIM-To find the Area of Interest during Surgery and give the alarm in needful distorted conditions.
Glimpse-
 Capture the fine tuned Video during the Surgery.
 Select the Area of Interest and extract surgical actions
 Train network to identify distance of tool to classify critical actions and potentially hazardous actions .
 Developing a visualization and notification method for Predictive analysis of surgical actions.
 Try to implement in Real Time Surgery.
4

5. Work plan
MONTH WORK
1 Literature survey and Concept Learning
2 Preprocessing of exiting videos and extracting the surgical actions and
classifying severity
3-4 Training network to identify proximity of tool to critical actions and
potentially hazardous actions
5 Developing a visualization and notification method
6-7 Try to Implement it with real-time tracking information
8 Collecting user feedback and improvement
9-10 Thesis writing and reporting
5

6. References
6
1 Bernhardt, Sylvain, Stéphane A. Nicolau, Luc Soler, and Christophe Doignon. "The status of
augmented reality in laparoscopic surgery as of 2016." Medical image analysis 37 (2017): 66-90.
2 hessian.AI health.Care #7: Towards Understanding Surgical Scenes Using Computer Vision

7. THANK YOU
7