IROS 2009 - Minimally Invasive Surgery Maneuver Recognition Based On Surgeon Model

SystemEngineeringand
AutomationDepartment
M. Belén Estebanez Campos
belen@uma.es
IROS 2009 : INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS
MINIMALLY INVASIVE SURGERY MANEUVER
RECOGNITION BASED ON SURGEON MODEL
B. Estebanez, G. Jimenez, V. Muñoz, I. Garcia-Morales, E. Bauzano and J.
Molina
Dpto. Ingeniería de Sistemas y Automática
http://www.isa.uma.es
Universidad de Málaga (Spain)

belen@uma.es
Minimally Invasive Surgery Maneuver Recognition Based On Surgeon Model
OUTLINE
I. Introduction
II. Intervention Model
III.Surgeon Behavior Model
IV.Implantation and Experiments
V. Conclusions and Future Works

belen@uma.es
I. INTRODUCTION

belen@uma.es
I. Introduction
Laparoscopic Surgery:
“Accessing to the abdominal
cavity with especial long
instruments and by means of an
endoscope with camera can be
seen the area of the operation.”
LAPAROSCOPIC SURGERY
Advantages
• Reduce recovery time
• Limit post-operative complications
• Lower scars
Constraints
• Movement limitations
•Techniques require high skill
• Loss of touch and 3D perception
• Hand-eye coordination problems

belen@uma.es
I. Introduction
COMMERCIAL SYSTEMS OF SURGICAL ROBOT
Zeus
Da Vinci

belen@uma.es
I. Introduction
INTERFACE OF THE ROBOTIC ASSISTANT OF TWO ARMS
Robotic Assistant of Two Arms
Surgeon and tools with marks
Computer with
Recognition System
Operation Room
Surgical
Tool
Surgeon
3D Tracker

belen@uma.es
II. INTERVENTION MODEL

belen@uma.es
THE WORKFLOW OF A SURGICAL PROCEDURE
START
Dissecting tissue to expose
cystic duct
Clipping cystic duct
Cutting cystic duct
Dissecting/Separating
Gallbladder
Collecting Gallbladder
END
First Maneuver
Second Maneuver
Third Maneuver
Fourth Maneuver
Fifth Maneuver
Sixth Maneuver
Final Maneuver

belen@uma.es
SET OF BASIC ACTIONS OF A MANEUVER
Four degrees of freedom
Maneuver: set of basic actions
that represent the instrument
movements: left-right, up-down, to
rotate on itself and insertion-
removal.
Others functions are clamp
opening and closing.
Each maneuver is identified as a
specific sequence of these basic
actions.

belen@uma.es
INTERVENTION MODEL
UPPER LAYER INNER LAYER

belen@uma.es
III. SURGEON BEHAVIOR MODEL

belen@uma.es
LIBRARY OF MANEUVER MODELS
III. Surgeon Behavior Model
Off-line process
MODULE OF
MANEUVER
TRAINING
LIBRARY OF
MANEUVER
MODELS
On-line process
MODULE OF
MANEUVER
RECOGNITION
DATA
ACQUISITION
AND CODING
MODULE

belen@uma.es
DATA ACQUISITION AND CODING MODULE
CHARACTERISTIC
EXTRACTION
FUZZY LABELING
Characteristic Vector
of a Movement
Signals of Movements of
surgical tools
Current
Movement
Label
Observable
Output

belen@uma.es
MODULE OF MANEUVER TRAINING
DATA
ACQUISITION
AND CODING
MODULE
MODULE OF
MANEUVER
TRAINING
LIBRARY OF
MANEUVER
MODELS
Action
Type
Position and Orientation
of Tools
Outputs
Sequence
Parameters of a
Maneuver Model
Surgeon

belen@uma.es
MODULE OF MANEUVER RECOGNITION
Surgeon
DATA
ACQUISITION
AND CODING
MODULE
Change Condition
of Maneuver
Position and
Orientation
of Tools
MODULE OF
MANEUVER
RECOGNITION
LIBRARY OF
MANEUVER
MODELS
Outputs
Sequence
INTERVENTION
MODEL
Current
Maneuver
Parameters
of a Maneuver
Model

belen@uma.es
MODULE OF MANEUVER RECOGNITION
Change Condition
of Maneuver
Outputs
Sequence
INTERVENTION
MODEL
Current
Maneuver
Maneuver with
High
Probability
LIBRARY OF MANEUVER
MODELS
Maneuver 1
(A1 , B1 matrixes)
Maneuver 2
(A2 , B2 matrixes)
Maneuver 3
(A3 , B3 matrixes)
Maneuver N
(AN , BN matrixes)
ViITERBI ALGORITHM

belen@uma.es
IV. IMPLEMENTATION AND EXPERIMENTS

belen@uma.es
• Simplification of the reality.
• Recognition of trajectories of tool tip.
• The basic actions are lines with different angles
and directions.
• Library has geometrical figures like
circle, square and triangle.
• A protocol is a sequence of these maneuvers.
1800
1900
200
-250
-200
-150
-100
-50
-300
-200
-100
0
100
200
300
400
xy (mm)
z(mm)
EXPERIMENTAL SYSTEM SETUP
IV Implementation and Experiments
Tool Tip
Fulcrum
Hand

belen@uma.es
MANEUVER MODEL
IV Implementation and Experiments
RECOGNITION RESULTS
End
A 4
A 2A 1
A 3
Start
Table 1. Isolated Maneuvers
Trajectory % Real Correct
Answer (100 essays /
maneuver)
CIRCLE 98.35 %
SQUARE 100.00 %
TRIANGLE 95.87 %
Table 2. Protocols
Trajectory % Ideal Correct
Answer (50
essays /protocol)
% Real Correct
Answer (50
essays / protocol)
90.39 % 85.00 %
91.20 % 82.50 %
94.29 % 81.00 %

belen@uma.es
V. CONCLUSIONS AND FUTURE WORKS

belen@uma.es
CONCLUSIONS
A methodology has been established for modeling the surgical
behavior: acquisition and coding, off-line training and on-line
recognition .
The HMM are useful for surgeon maneuver recognition.
Limitations
The performance of the system can be affected by the shadows
during the instrument tracking.
 Improvement of the maneuver ending condition detection.
The experiments are about fictitious maneuver and are dependent on
the figure orientation.

belen@uma.es
FUTURE WORKS
Acquisition of information inside an
operation room.
Inclusion of other useful information.
Signal reconstruction in instrument
shadows situation.
Design of new real maneuver
models, including the movements of two
instruments.
Integration of this recognition module
in the global system of two arms.

belen@uma.es
THANKS FOR YOUR ATTENTION
IROS 2009 : INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS

IROS 2009 - Minimally Invasive Surgery Maneuver Recognition Based On Surgeon Model

Recommended

Recommended

More Related Content

More from Robótica Médica UMA

More from Robótica Médica UMA (11)

Recently uploaded

Recently uploaded (20)

IROS 2009 - Minimally Invasive Surgery Maneuver Recognition Based On Surgeon Model