Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MiroSurge: Research Platform for Robotic Surgery

5,278 views

Published on

Presented by: Sergey Tarassenko, Research Assistant, Institute of Robotics and Mechatronics, DLR

Minimally invasive surgery (MIS) challenges the surgeon's skills due to his separation from the operation area which can be reached by long laparoscopic instruments only. To overcome the drawbacks of conventional MIS the DLR works on a research system for telerobotic endoscopic surgery. The DLR telesurgery scenario MiroSurge system includes a master console with a 3D-display and two haptic devices to control three MIRO robot arms. RTI Connext DDS provides the communications infrastructure between robots, controllers and user interfaces, facilitating synchronization and coordination between them.

Published in: Technology
  • Water Hack Burns 2lb of Fat Overnight (video tutorial) ✱✱✱ https://t.cn/A6wnCpvk
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ❤❤❤ http://bit.ly/39pMlLF ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❶❶❶ http://bit.ly/39pMlLF ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

MiroSurge: Research Platform for Robotic Surgery

  1. 1. Deutsches Zentrum DLR fiir Luft- und Raumfahrt German Aerospace Center . . LONDON CONNEX_T 2015 Mirosurgez Research Platform for Robotic Surgery Sergey Ta rassenko Institute for Robotics and Mechatronics German Aerospace Center
  2. 2. Outline ° German Aerospace Center I J - From Space to I/ Iedical robotics ~ I. /Iotivation - I/ IIroSurge System ° Con1ponents * Sofumanearchuecture ° Appficafions
  3. 3. DLR, German Aerospace Center ° 8000 employees We. - 32 institutes and facilities W. _m ° 16 locations in Germany Wm_ M"- - Key research topics Neustielitz I I Gottlngen ° Aeronautics MI ‘W ° Space ”°“" ' ° Energy ° Transport ° Defence smear- ; S e C U r Augsburg I I Oberpfaffenhofen y Weilheim I
  4. 4. DLR Oberpfaffenhofen ‘ / , . '5- 4 Institute of Robotics and Mechatronics (RM) Muenchner Str. 20, 82234 Wessling
  5. 5. Institute of Robotics and Mechatronics { / ’ Mobile Robots and DEOS 2; ; Eirt t“”"3’t‘”: t“: "
  6. 6. Institute of Robotics and Mechatronics . { / ’ Rollin‘ Justin and HaSy 2; ; I E S i53.; é2I’iF
  7. 7. Institute of Robotics and Mechatronics . { / ’ / Spacelustin E; S ; / J I 1/ fr? ?? / F‘. 'ii~t I 1/ . ‘*" 1. “"17 / ;; ”)»3x : ,4”, E5 / ' (. L§, '_i - . ‘ . , ‘ Fizyyglig ‘ * E i’ ‘.4 * ‘-‘E , - ‘ " "‘ i S l «('‘-; t_« l I1 If . ‘ ’ / /2 I ~». '—~~ ti’ ’, /if I’ in ‘,4 Telepresence for hazardous or unreachable environments
  8. 8. Institute of Robotics and Mechatronics { / ” Structure 2; ; I J E‘ ‘r/ Department Department Department Department Mechatronic Analysis and Perception and Autonomy and Components and Control of Cognition Teleoperation Systems Advanced Robotic Systems Mechatronics Communication Vision I Simulation Application if Design l'll= i°fil? ~?? l *'('l': 'i*lila+: l Control Telepresence Safey l Au mentation 7;"%. ,:E’iF
  9. 9. Outline ° German Aerospace Center t 5,) ~21’ - From Space to Medical robotics t I/ Iotivation ~ I/ IiroSurge System ° Con1ponents t Sofnmanearchnecture t Appficafions
  10. 10. From Space to Medical Robotics Motivation Central Mission of the Institute: Space Robotics Terrestrial Domains: T h I K h e. g. Medical Robotics ec no ogy’ now ow Technology, Knowhow Sidetrack: Medical Robotics ° No medical robotics department ° Medical robotics at DLR is a research group of about 20 people ° Research performed by PhD students and post—docs
  11. 11. From Space to Medical Robotics An a logy Medical Robotics Space Robotics Body wall T€l€f0b0tiCS Distance Risk of comsions with Low Inertia, Risk of collisions patient or staff Sensors with other devices Lightweight Design High costs and limited payload Variable setups Each patient is Unknown different environment Medical device & Safety, |nacce55ib| e direct HRI Robustness fie| d 11 of58 # I I
  12. 12. From Space to Medical Robotics / Surgery in space perspective / x ° Medical Evacuation not ______________________________ A Possible on O interplanetary missions ------ -f """""""""" . . °°"‘“4‘:2°: .,. ,,; .,. . V “X ‘ . fl . ‘x . , / 1 ° , ,The focus of an A n_mmg~. ' emergency medical wry” ' system becomes tt"~t- ‘- autonomy, intelligence, " I ”” . __ "'-. ,_ and reliability” : :5,, ::. _ mg‘; Nomi [Haidegger and Benyo: ,, Extreme Telesurgery", 2008] ° Concept: support astronaut with robotic technologies NASA: Guidelines and Capabilities for Designing Human Missions, 2003
  13. 13. Outline * German Aerospace Center t , l - From Space to I/ Iedical robotics ° Motivation - I/ IiroSurge System s Con1ponents * Sofumanearchnecture t Appficafions
  14. 14. Minimally Invasive Surgery + Lower patient traumatization + Faster recovery period + Cosmetic advantages — Loss of hand—eye coordination — No direct access to the OP field — Loss of degrees of freedom — Limited feedback — Unergonomic pose —> Intensive training necessary —> Complex tasks are time—consuming K 2‘ [Courtesy of Richard Wolf © 2015]” tel as i“"’*”'e7t‘ '”a'»"_, :
  15. 15. Minimally Invasive Robotic Surgery + Hand-eye coordination Full intracorporeal dexterity Master-slave decoupling + Motion Scaling + Tremor filtering Semiautomatic functions Comfortable surgeon position Increased operation duration Complete loss of haptic feedback 0 '1 5;: . T ‘ [Courtesy of Intuitive Surgical, Inc. 2015] I ii, ‘i I 5 " ’ I ll‘? 3 1 it I ‘E. s, i , , 1‘. IE? / V l 5 ale. L‘ ‘ J A V/ ' I ‘ ‘Ag.
  16. 16. Intuitive Surgical J / ’ da Vinci System 2; ; Q ‘/ V__/ S ta _ ffi . . incl ‘ E Nu 'l, . 6 I i V I T IIVV 1 W. ’ 71.‘, J/ K‘ . A . ’ r 7 .1 . _ ‘ -G R l 1 13.. . ~ t i ‘ - ,4 ‘«‘. : 't‘_ » __, .- _ W s ma . , , "i I ; ' h. /1‘ , _ V lip p/ /A , p . /E, f H’ T I VT": -,7- P ‘ft ,1: T. C’ ti [Courtesy of Intuitive Surgical, Inc. © 2015]
  17. 17. Outline * Gemnan Aerospace Center ~ , ) ~’ ° Fi'om Space to Medical robotics ° i/ iotivation - Mirosurge System - Components * Sofhmanearcnnecture ° Applications
  18. 18. Mirosurge { / ” ’ . / OVerVleW : / VJ < . _ Research platform for MIS and another applications ) , ‘V / V Three MIRO robotic arms Two MICA robotic instruments Endoscope Surgeon console with two force feedback input devices and 3D Monitor (Q ’ i‘ I . 14331
  19. 19. MIRO Features At ° Lightweight robot ° Payload: 3k? ° Weight: 9,8 kg ° Redundant, anthropomorphic kinematics ° 7DoF ° Optimized for medical procedures ° Integrated electronics . ° Standardized instrument interface AA, ° Torque sensors ° Spacewire network ° Control cycle U. Hagn: The Aspect of Versatility in the Design of a Lightweight Robot for Surgical Applications, doctor thesis, University of Hanover, Z011.
  20. 20. MIRO Features Safe interaction - Reduced mass ° Compliant Adaptable and flexible - Magnetic instrument interface ° Kinematic redundancy Different control modes - Accurate and stiff Cartesian position control ° Compliant Cartesian impedance control ° , ,Hands on” mode
  21. 21. MIRO Variable Setups @
  22. 22. MICA Features ° Weight: 0,9 kg , ' Manipulation forces: up to 10 N " ° 3 DoF Cable Driven ° Integrated electronics ° Control frequency: 3kHz ' Shaft length: 300 mm ° Shaft diameter: 10 mm ° Various end—effectors: Needle Holder, Maryland, Scissors ° Range of motion: i 80° ‘ ii ° 6+1 force/ torque sensor task specific tool dTlVe Unit S. Thielmann, et aI. : MICA —A new generation of versatile instruments in robotic surgery. In Proc. IEEE IROS, 2010. 2 2 of U. Seibold et al. : MICA — Prototype of lnstrumentfor Minimally Invasive Surgery with 6—axis Force Sensing I Capability. In Proc. IEEE ICRA, 2005
  23. 23. x) V’ I MIRO and MICA , i / _ ‘ V 6 DoF and gripper 2’ inside the patient V "Miro A Nullspace motion 7oorMiRo A . .. /V pT"’c"’ 2 DoF Trocar ]/ L2 DoF MICA (+ gripper) 6 D F TCP xTcp O r 1 DoF Nul| space‘/ /, / V ’ ? f /
  24. 24. Surgeon Console Overview ° Open console concept ° Sigma.7 haptic devices ° Adjustable monitor arm - Mobile base - Dimensions to pass standard doorways ° Compartment for computing units
  25. 25. Surgeon Console { / ’ / Sigma.7 : / 71 < . _ Transparent haptic device for I . force and torque feedback Rotational workspace is decoupled from translational workspace Gravity compensated Custom design with force/ torque sensor 4 kHz control rate Integrated force/ torque sensor I W :9
  26. 26. Mirosurge Overview °= .12?‘ "‘ ; ‘its: air 1*‘ Jim
  27. 27. Outline ° Gerrnan Aerospace Center ~ , l J ‘i Fiom Space to Medical robotics ° Motivation - MiroSurge System * Conwponents ° Software architecture ° Applications
  28. 28. Outline - Mirosurge System I iii :39 0 Conwponents ° Software architecture ° System Overview * Cfommunicatlon e dds_, .aemon ° DDS Library usage 8 Applications
  29. 29. System Overview { / / OS Environment . ; i; s / —4 QNX 6.5 Hi. ) ’ Rea| —time critical code ’ C++ MATLAB Simulink models with Real Time Workshop SUSE Linux Enterprise Desktop 11 Development platform User interface High level workflow control C++ Python MATLAB Simulink models in interpreted mode
  30. 30. System Overview { / ’ i / Typical system setup cg ; Hardware DJ 3 MIRO Robots ii 2 MICA Robotic instruments 2 Haptic input devices Software 45 software modules 10+ workstations Network 2 separate Gigabit LANs _ 4’ Communication A 170 topics I y Message size between 10 and 1000 bytes , T ' Q , fig" * Rates up to 4 kHz l QQ-(gfii-i_ . ; _ 1 . '3‘ 3:, ;'
  31. 31. System Overview . { / ’ ’ / Software layers 4 * V I V / ' ——' User Interface Viewer Surgeon Ul l Engineer Ul / ll _ Workflow MiroSetup I [MiroWorkf| ow] Control MiroContro| HAL Hardware
  32. 32. Outline - Mirosurge System I , ) 0 Conwponents ° Software architecture *> System Overview ° Communication ° dds_, .aemon ° DDS Library usage 8 Applications
  33. 33. 0 Communication ’ History — aRD @ ° Input or output port , ../ ° UDP or shared memory / ° Direct port connection or point-to-point bridge between ports as separate process - Shared memory interface for simulink + Veryfast - inflexible - Difficult to maintain all connections and check state of the system - Unstructured data, usually array of doubles B. Baeuml, et aI. : Agile Robot Development (aRD): A Pragmatic Approach to Robotic Software. In Proc. IEEE IROS, 2006. U I
  34. 34. Communication . { / ’ ’ Spacewire I USB Ethernet UDP, TCP/ lP RS232 HAL I Mir AL I [ Sig HAL ] [ Mic AL ] 3 iii 3 Hardware MA
  35. 35. Communication HAL 9 Control @m - Low latency _ l ,2 ° Always 1:1 communication ° Same host = > shared memory control Sismaantral I fiitfifil 6.? E? 6? HAL MiroHAL SigmaHAL MicaHAL
  36. 36. Communication . { / ’ ’ / Shared memory interface cg ; POSIX API ) T QNX, Linux ' Single writer, multiple readers Double buffered Lock—free write Blocking or non-blocking read Shared mutex and condition Common Data Representation or direct memcpy Contro| Miro ntrol R? ‘ lsigmaéantrol Micagntrol ] HAL '3 MiroHAL V [ SigmaHAL ] [ MicaHAL I “I N__; ,. . J, L, ‘ 31.32 an —a *3" *7‘ TFZ~1’. .‘. . IJl. ;:t
  37. 37. Communication Control (A Workflow (-9 UI @ Different types of data I g ,2 ° Streams of real—time data (best—effort) ° Cyclical events (best—effort) ° Events, commands (reliable) User Interface Surgeon Ul Engineer UI M irosetu p M iroworkflow Control Mirocontrol
  38. 38. Communication { / ’ / Requirements Real-time capability ) ‘Q ' Low latency and high throughput Scalability Encourage modular approach a Dynamic configuration of the system Certification for safety-critical systems Simulink and python support : > RTI Connext DDS ‘iij”'i“£. ‘:‘i
  39. 39. Communication { / ’ History — DDS / ‘ 1 / gr L Input or output port a Shared memory One-to-many bridge process via DDS between ports Less bridge processes Better system overview Difficult to maintain all connections and check state of the system Unstructured data (DDS_Octets) High load during discovery A 4 « ONDON CONNEXWOI r » Y
  40. 40. 0 Communication ’ Solution j ° Some real time critical connections are L L, static and on the same host " ° Dynamic discovery causes additional network and CPU| oad ° Local communication via shared memory ° Remote communication via DDS through additional process (dds_daemon) ° Same interface via shared memory ° Solution for QNX network stack limitation
  41. 41. Outline - Mirosurge System I , ) d Conaponents ° Software architecture a System Overview d Communic: atloi‘1 ° dds_daemon a DDS Library usage a Applications
  42. 42. dds_daemon Local approach MiroContro| shm: miro_info I shm: miro_command MiroWorkflow shm: workf| ow_miro_command A V / / —4‘I l Tr / dds_daemon dds: miro01_miro_info dds: miroO1_miro_command / A V V
  43. 43. dds_daemon Overview Connections In/ out connector TypeCode Rate, QoS, etc. CDR serialization Communication status Save system logs API for dynamic connection control l / A ONDON EONNE ‘_ 01 ~ V Global Data. Space dds daemon MiroSetup Miroworkflow MiroContro|
  44. 44. dds_daemon . { / ’ / Features . ; 5;. / + Fast discovery + Fast and deterministic local communication 7 + Automation tasks + Easy rerouting of the existing connections + Additional connectors allow interfacing with another systems (aRD, LN) Higher round-trip duration - Some Qualities of Service lost — Reliability, TimeBasedFi| ter still possible - Single point of failure per workstation
  45. 45. Outline / - Mirosurge System I iii :39 d Condponents ° Software architecture a System Overview d Communicatloi‘1 a dds_“ . aemon ° DDS Library usage a Applications
  46. 46. D D5 / A X / Development process I; ; RTI Code Generator 2 extension I : Automatic compilation of C++ Library a Automatic generation of Simulink Library : Automatic generation of serialized TypeCodes _ Simulink C++ library library Serialized Python TypeCode bindings
  47. 47. D D5 / f / Simulink example ‘: struct Basecommand '4 ‘AV 5 Simulink library uses shared ‘ boolean start: ) , ,,. .x - oolean 5 o : ' memory interface Ewan misfit. boolean resetsensor: 3 types of blocks are generated: Publisher "'°"“‘“""““" Blocking subscriber L Non—blocking subscriber st : cdr: sigrnacontrolzzaase l: ommand5ubB Sm , ["shmName"] raj ' cdr: 3igmaCon| ro| ::BaseCommardPub status . Dynamic simulink blocks using status * serialized TypeCode «an cdr: sign1aControl: :Basel: ornmandsubNB st , [''shmName''i ‘'9 reset ' reset Sensor '
  48. 48. D DS I / f / Python bindings : htt s: ithub. com forrestv DDS I ctypes Builtin topics subscriber to discover all types and topics DynamicData API with serialized typecode Typed API for generated shared library In progress dds_daemon API for automation ‘-1 i? ;': ‘»‘.
  49. 49. DDS C++ helper library - Template DataReader, DataWriter . I J ° Available in 5.2.0 ° Serialization functions - CDR navigation functions to convert to another serialization modes - RTI Open Community Source ° Compile serialization functions for any platform, e. g. QNX 6.5 with Dinkum C++ libraries ° Update for RTICdrTypeCode_seria| ize
  50. 50. DDS . { Network structure and discovery / V . < / ‘ I General purpose network Most workstations in same subnet Discovery via multicast and unicast to the peers outside the subnet Dedicated real—time network All workstations in same subnet Discovery via multicast Automatic configuration of NDDS_D| SCOVERY_PEERS based on subnet / / / I - ONDONCONNEXWOI ‘ . V Y iii PZ«‘. ... ... . in) CI: : ‘#733’? 1
  51. 51. DDS / Quality of Service XML QoS profile library . I D? - Best—effort for continuous data - Reliable for user input and internally generated commands - Reliable large data to transmit log files - Best—effort with longer history for connection status
  52. 52. DDS Performance 10‘ 9 V0‘ 9 app with dds daemon‘ app with dds daemonl HI, ‘ can ~C'". -cni> ‘ " , ' simulink 3 10 Duration (microseconds) and Duration (microseconds) 6_ SN ll‘. 4 X / -I I t-/ ""‘ 2 O 10 r 10 2 3 4 s 2 3 1 > 10 10 10 10 10 10 10 10 Packet size (bytes) Padmt slzr: (bytesi Mean roundtrip Mean write blocking duration
  53. 53. Outline a German Aerospace Center r I - From Space to Medical robotics a l. /lotivation - MiroSurge System a Components a Software architecture - Applications
  54. 54. Robot Guided Waterjet surgery / _ ( ° Selective, tissue conserving dissection I - Standard handling from open surgery limits the use in MIS : > Assistance function under full control of the surgeon Bahls, Thomas und Friihlich, Florian und Albu-Schéiffer, Alin: Toward robot guided waterjet surgery. In: CURAC, 28.—30. Nov. 2013, Innsbruck Dstereich.
  55. 55. New input modalities * / @m: ° Optica| —inertia| tracking All input device ° active infrared markers ° inertial measurement unit ° Occlusion of markers is possible ° Large workspace F. Steidle, et aI. : Optical-Inertial Tracking with Active Markers and Changing Visibility. In Proc. IEEE IROS, 2012.
  56. 56. Integrating kinematics of the operating table ° Enable intraoperative patient repositioning ° Use gravity acting on organs to reach the area of interest maintain hand—eye coordination preserve the fulcrum point F. Steidle, et aI. : Adaption of the DLR MiroSurge System's Kinematics to Surgical Table Based Patient Repositioning in Minimally Invasive Surgery. In Proc. of CURAC Annual Meeting, 2014.
  57. 57. System Simulation Master Any parts of the system can be replaced by simulation
  58. 58. Thank you for your attention! Questions?

×