Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Agrirobot presentation by George Adamides

3,735 views

Published on

Agricultural Robot Sprayer and Evaluation of User Interfaces in Field Experiments

Published in: Technology
  • Be the first to comment

Agrirobot presentation by George Adamides

  1. 1. Agricultural Robot Sprayer and Evaluation of User Interfaces in Field Experiments George Adamides Senior Agricultural Research Officer Agricultural Research Institute
  2. 2. Presentation overview Technical characteristics of the Agricultural Robot Sprayer User Interfaces for HRI Design & implementation Field Experiment design & implementation Findings Conclusion
  3. 3. Summit XL by Robotnik Technical characteristics of the Agricultural Robot Sprayer http://www.robotnik.es/en/products/mobile-robots/summit-xl
  4. 4. Summit XL technical characteristics • The SUMMIT XL is a medium-sized high mobility all-terrain robot with extreme performance. • The Summit XL has skid-steering kinematics based on 4 high power motor wheels – Dimensions 693x626x417 mm – Weight 30 Kg – Load capacity 20 Kg – Speed 3 m/s – Traction system 4 wheels – Batteries 8x3.2V LiFePO4 (~5 hours autonomy) • 45 minutes for full charge – Temperature range 0oC +50oC – Max climbing angle 45o – Controller ROS embedded PC with Linux Real Time – Communications WiFi 802.11n – Connectivity Internal: USB, RS232, GPIO y RJ45 – External: USB and power supply 12 VDC Technical characteristics of the Agricultural Robot Sprayer
  5. 5. Main parts of Summit XL robot
  6. 6. Building the Robot Sprayer Technical characteristics of the Agricultural Robot Sprayer September 2012
  7. 7. The Sprayer • Serena electric sprayer Weight (with full tank) 13.6 kg Measurements: 315x145x400 Water flow rate with a fan nozzle 26 liters/h Battery life 11.30 h Full cycle charging 10h Capacity 10 liters Technical characteristics of the Agricultural Robot Sprayer
  8. 8. Sprayer installation • Modbus IO • Sprayer case • ROS module programming <?xml version="1.0"?> <launch> <!-- start modbus_io node --> <param name="modbus_io_node/ip_address" value="192.168.2.185" /> <param name="digital_outputs" value="8"/> <param name="digital_inputs" value="8"/> <param name="analog_outputs" value="2"/> <param name="analog_inputs" value="2"/> <node pkg="modbus_io" type="modbus_io_node" name="modbus_io_node" output="scr een"/> </launch> ssh summit@192.168.0.200 summit> ping 192.168.0.185 3. Launch the modbus_io module roslaunch modbus_io test_io.launch 4. Test the digital outputs rosservice call /modbus_io/write_digital_output 0 false rosservice call /modbus_io/write_digital_output 0 true rosservice call /modbus_io/write_digital_output 1 false rosservice call /modbus_io/write_digital_output 1 true ... Outputs 5,6,7 and 8 are RELAYS 5. Once it is working, modify the summit_xl_complete launch file in order to launch t Sprayer case and installation by AgroWise
  9. 9. Building the Robot Sprayer – modified version December 2012 Technical characteristics of the Agricultural Robot Sprayer
  10. 10. Initial trial-out findings • Issues with the camera – Viewing angle – Drops on dome cover • Issues with PC screen – Lighting/shading/reflection • Issues with wireless connection – Distance • Issues with Bluetooth connection – PS3 (distance) Technical characteristics of the Agricultural Robot Sprayer
  11. 11. Improving the Agrirobot • Hardware – Installation of two USB cameras to improve Peripheral vision and End-effector vision – Bigger wifi antenna – Moved higher the spraying nozzle • Software – Installation and configuration of the mjpeg_server ROS module and the Apache webserver – Installation and programming the pr2_keyboard_teleop ROS module for the Summit XL navigation and the sprayer on/off state Technical characteristics of the Agricultural Robot Sprayer
  12. 12. Peripheral Visual aid Technical characteristics of the Agricultural Robot Sprayer
  13. 13. End-effector visual aid Technical characteristics of the Agricultural Robot Sprayer
  14. 14. WiFi Antenna
  15. 15. Building the Robot Sprayer – current version Technical characteristics of the Agricultural Robot Sprayer May 2013
  16. 16. Technical issues and troubleshooting Short-movie Technical characteristics of the Agricultural Robot Sprayer
  17. 17. Presentation overview Technical characteristics of the Agricultural Robot Sprayer User Interfaces for HRI - Design and Implementation Field Experiments Design & Implementation Findings Conclusion
  18. 18. Reality Based Interaction (RBI) styles [1] New interaction styles that draw strength by building on users’ pre-existing knowledge of the everyday, non-digital world to a much greater extent than before. Examples of RBI: VR, AR, TUI, ubiquitous and pervasive computing, handheld or mobile interaction… [1] Jacob, Robert JK, et al. "Reality-based interaction: a framework for post-WIMP interfaces." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2008. User Interfaces for HRI – Design and Implementation
  19. 19. User Interface for Robot Teleoperation – Development Phases User Interfaces for HRI – Design and Implementation Spraying
  20. 20. Designing for HRI Awareness HRI Awareness [2] Given one human and one robot working on a task together, HRI awareness is the understanding that the human has of the location, activities, status, and surroundings of the robot; and the knowledge that the robot has of the human’s commands necessary to direct its activities and the constraints under which it must operate. LASSO technique [3] • Location Awareness • Activity Awareness • Status Awareness • Surroundings Awareness • Overall Mission Awareness [3] Jill L. Drury, Holly A. Yanco & Keyes, B 2007, 'LASSOing HRI: analyzing situation awareness in map-centric and video-centric interfaces', Proceedings of the ACM/IEEE international conference on Human-robot interaction. [2] Scholtz, J.; Young, J.; Drury, J.L.; Yanco, H.A., "Evaluation of human-robot interaction awareness in search and rescue," Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 IEEE International Conference on, vol.3, no., pp.2327,2332 Vol.3, 26 April-1 May 2004
  21. 21. Robot teleoperation through Human-Robot user interfaces Mental model User Interfaces for HRI – Design and Implementation
  22. 22. Phase 1. Using ROS command line ~> roslaunch usb_cam low_res.launch User Interfaces for HRI – Design and Implementation ~> export ROS_MASTER_URI=http://V3:11311 ~> rostopic list ~> rosrun image_view image_view image:=/logitech_usb_webcam/image_raw Robot PC Remote PCssh summit@V3
  23. 23. • Using ROS command line to display Peripheral and End-Effector cameras • Web interface of Axis Ethernet camera ROS environment with three cameras User Interfaces for HRI – Design and Implementation
  24. 24. • Installed the mjpeg_server module • Installed the Apache web server The mjpeg_server is a streaming server that subscribes to requested image topics in ROS and publishes those topics as MJPEG streams via HTTP Preparing for phase 2 User Interfaces for HRI – Design and Implementation
  25. 25. Phase 2. First attempt for Web UI in HTML User Interfaces for HRI – Design and Implementation
  26. 26. Improved version of the Web UI in PHP UI Design and Implementation by Istognosis
  27. 27. UI for driving Main camera & Peripheral camera UI for spraying (rejected) Main camera & spraying camera UI for spraying Main camera & ssupport cameras
  28. 28. Phase 4. Using a patriot wireless tracker & digital glasses User Interface Design and Implementation by Istognosis
  29. 29. Presentation overview Technical characteristics of the Agricultural Robot Sprayer User Interfaces for HRI – Design & Implementation Field Experiments Design & implementation Findings Conclusion
  30. 30. Field experiments Design & Implementation
  31. 31. Setting up the stage Field experiments Design & Implementation
  32. 32. Preparing the robot Field experiments Design & Implementation
  33. 33. Path and obstacles Field experiments Design & Implementation
  34. 34. Grape clusters (targets) Field experiments Design & Implementation
  35. 35. Experiment design & implementation 1 PC Screen + PS3 + Main Camera Only 2 PC Screen + PS3 + Main & Support Cameras 3 PC Screen + Keyboard + Main Camera Only 4 PC Screen + Keyboard +Main & Support Camera 5 AR Glasses + PS3 + Main Camera Only 6 AR Glasses + PS3 + Main & Support Cameras 7 AR Glasses + Keyboard + Main Camera Only 8 AR Glasses + Keyboard + Main & Support Cameras Field experiments Design & Implementation Tasks were randomized and were conducted in two day visits. Four tasks were carried out on day 1 and the remaining tasks on day 2 (not consecutive, period between experiments varied 2 to 10 days) 25 participants: Agronomists Agricultural Technicians Agricultural Laborers
  36. 36. Experiment procedures • Consent form • Pre-Questionnaire – Demographics, Immersion Tendency Questionnaire, General Self-Efficacy Scale, Santa Barbara Sense of Direction Scale, CEW Fluency Scale • Briefing and getting familiar with the UIs • Post-Questionnaire after each run – SUS, Presence, NASA TLX – Metrics (collisions, path divergence, targets sprayed, targets missed, percent completed, duration) • Experiment duration ~3 hours per participant to complete 8 tasks Field experiments Design & Implementation
  37. 37. The Agrirobot in the field Photo Album Field experiments Design & Implementation
  38. 38. Spraying – extended nozzle antenna
  39. 39. The Agrirobot UI findings Screenshots
  40. 40. Findings Main camera VS Main Camera and Support Cameras
  41. 41. Identifying obstacles
  42. 42. Identifying obstacles
  43. 43. Peripheral visionIdentifying obstacles
  44. 44. Shading Issues (left wheel) Spraying
  45. 45. Stopped Spraying Shading Issues (left wheel)
  46. 46. Shading issuesIdentifying obstacles
  47. 47. Shading Issues (wheels)
  48. 48. Identifying red grape clusters Identifying green grape clusters
  49. 49. The Agrirobot VS obstacles in the field Image album
  50. 50. Keep going…
  51. 51. Presentation overview Technical characteristics of the Agricultural Robot Sprayer User Interfaces for HRI – Design & Implementation Field Experiments Design & Implementation Findings Conclusion
  52. 52. Preliminary results NASA Task Load Index per UI Findings User Interfaces with PS3 controller User Interfaces using keyboard controller
  53. 53. Preliminary results Effectiveness: Number of grape clusters sprayed per UI Findings User Interfaces with main and Support (3) cameras User Interfaces with main (1) camera
  54. 54. Preliminary findings Mean number of collisions per UI UI # Cameras N Minimum Maximum Mean Std. Deviation UI1 - 1 Camera 25 0 3 ,76 ,779 UI2 - 3 Cameras 25 0 4 ,60 ,913 UI3 - 1 Camera 25 0 7 1,36 1,777 UI4 - 3 Cameras 25 0 2 ,56 ,861 UI5 - 1 Camera 25 0 5 1,28 1,242 UI6 - 3 Cameras 25 0 4 ,84 ,987 UI7 - 1 Camera 25 0 4 ,80 1,118 UI8 - 3 Cameras 25 0 3 ,28 ,678
  55. 55. Presentation overview Technical characteristics of the Agricultural Robot Sprayer User Interface Design Field Experiment design & implementation Findings Conclusion
  56. 56. In Summary – Problems faced and overcome • Transformation of a “off-the-shelf” robot into a robotic sprayer • Pilot trials revealed issues with WiFi, Bluetooth, camera view points • A lot of –smaller or bigger- practical, “non-research” problems, turned into valuable experience for the future
  57. 57. In Summary - what we did • Designed and implemented several user interfaces • Used WIMP and RBI interaction styles • Field experiments
  58. 58. Conclusions • Yes, it is feasible! (Agri Robot tele-operation ) • The user interface design does make a difference • There are many small, practical issues to resolve – Agricultural task are demanding and take place in a difficult environment – Many issues to overcome • PS3 Bluetooth, WiFi, monitor shading/light, web cameras • Robot wheels, sprayer hose • ROS module programming • Promising findings
  59. 59. Future work • Incorporation of sensor information in the UI to include ultrasound sensor information, battery-life (under development) • Robot improvements – Servos for extending sprayer antenna and USB/Ethernet cameras control/rotation – Sprayer antenna with multiple nozzles • Learnability issues need further study • Long hours? • Cost-benefit analysis
  60. 60. Spraying short movie
  61. 61. Thank you for your attention! Discussion / Coffee time

×