This document describes the development of an agricultural robot sprayer and evaluation of different user interfaces for human-robot interaction. It discusses the technical specifications of the robot platform used and modifications made to integrate a sprayer. Various user interface designs were implemented and tested in field experiments, including web-based and augmented reality interfaces. Preliminary findings showed that interfaces providing views from multiple cameras led to better task performance and fewer collisions compared to a single camera view. Further work is still needed to address additional technical challenges of agricultural robotics and improve the usability of interfaces.

1. Agricultural Robot Sprayer and
Evaluation of User Interfaces in Field
Experiments
George Adamides
Senior Agricultural Research Officer
Agricultural Research Institute

2. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI Design &
implementation
Field Experiment design &
implementation
Findings
Conclusion

3. Summit XL by Robotnik
Technical characteristics of the Agricultural Robot Sprayer
http://www.robotnik.es/en/products/mobile-robots/summit-xl

4. Summit XL technical characteristics
• The SUMMIT XL is a medium-sized high mobility all-terrain
robot with extreme performance.
• The Summit XL has skid-steering kinematics based on 4
high power motor wheels
– Dimensions 693x626x417 mm
– Weight 30 Kg
– Load capacity 20 Kg
– Speed 3 m/s
– Traction system 4 wheels
– Batteries 8x3.2V LiFePO4 (~5 hours autonomy)
• 45 minutes for full charge
– Temperature range 0oC +50oC
– Max climbing angle 45o
– Controller ROS embedded PC with Linux Real Time
– Communications WiFi 802.11n
– Connectivity Internal: USB, RS232, GPIO y RJ45
– External: USB and power supply 12 VDC
Technical characteristics of the Agricultural Robot Sprayer

5. Main parts of Summit XL robot

6. Building the Robot Sprayer
Technical characteristics of the Agricultural Robot Sprayer
September 2012

7. The Sprayer
• Serena electric sprayer
Weight (with full tank) 13.6 kg
Measurements: 315x145x400
Water flow rate with a fan nozzle 26 liters/h
Battery life 11.30 h
Full cycle charging 10h
Capacity 10 liters
Technical characteristics of the Agricultural Robot Sprayer

8. Sprayer installation
• Modbus IO
• Sprayer case
• ROS module programming
<?xml version="1.0"?>
<launch>
<!-- start modbus_io node -->
<param name="modbus_io_node/ip_address" value="192.168.2.185" />
<param name="digital_outputs" value="8"/>
<param name="digital_inputs" value="8"/>
<param name="analog_outputs" value="2"/>
<param name="analog_inputs" value="2"/>
<node pkg="modbus_io" type="modbus_io_node" name="modbus_io_node" output="scr
een"/>
</launch>
ssh summit@192.168.0.200
summit> ping 192.168.0.185
3. Launch the modbus_io module
roslaunch modbus_io test_io.launch
4. Test the digital outputs
rosservice call /modbus_io/write_digital_output 0 false
rosservice call /modbus_io/write_digital_output 0 true
rosservice call /modbus_io/write_digital_output 1 false
rosservice call /modbus_io/write_digital_output 1 true
...
Outputs 5,6,7 and 8 are RELAYS
5. Once it is working, modify the summit_xl_complete launch file in order to launch t
Sprayer case and installation by AgroWise

9. Building the Robot Sprayer – modified version
December 2012
Technical characteristics of the Agricultural Robot Sprayer

10. Initial trial-out findings
• Issues with the camera
– Viewing angle
– Drops on dome cover
• Issues with PC screen
– Lighting/shading/reflection
• Issues with wireless
connection
– Distance
• Issues with Bluetooth
connection
– PS3 (distance)
Technical characteristics of the Agricultural Robot Sprayer

11. Improving the Agrirobot
• Hardware
– Installation of two USB cameras to improve
Peripheral vision and End-effector vision
– Bigger wifi antenna
– Moved higher the spraying nozzle
• Software
– Installation and configuration of the
mjpeg_server ROS module and the Apache
webserver
– Installation and programming the
pr2_keyboard_teleop ROS module for the
Summit XL navigation and the sprayer on/off
state
Technical characteristics of the Agricultural Robot Sprayer

12. Peripheral Visual aid
Technical characteristics of the Agricultural Robot Sprayer

13. End-effector visual aid
Technical characteristics of the Agricultural Robot Sprayer

14. WiFi Antenna

15. Building the Robot Sprayer – current version
Technical characteristics of the Agricultural Robot Sprayer
May 2013

16. Technical issues and
troubleshooting
Short-movie
Technical characteristics of the Agricultural Robot Sprayer

17. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI - Design
and Implementation
Field Experiments Design &
Implementation
Findings
Conclusion

18. Reality Based Interaction (RBI) styles [1]
New interaction styles that draw strength
by building on users’ pre-existing
knowledge of the everyday, non-digital
world to a much greater extent than
before.
Examples of RBI: VR, AR, TUI, ubiquitous and
pervasive computing, handheld or mobile
interaction…
[1] Jacob, Robert JK, et al. "Reality-based interaction: a framework for post-WIMP
interfaces." Proceedings of the SIGCHI conference on Human factors in computing systems.
ACM, 2008.
User Interfaces for HRI – Design and Implementation

19. User Interface for Robot Teleoperation –
Development Phases
User Interfaces for HRI – Design and Implementation
Spraying

20. Designing for HRI Awareness
HRI Awareness [2]
Given one human and one
robot working on a task
together, HRI awareness is
the understanding that the
human has of the location,
activities, status, and
surroundings of the robot;
and the knowledge that
the robot has of the
human’s commands
necessary to direct its
activities and the
constraints under which it
must operate.
LASSO technique [3]
• Location Awareness
• Activity Awareness
• Status Awareness
• Surroundings
Awareness
• Overall Mission
Awareness
[3] Jill L. Drury, Holly A. Yanco & Keyes, B 2007,
'LASSOing HRI: analyzing situation awareness in
map-centric and video-centric interfaces',
Proceedings of the ACM/IEEE international
conference on Human-robot interaction.
[2] Scholtz, J.; Young, J.; Drury, J.L.; Yanco, H.A., "Evaluation of human-robot interaction
awareness in search and rescue," Robotics and Automation, 2004. Proceedings. ICRA '04.
2004 IEEE International Conference on, vol.3, no., pp.2327,2332 Vol.3, 26 April-1 May 2004

21. Robot teleoperation through
Human-Robot user interfaces
Mental model
User Interfaces for HRI – Design and Implementation

22. Phase 1. Using ROS command line
~> roslaunch usb_cam low_res.launch
User Interfaces for HRI – Design and Implementation
~> export ROS_MASTER_URI=http://V3:11311
~> rostopic list
~> rosrun image_view image_view image:=/logitech_usb_webcam/image_raw
Robot PC Remote PCssh summit@V3

23. • Using ROS
command line to
display Peripheral
and End-Effector
cameras
• Web interface of
Axis Ethernet
camera
ROS environment with three
cameras
User Interfaces for HRI – Design and Implementation

24. • Installed the mjpeg_server module
• Installed the Apache web server
The mjpeg_server is a streaming
server that subscribes to requested
image topics in ROS and publishes
those topics as MJPEG streams via
HTTP
Preparing for phase 2
User Interfaces for HRI – Design and Implementation

25. Phase 2. First attempt for Web UI in HTML
User Interfaces for HRI – Design and Implementation

26. Improved version of the Web UI
in PHP
UI Design and Implementation by Istognosis

27. UI for driving
Main camera & Peripheral
camera
UI for spraying (rejected)
Main camera & spraying
camera
UI for spraying
Main camera & ssupport
cameras

28. Phase 4. Using a patriot wireless tracker &
digital glasses
User Interface Design and Implementation
by Istognosis

29. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI – Design &
Implementation
Field Experiments Design &
implementation
Findings
Conclusion

30. Field experiments Design & Implementation

31. Setting up the stage
Field experiments Design & Implementation

32. Preparing the robot
Field experiments Design & Implementation

33. Path and obstacles
Field experiments Design & Implementation

34. Grape clusters (targets)
Field experiments Design & Implementation

35. Experiment design &
implementation
1 PC Screen + PS3 + Main Camera Only
2 PC Screen + PS3 + Main & Support Cameras
3 PC Screen + Keyboard + Main Camera Only
4 PC Screen + Keyboard +Main & Support Camera
5 AR Glasses + PS3 + Main Camera Only
6 AR Glasses + PS3 + Main & Support Cameras
7 AR Glasses + Keyboard + Main Camera Only
8 AR Glasses + Keyboard + Main & Support Cameras
Field experiments Design & Implementation
Tasks were randomized and
were conducted in two day
visits. Four tasks were carried
out on day 1 and the
remaining tasks on day 2 (not
consecutive, period between
experiments varied 2 to 10
days)
25 participants:
Agronomists
Agricultural Technicians
Agricultural Laborers

36. Experiment procedures
• Consent form
• Pre-Questionnaire
– Demographics, Immersion Tendency Questionnaire,
General Self-Efficacy Scale, Santa Barbara Sense of
Direction Scale, CEW Fluency Scale
• Briefing and getting familiar with the UIs
• Post-Questionnaire after each run
– SUS, Presence, NASA TLX
– Metrics (collisions, path divergence, targets
sprayed, targets missed, percent completed,
duration)
• Experiment duration ~3 hours per participant
to complete 8 tasks
Field experiments Design & Implementation

37. The Agrirobot in the field
Photo Album
Field experiments Design & Implementation

38. Spraying – extended nozzle antenna

49. The Agrirobot UI findings
Screenshots

50. Findings
Main camera VS Main Camera and Support Cameras

51. Identifying obstacles

52. Identifying obstacles

53. Peripheral visionIdentifying obstacles

54. Shading Issues (left wheel)
Spraying

55. Stopped Spraying
Shading Issues (left wheel)

56. Shading issuesIdentifying obstacles

57. Shading Issues (wheels)

58. Identifying red grape clusters
Identifying green grape clusters

59. The Agrirobot VS
obstacles in the field
Image album

68. Keep going…

69. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI – Design &
Implementation
Field Experiments Design &
Implementation
Findings
Conclusion

70. Preliminary results
NASA Task Load Index per UI
Findings
User Interfaces
with PS3 controller
User Interfaces
using keyboard
controller

71. Preliminary results
Effectiveness: Number of grape clusters sprayed per UI
Findings
User Interfaces
with main and
Support (3) cameras
User Interfaces
with main (1) camera

72. Preliminary findings
Mean number of collisions per UI
UI # Cameras N Minimum Maximum Mean Std.
Deviation
UI1 - 1
Camera
25 0 3 ,76 ,779
UI2 - 3
Cameras
25 0 4 ,60 ,913
UI3 - 1
Camera
25 0 7 1,36 1,777
UI4 - 3
Cameras
25 0 2 ,56 ,861
UI5 - 1
Camera
25 0 5 1,28 1,242
UI6 - 3
Cameras
25 0 4 ,84 ,987
UI7 - 1
Camera
25 0 4 ,80 1,118
UI8 - 3
Cameras
25 0 3 ,28 ,678

73. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interface Design
Field Experiment design &
implementation
Findings
Conclusion

74. In Summary – Problems faced and
overcome
• Transformation of a “off-the-shelf”
robot into a robotic sprayer
• Pilot trials revealed issues with WiFi,
Bluetooth, camera view points
• A lot of –smaller or bigger- practical,
“non-research” problems, turned into
valuable experience for the future

75. In Summary - what we did
• Designed and implemented several
user interfaces
• Used WIMP and RBI interaction
styles
• Field experiments

76. Conclusions
• Yes, it is feasible! (Agri Robot tele-operation )
• The user interface design does make a
difference
• There are many small, practical issues to
resolve
– Agricultural task are demanding and take place in a
difficult environment
– Many issues to overcome
• PS3 Bluetooth, WiFi, monitor shading/light, web
cameras
• Robot wheels, sprayer hose
• ROS module programming
• Promising findings

77. Future work
• Incorporation of sensor information in the
UI to include ultrasound sensor
information, battery-life (under development)
• Robot improvements
– Servos for extending sprayer antenna and
USB/Ethernet cameras control/rotation
– Sprayer antenna with multiple nozzles
• Learnability issues need further study
• Long hours?
• Cost-benefit analysis

78. Spraying short movie

79. Thank you for your
attention!
Discussion / Coffee time