This document discusses remote mining and CSIRO's research into mining automation and teleoperation technologies. It describes CSIRO's work developing intelligent systems for autonomous operation of mining equipment, including examples of dragline swing assist, terrain following, auto tramming, and explosive loading systems. The document outlines CSIRO's focus on improving safety, productivity, and equipment utilization through mixed-reality teleoperation, shared autonomy between humans and machines, distributed sensing networks, and augmented virtual control room interfaces.

1. Remote Mining
Elliot Duff
Senior Research Scientist
Nov 2007

2. CSIRO. TeleRobotics
Mining
Equipment
Shovels
Trucks
Draglines
Excavators
Drilling
Shearers LHD
Loading
Exploration & Mining
Sensors &
Communications
INS
LasersGPS
RadarVision
WiFi
GSM/GPRS CDMA/3G
Adhoc
UWB
TIP/MS/MST
Traditional Roles in CSIRO

3. CSIRO. TeleRobotics
Sensors &
Communications
New navigation and position
systems
New distributed sensor
technologies
Mining
Equipment
Intelligent control systems
Real-time reasoning
Specialising in the development of intelligent
systems for the autonomous operation of mining equipment
10 Years of Collaboration

4. CSIRO. TeleRobotics
Example 1. Dragline Swing Assist

5. CSIRO. TeleRobotics
Example 1. Dragline Terrain Following

6. CSIRO. TeleRobotics
Example 2. LHD Auto Tramming

7. CSIRO. TeleRobotics
Example 3. Explosive Loading

8. CSIRO. TeleRobotics
Mining
Equipment
Sensors &
Communications
Humans
SafetyHMI
Mine Awareness
• Failsafe communications
• Reliable localization
• Asset management
Mine Telerobotics
• Visualization
• Immersion
• Haptic Feedback
OH&STraining
Productivity
Improving health and safety for workers in hazardous environments,
as well as, enhancing equipment utilization and consistency of production.
Mine Automation
• Shared Autonomy
• Cooperative Behavior
• Material Handling
New Directions in Mining

9. CSIRO. TeleRobotics
Mine Tele-Robotics and Remote Mining
Mixed-RealityTele-Robotic
Robot
Tele-Operation
Machine
Autonomy
Shared
Autonomy
Autonomy
Manual
User
Interface
Intelligent
Behavior
Extent of
Knowledge
Communications
Latency
Global
LocalReactive
Proactive
Augmented
Reality
Augmented
Virtuality
Supervisory
Assistive
Real
Virtual
Mine Automation Mine Sensing and Comms
Mine Tele-Robotics
Development of shared perceptive space
(distributed model of the world) that facilitates
communication and task planning between
humans and machines with different levels of
sensing and autonomy.
Development of an augmented virtual user
interface that is capable of supervising a
number of remote machines and integrating
complex geological data with real-time data
streams from multiple remote sensors

10. CSIRO. TeleRobotics
Future Control Room
• Industry Drivers
• Underground - Remove people from dangerous environment
• Surface - Improve productivity (dig-to-plan & cooperative)
• Requirements
• Visual and Haptic user interfaces that are capable of integrating
complex geological data with real-time data streams from multiple
and remote machine sensor
• Visualization
• Putting disparate and numerous data sets into a coherent hole.
• NOT Virtual or Simulated data - augmented virtuality
• Based upon actual state of the machine/environment in real-time
• Immersive and Interactive
• Experience of being fully present - Not necessarily HMD or stereo
• Appropriate presentation of data to the user - single modality
• Engage operator in a natural and intuitive manner - 3D and Haptic

11. CSIRO. TeleRobotics
Enabling Technology – Localization
• Inertial Navigation Systems
• Global Positioning System
• Beacons / Tracking
• Laser
• Radar
• Vision

12. CSIRO. TeleRobotics
Enabling Technology - Environmental Sensing
• Conventional sensors
• Vision / Stereo
• Scanning Range Laser
• Ad-Hoc Sensing networks
• Radar

13. CSIRO. TeleRobotics
Enabling Technologies – Digital Terrain Mapping

14. CSIRO. TeleRobotics
Enabling Technologies – Active Controls
Dragline
Controls
Haptic
Workbench
• Seamless Set-point Transfer
• Intuitive Feedback
• Operator Acceptance
• Interchangeable

15. CSIRO. TeleRobotics
Example 4. Excavator Guidance
Real-time 3D Digital Terrain Map
Simulation based upon measured joint angles
Laser Bucket Tracking
Bucket
Ground
Excavator
Truck

16. CSIRO. TeleRobotics
Example 4. Excavator Reconciliation

17. CSIRO. TeleRobotics
Example 5: Shovel Truck Interaction

18. CSIRO. TeleRobotics
Example 5: Shovel DigAssist

19. CSIRO. TeleRobotics
Example 3: Scale Dragline

20. CSIRO. TeleRobotics
GUI Development
Trees
Machine
Centre
Trailer
Dig face

21. CSIRO. TeleRobotics
GUI Contour Plot
Mark dig and
dump point

22. CSIRO. TeleRobotics
Tele-Excavation over the Internet
QCAT
Redbank
ADSL
WiFi
JPL
Internet

23. CSIRO. TeleRobotics
Full Automation

24. CSIRO. TeleRobotics
Before and After

25. Contact Us
Phone: 1300 363 400 or +61 3 9545 2176
Email: enquiries@csiro.au Web: www.csiro.au
Thank you
ICT Centre
Elliot Duff
Senior Research Scientist
Phone: +61 7 3327 4632
Email: elliot.duff@csiro.au
Web: www.csiro.au/asl

26. CSIRO. TeleRobotics
Autonomy Spectrum (Digging)
Manual AutonomousDigToPlan DigAssist
Plan
Safety
Tool
Position
Encoders
3D
Display
Realtime
DTM
Lasers
Immersive
Display
Machine
State
Haptics
INS etc
Force
Feedback
Visualization
GPS
2D
Display
Offline
DTM
Situation
Awareness
Comms
Mission
Collaborative
Interfaces

27. CSIRO. TeleRobotics
State of the art in Excavation?
0
50
100
Digging
Swing
Dump
Walk
Maintenance
Integration
Autonomous
Manual
DigAssist
DigToPlan

28. CSIRO. TeleRobotics
Longwall Automation
Remote visualization of longwall driven by real data in realtime

29. CSIRO. TeleRobotics
Where to now?

30. CSIRO. TeleRobotics
Gaming ?
Eve - Asteroid MiningSecond Life