TeleRescuer is an innovative system for inspecting coal mine roadways, especially those affected by catastrophes such as fire, explosion of methane or coal dust, and the others. The system allows virtual teleportation of a mining rescuer to those areas of a coal mine, in which he could not remain due to hazards for life or health.

1. TeleRescuer
Project RFC-CT-2014-00002
TELERESCUER
SYSTEM FOR VIRTUAL TELEPORTATION OF RESCUER FOR INSPECTING COAL MINE
AREAS AFFECTED BY CATASTROPHIC EVENTS

2. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 2
SCHEDULE
1. Introduction
2. Work Packages and Bar Chart
3. Budget information
4. Deliverables
5. Detailed identification of needs, formulating requirements (WP1)
6. Research into the UV (WP 2)
7. Research into virtual teleportation technology… (WP3)
8. Dissemination of results (WP5)
9. Management and Coordination (WP6)
10.Conclusions

3. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 3
INTRODUCTION
GENERAL IDEA
 TeleRescuer is an innovative system for inspecting coal mine roadways,
especially those affected by catastrophes such as fire, explosion of methane or
coal dust, and the others.
 The system allows virtual teleportation of a mining rescuer to those areas of a coal
mine, in which he could not remain due to hazards for life or health

4. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 4
INTRODUCTION
MAIN PARTS

5. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 5
WORK PACKAGES AND BAR CHART

6. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 6
WORK PACKAGES AND BAR CHART
WP1-WP3
Added to initial bar chart in official ammendment no 1 (accepted by EC)

7. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 7
WORK PACKAGES AND BAR CHART
WP4-WP6

8. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 8
DELIVERABLES
ACHIEVED DELIVERABLES
Deliverable
number
Deliverable name Foreseen
finalisation date
Real finalisation
date
Form Location
D1.1. Formal specification of
requirements
M6 31.12.2014 Written
document
Appendix 2 to the first annual report
S1.1. A decision as to whether the
system has to be ATEX-
ready/ATEX-certified
M6 31.12.2014 Written
document
(minutes from
the meeting)
Appendix 3 to the first annual report
D3.1 A report on rescuers’ knowledge
acquisition and representation
M6 31.12.2014 Written
document
Appendix 4 to the first annual report
D3.3 A report on the simulations of the
operations of rescuers’ in a
hazardous area of the coal mine
with augmented reality elements
M18 Written
document
Appendix 2 to the mid-term report
D5.1 A TeleRescuer project official
webpage
M3 30.09.2014 Website Appendix 5 to the first annual report
and internet (www.telerescuer.
Polsl.pl)
D5.3 Articles about the obtained results
in scientific and industry journals
M12-M36 - Published
papers
List of papers presented in section 2
(Project overview table)
D5.4 A scientific seminar presenting the
theoretical results achieved to-
date
M18 23.09.2015 Meeting -
D6.1 A Consortium Agreement M3 07.07.2014 Written
document
Appendix 6 to the first annual report
D6.2 A first annual report M9 (initial
deadline: M15)
31.03.2015 Written
document
CIRCABC
D6.3 Mid-term financial and technical
reports
M21 Written
document
CIRCABC

9. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 9
DELIVERABLES
FUTURE DELIVERABLES
Deliverable
number
Deliverable name Foreseen finalisation
date
D2.1. A prototype of a mechatronic platform of the UV M24
D2.2. A prototype of a communication system and a sensory system M27
D2.3. A prototype of a control system M27
D2.4. A prototype system for building maps M27
D2.5 A method and a prototype system for the autonomous operation of the UV in a
known environment
M27
D3.2 A prototype of an effective human-machine interface for virtual teleportation M19
D3.4 A prototype of the training simulator M30
D4.1 A report on tests of the system and its components M36
D5.2 A brochure about the TeleRescuer project (as an electronic pdf file and in print) M33
D5.5 A promotional seminar intended for the potential recipients of the project’s results M35
D6.4 A second annual report M27
D6.5 Final technical and financial reports M36+9

10. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 10
DETAILED IDENTIFICATION OF NEEDS, FORMULATING REQUIREMENTS (WP1)
The identification of the needs has been carried out in close collaboration with the Central
Mining Rescue Station (CMRS). Results of those activities have been summarized, reported and
converted into formal requirements.
Requirements have been specified for each subsystem of the UV, including:
• Robot platform:
 Platform’s mobility, dimensions, weight, protection level, operating time and range;
 Required external mechanical equipment and it’s parameters;
• Communication system:
 Optical fiber communication;
 Wireless communication with motes;
• Sensory system:
 Internal sensors: orientation sensors, robot state sensors, protection sensors;
 External sensors: gas sensors (CH4, CO, CO2, O2), air velocity senor and also
requirements on their placement;
 Cameras: quantity , type and optimal placement;
• Control system with autonomous operation:
 Autonomous operation modes and conditions for autonomous operation;

11. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 11
RESEARCHINTOTHE UV (WP2)
MECHATRONIC SUBSYSTEM OF THE UV (T2.1)
Mechatronic subsystem is composed of:
 Base robotic platform:
• High mobility tracked platform with
independent tracks’ flippers
• Designed to meet ATEX standards
requirements
• Capable of being used in various missions
due to different configurations of external
equipment
 External equipment:
• Cylinder arm with sensors, cameras and
lights
• Laser scanner
• Mote dispenser
• Fiber unwinder

12. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 12
Managed 2 microcontrollers:
I. 1st C: Arduino Uno + Arduino Ethernet Shield
Connections:
• SPI:Voltage sensor (fuel gauge) to control
batteries;
• SPI: Ethernet shield;
• I2C: Inertial Measurement Unit (IMU) - MPU-
6050;
• UART RS-485:
II. 2nd C: PIC Microcontroller series 30GP
or 33MC from Microchip
Sensor control module (outside the
cylinder) managing gas sensors:
• Methane (min. 0 ÷ 20%V/V)
• Mass flow (0 ÷ 20 m/s)
• O2 (0 ÷ 25%)
• CO (0 ÷ 10000 ppm)
• CO2 (0 ÷ 5% vol)
• Humidity/Temperature (0÷100% /-20÷+60ºC)
• Free pins: 4 PWM pins (illumination), 4 I/O
digital pins (relay, …)
Arduino Uno + Eth shield
MPU-6050
SENSORS
SENSORY SYSTEM (T2.2)
RESEARCH INTOTHE UV (WP2)

13. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 13
Constraints:
• Reduced space for all the camera electronics (130 x 110 x 160 mm box)
• Need to work under fast Ethernet protocol (Not Giga because of the possible danger caused by a laser: IEC-
68070-28)
• Ability to change online the video bandwidth output of each camera
• Real time low latency video compression
• On-off capabilities for each camera
• PoE with the lower power consumption as possible
• At least one thermal sensor for smoke
• Illumination will be outside the safety box so it must be ATEX ready
Visible light cameras: NC353L Elphel model
 2 fish-eye (180º): at front and at rear
• Lens
• 5Mpix color sensor board model 0353-00-17
• IP fast Ethernet camera main board 10353
 1 stereoscopic camera (2 lenses)
• Additional Synchronization board for the stereo rig model 10359
Thermal Camera: FLIR AX8
 With converter to Ethernet cable
Illumination: AdaroTecnologia
 M1 enclosure
 Own electronics (PWM, omnidirectional LEDs)
VISION SYSTEM (T2.2)
RESEARCH INTOTHE UV (WP2)

14. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 14
Spool of Sedi-Ati
Possible Secondary
Switches
WIRED COMMUNICATION SYSTEM (T2.2)
RESEARCH INTOTHE UV (WP2)
Initial Possibility:
• Own solution;
• A simple unwinder (with an Optical Fiber Rotatory Joint – OFRJ of 2 channels) + secondary system to
wind the optical fiber once the robot is outside;
Problems:
• OFRJ: Much expensive;
• Dimensions of a reel with strong optical cable;
Solution: Spool for unmanned ground vehicles with not much strong cable;
100Base-FX or 1000Base-FX?
• 1000Base-FX is not easy to atexize;
• 1000Base-FX normally uses lasers: difficult to satisfy the regulation IEC-68070-28;
• 100Mbps is enough for cameras with an acceptable resolution;
3 switches needed:
1) Primary station (in the safe place):
• 1 Optical Fiber port (to the robot);
• 1 Ethernet port: ATEX Ethernet (80 m máx.) for the first mote;
• 1 port for the operator;
2) Main switch (in the chassis): 4 ports (PC board, 1st mote, secondary switch, optical (for the O.F.));
3) Secondary Switch (in the cylinder):
• Small (112 mm x 155 mm x 134 mm OR 106 mm x 160 mm x 134 mm);
• Managed;
• Able to bear high temperatures;
• PoE ports (4 at least for cameras);
• 8 ports (4 cameras, 1 C, 1 primary switch, 1 relay, 1 free);

15. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 15
Requirements:
• Able to work in hazardous
environment;
• Size constraints;
• Range distance;
• Robust;
Technical requirements:
• Ad-hoc network: just forwarding
elements, no routers, no Aps;
• Self-organizing network: Same LAN,
pre-configured IP addresses;
Technologies:
1) UWB-Decawave: Data rate (6.8 Mb/s
máx. Non effective!!! ;  Max 10m ).
Not optimized for communications;
2) IEEE 802.11 (Atexized Raspberry
Pi);
ATEX-ready Batteries:
• Lithium iron phosphate batteries
(LiFePO4)
WIRELESS COMMUNICATION SYSTEM (T2.2)
RESEARCH INTOTHE UV (WP2)
First prototype

16. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 16
RESEARCH INTOTHE UV (WP2)
CONTROL SYSTEM (T2.3)
Mote #n
Chassis – inner space
4x RoboteQ FBL 2360
- Track + Track Arm motors
- Absolute sensor tilt of arms
(connected to RoboteQ AI)
- 2 x IRC + Hall (connected to RoboteQ)
PC Board
Vision (2D/3D)
- 1 IR, 2x for stereo (common Eth.channel), 2 x fish = 5
cameras with 4 IP interface
UC3M
Cylinder Control System
- Communication with external Sensor Control System (RS485) and MCS (Ethernet)
- Cameras power ON/OFF
- Lighting – front and rear
- Voltage of power supply measuring,
- IMU – Tilting x,y,z,..- for info about head tilt – artificial horizon for operator
UC3M
Ethernet Switch
3D Laser scanner
12V, 20W
endcap
26+2 Temperature
Sensors (DALLAS)
Mote #02
Mote #01
(CH4, CO, CO2, O2, temperature
outer, humidity,.....
Sensor
Control
System
CPU
CH4, CO, CO2, O2, temperature
outer, humidity,.....
Metalic Ethernet (4 wire)
Optical serial communication (2 fibres)
Metalic CAN bus (2 wire)
Metalic serial communication + power(2 + 2 wires)
Metalic 1wire bus (3 wire)
Optical Fibre Interface
USB/CAN interface
USB/1Wire Interface
Main IMU
Autonomy
Lock
Button
Central
Stop
Button
Metalic USB (4 wire)
endcap
endcap
endcap
IMU Configuration
USB Reserve
RS485 Reserve
Battery
Management
System
CAN
CAN
DI ENABLE
USB (2xRS232)
USB
USB
USB
RS485
USB
ETH
ETH ETH ETH
1WI
Optical
Interface
Optical Ethernet (1 - 2 fibres)
ETH
ETHETHETH
Optic Fibre Reel
UC3M
ETH
ETH
RS485
RS232/RS485
DC/DC
converter
Ethernet switch
Main Switch
Flameproof enclosure bushing
ETHETH
UC3M
UC3M
SUT
SUT
Optical Fibre
Interface +
ENABLE control
(VŠB)
UC3M
SUT
SUT
SUT
SUT
VSB
VSB
SUT
SUT SUT
UC3MFinal Assebling + cabling: SUT, VSB
UC3M
SUT,
UC3M
MOTE
Relaser
SUT
Wireless
comunication
module (UC3M)
RoboteQ
Sensor arm Tilting, – Sens.arm
Cylinder, Methane arm
3 axes= 1 drive, 1 motor,
2x solenoid (1 clutch, 1 brake)
2 x Absolute sensors – yes SUT
VPwrCtrl
(ENABLE)
RS232
Methane sensor
on methane arm
Main Control System and its Connectivity
with All Subsystems

17. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 17
RESEARCH INTOTHE UV (WP2)
METHODOLOGIES FOR BUILDING MAPS (T2.4)
 Laser Range Finder (Sick LMS111) + Positioning unit;
 Methane sensor SC-CH4 with ATEX switches off 3D LRF,
when methane concentration exceed the limit;
 A few methods were programmed for improvement of
visualization and coloring output with additional
information;
 Prototype tested in Królowa Luiza Coal Mine (October
2015): no failures were encountered;

18. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 18
RESEARCH INTOTHE UV (WP2)
AUTONOMOUS OPERATION OF THE UV (T2.5)
• Localization in a (not) known environment,
• Path planning in a known environment,
• Movement realization according to a
planned path in a constantly changing
environment
Main challenges
Growing uncertainty
over time
Reduced uncertainty
thanks to map matching
Based on R. Siegwart (ETH Zurich)
When autonomy will be used?

19. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 19
RESEARCH INTOTHE UV (WP2)
AUTONOMOUS OPERATION OF THE UV (T2.5)
Tests in simulated environmentsTests of inertial navigation
Robot localization based on 3D scans

20. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 20
RESEARCH INTOVIRTUAL TELEPORTATION TECHNOLOGY…(WP3)
OPERATOR STATION
In the operator station the following technolgies are used:
 3 monitors (144 Hz)
 3D technology
 Nvidia 3D vision

21. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 21
RESEARCH INTOVIRTUAL TELEPORTATION TECHNOLOGY…(WP3)
OPERATOR STATION - SOFTWARE STRUCTURE

22. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 22
DISSEMINATION OF RESULTS(WP5)
RESULTS ACHIEVED (1/2)
 The officialTeleRescuer logo;
 TeleRescuer website: www.telerescuer.eu (D5.1);
 Template for multimedia presentations
and official documents;

23. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 23
DISSEMINATION OF RESULTS(WP5)
RESULTS ACHIEVED (2/2)
 A brochure on the main project’s findings - with the
use of augmented reality techniques (D5.2);
 A scientific seminar presenting the theoretical results
(D5.4):
 Seminar on 23th of September, 2015 (Gliwice,
Poland);
 Seminar on 9th March, 2016 (Gliwice, Poland);
 Webinars/teleconferences (in average one per
month);
 Scientific papers: 10 publications) (D5.3);

24. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 24
DISSEMINATION OF RESULTS(WP5)
PUBLICATIONS IN 2014-2015 (D5.3)
1. W. Moczulski, K. Cyran, P. Novak, A. Rodriguez, M. Januszka, TeleRescuer - a concept of a system for
teleimmersion of a rescuer to areas of coal mines affected by catastrophes. Abstract, 28 November 2014 (full
paper will be published in: Proc. of the Institute of Vehicles, Warsaw University of Technology in 2015).
2. Smart Autonomous Mobile Systems for Exploring the Unknown. Poster presented at the 1st PERASPERA
Workshop, Noordwijkerhout, The Netherlands, 11-12 February 2015.
3. P. Novák, J. Babjak, T. Kot, W. Moczulski, Control System of the Mobile Robot TELERESCUER. Proc. of the
Optirob Conference, Bucharest, Romania, 27-30 June, 2015.
4. D. Myszor, W. Moczulski, K. Cyran: Innowacyjny interfejs ratownika umożliwiający wirtualną teleportację w
celu inspekcji wyrobiska kopalni dotkniętego katastrofą (in Polish). Proc. of the 42nd Symposium on
Technical Diagnostics (Abstracts), Silesian University of Technology, Faculty of Transport, Wisła, Poland, 02-
06 March, 2015.
5. Novák P., Babjak J., Moczulski W. Control System of the Mobile Robot TELERESCUER. Applied Mechanics
and Materials. 2015, vol. 772, pp. 466-470, doi : 10. 4028, ISSN : 1662-7482.
6. Novák P., Babjak J., Kot T., Olivka P. Exploration Mobile Robot for Coal Mines. In Modelling and Simulation
for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April 29-30, 2015,
209-215, ISBN 978-3-319-22383-4.
7. Kot T., Novák P., Babjak J. Virtual Operator Station for Teleoperated Mobile Robots. In Modelling and
Simulation for Autonomous Systems. International Workshop, MESAS 2015, Prague, Czech Republic, April
29-30, 2015, 144-153, ISBN 978-3-319-22383-4.
8. Timofiejczuk A., Adamczyk M., Bagiński M., Golicz P., Wymagania dla robotów uczestniczących w akcjach
ratowniczych w podziemnych kopalniach węgla kamiennego. Mechanizacja, automatyzacja i robotyzacja w
górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w
Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum
Badań i Dozoru Górnictwa Podziemnego, 2015, s. 59-65
9. Timofiejczuk A., Adamczyk M., Mura G., Nocoń M., Moczulski W., Układ mobilny specjalistycznego robota do
inspekcji wyrobisk kopalnianych dotkniętych katastrofą. Mechanizacja, automatyzacja i robotyzacja w
górnictwie. Monografia. Krzysztof Krauze (Red.). Centrum Badań i Dozoru Górnictwa Podziemnego w
Lędzinach, Katedra Maszyn Górniczych, Przeróbczych i Transportowych AGH w Krakowie. Lędziny: Centrum
Badań i Dozoru Górnictwa Podziemnego, 2015, s. 66-74
10. Mura G., Adamczyk M., Nocoń M.. Numerical simulation of mobility of miners rescue robot. 13th Conference
on Dynamical Systems Theory and Applications. DSTA 2015, Łódź, December 7-10, 2015, Poland. Eds. J.
Awrejcewicz, M. Kaźmierczak, P. Olejnik, J. Mrozowski. Łódź: Wydaw. Politechniki Łódzkiej, 2015, s. 222

25. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 25
MANAGEMENT AND COORDINATION(WP6)
 Collaboration with the use of knowledge management system: MOBIROB
platform on the base of OpenKM;
 At least 20 technical meetings since the start of the project: including
videoconferences and live meetings;
 4 official meetings: kick-off meeting (Gliwice, July 2014), first annual
meeting (Madrid, March 2015), half-year meeting in 2015 (Ostrava, July
2015), mid-term meeting (Gliwice, January 2016);
 All actions inTeleRescuer project related to WP6 have been strongly
supported by the Project Management Centre at the Silesian University of
Technology

26. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 26
CONCLUSIONS
PROBLEMS ENCOUNTERED AND CORRECTIVE ACTIONS
 Due to financial issues at AITEMIN, this partner had to leave the consortium,
with effect on the 31st of March 2015.
 The withdrawal of AITEMIN was organized in an orderly manner.
 A suitable replacement partner, with capacity to carry out the work initially
allocated to AITEMIN was proposed, and accepted by the consortium.
 AITEMIN will cooperate with the new partner in order to achieve a seamless
transition.
 Instead of AITEMIN the consortium included two new beneficiaries: UC3M and
KOPEX.
 KOPEX represents mining experience and knowledge necessary for
completing the project.

27. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 27
CONCLUSIONS
 All tasks in the framework of the
TeleRescuer project are carried out in
accordance with the schedule and
budget:
 All tasks from WP1 have been
finished;
 Tasks from WP2, WP5 and WP6 have
been started and are still in
progress.;
 Tasks from WP4 are planned in the
future;
 All planned objectives, deliverables and
milestones have been achieved.
 Since the aproval of the amendment has
taken quite a long time, the Consortium
will apply for the extension of the project.

28. T e l e R e s c u e r
P r o j e c t R F C - C T - 2 0 1 4 - 0 0 0 0 2 28
CONCLUSIONS
FUTURE WORK
 Regarding toWP2:
 Mechatronic subsystem of the UV;
 Communication system and sensory system;
 Control system;
 System for map building;
 System responsible for autonomous operation of UV in a known
environment;
 Regarding toWP3:
 Software and hardware of the human-machine interface (HMI) for virtual
teleportation technology;
 Software and hardware for training simulator;
 Regarding toWP4:
 Plan of validation tests.
In the next one year period the following results will be achieved:

29. TeleRescuer
Project RFC-CT-2014-00002
Silesian University of Technology
Coordinator
TeleRescuer Project Office
D.Sc. (habil) Anna TIMOFIEJCZUK
Silesian University of Technology
Faculty of Mechanical Engineering
Vice Dean for Organisation and Development
Project Coordinator
Akademicka Street 2A
44-100 Gliwice
Konarskiego Street 18a
44-100 Gliwice
Phone: +48 32 237 24 26
Fax: +48 32 237 13 60
e-mail: anna.timofiejczuk@polsl.pl
www.telerescuer.eu
THANK YOU FOR ATTENTION