ROBOT AIDED TUNNEL INSPECTION AND MAINTENANCE SYSTEM full presentation

1. ROBOT AIDED TUNNEL
INSPECTION AND MAINTENANCE
SYSTEM
PRESENTED BY
SREETHU M B
S3 CASE
DEPT. OF CIVIL ENGINEERING

2. CONTENTS
1. INTRODUCTION
2. TUNNEL DEFECTS
3. TUNNEL INSPECTION METHODS
4. ROBOT AIDED TUNNEL INSPECTION AND MAINTENANCE
5. CASE STUDY
6. CONCLUSIONS
REFERENCES
2

3. 1. INTRODUCTION
• One of the greatest challenges engineers face is the inspection, assessment,
maintenance and safe operation of the existing civil infrastructure.
• It includes large - scale constructs such as tunnels, bridges, roads and
pipelines
• Tunnels nowadays are designed and built to last hundreds of years
• Change in use, new load criteria, and impact and damage caused by natural
and human factors can drastically reduce a tunnel's service life
• Tunnels throughout the world and in europe in particular become older,
matters of inspection and maintenance adopt an ever increasing degree of
importance

4. NEED OF ROBOT AIDED TUNNEL INSPECTION !!!
• Inspection and maintenance operations in tunnels depend heavily upon
time and space constraints:
▫ Tunnel's intrinsic conditions (reduced space, possible existence of
service pipes)
▫ Traffic flow
▫ Presence of aerial electric cable in railway tunnels
• Working conditions in these subterranean infrastructures can be slow and
tedious
• Dust, humidity, and complete absence of natural light create uncomfortable
and at times unhealthy working conditions

5. 2. TUNNEL DEFECTS
• TOMIE Manual (Tunnel Operations, Maintenance, Inspection and
Evaluation) created by the Federal Highway Administration (FHWA)
1. Concrete structures: scaling, cracking (traverse, longitudinal, horizontal,
vertical, diagonal, pattern, d-cracks, random), spalling, joint spall, pop-
outs, efflorescence, staining, delamination, honey-comb, leakage.
2. Steel structures: corrosion, cracks, buckles and kinks, leakage, protective
layer fail.
3. Masonry structures: masonry units (displaced, cracked, broken, crushed,
or missing), mortar, shape, alignment, leakage.
4. Timber structures: decay, insects, checks/splits, fire damage, hollow area,
leakage.

6. Fig 1: Spalling and cracking with efflorescence

7. 3. TUNNEL INSPECTION METHODS
• Most common structural material in tunnels is concrete, thus the following
inspection processes are usually applied to concrete tunnels
• NDI (non-destructive inspection) methods in structures can be divided in
1. Visual Methods
2. Strength Based Methods
3. Sonic and Ultrasonic Methods
4. Magnetic Methods
5. Electrical Methods
6. Thermography Methods
7. Radar Methods
8. Radiography Methods
9. Endoscopy Methods

8. 1. VISUAL METHOD
• Visual testing is probably the most important of all non-destructive tests
• It can often provide valuable information to the well trained eye
• Information can be gathered from visual inspection to give a preliminary
indication of the condition of a structure and allow the formulation of a
subsequent testing program

9. 2. STRENGTH BASED METHODS
• Measure the surface hardness of materials and provide an estimation of
surface compressive strength, uniformity and quality of the structure
• Done by Schmidt Hammer, the Windsor Probe and Flat Jack Testing
• Schmidt rebound hammer is principally a surface hardness tester
• It works on the principle that the rebound of an elastic mass depends on the
hardness of the surface against which the mass impinges
• Hammer pushed hard against the concrete, the body is allowed to move
away from the concrete until the latch connects the hammer mass to the
plunger

10. Schmidt Hammer Test

11. • Windsor probe is a hardness tester
• The penetration of the probe reflects the precise compressive strength in a
localized area
• A depth gauge measures the exposed lengths of the individual probes. The
manufacturer of the Windsor probe test system has published tables
relating the exposed length of the probe with the compressive strength of
the concrete
• For each exposed length value, different values for compressive strength
are given, depending on the hardness of the aggregate
• Flat Jack Testing used to determine the acting stresses or to evaluate the
mechanical parameters of the masonry/concrete structures

12. • Flat jack testing is a non-destructive way to evaluate the stress condition of
in-situ
• Two methods of testing are utilized: measurement across one cut to
evaluate acting stress and between two cuts to evaluate deformation or
modulus of elasticity of the concrete wall
12

13. 3.SONIC AND ULTRASONIC METHODS
• It is also known as impact-echo tests.
• Hammer blows create impulses, and the time of travel of these sonic pulses
is measured with pickups placed on the wall.

14. 4.MAGNETIC METHODS
• Magnetic methods are used to determine the position of reinforcements and
are not techniques for detecting defects or deterioration directly
• Methods are the Magnetic Flux Leakage method or the Magnetic Field
Disturbance method
• The basic principle is that a powerful magnet is used to magnetize the steel.
At areas where there is corrosion or missing metal, the magnetic field
"leaks" from the steel

15. 5.ELECTRICAL METHODS
• Electrical methods for inspection of tunnel components include resistance
and potential measurements.
• Electrical resistance has been used for measuring the permeability of deck
seal coats and involves measuring the resistance between the reinforcing
steel and surface, while electrical potential differences are caused by
corrosion of reinforcement.

16. 6.THERMOGRAPHY METHODS
• Infrared thermography measures the thermal radiation emitted by the
tunnel’s walls. Infrared registration techniques allow visual presentation of
the temperature distribution on the surface.
• Temperature on the surface represents the thermal flow through the surface,
which in turn is influenced by the mechanical and/or hydraulic
discontinuities of the structure.
• Thermal discontinuities on a surface reflect abnormalities within the
underlying structure.

17. 7.RADAR METHODS
• Radar methods have been widely used to detect defects in tunnels and other
structures, and the most used is the Ground-Penetrating Radar (GPR).
• GPR is the electromagnetic analogue of sonic and ultrasonic pulse echo
methods.
• It is based on the propagation of electromagnetic energy through materials
of different dielectric constants at an interface between two materials.

18. 8.RADIOGRAPHY METHODS
• X-rays, gamma radiation or neutron rays can penetrate structural materials
and therefore can be used on inspection purposes.
• The amount of radiation absorbed by the material is dependent upon the
density and thickness.
• This radiation can be detected and recorded on either film or sensitized
paper
• With this method, limitations are imposed by accessibility to both sides of
the object, long exposure times.

19. 9. ENDOSCOPY METHODS
• Endoscopes or video scopes consist of rigid or flexible viewing tubes that
can be inserted into pre- drilled boreholes of an element under
investigation.
• These allow close examination of parts of the structure which could not be
otherwise viewed

20. WHY ROBOTIC TUNNEL INSPECTION
AND MAINTANANCE ? ? ?
20

21. • Structural tunnel inspection is predominantly performed through
▫ Scheduled
▫ Periodic
▫ Tunnel-wide visual observations by inspectors who identify structural
defects and rate these defects
• This process is
▫ Slow
▫ Labor intensive
▫ Experience and fatigue of the inspector
▫ Working in an unpleasant environment due to dust, absence of natural
light
▫ Uncomfortable conditions or even toxic substances such as lead and
asbestos
21

22. 4. ROBOT AIDED TUNNEL INSPECTION AND
MAINTENANCE SYSTEMS
• Robotics systems in the construction field had been a common research area,
and several studies review the advantages in the use of robotic platforms for
construction and underground construction purposes.
• Robotic systems can complete the inspection process with
▫ Objective results and high efficiency.
▫ Improve safety in dangerous environments and more accurate results
• Manual and (human) visual inspection are being replaced with more precise
methods using mechanical, electronic and robotic systems and processing data
provided by cameras, laser, sonar, etc.
22

23. a) MAINTENANCE PROCEDURES TO BE
AUTOMATED
• practically all inspection and maintenance operations in tunnels are
performed manually.
• Frequently, traffic flow must be cut, and scaffolds mounted, implying the
subsequent loss of global productivity.
• The maintenance operations studied for automation call for the following
set of tasks:
a) Superficial preparation
b) Fissure injection
c) FRP composite adhesion.
23

24. 1) SUPERFICIAL PREPARATION
• This includes all of the processes needed to eliminate concrete in bad state
• Loss of mechanical capacity or of stability within the rest of the structure
can be a possible reason for this need
• The surface must be prepared before concrete reparation and/or restoration
• Common methods include compressed air blowing, sand abrasion, and
hydro- demolition (preparation with high-pressured water)
• The final prepared surface must be dry and free from sand and dust
24

25. 25

26. 2) FISSURE INJECTION
• Current developments of advanced materials such as low-viscosity epoxy
resins allow pressure injection of these materials through fissures as small as
0.2 mm thick.
• They are usually thixotropic, solvent-free, and bi-component.
• Components are included in different compartments of a same cartridge and
combined through a static mixer at application time
• The main objective of these injections is to re-establish continuity in concrete
sections
• Injections can be used to fill in internal or hard-to-access zones
• These techniques have been used and tested on tunnels, bridges, and several
nuclear centrals over the past few years.
26

27. 27

28. 3) FRP COMPOSITE ADHESION
• FRP adhesion now is also becoming a common practice among
automotive, marine, and aerospace industries
• Carbon or aramid fibers allow high mechanic, thermal, electric and
chemical resistance, high modulus of elasticity, and low density
• Commercially available FRP includes rolls that have been pre-impregnated
with epoxy resin, and dry rolls
• Basic manual procedures for adhesion of dry-roll FRP can be decomposed
into a sequence of three steps
28

29. 1. Operator spreads epoxy resin homogeneously on the
surface to be treated by paintbrush
Applied resin is specific to the FRP and, as bi-
component mixture, has been mixed by the operator in a
separate pail and must be applied before its curing time
2. FRP is laid over the resin-impregnated surface, and
is pressed against it with a metallic roller
This roller is hard enough to press the FRP until it
gets moist with resin, and its diameter is small enough to
adapt to a tunnel's curved surfaces
3. After this, a finishing layer of resin is applied. The
resin applied in this last layer is of the same type of that of
the first one
29

30. b) INTEGRATED PROCESS AUTOMATION
• Robot-Aided Tunnel Inspection and Maintenance (RATIM) system is the
result of one of the first attempts ever to automate superficial preparation,
fissure injection, and FRP composite adhesion procedures
• An integrated system is mounted with a
1. Robot arm
2. Wheeled vehicle with a platform lifts
3. Specially designed tool
• Inspection in the RATIM system is performed through a simple and very
intuitive Human Machine Interface (HMI), where the camera image stream
is displayed at all times and operation procedures are solicited
• Visual surveying is based on images and the depth measure captured by the
tool's laser telemeter
30

31. 31

32. 1. ROBOT ARM
• For dexterous and light-weight operation, the robot arm chosen for the
application was the Mitsubishi PA-10, a 7 DOF(Degrees of Freedom)
manipulator with a 10kg load capacity and a 1 meter maximum extension
range, weighting only 35 kg itself
• Global increment of this range is achieved by mounting the Robot–Tool set
on a 5 meter extensible articulated lift platform
32

33. 2. WHEELED VEHICLE
• HMI is installed in the wheeled vehicle's cabinet to which the articulated
lift platform is attached
• Power for the system can be supplied from an on-board generator, the
wheeled vehicle's motor, or the tunnel's basic provided services.
• The surface to treat is inside the reach volume, user extends the vehicle's
stabilizers, directs the platform towards the concrete surface, and then
follows the HMI's on-screen instructions
33

34. 3. SPECIALLY DESIGNED TOOL
• A light-weight sensed integrated tool was designed and manufactured for the
automation of the described processes
• OBJECTIVES
 Minimum changes should be needed to be effectuated on the tool to change
between processes of superficial preparation and resin injection, and processes
of superficial preparation and FRP adhesion
 A single-pass FRP adhesion process is desirable. This implies FRP must be
dispensed and pressed against the surface simultaneous to the advance of the
tool and release of sufficient resin
 Robot tool interconnection flange must allow simple mounting and demounting.
Vision, laser distance, and security systems should be intrinsic to the tool
34

35.  Maintaining the tool light-weight and compact throughout the design
process would allow it to be attached and used within a wide range of
commercial and non-commercial robots
• Objectives were achieved, allowing the tool to benefit from the advantages
of repeatability, reliability, and precision provided by modern-day robotics
• Final designed tool is composed by two complementary systems:
1. Material application system composed by mechanical subsystems and
actuators, and a vision
2. Security system composed by camera, laser distance sensor, and
security micro-switches
35

36. • The option chosen for superficial preparation is compressed air blowing
• It is provided by a non-board compressor mounted on the wheeled vehicle
• Most tunnels provide this service, but the compressor is incorporated to
increase the flexibility of the system
• Air flow is digitally controlled by low power-consumption mini electro
valves
• The nozzle is basically an empty cylinder with one resin input bore
(ϕ5mm) and many punctures (ϕ1mm) for the tunnel side
• Availability of compressed air also led us to one of the system's
fundamental breakthroughs
• Thrust of the resin towards the tip is triggered by the canalization of
compressed air towards the piston of the resin cartridge
36

37. • Therefore, the need of a motor, gear reductions, and additional mechanical
components is obviated This frees the tool from extra weight,
size, complexity, and cost
37

38. 38

39. 39
 FRP
• Commercial FRP is distributed in the form of long rolls in both dry and
pre-impregnated rolls are available
• Use of pre-impregnated FRP rolls was discarded due to possible undesired
FRP tool interaction caused by the adhesive properties of the composite
• Dry FRP must be adhered with epoxy resin at application time
• The strips must be pressed against the resin soaked surface until they are
moistened with resin on internal and external sides
• This means that the resin must penetrate the FRP and soak the strips
internally too
• A sophisticated gear-and-roller-set system was designed for the
dispensation of dry FRP strips, maintaining the initial single-pass objective

40. • No motor is needed, as motion is achieved through the rotation of a roller
by friction with the surface
• In the designed system, rollers, gears, and resin dispensation work in
conjunction as the tool advances to obtain compact resin–FRP–resin layers
mounted on the tunnel's surface
• The simultaneity of processes makes the system act as a mini-production
line, where the traction roller is always in contact with the dry surface
• FRP dispensation and tool advance velocity radius is adjusted for
correctness of functionality
• In order to cut the roll of FRP into strips of the desired length, the gear-
and-roller system is combined with a guillotine-like cutting mechanism that
is activated by relative movements between the tool's internal mobile parts
and the tunnel's surface
40

41. • Linear actuators (pneumatic cylinders) were incorporated to automate the
cutting process without having to depend on the robot or manipulator's
strength against this surface
• A linear dispenser, very similar to the one used for air in the superficial
preparation process
41

42. 42
• VISION SYSTEM
• Two main components of the vision system
1. Axis IP surveillance camera
2. Ultra-light precision diode laser telemeter sensor
• Sensor ray and the camera's central pixel point are aligned and oriented parallel to
the axis of the resin cartridge
• Stereoscopic vision was discarded due to its ineffectiveness on homogeneous
patterns, such as tunnel concrete surfaces
• IP Surveillance Camera obtains images with a resolution up to 1280×1024 pixels
streaming at 30 frames per second, sufficient for having real-time visual
information in the control cabin
• Micro- switches activated by contact with the tool's mobile parts assure mechanical
safety

43. HMI AND CONTROL SOFTWARE
• HMI and control software were coded entirely in C++, a powerful multi-
platform object oriented programming language
• User-friendly HMI guides the system operator graphically and intuitively
through the inspection and identification tasks
• At program initialization subsystems and communications are initialized,
and a task selection box is deployed
• In the final version of the software, two options are available:
1. Superficial preparation and epoxy resin injection
2. Superficial preparation and FRP adhesion
• This value is kept for the rest of the execution of the program, forcing the
operator to reboot to confirm having preformed changes on the tool
43

44. • Once this step is confirmed, the interface passes to the main screen
44

45. 45
• Instructions are present at all times throughout the screens
• A virtual traffic light (circled in red) informs the user if the tool tip is too close
or too far away from the tunnel surface
• Exact measure of the distance from the tool tip to the surface is also presented
to the user.
• In the first main screen, two degrees of freedom are permitted to the operator
for directing the robot
▫ LEFT and RIGHT buttons represent a roll movement of the base joint of
the robot
▫ UP and DOWN imply a change of pitch of the end effectors
▫ The operator when satisfied press the CONTINUE button

46. • A zoomed vision of the camera images is displayed, and fine adjustment of
the robot tip is performed with the same degrees of freedom
46

47. • Operator presses the ALIGN ROBOT button, the robot performs a vertical
and a horizontal scan of the surface (t-like shape) and returns to a centred
position and orientation
• Operator uses these buttons accordingly until he or she is newly satisfied,
and then presses FIX ROBOT to finally be allowed to mark the points of
interest for trajectory generation
• If FRP adhesion was previously selected, the operator marks the limits of
the FRP strip to be adhered
• Resin injection was previously selected, A spline-type curve is generated as
operator marks points through the interface
47

48. 48

49. • A pre visualization of the application can be seen on-screen for either of
the two procedures
• Final step for the operator is to press the DO PROCEDURE button
• The 2D data introduced by the operator is enough for the system to
generate the final 3D robotic trajectories
• The system executes these trajectories while coordinating the automatic
actuators for task completion in real, non-ideal environments such as
tunnels.
• The operator simply waits for procedures to be performed.
• On-screen messages inform the operator of the current state of procedure
performance.
49

50. • After task accomplishment, the robot safely returns to its initial position,
and the interface returns to the main screen
• Internally, software parameters are set in headers, and additional calibration
information is calculated at run-time
• Communication schemes between HMI and control software are
implemented over TCP/IP, the Internet Protocol
• This means the operator could, technically, run the HMI software to
identify fissures and weakened surfaces from any place
• Graphical interface could also be used for applying machine vision
algorithms for the identification of cracks, relieving this task from the user
or providing support through augmented reality
50

51. 51

52. • Additionally, the following results from the experiments in the tunnels
were obtained.
1. A single operator was capable of successfully performing the
proposed inspection and maintenance tasks, conventional methods
often require from 2-5 operators
2. Mobile platform could be stabilized at the working position in less
than half an hour ,conventional tubular scaffold structure takes several
hours to mount at least
3. Performance time of resin application was reduced from a 10 min
average (manual) to less than 7, mainly due to the high pressure resin
output provided by the system.
52

53. 4. Test samples were extracted from the tunnel's surface. Three were
results from a manual conventional treatment, and three from the
results of the RATIM system
5. Analysis indicated no significant difference of the disparity of the
stress and strength parameters between the total of six result test
samples
53

54. 5.CASE STUDY
• In Korea, Approximately 70% of Korea consists of mountainous terrain
necessitating the use of many tunnels for railroads and roads.
• A concern exists for the safety of concrete structures, such as tunnels, in traffic
environments.
• To determine the safety of such structures, periodic inspections have been
conducted using non-destructive tests.
• It was very slow and less accurate.
• To solve these problems, a semi-automatic inspection using image processing
has been developed.
• These methods have been applied in practical settings including Tunnels, roads,
bridges.
54

55. 55
• Komatsu Engineering Corp. has developed and commercialized an image
acquisition system
• It can acquire the images of road and tunnel linings by using a laser-
scanning device.
• The mobile robot system consisted of optical, mechanical, and data storage
devices.
• These devices store images of the surface of the concrete structure
• The software extracted the length, width and orientation of cracks.
• The crack detecting system provide informations that helps to determine if
additional inspection of a structure is needed.

56. 6.DRAWBACKS
• Need skilled operators for the inspection.
• Operator should be present in the same location.
• Because of this the operator is exposed to dangerous environment.
56

57. 7.FUTURE TRENDS
• ROBINSPECT (ROBotic System with Intelligent Vision and Control for
Tunnel structural INSPECTion and Evaluation)
• It is a project Co-Founded by the European Commission under 7th
framework.
• Started in october 2013 and will be launched in 2017.
• The project comprises an autonomous robotic system
• Which is capable of inspection and assessment of tunnel in one pass.
57

58. ROBINSPECT
58

59. 59
• ROBINSPECT is driven by the tunnel inspection industry
adapts and integrates recent research results in intelligent
control in robotics.
• Computer vision tailored with semi-supervised and active
continuous learning and sensing.
• For detecting and measuring
 Radial deformation in cross-sections,
 Distances between parallel cracks, and
 cracks in open joints that impact tunnel stability.
• With millimeter accuracies.

60. 60
• ROBINSPECT extended mobile robotic system.
• It is a wheeled robotic system that will be able to extend an
automated crane to the lengths upto 4 to 7 metres
• This robotic system is composed of three subsystems:
 A mobile robot, an automated crane arm
 An industrial-quality robot manipulator
 Basic 1D,2D and 3D Sensors
• Apart from that, a special new ultrasonic sensor will be
equipped to measure cracks width and depth.

61. 61
• At the software level, Component Based Software
Engineering (CBSE) techniques is applied.
• Currently, several robotic software architectures YARP (Yet
Another Robot Platform) , ROS (Robot Operating System),
OROCOS (Open Robot Control Software, etc. are available
• For implementing CBSE and are in operation.
• Controllability is improved as much as possible to achieve an
accurate path in tunnels.

62. 62
• In future difficulties in doing inspection with the use of
wheeled platforms will be replaced.
• With the use of legged robots with insect-like legs.
• It can through rough and uneven terrain and can avoid
unexpected obstacles easily.
• Another technology which is used in tunnel inspection
process which avoids the mobility limitations by Unmanned
Aerial Vehicles (UAV).
• Advantage is that these robots can be produced at low cost.
• It can be used simultaneously as they do not interrupt traffic.

63. 63

64. 64
• Robots of reduced dimension are used with advances in the
field of nanotechnology known as Nano-robots.
• Nano-robots could penetrate tunnel walls through cracks or
small fissures
• Perform inspection and structural assessments of the
materials from the inside.
• Tests that are currently destructive could be performed
through the use of non-destructive Nano-bot mechanisms.

65. 8.CONCLUSION
• The Robot-Aided Tunnel Inspection and Maintenance System is the first
system of its kind.
• The system have ability to perform operations without cutting traffic flow
• Mounting and dismantling tremendous scaffolds are avoided
• RATIM system reduces the operation time.
• It is a quick system.
• RATIM is more safe to environment and less budget spent on repairing.
• RATIM is a reliable, accurate and cost effective system for inspection and
maintenance of tunnels
65

66. REFERENCES
• C.Balaguer, R.Montero, J.G Victores, S.Martinez, A.Jardon, Towards Fully
Automated Tunnel Inspection: A Survey and Future Trends,The 31st International
symposium on Automation and Robotics in Construction and Mining ,2014
• C.Balaguer, R.Montero, J.G Victores, S.Martinez, A.Jardon, Past, Present and
Future of Robotic Tunnel Inspection
• J.G. Victores, S. Martínez, A. Jardón, C. Balaguer, Robot-aided tunnel inspection
and maintenance system by vision and proximity sensor integration, Automation in
Construction Vol 20, 2011,pp 629–636.
• Seung-Nam Yu, Jae-Ho Jang, Chang-Soo Han ,Auto inspection system using a
mobile robot for detecting concrete cracks in a tunnel, Automation in Construction
Vol16 ,2007 pp 255–26.
66

67. 67
• Federal Highway Administration: Tunnel Operations,Maintenance,
Inspection and Evaluation (TOMIE).
• International Atomic Energy Agency: Guidebook on non-destructive
testing of concrete structures. 2002.
• Fujita, M. et al.: Non-Destructive Remote Inspection for Heavy
Constructions. In: Conference on Lasers and Electro-Optics 2012.
Washington, D.C.: Osa, 2012.
• Asakura, T., Kojima, Y.: Tunnel maintenance in Japan. Tunnelling and
Underground Space Technology, 18 (2-3), p. 161–169, 2003.
• Esser, B. et al.: Wireless inductive robotic inspection of structures. In:
Proceedings of STED International Conference Robotics and Applications.
Honolulu, Hawaii, USA, 2000, p. 14– 6.
• Berns, K., Hillenbrand, C.: A climbing robot based on under pressure
adhesion for the inspection of concrete walls. In: Proc. ISR. 2004, p. 5.

68. THANK YOU…
68