2. Z. Luo et al. / Journal of Manufacturing Systems 31 (2012) 2–7 3
sensors typically project a single laser stripe onto the seam, and
provide limited look-ahead information [3–5]. Most recently, Sung
et al. [6] demonstrated ultra-high-speed seam tracking with newly
developed multi-line laser vision sensor. The reported average
tracking error for linear seam is 0.3 mm at a speed of 20 m/min.
The Precitec Company (Germany) has demonstrated online its
robot tracking system with a multi-line laser sensor. In principle,
it is possible to estimate the TCP offset-error by extrapolating
information of multiple laser stripes, and combining the benefits
of the above two types of sensors. Nevertheless, such sensors have
not become off-the-shelf until recently.
Depending on the information extractable from tracking
sensors and ways to translate the information to robot control
commands, the typical seam position prediction schemes can
be classified into two types, namely, incremental prediction
and absolute prediction. These schemes can lead to more
advanced and complicated tracking methods when enhanced
with other techniques, such as feedback control [7], feedforward
compensation [3,8], or intelligent control methods [2,9]. The
readers are referred to [10] for a slightly different classification
where seam tracking methods are classified as the visual servoing
and the trajectory-based control.
The incremental prediction scheme uses measured (or esti-
mated) TCP offset-errors with respect to certain local coordinate
frames to generate incremental commands. Typical local coordi-
nate frames are image coordinate frames used in many visual ser-
voing methods. Case and Ringwall proposed combining a 3D pro-
file sensor and a TCP co-axial sensor to calculate the micro-vector
motion of a welding robot [1]. Their method was straightforward
but required that the image capture cycle be shorter than the robot
interpolation cycle. Xu et al. [11] proposed a hybrid visual ser-
voing control, in which a position-based servoing control makes
the robot move along the direction of the welding-seam, while an
image-based visual servoing control adjust the robot to approxi-
mate the ideal pose. Nevertheless the image-based visual servoing
does not consider the look-ahead distance of 3D profile sensors and
hence may be unsuitable for nonlinear seams. Zhou et al. [7] en-
hanced a visual feedforward control with a PD feedback by convert-
ing visual incremental offset-errors into robot speed commands.
In such a manner the image capture cycle can be slower than the
robot interpolation cycle.
The absolute prediction scheme is widely known thanks to
the monograph of Nayak and Ray [5] published in 1993. In
their control method, seam geometry is represented as a hybrid
model, that is, the positions of seam root points are modeled
continuously as cubic polynomials and the seam coordinate frames
are modeled discretely; both are relative to the world coordinate
frame. They suggested a tracking system based on real-time
offline programming, for the robot controller to get the new
target position from a master controller at certain time intervals.
Their method had been successfully applied to arc welding
with a relatively high speed of 60 in./min but a slow control-
cycle period of 1000 ms. They emphasized that the tracking
methods requiring pre-taught paths are not desirable. However,
this is arguable for tasks with steady fixtures, where adapted
Tractrix algorithm [12] or iterative learning algorithms [10] can
be applied to obtain high quality pre-taught paths of industrial
robots automatically. In the industrial field, Taft and Ellison [4]
patented a quite similar system, where the seam and TCP path
were represented as spline using both coordinate extrapolation
and interpolation. It is noted that the above two methods lack a
feedback control and assume that the robot kinematical model
and the camera-to-tool frame transformation can be precisely
calibrated. Therefore, even though these methods are able to
converge with anticipated upper bounds of tracking errors, their
tracking errors are not optimized. Bauchspiess and Alfaro [13]
proposed a model-predictive method based on mixed optimization
criteria for seam tracking. This method allows the prediction of
the dynamic behavior of each robot joint for a given reference
trajectory. They applied this method successfully to a sensor-
guided hydraulic robot for path tracking with a customized
controller, but reported difficulties with the industrial robot in
the interfaces. Lange and Hirzinger [2] proposed an adaptation of
the positional feedforward controller in order to obtain optimized
feedforward commands. In their positional feedforward controller,
a linearized robot dynamic model and parallel neural nets were
used to optimally compensate nonlinear coupling effects. Their
robot system is able to track a seam at a speed of 0.7 m/s with
a tracking error of 1 mm. Most recently, Graff [10] extended
Nayak and Ray’s method to high-speed robotic laser welding.
In his tracking system, a Stäubli RX90 robot with a low-level
interface was employed, and allows a user application to set joint
commands at every 4 ms. A real-time setpoint generator based
on cubic spline interpolation was developed to generate a smooth
trajectory. Graff’s experiments demonstrated that the tracking
errors are mainly within ±0.1 mm at a velocity of 50 mm/s,
but the maximum value approaches 0.2 mm. Nevertheless, the
performance is still improvable considering that the repeatability
of the Stäubli RX90 robot is 0.025 mm. Moreover, it is preferable to
investigate robots larger than the RX90, so that large parts can be
welded.
Note that the scheme of incremental pose prediction is
preferred in some commercial robotic laser welding systems. For
example, Servo-Robot Inc. (hereafter abbreviated as SRI) of Canada
sells their robotic laser welding systems based on macro–micro-
robot systems, in which an industrial robot produces macro-
motions with external axes installed at the robot end flange
producing the micro-motions [8]. SRI’s patented tracking method
also introduces a feedforward compensation to reduce tracking
errors.
It is beneficial to combine the approaches from both academic
and industrial fields. We therefore employ iterative learning
methods to optimize the procedures of SRI and develop the
enhanced tracking algorithms [3]. In our system, a TCP co-axial
sensor and a laser sensor are combined to detect robot path shifts
and seam positions.
3. Feedforward compensation and predictive tracking
3.1. Residual errors in a high-precision seam tracking
For a precise laser welding, it is important to accurately
evaluate the TCP offset-error across the seam. Some conventional
approaches tempt to estimate the TCP offset-error during the
autonomous laser welding but meet with difficulties. When a TCP
co-axial sensor is used, it is possible to extrapolate seam points
measured near the TCP to estimate the TCP offset-error, but the
estimation error can easily exceed 0.1 mm due to noises. When a
laser sensor is used, the estimation of TCP offset-error is greatly
affected by the look-ahead distance, especially for nonlinear seams,
as has been noted by Graff and others [10]. Nevertheless, our
no-welding experiments of typical tracking algorithms indicate
that the mean of the TCP offset-error curves of multiple runs is
constant and should not be ignored for laser welding. It is therefore
important to minimize such errors. The mean TCP offset-error is
termed here as residual errors.
It is also important to understand the path accuracy and
repeatability of modern industrial robots. Fig. 1 illustrates an ABB
IRB4400 robot which is set up to track a straight seam 1600 mm
long. The TCP offset of the robot’s linear motion at a speed of
120 mm/s with respect to the seam is measured by a laser sensor.
Fig. 2 illustrates the averaged path of 30 repeated runs and an
3. 4 Z. Luo et al. / Journal of Manufacturing Systems 31 (2012) 2–7
Fig. 1. A typical linear seam in robotic welding.
Fig. 2. Typical robot path accuracy and repeatability.
individual path. It is observed that the robot drifts its path up to
0.6 mm in a short distance, and the path repeatability near position
550 mm is up to 0.2 mm. The illustrated fast and repeatable
path drifts are normally caused by inherited joint backlashes and
frictions in the robot, providing external disturbances are carefully
minimized. They are hereafter termed bursting-type path shifts.
3.2. SRI’s tracking and compensation strategy
It is understood that the bursting-type path shifts and the worse
repeatability are difficult to be modeled as an ideal robot kine-
matic model or a dynamic model. Therefore the classic tracking
methods based on absolute prediction scheme using the ideal mod-
els are prone to the wrong measurement of seam positions, and
can lead to large TCP offset-errors. In view of the problem, SRI
patented a new tracking method which simply extends the typ-
ical incremental pose prediction schemes of a single robot to a
macro–micro-robot [8]. In the control system, only a single laser
sensor is employed. Rather than being corrected by a feedback
control, the TCP offset-errors are measured and then compen-
sated feedforwardly through a three-pass procedure explained as
follows.
At the first pass when the robot travels along the seam, a real-
time seam tracking without actual welding is carried out. The
robot follows a pre-taught path and the external axes execute
micro-motion tracking commands generated by a typical tracking
algorithm. The interpolated seam data is recorded and sent to a
robot controller just before the effective arrival of the TCP to the
recorded portion of the seam path. These tracking commands sent
to external axes are recorded by the controller. At the second
pass, the laser welding head is at first manually moved back to
a fixed distance such that the attached look-ahead laser sensor
is shifted to the place of TCP. The robot then follows the pre-
taught path and the recorded tracking commands are played
back at their corresponding time stamps. The newly sensed TCP
offset-errors are recorded and saved as a compensation table. At
the third pass, the look-ahead laser sensor is manually moved
forward to its original place, and a real-time tracking with actual
welding is carried out. The robot follows the pre-taught path;
Fig. 3. Two schemes for robot tracking control: (a) input shaping scheme; (b)
iterative learning control scheme.
meanwhile, TCP offset-errors interpolated from the recorded
compensation table at the second pass are superimposed to the
tracking commands of the external axes. In the experiment, the
first two passes can also be run multiple times to obtain averaged
results. Note that those runs with large deviations to averaged
results are discarded as outliers. Additionally for a batch welding
with steady part fixtures, the first two passes are only needed for
a nominal path. The control scheme of SRI can be summarized
as ‘‘compensation of the small residuals left by tracking large
deviations’’.
Although SRI’s feedforward compensation approach can effec-
tively decrease the tracking errors, the residual errors are still no-
table, as can be seen from Fig. 4. In the following two subsections,
we will reveal the causes of such errors and present corresponding
solutions.
3.3. Optimized feedforward compensation
A major cause is that the dynamic cross-coupling effect
between the external axes and the industrial robot is not
ignorable at the place where the Cartesian stiffness of the robot
is low. To some extent, the dynamic cross-coupling effect in the
macro–micro-robot system can be approximated as a dynamic
stiffness model shown in the right part of Fig. 1. The motion
command u of the external axis induces a reaction force to the
robot end flange, and leads to a deflection y of the end flange. Thus
the final position of the end-effector sums results in u’.
There are two common schemes for dealing with the cross-
coupling effect. Fig. 3(a) shows a scheme where an input shaper
is used to suppress the vibration induced by the cross-coupling.
Normally classic input shapers like ZV, ZVD or EI filter [14] can
be used to suppress the vibration y, but unfortunately the desired
input w is also modified and the final position u′
will not match
the desired input. To deal with the dilemma, Kamel et al. recently
proposed a new input shaping algorithm for a different application,
in which a robot with a flexible torque sensor between the robot
end flange and the end-effector is controlled to track a moving
target [14]. They formulated the problem as a minimization of the
tracking residual r subject to constraint equations for computing
input shaping parameters. The benefit of such an algorithm is
that it can be applied online. Nevertheless, both filter length
and a weighting matrix for input shaping parameters have to
be adjusted in order to tune the tracking performance. Note
that the input shaping methods approximate a dynamic system
using transfer functions of the second order. However this may
not be suitable for high-precision applications, considering that
nonlinear factors like backlashes and frictions are hard to model,
4. Z. Luo et al. / Journal of Manufacturing Systems 31 (2012) 2–7 5
and this renders the above optimization approach to be quite
challenging.
The second scheme shown in Fig. 3(b) is based on the
iterative learning control (ILC) principle. The ILC scheme has been
intensively studied in the field of robot control, and is often used
in robot tasks when the desired output is a priori known and
the task is repeatedly executed. ILC algorithms normally take the
form:
ui+1(t) = ui(t) + g(ei(t), ˙ei(t)) 0 ≤ t ≤ T (1)
where T is the total time for a run, ui(t) and ui+1(t) are inputs
applied at the ith iteration and (i + 1)th iteration respectively,
ei(t) is the error obtained at the ith iteration, and g(·) is the
learning function. Note that for typical ILC algorithms, the error
will converge to zero but not necessarily monotonically. Therefore
it is often desirable to use intelligent and enhanced algorithms to
reduce the number of iterations [15]. Among others, An et al. [16]
proposed a Proportional–Differential-type ILC formulation for
MIMO system as follows:
ui+1(t) = ui(t) +
1
Ts
G(t)ei(t) + G(t)˙ei(t) (2)
where Ts is the sampling time, and the learning matrix G(t)satisfies
the following constraint:
max
k
ρ
I +
1
Ts
B(k)G(k)C(k + 1)−1
< 1 (0 ≤ k ≤ N − 1)
where B(k) and C(k) are linearized input and output transmission
matrices respectively, at the kth sampling time, ρ indicates the
matrix spectrum, and N is the number of samplings to constitute
a prescribed path of a nonlinear time-variant system. It is proved
that the errors at all sampling time will converge to zero even
when the initial error at time t = 0 is not zero [16]. Nevertheless
this formulation requires that the error signal is smooth and
differentiable.
In our application, the compensation table is the input for using
iterative learning. For each modified compensation table, the third
pass is run multiple times to get an averaged residual curve. A
modified Proportional-type learning function for our SISO system
is utilized as follows:
ui+1(t) = ui(t) + p(t)ei(t − τ) (3)
where τ is the small lead time, and p(t) is the learning gain at time
t. The above equation can be expressed in the frequency domain
as:
Ui+1(s) = Ui(s) + P(s)Ei(s) exp(τs). (4)
Expanding exp(τs) as Maclaurin series and considering that τ
is normally in an order of 0.01 s, one yields:
exp(τs) = 1 +
1
1!
τs +
1
2!
τ2
s2
+
1
3!
τ3
s3
+ · · ·
≈ 1 + τs. (5)
Hence, Eq. (4) can be written as
Ui+1(s) ≈ Ui(s) + P(s)Ei(s)(1 + τs). (6)
Transforming Eq. (6) back to time domain gives
ui+1(t) ≈ ui(t) + P(t)ei(t) + τP(t)˙ei(t). (7)
This indicates that Eq. (3) is almost identical to Eq. (2), except
for several minor differences. First, τ is not limited to the sampling
time of the control system, and one can take into account the time
delay for the external axes to move to the commanded position.
Second, Eq. (3) does not require the error signal to be smooth since
the derivative of ei(t) is computed implicitly. Note that a low-pass
Fig. 4. An example of the evolution of the compensation table based on iterative
learning.
filter may be applied to the second term in Eq. (3) to smooth out
both the sensor noises and the numerical noises introduced from
higher-order terms in Eq. (5).
In practice, at each learning stage, multiple values of τ and
p(t) are selected for evaluation; these values may be different
for different positions of a seam. Normally, p(t) is chosen to be
around 0.6, and τ around 15–20 ms based on experience. Bootstrap
techniques [17] are also tested to decrease the number of τ and
p(t) to be evaluated. According to our experiments, convergence
is normally obtained after four iterations. Fig. 4 illustrates an
example of the evolution of the compensation table (abbreviated as
CT) based on iterative learning. For clarity, only the middle portion
of the seam which has most of the path shifts is displayed.
It can be seen that when it is near positions 800 and 1080 mm,
there are self-excited vibrations with the first compensation table
applied. However the compensation table obtained after a few
iterations can be used to suppress vibrations at these positions, as
will be demonstrated in Fig. 7.
The optimized compensation table and the tracking commands
recorded in the first pass of SRI’s three-pass procedure can be
superimposed to obtain a new compensation table termed the total
compensation table as in Fig. 5. Note that the total compensation
table obtained for a linear seam is close to the robot path curve
shown in Fig. 2. It is almost independent of the tracking algorithm
used in the three-pass procedure, and can be applied alone to
compensate a nominal seam without adding a tracking. However
it cannot be combined directly with existing tracking algorithms
for batch processing since robot path variation and seam variation
have to be properly separated from the data measured by the look-
ahead sensor.
3.4. Predictive tracking combined with feedforward compensation
There are two additional causes which lead to the notable
residual errors. First, the variations of robot paths from pass to
pass due to path repeatability, macro–micro-cross-coupling and
external disturbances should be considered. A robot path variation
will be mistaken by the laser sensor as a change of seam position,
even though the seam is indeed fixed. Second, in our system the
feedforward compensation data is constructed according to the
estimated elapsed distance that may be imprecise since the robot
path shifts can vary slightly from pass to pass. Robot controllers
with a fast position feedback may improve performance though.
5. 6 Z. Luo et al. / Journal of Manufacturing Systems 31 (2012) 2–7
To solve these issues, a comprehensive architecture is presented. A
look-ahead laser sensor and a TCP co-axial sensor are installed onto
the end-effector, i.e. a laser welding head, with their coordinate
frames being parallel.
Fig. 5 illustrates the new architecture and its working flow. In
the figure, X denotes the elapsed distance along the seam, and Y
denotes the axis perpendicular to the seam and the tool axis. Note
that the Y notation can be readily extended to the vertical transla-
tion and the rotation of the tool axis (normally denoted as Z-axis).
In the architecture, two data tables are constructed before the
final welding pass. In the lower block of the figure, the ‘‘TCP
(total compensation table)’’ is mentioned in Section 3.3. In the
upper tracking block, the table of ‘‘seam sensor reference value’’
is constructed through a no-welding pass. For generality, the first
seam to be welded is chosen as a nominal seam. At each position
along the nominal seam, the seam offset is measured in the
local coordinate of the look-ahead sensor, and the simultaneously
measured TCP offset by the TCP co-axial sensor is subtracted
from the above value to obtain a reference value YRef. Since both
sensors are fixed to the end-effector, YRef at each seam position is
independent of robot path shifts and external axes motions. One
may assume that YRef is what the look-ahead sensor will see when
the robot TCP is precisely on the seam.
In the figure, t denotes the elapsed time, V denotes the TCP
speed of the robot, and Xt denotes the elapsed distance from the
starting point of the seam at time t. Xt is computed by integration
of V with respect to time t. YL denotes the seam position measured
in the local coordinate of the look-ahead sensor, YE denotes the
encoder feedback of the external Y-axis actuator, and YAbs is the
sum of YL and YE . Note that YAbs is independent of external axes
motions, since a positional change of the external axis will produce
an opposite signal change in the look-ahead sensor.
The new control scheme can be summarized as ‘‘compensating
large deviations to a nominal seam, and then predicatively tracking
small variances’’. Its working flow will be explained as follows.
At time t, the elapsed distance Xt is first integrated. The
TCP compensation command YTC is extracted from the ‘‘TCP
(total compensation table)’’ based on Xt and ∆XD via look-up
and interpolation. Note that ∆XD is initially zero. Similarly the
reference value YRef is extracted from the table of ‘‘seam sensor
reference value’’. YAbs and ∆YS can be also easily computed.
Suppose a feedforward compensation using the total compen-
sation table is applied to the nominal seam, the difference between
YAbs and YRef should be identical to zero if the following two varia-
tions are zero. They are the variation of robot path with respect to
the averaged robot path, and the variation between the seam to be
welded and the nominal seam. Even if the two variations are not
zero in reality, their amplitudes are normally quite small and can
be handled by a predictive tracking algorithm explained as follows.
Data set ∆YS is queued to a buffer and the queue is smoothed by
a filter to obtain data set ∆YF . If it changes suddenly, the robot path
repeatability is deteriorating. A predictive algorithm conjectures
that the robot shifts its path earlier or later by ∆T, and then assign
V∗
∆T to ∆XD and sends the value to the compensation block in
order to shift the compensation data correspondingly. Predictive
command ∆YP is simply a modification of ∆YF to remove the rapid
changed outlier, and it will be sent to the external actuator after the
look-ahead distance is passed.
The final command (YMotor) sent to the Y-axis external actuator
is the sum of feedforward compensation command (YTC ) and the
predictive tracking commands (∆YP ).
Fig. 5. Working flow with feedforward compensation and predictive tracking.
Fig. 6. (a) The robotic seam tracking system. (b) Laser welding results of a nonlinear
seam.
4. Prototype system and experiment results
Fig. 6(a) shows the robotic seam tracking system for Nd:YAG
laser welding of tailored blanks. In the system, an ABB IRB4400
robot with a load capacity of 40 kg is employed. At least one
external axis is installed to the end-effector of the robot. The
external Y-axis actuator provides a travel range of 30 mm, a load
capacity of 30 kg, and positional accuracy of 0.003 mm.
The laser sensor purchased from SRI can output data at
200 Hz to SRI’s proprietary control system, but cannot output
quickly enough to other controllers. In order to detect rapid
path shift another self-made laser camera with an image rate
of 75 Hz is installed, and the look-ahead distance is 30 mm.
Moderate reductions on TCP offset-errors are obtained. Fig. 6(b)
demonstrates the welding results on a nonlinear seam formed by
a pair of tailored blanks. The welding speed is 80 mm/s for the
curved portion and 100 mm/s for the linear portion. For nonlinear
seams, robot path shifts may be not quite notable provided that
the fifth axis of the robot does not change its rotational direction
during the welding.
Three figures in Fig. 7 illustrate three typical curves of the TCP
offset-errors resulted from tracking the seam in the way as in
Fig. 1. The first figure was recorded in two runs out of 100 runs,
the third figure was recorded in ten runs out of 100 runs, while
the left runs are close to the second figure or the third figure. The
average tracking error is with ±0.1 mm, and the maximum error
is about 0.15 mm. Note that vibrations at position 800 mm can
6. Z. Luo et al. / Journal of Manufacturing Systems 31 (2012) 2–7 7
Fig. 7. Typical TCP offset-errors for tracking of linear seams.
be suppressed in most cases. Nevertheless vibrations at position
1080 mm may be still excited, and large path shifts at position
700 mm that requires a faster detection using laser sensors or
accelerometers.
5. Conclusions
In this paper, a novel architecture is presented for the high-
precision robotic seam tracking using an industrial robot and off-
the-shelf sensors. Its working principle, operational procedures,
and control algorithms are explained in detail. The experiments
demonstrate that the new architecture outperforms most classic
seam tracking methods and meets demanding requirements of
high-precision industrial laser welding applications. It is expected
that when provided with a fast laser sensor for detecting bursting-
type path shifts, predictive tracking algorithms could lead to
a superb performance. The new algorithm and procedure are
expected to have a significant use in the seam tracking to reduce
the errors and increase the repeatability.
Acknowledgment
This work is supported by the National Natural Science
Foundation of China under the grant number 50705095.
References
[1] Case AW, Ringwall CG. Microvector control for edge and joint following. US
patent 4542279. 1983.
[2] Lange F, Hirzinger G. Predictive visual tracking of lines by industrial robots. Int
J Robot Res 2003;22(10–11):889–903.
[3] Luo ZJ, Tian YL. Device and mthod for high precision tracking of specific path
by robots. PCT patent CN2008071687. 2008.
[4] Taft JD, Ellison JF. Parametric path modeling for an optical automation seam
tracker and real time robot control system. US patent 4965499. 1988.
[5] Nayak N, Ray A. Intelligent seam tracking for robotic welding. Springer-Verlag;
1993.
[6] Sung K, Lee H, Choi YS. Development of a multiline laser vision sensor for joint
tracking in welding. Weld J 2009;(Supplement):79–85.
[7] Zhou L, Lin T, Chen SB. Autonomous acquisition of seam coordinates for arc
welding robot based on visual servoing. J Intell Rob Syst 2006;47:239–55.
[8] Boillot JP, Villemure D. Robot feature tracking devices and methods. US patent
6430472. 1999.
[9] Wang W, Zou QS, Zhu LM, Song GJ. The development state and implementing
method of seam tracking system with optical sensor. Chinese J Electr Weld
Machine 2002;32(5):2–8.
[10] Graff M. Sensor-guided robotic laser welding. Ph.D. thesis. University of
Twente. 2007.
[11] Xu D, Wang L, Tan M. Image processing and visual control method for arc
welding robot. In: IEEE int. conf. on robotics and biomimetics. 2004. p. 1–6.
[12] Hakvoort W. Iterative learning control for LTV systems with applications to an
industrial robot. Ph.D. thesis. University of Twente. 2005.
[13] Bauchspiess A, Alfaro SC. Predictive sensor guided robotic manipulators in
automated welding cells. J Mater Process Technol 2001;109(1–2):13–9.
[14] Kamel A, Lange F, Hirzinger G. An industrial-robots suited input shaping
control scheme. In: The 9th int. conf. on motion and vibration control. MOVIC.
2008.
[15] Gopinath S, Kar IN. Experience inclusion in iterative learning controllers: fuzzy
model based approaches. Eng Appl Artif Intell 2008;21:578–90.
[16] An G, Zhang L, Liu JT. An iterative learning control algorithm for robot
trajectory tracking. J Robot (China) 2001;23(1):36–9.
[17] Luo ZJ, Dai J. Pattened bootstrap: a new method that gives efficiency for some
precision position synthesis problems. Trans ASME, J Mech Des 2007;129:
173–83.