Modern software systems are now being built to be used in dynamic environments utilizing configuration capabilities to adapt to changes and external uncertainties. In a self-adaptation context, we are often interested in reasoning about the performance of the systems under different configurations. Usually, we learn a black-box model based on real measurements to predict the performance of the system given a specific configuration. However, as modern systems become more complex, there are many configuration parameters that may interact and, therefore, we end up learning an exponentially large configuration space. Naturally, this does not scale when relying on real measurements in the actual changing environment. We propose a different solution: Instead of taking the measurements from the real system, we learn the model using samples from other sources, such as simulators that approximate performance of the real system at low cost.

1. Transfer Learning for Improving Model Predictions
in Robotic Systems
Pooyan Jamshidi, Miguel Velez, Christian Kästner
Norbert Siegmund, Prasad Kawthekar

2. Overview of our approach
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
2

3. Overview of our approach
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
Data
3

4. Overview of our approach
We get more cheaper samples from the simulator and only few expensive from the
real robot to learn an accurate model to predict the performance of the robot.
Here, performance may denote “battery usage”, “localization error” or “mission time”.
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
4

5. Traditional machine learning vs. transfer learning
Source Domain
(cheap)
Target Domain
(expensive)
Learning
Algorithm
Different Domains
Learning
Algorithm
Learning
Algorithm
Learning
Algorithm
Traditional Machine Learning Transfer Learning
Knowledge
Data
Source
Target
5

6. Traditional machine learning vs. transfer learning
Source Domain
(cheap)
Target Domain
(expensive)
Learning
Algorithm
Different Domains
Learning
Algorithm
Learning
Algorithm
Learning
Algorithm
Traditional Machine Learning Transfer Learning
The goal of transfer learning is to improve learning in the target
domain by leveraging knowledge from the source domain.
Knowledge
Data
Source
Target
6

7. Performance
prediction
for CoBot
5 10 15 20 25
5
10
15
20
25
14
16
18
20
22
24
5 10 15 20 25
5
10
15
20
25
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25
5
10
15
20
25
5
10
15
20
25
30
CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
(a) Source response
function (cheap)
(b) Target response
function (expensive)
7

8. Performance
prediction
for CoBot
5 10 15 20 25
5
10
15
20
25
14
16
18
20
22
24
5 10 15 20 25
5
10
15
20
25
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25
5
10
15
20
25
5
10
15
20
25
30
CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
(a) Source response
function (cheap)
(b) Target response
function (expensive)
(c) Prediction without
transfer learning
(d) Prediction with
transfer learning
8

9. Prediction
accuracy and
model reliability
2
2
2
2
2
2
2
2
2
44
4
4
6
6
6
8
8 1012
14 16
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
5
5
5
5
5
5
5
5
5
10
10
10
10
15
15
15
20
20
25
303540
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
(a) (b)
0.5 0.5 0.5
1 1 1
1.5 1.5 1.5
2
2 2
2.5 2.5
2.5
3
3
3
3
3.5
3.5
3.5
3.5
4
4
4
4
4
4.5 4.5
4.5
5
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
0.005 0.005 0.005
0.01 0.01 0.01
0.015 0.015 0.0150.02 0.02 0.02
0.02
0.025
0.025
0.025 0.025
0.025
0.025
0.025
0.03
0.03
0.03
0.03
0.03
0.03
0.03
0.035
0.035
0.035
0.04
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
(c) (d)
(a) Prediction error
• Trade-off among using more
source or target samples
(b) Model reliability
• Transfer learning contributes
to lower the prediction
uncertainty
9

10. Prediction
accuracy and
model reliability
2
2
2
2
2
2
2
2
2
44
4
4
6
6
6
8
8 1012
14 16
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
5
5
5
5
5
5
5
5
5
10
10
10
10
15
15
15
20
20
25
303540
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
(a) (b)
0.5 0.5 0.5
1 1 1
1.5 1.5 1.5
2
2 2
2.5 2.5
2.5
3
3
3
3
3.5
3.5
3.5
3.5
4
4
4
4
4
4.5 4.5
4.5
5
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
0.005 0.005 0.005
0.01 0.01 0.01
0.015 0.015 0.0150.02 0.02 0.02
0.02
0.025
0.025
0.025 0.025
0.025
0.025
0.025
0.03
0.03
0.03
0.03
0.03
0.03
0.03
0.035
0.035
0.035
0.04
1 2 3 4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
(c) (d)
(a) Prediction error
• Trade-off among using more
source or target samples
(b) Model reliability
• Transfer learning contributes
to lower the prediction
uncertainty
(c) Training overhead
(d) Evaluation overhead
• Appropriate for runtime
usage
10

11. Prediction accuracy
• The model provides more
accurate predictions as we
exploit more data
• Transfer learning may (i) boost
initial performance, (ii)
increase the learning speed,
(iii) lead to a more accurate
performance finally
correspondences. In much of the work on transfer learning, a human p
this mapping, but some methods provide ways to perform the mappin
matically. Another section of the chapter discusses work in this area.
performance
training
with transfer
without transfer
higher start
higher slope higher asymptote
Fig. 2. Three ways in which transfer might improve learning.
2
[Lisa Torrey and Jude Shavlik]
11

12. Current priority
We have looked into transfer learning from simulator to real robot, but
now, we consider the following scenarios:
• Workload change (New tasks or missions, new environmental
conditions)
• Infrastructure change (New Intel NUC, new Camera, new Sensors)
• Code change (new versions of ROS, new localization algorithm)
12

13. Backup slides
13

14. Problem-solution overview
• The robotic software is considered as a highly configurable system.
• The configuration of the robot influence the performance (as well as
energy usage).
• Problem: there are many different parameters making the configuration
space highly dimensional and difficult to understand the influence of
configuration on system performance.
• Solution: learn a black-box performance model using measurements from
the robot. performance_value=f(configuration)
• Challenge: Measurements from real robot is expensive (time consuming,
require human resource, risky when fail)
• Our contribution: performing most measurements on simulator (Gazebo)
and taking only few samples from the real robot to learn a reliable and
accurate performance model given a certain experimental budget. 14

15. Synthetic
example
(a) Source and target
samples
(b) Learning without
transfer
(c) Learning with
transfer
(d) Negative transfer
1 3 5 9 10 11 12 13 14
0
50
100
150
200
source samples
target samples
(a) (b)
(c) (d) 15

16. Integrating cost model with transfer learning
1. Model
Learning
2. Predictive
Model
4. Transfer
Learning
3. Cost
Model
Improved
Predictive
Model
Conf.
Parameters
Samples
0 500 1000 1500
Throughput (ops/sec)
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Averagewritelatency(µs)
TL4CO
BO4CO
We have only a
limited
experimental
budget and we
need to spend that
wisely
16

17. Assumption: Source and target are correlated
0 2 4 6 8 10
0
0.5
1
1.5
2
2.5
10
4
0 2 4 6 8 10
0
0.5
1
1.5
2
2.5
3
10
4
0 2 4 6 8 10
0
0.5
1
1.5
2
2.5
10
4
(a) (b) (c)
1.5
2
2.5
3
104
(d)0 2 4 6 8 10
-0.5
0
0.5
1
1.5
2
2.5
10
4
1.5
2
2.5
3
104
1.5
2
2.5
3
104
ntime
it is
cisely
using
ration
i). In
uch as
rithm
binary
There-
tesian
terest
space
R is
ations
✓ X.
noise,
words,
from
c that
s real
dimensional spaces. Relevant to software engineering commu-
nity, several approaches tried different experimental designs
for highly configurable software [14] and some even consider
cost as an explicit factor to determine optimal sampling [26].
Recently, researchers have tried novel way of sampling with
a feedback embedded inside the process where new samples
are derived based on information gain from previous set of
samples. Recursive Random Sampling (RRS) [33] integrates
a restarting mechanism into the random sampling to achieve
high search efficiency. Smart Hill Climbing (SHC) [32] inte-
grates the importance sampling with Latin Hypercube Design
(lhd). SHC estimates the local regression at each potential
region, then it searches toward the steepest descent direction.
An approach based on direct search [35] forms a simplex in
the parameter space by a number of samples, and iteratively
updates a simplex through a number of well-defined operations
including reflection, expansion, and contraction to guide the
sample generation. Quick Optimization via Guessing (QOG)
in [23] speeds up the optimization process exploiting some
heuristics to filter out sub-optimal configurations. Some recent
work [34] exploited a characteristic of the response surface of
the configurable software to learn Fourier sparse functions by
only a small sample size. Another approach also exploited this
fact, but iteratively construct a regression model representing
performance influences in an active learning process.
time constrained environments (e.g., runtime
making in a feedback loop for robots), it is
to select the sources purposefully.
ormulation: Model learning
ntroduce the concepts in our approach precisely
we define the model learning problem using
notation. Let Xi indicate the i-th configuration
hich ranges in a finite domain Dom(Xi). In
ay either indicate (i) an integer variable such as
iterative refinements in a localization algorithm
orical variable such as sensor names or binary
local vs global localization method). There-
figuration space is mathematically a Cartesian
of the domains of the parameters of interest
) ⇥ · · · ⇥ Dom(Xd).
ation x resides in the design parameter space
black-box response function f : X ! R is
a performance model given some observations
performance under different settings, D ✓ X.
hough, such measurements may contain noise,
xi) + ✏i where ✏i ⇠ N (0, i). In other words,
e model is simply a function (mapping) from
space to a measurable performance metric that
val-scaled data (here we assume it produces real
dimensional spaces. Relevant to software engineering commu-
nity, several approaches tried different experimental designs
for highly configurable software [14] and some even consider
cost as an explicit factor to determine optimal sampling [26].
Recently, researchers have tried novel way of sampling with
a feedback embedded inside the process where new samples
are derived based on information gain from previous set of
samples. Recursive Random Sampling (RRS) [33] integrates
a restarting mechanism into the random sampling to achieve
high search efficiency. Smart Hill Climbing (SHC) [32] inte-
grates the importance sampling with Latin Hypercube Design
(lhd). SHC estimates the local regression at each potential
region, then it searches toward the steepest descent direction.
An approach based on direct search [35] forms a simplex in
the parameter space by a number of samples, and iteratively
updates a simplex through a number of well-defined operations
including reflection, expansion, and contraction to guide the
sample generation. Quick Optimization via Guessing (QOG)
in [23] speeds up the optimization process exploiting some
heuristics to filter out sub-optimal configurations. Some recent
work [34] exploited a characteristic of the response surface of
the configurable software to learn Fourier sparse functions by
only a small sample size. Another approach also exploited this
fact, but iteratively construct a regression model representing
performance influences in an active learning process.
in some time constrained environments (e.g., runtime
decision making in a feedback loop for robots), it is
important to select the sources purposefully.
C. Problem formulation: Model learning
In order to introduce the concepts in our approach precisely
and concisely, we define the model learning problem using
mathematical notation. Let Xi indicate the i-th configuration
parameter, which ranges in a finite domain Dom(Xi). In
general, Xi may either indicate (i) an integer variable such as
the number of iterative refinements in a localization algorithm
or (ii) a categorical variable such as sensor names or binary
options (e.g., local vs global localization method). There-
fore, the configuration space is mathematically a Cartesian
product of all of the domains of the parameters of interest
X = Dom(X1) ⇥ · · · ⇥ Dom(Xd).
A configuration x resides in the design parameter space
x 2 X. A black-box response function f : X ! R is
used to build a performance model given some observations
of the system performance under different settings, D ✓ X.
In practice, though, such measurements may contain noise,
i.e., yi = f(xi) + ✏i where ✏i ⇠ N (0, i). In other words,
a performance model is simply a function (mapping) from
configuration space to a measurable performance metric that
produces interval-scaled data (here we assume it produces real
dimensional spaces. Relevant to software engineering commu-
nity, several approaches tried different experimental designs
for highly configurable software [14] and some even consider
cost as an explicit factor to determine optimal sampling [26].
Recently, researchers have tried novel way of sampling with
a feedback embedded inside the process where new samples
are derived based on information gain from previous set of
samples. Recursive Random Sampling (RRS) [33] integrates
a restarting mechanism into the random sampling to achieve
high search efficiency. Smart Hill Climbing (SHC) [32] inte-
grates the importance sampling with Latin Hypercube Design
(lhd). SHC estimates the local regression at each potential
region, then it searches toward the steepest descent direction.
An approach based on direct search [35] forms a simplex in
the parameter space by a number of samples, and iteratively
updates a simplex through a number of well-defined operations
including reflection, expansion, and contraction to guide the
sample generation. Quick Optimization via Guessing (QOG)
in [23] speeds up the optimization process exploiting some
heuristics to filter out sub-optimal configurations. Some recent
work [34] exploited a characteristic of the response surface of
the configurable software to learn Fourier sparse functions by
only a small sample size. Another approach also exploited this
fact, but iteratively construct a regression model representing
performance influences in an active learning process.
in some time constrained environments (e.g., runtime
decision making in a feedback loop for robots), it is
important to select the sources purposefully.
C. Problem formulation: Model learning
In order to introduce the concepts in our approach precisely
and concisely, we define the model learning problem using
mathematical notation. Let Xi indicate the i-th configuration
parameter, which ranges in a finite domain Dom(Xi). In
general, Xi may either indicate (i) an integer variable such as
the number of iterative refinements in a localization algorithm
or (ii) a categorical variable such as sensor names or binary
options (e.g., local vs global localization method). There-
fore, the configuration space is mathematically a Cartesian
product of all of the domains of the parameters of interest
X = Dom(X1) ⇥ · · · ⇥ Dom(Xd).
A configuration x resides in the design parameter space
x 2 X. A black-box response function f : X ! R is
used to build a performance model given some observations
of the system performance under different settings, D ✓ X.
In practice, though, such measurements may contain noise,
i.e., yi = f(xi) + ✏i where ✏i ⇠ N (0, i). In other words,
a performance model is simply a function (mapping) from
configuration space to a measurable performance metric that
produces interval-scaled data (here we assume it produces real
dimensional spaces. Relevant to s
nity, several approaches tried dif
for highly configurable software [
cost as an explicit factor to deter
Recently, researchers have tried
a feedback embedded inside the
are derived based on information
samples. Recursive Random Sam
a restarting mechanism into the
high search efficiency. Smart Hil
grates the importance sampling w
(lhd). SHC estimates the local
region, then it searches toward th
An approach based on direct sea
the parameter space by a numbe
updates a simplex through a numb
including reflection, expansion, a
sample generation. Quick Optimi
in [23] speeds up the optimizati
heuristics to filter out sub-optimal
work [34] exploited a characterist
the configurable software to learn
only a small sample size. Another
fact, but iteratively construct a re
performance influences in an acti
a) Source may be a shift to the target
b) Source may be a noisy version of the target
c) Source may be different to target in some extreme positions in the configuration space
d) Source may be irrelevant to the target -> negative transfer will happen!
17

18. Level of correlation between source and
target is important
10
20
30
40
50
60
AbsolutePercentageError[%]
Sources s s1 s2 s3 s4 s5 s6
noise-level 0 5 10 15 20 25 30
corr. coeff. 0.98 0.95 0.89 0.75 0.54 0.34 0.19
µ(pe) 15.34 14.14 17.09 18.71 33.06 40.93 46.75
TABL
colum
datase
measu• Model becomes more accurate
when the source is more
related to the target
• Even learning from a source
with a small correlation is
better than no transfer
18

19. Active learning + transfer learning
Learned function with bad
samples -> overfitting
Goal: find best sample points iteratively by gaining
knowledge from source and target domain
Learned function with
good samples
19

20. Conclusion: Our cost-aware transfer learning
• Improves the model accuracy up to several orders of magnitude
• Is able to trade-off between different number of samples from source
and target enabling a cost-aware model
• Imposes an acceptable model building and evaluation cost making
appropriate for application in the robotics domain
20