The document discusses using transfer learning to improve model predictions for highly configurable software systems. Transfer learning allows data from similar, previously measured systems to be reused to enhance predictions for a new target system with limited data. This improves prediction accuracy while requiring fewer measurements of the target system. The approach was evaluated on autonomous robots and streaming applications and was shown to produce accurate models with few target system samples. Transfer learning could enable self-optimizing software by improving models during each adaptation cycle.

1. Transfer Learning for
Improving Model Predictions
in Highly Configurable Software
Pooyan Jamshidi, Miguel Velez, Christian Kästner,
Norbert Siegmund, Prasad Kawthekar

4. Highly Configurable Systems In Dynamic
Environments
[Credit to CMU CoBot]

5. Options Influence Performance

6. Adapt to Different Environments
TurtleBot

7. Adapt to Different Environments
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot

8. Adapt to Different Environments
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot

9. Classic Sensitivity Analysis
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot

10. Classic Sensitivity Analysis
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure

11. Classic Sensitivity Analysis
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Learn

12. Classic Sensitivity Analysis
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Learn

13. Applications
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Learn
Optimization
Adaptation +
Reasoning +
Debugging +

14. Measuring Performance is Expensive
25 options × 10 values = 1025 configurations
Measure
[Credit to Peng Hou]

15. Transfer Learning

16. Reuse Data From Similar System
TurtleBot
Measure

17. Reuse Data From Similar System
TurtleBot
Measure
Simulator (Gazebo)

18. Reuse Data From Similar System
TurtleBot
Measure
Measure
Simulator (Gazebo)

19. Reuse Data From Similar System
TurtleBot
Measure
Reuse
Measure
Simulator (Gazebo)

20. Reuse Data From Similar System
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Reuse
Measure
Learn
with TL
Simulator (Gazebo)

21. Reuse Data From Similar System
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Reuse
Measure
Learn
with TL
Simulator (Gazebo)

22. Transfer Between Different Systems

23. Transfer Between Different Systems

24. Transfer Between Different Systems

25. Transfer Between Different Systems
(0, 0) → (7, 12)(0, 0) → (0, 10)

26. Exploiting Similarity

27. Function Prediction
1 3 5 9 10 11 12 13 14
0
50
100
150
200
Target function
Target samples

28. Prediction without Transfer Learning

29. Prediction with More Data but without Transfer
Learning

30. Prediction with Transfer Learning

31. Technical Details

32. Prediction With Transfer Learning
true function
GP mean
GP variance
observation
selected point
true
minimum
Motivations:
1. mean estimates + variance
2. all computation are linear algebra
3. good estimations with few data

33. Prediction With Transfer Learning
true function
GP mean
GP variance
observation
selected point
true
minimum
Motivations:
1. mean estimates + variance
2. all computation are linear algebra
3. good estimations when few data
https://github.com/pooyanjamshidi/transferlearning

34. Prediction With Transfer Learning
true function
GP mean
GP variance
observation
selected point
true
minimum
Motivations:
1. mean estimates + variance
2. all computation are linear algebra
3. good estimations when few data
https://github.com/pooyanjamshidi/transferlearning
pjsamshid@cs.cmu.edu

35. Evaluation

36. RQ1: Improve prediction accuracy?
RQ2: Tradeoffs among number of source and
target samples?
RQ3: Fast enough for self-adaptive systems?
Case Study and Controlled Experiments

37. Autonomous service robot
Environmental change
3 stream processing apps
Workload change
NoSQL DBMS
Workload & hardware changes
Analyzed Systems

38. Prediction Accuracy

39. 5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Performance Prediction for CoBot
Source
Model

40. 5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Performance Prediction for CoBot
Source
Model
Target
Model

41. 5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Performance Prediction for CoBot
Source
Model
Target
Model
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Prediction
with 4 samples
CPU usage[%]

42. 5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Performance Prediction for CoBot
Source
Model
Target
Model
5 10 15 20 25
5
10
15
20
25
0
5
10
15
20
25
Prediction
with 4 samples
Prediction
with TL
CPU usage[%] CPU usage[%]

43. Number of Source and Target
Samples

44. Prediction Error with Different Source
and Target Samples
%Source
%Target

45. Prediction Error with Different Source
and Target Samples
%Source
%Target

46. Prediction Error with Different Source
and Target Samples
%Source
%Target

47. Prediction Error with Different Source
and Target Samples
%Source
%Target

48. Prediction Error with Different Source
and Target Samples
%Source
%Target

49. Prediction Error with Different Source
and Target Samples
%Source
%Target

50. Accuracy and Costs
%Source
%Target
B
udget

51. Prediction error of other systems

52. Applicable in Self-Adaptive
Systems

53. Low model training overhead
Produces accurate models
Models can improve at each adaptation cycle
Transfer Learning in Self-Adaptive Systems

54. Future Work, Insights, and
Ideas

55. Selecting from Multiple Sources
Source Simulator Target Simulator
Source Robot Target Robot
(1)
(2)
(3)
(4)
(5)
C1
C2
C3

56. Active Learning with Transfer Learning
50 + 3*C1 + 15*C2 - 7*C2 *C3
TurtleBot
Measure
Reuse
Measure
Learn
with TL
Simulator (Gazebo)
Iteratively find best
sample points that
maximize knowledge

57. Integrating Transfer Learning in MAPE-K
Contribute to Knowledge
Assist in self-optimization
Support online learning

58. Summary

59. Transfer Learning for
Improving Model Prediction
in Highly Configurable Software
Reuse data from similar system
Improves model accuracy
Applicable in self-adaptive systems

60. Transfer Learning Improves Sampling