Machine learn models efficiently under budget constraints to adapt to perturbations such as environmental changes or changes in the internal resources. Modern software-intensive systems are composed of components that are likely to change their behaviour over time (e.g., adding/removing components).  For software to continue to operate under such changes, the assumptions about parts of the system made at design time may not hold at runtime due to uncertainty. Mechanisms must be put in place that can dynamically learn new models of these assumptions and use them to make decisions about missions, configurations, etc.

1. Integrated Model Discovery and
Self-Adaptation of Robots
BRASS CMU MARS
Model-based Adaptation
for Robotic Systems
DARPA BRASS
Platform Demonstration Workshop
July 30, 2018
School of Computer Science
Pooyan Jamshidi, Christian Kästner, Javier Cámara,
Bradley Schmerl, Miguel Velez

2. Big Picture: Model-Based Adaptation
2
Models of Intent
I = f(A,T,S)
Mission intent
Design properties
Design structure
Architect
specifies
Robot
System
implementation
writes
discovery
Impact of
adaptations
Adaptation space
Evaluation of
adaptations
Adaptation
planner
:
-
)
Adapted
system
Adaptations

3. Big Picture: Model-Based Adaptation
3
Models of Intent
I = f(A,T,S)
Mission intent
Design properties
Design structure
Architect
specifies
Robot
System
implementation
writes
discovery
Impact of
adaptations
Adaptation space
Evaluation of
adaptations
Adaptation
planner
:
-
)
Adapted
system
Adaptations
CP1: Model
discovery
CP2: Code-
level
adaptation
CP3:
Architectural
adaptation

4. Challenge Problem
• Machine learn models efficiently under budget constraints
to adapt to perturbations such as environmental changes
or changes in the internal resources.
• Modern software-intensive systems are composed of
components that are likely to change their behavior over
time (e.g., adding/removing components).
• For software to continue to operate under such changes,
the assumptions about parts of the system made at design
time may not hold at runtime due to uncertainty.
• Mechanisms must be put in place that can dynamically
learn new models of these assumptions and use them to
make decisions about missions, configurations, etc.
4

5. Robots as highly-configurable systems
5
Many components will be
added over 100 years
We need to scale to large
configuration spaces

6. Research Questions
• RQ1: How models that have been learned under budget
constraint improve the quality of missions?
• RQ2: How can learning generate better adaptation strategies
that impact the satisfaction of mission goals and qualities?
6

7. Approach: Model Discovery and Adaptation
Key idea: Learning models and use it for adaptation
• Configuration adaptation:
1. Add a model of power usage
2. Monitor power state of robot
3. Analyze power as part of plan feasibility
4. Plan with an additional adaptation option
• Reconfigure to a conservative configuration
• Recharge at a power station
• Reroute
5. Execute the plan that is evaluated to be best
7
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
Benefits
Learning that scales
Planner automatically
handles unanticipated
combinations of obstacle,
power adaptations
Optimized path planning

8. Demo
8
https://youtu.be/ec6BhQp2T0Q
Integrated Model Discovery and
Self-Adaptation of Robots

9. Integrated Model Discovery and Adaptation
9
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
Benefits
Learning that scales
Planner automatically
handles unanticipated
combinations of obstacle,
power adaptations
Optimized path planning
Accuracy
Architecture TaskConsider
adaptations to
configuration
Consider effect
of adaptation
on power
consumption

10. Model Discovery
10
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
Innovative aspects:
• Learning under budget constraint
• Restricting the possible configurations for scaling adaptation

11. Model Discovery
11
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
Data
Innovative aspects:
• Learning under budget constraint
• Restricting the possible configurations for scaling adaptation

12. Test Environment
13
Independent evaluation by a third-party (MIT Lincoln Lab)

13. Integration with the Test Harness
14

14. Evaluation Results
• After the bug fixes, the invalid tests are now down less
than 15% of the overall tests ran so far.
• In 98% of the valid tests, the robot could either fully
recover the intent (pass) or partially recover the intent
(degraded).
• Only 2% of the results the robot failed, which means that
it could not recover the intent.
15

15. Evaluation Results
• High test coverage
• Enabled additional
introspection and analysis
• Adaptations recovers intent
in most tests
16

16. Limitations
• Adaptation planning rely on model checking which does
not scale to large number of configurations.
• We select Pareto optimal solutions out of millions of
possible configurations which may not necessarily the
most accurate ones.
• Inaccurate predictions may lead to misleading adaptations
• Go to charging that may not necessary
• Reconfigure to a bad configuration
• We didn’t consider reconfiguration cost
17

17. Checkout our Open Source Software
18

18. Checkout our Open Source Software
19https://github.com/search?q=topic%3Acp1+org%3Acmu-mars+fork%3Atrue

19. TECHNICAL RESEARCH
20

20. Transfer Learning for Model Discovery
21
ML Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Reason
Transfer learning combines:
• Lots of data gathered cheaply from the simulator
• With much less data gathered expensively from the robot
We will learn models based on previous runs, e.g., early tasks in the
missions to learn models and use for adaptation for the latest tasks

21. Intuition behind our transfer learning
22
xf
So, the kernel helps to get accurate predictions for s
configurations. We now need to exploit the relationsh
tween the source and target functions, g, f, using the c
observations Ds, Dt to build the predictive model ˆf. To c
the relationship, we define the following kernel functio
k(f, g, x, x0
) = kt(f, g) ⇥ kxx(x, x0
),
where the kernels kt represent the correlation between s
and target function, while kxx is the covariance functio
inputs. Typically, kxx is parameterized and its paramete
learnt by maximizing the marginal likelihood of the
given the observations from source and target D = Ds
ConfigurationsModels
Performance
Intuition: Observations on the
source(s) can affect predictions
on the target
Example: Learning the chess
game make learning the Go
game a lot easier!

22. Results: Making Quality Tradeoff at Runtime
Energy
constraint
Safety
constraint
Pareto
front
Sweet
Spot
better
better
no_of_particles=x
no_of_refinement=y

23. Model Discovery: Transfer Learning Results
24
Approximation
(from simulation)
Ground Truth
(from robot)
Using only a
few real data
points to
predict yields
poor results
Using transfer
learning to
combine the few
real data points
with lots of
approximate data
yields a good
model

24. PHASE III
25

25. Offline Learning (Traditional Learning)
26
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
RuntimeOffline
Learning models offline
Phase II:
Using models for adaptations
online
Model
Data
Learning

26. Transfer Learning
27
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
Model
RuntimeOffline
Benefits:
Increase prediction accuracyModel
Source
Data Data
Learning
Transfer
Learning
Target
Increase model reliability
Decrease learning cost

27. Phase III: Online (Transfer) Learning
28
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
Model
RuntimeOffline
Learning speed improvements
Benefits:
Model
Source
Data Data
Learning
Transfer
Learning
Target
Jumpstart improvements

28. Phase III: Continual Learning
29
Model(s)
Environment
Monitoring
(+power)
Analysis
(+power)
Planning
(+recharge)
Execution
Power
Model
Runtime
Experience is continually
reused
Benefits:
Model
Source
Data
Transfer
Learning
Target
Data
Data
Data