Talk presented at the 2018 machine learning for programming languages workshop colocated with ECOOP in Amsterdam. This work presents the use of reinforcement learning for the discovery of context objects to drive behavioral adaptations. Our technique uses information gathered from the systems environment to define context, associating the actions taken by the user in such situations to define and automate behavioral adaptations

1. Generating Software Adaptations Using
Machine Learning
Nicolás Cardozo Ivana Dusparic
n.cardozo@uniandes.edu.co ivana.dusparic@scss.tcd.ie
@ncardoz @ivanadusparic
2nd International Workshop on Machine Learning techniques for Programming Languages (ML4PL’18) - Amsterdam, the Netherlands

2. Adaptive software systems
2

3. Adaptive software systems
3
Base
application

4. Adaptive software systems
3
A1
Base
application

5. Adaptive software systems
3
A2
A1
Base
application

6. Adaptive software systems
3
A3
A2
A1
Base
application

7. Adapting a cruise control system
4
Speed control
Lane changing
Collision avoidance
Maintain a maximum speed
Switch lanes when possible
Break upon encountering cars
Respect the speed limit

8. Driving assistant with COP
5
Car {
drive : function() {
this.straight();
}
}
Proximity1 = new cop.Context({
Name = “proximity1”
})
Contextt
ProximityAdap = Trait({
drive: function() {
car.slowDown();
car.steerLeft();
car.speedUp();
car.speedUp();
car.steerRight();
}
}); Behavioral
adaptation
Proximity1.adapt(Car, ProximityAdap)
Adaptation

9. Driving assistant with COP
6
Car {
drive : function() {
this.straight();
}
}Proximity sensor
Proximity1.activate()
Context discovery Car {
drive: function() {
this.slowDown();
this.steerLeft();
this.speedUp();
this.speedUp();
this.steerRight();
}
}
Adapted behavior
3m

10. Static adaptation model
7
Car {
drive: function() {
this.slowDown();
this.steerLeft();
this.speedUp();
this.speedUp();
this.steerRight();
}
}
Can adaptations be conceived on the fly?
Adaptation are manually created by
developers, and limited to foreseen
circumstances only
Adaptations are fixed for all
situations/all users, there are no
customizations

11. 8
Enhance the adaptation model
to learn from previous uses of
the system
User actions under
different situations
Using unsupervised
learning

12. ML-based adaptation engine
9

13. ML-based adaptation engine
10
...
left 5 0 speed up
left 10 0 speed up
left 15 0 speed up
left 20 0 speed up
left 25 0 speed up
left 30 0 straight
left 30 0 speed up
left 35 0 speed up
left 40 0 speed up
right 45 0 overtake
left 45 2 speed up
left 50 2 slow down
left 45 1 speed up
right 50 1 overtake
left 50 0 stop
left 0 0 straight
left 0 0 speed up
left 5 0 speed up
…
ActionState
SpeedLimit30 = new cop.context({…})
Car {
drive: function() {
this.speedUp();
this.speedUp();
this.speedUp();
this.speedUp();
this.straight();
}
}
whenever: state becomes: left-5-0
SpeedLimit30.activate()
Adaptation engine

14. % of accidents1
Respect the speed limit
Validation
11
Steps at speed limit2
Avoid accidents

15. 12
Adaptation engine prototype

16. Initial results
13

17. Initial results
14

18. 15
Full adaptation engine
Open issues in ML-generated code
… automating the learning process specification
… integration of new sensors to accommodate
unknown situations

19. Open issues in ML-generated code
16
Impact on Development and Maintenance Cost
… learning, rather than pre-specifying adaptations
reduces development complexity and user effort, but
the use of ML could increase long-term software
maintenance cost

20. 17
Providing behavior guarantees
Open issues in ML-generated code
… difficulty of providing behavioral guarantees, not
just in terms of convergence to optimality, but
more crucially guaranteeing that system will not
significantly deviate from the expected safe
behavior

21. 18
Generating Software Adaptations Using Machine Learning

22. 18
Questions ?
Generating Software Adaptations Using Machine Learning