Talk presented at the 2018 machine learning for programming languages workshop colocated with ECOOP in Amsterdam.
This work presents the use of reinforcement learning for the discovery of context objects to drive behavioral adaptations. Our technique uses information gathered from the systems environment to define context, associating the actions taken by the user in such situations to define and automate behavioral adaptations
Generating software adaptations using machine learning
1. Generating Software Adaptations Using
Machine Learning
Nicolás Cardozo Ivana Dusparic
n.cardozo@uniandes.edu.co ivana.dusparic@scss.tcd.ie
@ncardoz @ivanadusparic
2nd International Workshop on Machine Learning techniques for Programming Languages (ML4PL’18) - Amsterdam, the Netherlands
7. Adapting a cruise control system
4
Speed control
Lane changing
Collision avoidance
Maintain a maximum speed
Switch lanes when possible
Break upon encountering cars
Respect the speed limit
13. ML-based adaptation engine
10
...
left 5 0 speed up
left 10 0 speed up
left 15 0 speed up
left 20 0 speed up
left 25 0 speed up
left 30 0 straight
left 30 0 speed up
left 35 0 speed up
left 40 0 speed up
right 45 0 overtake
left 45 2 speed up
left 50 2 slow down
left 45 1 speed up
right 50 1 overtake
left 50 0 stop
left 0 0 straight
left 0 0 speed up
left 5 0 speed up
…
ActionState
SpeedLimit30 = new cop.context({…})
Car {
drive: function() {
this.speedUp();
this.speedUp();
this.speedUp();
this.speedUp();
this.straight();
}
}
whenever: state becomes: left-5-0
SpeedLimit30.activate()
Adaptation engine
18. 15
Full adaptation engine
Open issues in ML-generated code
… automating the learning process specification
… integration of new sensors to accommodate
unknown situations
19. Open issues in ML-generated code
16
Impact on Development and Maintenance Cost
… learning, rather than pre-specifying adaptations
reduces development complexity and user effort, but
the use of ML could increase long-term software
maintenance cost
20. 17
Providing behavior guarantees
Open issues in ML-generated code
… difficulty of providing behavioral guarantees, not
just in terms of convergence to optimality, but
more crucially guaranteeing that system will not
significantly deviate from the expected safe
behavior