The document discusses the research focus of the TAO group at Inria Saclay, which includes machine learning and optimization applications for energy management. The group has one permanent member and others part-time. It collaborates closely with partners in Taiwan and the company Artelys. The research aims to address challenges in power grid simulation like variable demand and renewable energy sources using techniques like mathematical programming, reinforcement learning, and direct policy search combined with heuristics and Monte Carlo tree search.

1. TAO - Inria Saclay-IDF
● Machine Learning & Optimization
● Tao-uctsig:
● Sequential decision making
● One permanent, full time + others part time
● Applications to energy management
● Strong collaboration with
● Taiwan
● Artelys (Ilab Metis; joint software)
● Others
O. Teytaud, Research Fellow,
olivier.teytaud@inria.fr
http://www.lri.fr/~teytaud/

2. Power systems, high scale
Production
Network
Feedback mecanism
(smart grids)
em and
D

3. For choosing investments we want
to simulate systems
● Difficulties:
● Demand varying in time, bounded prediction
● Transportation introduces constraints
● Renewable ==> variability ++
● Problems:
● Limited previsibility has an impact ==> anticipative high-level
techniques underestimate the need for storage / smoothing
● Markovian assumptions ==> wrong
● A system which neglects “base ≠ peak” can not be used.
==> Model error >> optimization error
==> Machine Learning on top of Math. Programming

4. Math programming and machine
learning
● Math programming:
● Nearly exact solutions
● High-dimensional constrained action space
● But small state space & not anytime
● Reinforcement learning
● Unstable
● Small / simple action space
● But high dimensional state space & anytime

5. Stochastic dyn. Programming
Huge computation time
Assumes Markovian
Models.
Neglects non-linearities.
● Step 1: compute Bellman's function:
Can work with huge constrained
● Step 2: make decisions: action space

6. Direct Policy Search
● Define a parametric function
● Neural network
● Handcrafted function
● Non-linear optimization
● The best θ is the one which performs best on simulations
==> obtained by non-linear stochastic optimization
● Non-linearities ok, arbitrary stochastic process, large state
space ==> little model bias
● No solution for huge constrained action spaces

7. Math prog & reinforcement learning
● Here, we consider “math prog = heuristic”,
because it's fast but with strong model bias
● Proposals:
DPS-style:
● MCTS (Monte-Carlo Tree Search) + heuristic Little model bias,
arbitrary random
● DPS (Direct Policy Search) + heuristic process, large
state space
● Example Bellman-style;
ok for
large constrained
action spaces Non-linear ~ (θ, xt)
Linear ~ x(t+1)

8. Works in Tao
● Noisy non-linear optimization
● Fabian's algorithm
● Anytime properties (for bilevel problems)
● Evolutionary algorithms
● Reinforcement learning
● MCTS (Monte Carlo Tree Search) on top of heuristics
● DPS (combined with MCTS or heuristics)
● Links with Artelys:
● Joint software
● Experiments
– Non-anticipativity
– Non-linearities