This document discusses using imitative learning to develop online planning agents for microgrids. It proposes learning optimal sequences of actions for storage systems from linear programming solutions and using these to build a smart planning agent via machine learning. The goal is to automate extraction of planning strategies that can balance storage systems in real-time under uncertainties. Preliminary results show the imitative learning agent outperforms a novice agent in minimizing costs.

1. IMITATIVE LEARNING FOR ONLINE PLANNING IN MICROGRIDS
SAMY AITTAHAR, VINCENT FRANÇOIS-LAVET, STEFAN LODEWEYCKX, DAMIEN ERNST AND RAPHAËL FONTENEAU
UNIVERSITY OF LIÈGE, DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
FULLY OFF-GRID MICROGRID
For each storage systems included in the microgrid :
State system variable Decision making process
Challenge : Real-time balancing of several storage systems under uncertainties.
DATA
• Annual solar irradiance in Belgium ;
• Daily consumption pattern (18 kWh) ;
• Microgrid size.
INCENTIVES
• Load (e.g. house) proximity ;
• Drop in the cost of PV panels.
LINEAR DYNAMICS
Variables
∀t ∈ {1 . . . T}, σ ∈ Σ, T ∈ N :
• aσ,+
t , aσ,−
t → Storage system σ actions ;
• sσ
t = sσ
(t−1) + a−,σ
t−1 + a+,σ
(t−1), sσ
0 = 0 →
Storage system σ state ;
• Ft = dt − σ∈Σ (
a+,σ
t
ησ + ησ
a−,σ
t ) →
Power cut. dt = prodt − const is the net de-
mand. ησ
is storage system σ efficiency.
Objective function (operational costs)
Levelized Cost of Energy :
LEC =
T
t=1
− ψ∈Ψ k
ψ
t F
ψ
t
(1+r)y
+I0
n
y=1
y
(1+r)y
where
• I0 → Initial investment cost ;
• kψ
t Fψ
t → Cost of consumption not met for
load ψ ∈ Ψ at time t ∈ {1 . . . T}.
Linear programming
Minimization of objective function with
contraints related to the dynamics →
Optimization of planning strategies given a
complete scenario.
GOAL
Automated extraction of smart online planning
agents using imitative learning.
IMITATIVE LEARNING
Principle : learning near-optimal behavior with
optimal sequences of actions.
• Compute optimal sequences of actions from
production and consumption scenarios (lin-
ear programming) ;
• Build smart online planning agent with op-
timal sequences (machine learning).
FUTURE WORK
• Benchmarking of others machine learning
structures/algorithms ;
• Testing on others microgrids (e.g. con-
nected on main network) ;
• Transfer learning (i.e. adaptation of an ex-
isting strategy for new microgrids).
RESULTS
Discharge/recharge storage systems in
increasing order of efficiency.
LEC (e/ kWh) :
Expert Agent Novice
0.32 0.42 0.6
LEARNING STRUCTURE
Forest of regression binary trees.