Power systemsilablri

Ilab METIS
Optimization of Energy Policies
Olivier Teytaud + Inria-Tao + Artelys
TAO project-team
INRIA Saclay Île-de-France
O. Teytaud, Research Fellow,
olivier.teytaud@inria.fr
http://www.lri.fr/~teytaud/

Outline

●

●

Ilab Metis, working on power systems
Principles of power system simulation
● Classical model
● Reserves constraint
● Recourse
● Network constraints
● This simple model does not solve hydro
● It gets more complicated in the future

Alternate tools:
● Real-time control ?
● Games ?
● Bilevel problems

●

Ilab METIS
www.lri.fr/~teytaud/metis.html
●

Metis = Tao + Artelys
TAO tao.lri.fr, Machine Learning & Optimization
● Joint INRIA / CNRS / Univ. Paris-Sud team
● 12 researchers, 17 PhDs, 3 post-docs, 3 engineers
● Artelys www.artelys.com SME
- France / US / Canada
- 50 persons
==> collaboration through common platform
●

●

Activities
● Optimization (uncertainties, sequential)
● Application to power systems
O. Teytaud, Research Fellow,
olivier.teytaud@inria.fr
http://www.lri.fr/~teytaud/

Academic Highlights
●

●

Bandit algorithms
Pascal / Nips 06 challenge on dynamic bandits
MCTS for Games
●

First ever win against professional
players in the game of Go

●
●

●

State of the art in MineSweeper
Urban Rivals AI (17 M players)

Planning
Temporal satisficing track, IPC 2011
(International Planning Competition)

Industrial application
●

●

Building power systems is expensive
power plants, HVDC links, networks...
Non trivial planning questions
●

●

●

Compromise: should we move solar power to the
south and build networks ?
Is a HVDC connection a good idea between X and
Y?

What we do:
●

●

Simulate the operational level of a given power
system (this involves optimization of operational
decisions)
Optimize the investments

Specialization on Power Systems
●

Planning/control
●

Pluriannual planning: evaluate marginal costs of hydroelectricity

●

Taking into account stochasticity and uncertainties
==> IOMCA (ANR)

●

High scale investment studies (e.g. Europe+North Africa)
●

Long term (2030 - 2050)

●

Huge (non-stochastic) uncertainties

●

Investments: interconnections, storage, smart grids, power plants...
==> POST (ADEME)

●

Moderate scale (Cities, Factories)
●

Master plan optimization

●

Stochastic uncertainties
==> Citines project (FP7)

Example: interconnection studies
(demand levelling, stabilized supply)

The POST project – supergrids
simulation and optimization
Mature technology:HVDC links
(high-voltage direct current)

European subregions:
- Case 1 : electric corridor France / Spain / Marocco
- Case 2 : south-west
(France/Spain/Italiy/Tunisia/Marocco)
- Case 3 : maghreb – Central West Europe
==> towards a European supergrid

Related
ideas in Asia

Investment decisions through simulations
●

Issues
–
–
–

●

Methods
–
–

●

Demand varying in time, limited previsibility
Transportation introduces constraints
Renewable ==> variability ++
Markovian assumptions ==> wrong
Simplified models ==> Model error >> optimization error

Our approach
●

Machine Learning on top of Mathematical Programming

Hybridization reinforcement learning /
mathematical programming
●

Math programming
–
–
–

●

Nearly exact solutions for a simplified problem
High-dimensional constrained action space
But small state space & not anytime

Reinforcement learning
–
–
–
–

Unstable
Small model bias
Small / simple action space
But high dimensional state space & anytime

●

Students + Engineers
●

●

J-J Christophe (engineer)

●

A. Couëtoux (ph.D. Student)

●

M-L Cauwet (ph.D. Student)

●

J. Decock (ph.D. student)

●

J. Liu (ph.D. Student)

●

●

S. Astete Morales (ph.D. student)

D. Lupien Saint-Pierre (post-doc)

Old crocs:
●

M. Schoenauer & M. Sebag (Research Directors)

●

●

Who ?

O. Teytaud (Research Fellow)

Artelys www.artelys.com (most with Ph.D.)
==> many engineers and experts of power systems

●

Admin. side: A.-C. Lamballe, R. Ronchaud, M. Gilliaud

●

Many contacts worldwide

Outline

●

●

● Classical model
● Recourse
● Future is (even) more complicated

Alternate tools:
● Games ?

●

The problem in one slide
●

The assets:
●

●
●

●

●

I have plenty of (possibly faulty) power plants (each
of them located on a network)
I have plenty of (stochastic) electric loads
Total production = total consumption + loss
(otherwise many things broken!)
Many constraints on the network

Decisions:
●
●

●

Strategic decisions (investments)
Intermediate decisions: Hydro-power = bounded
resource
Tactical decisions: which power plant to switch
on/off

Let us start with short term
●

Thermal power plant producing p MW during
1h:
productionCost
=Cxp

(fuel cost)

+ startUpCost (if it was off and p>0)
+ C2 x p2 -

(a quadratic term for fun)

Several time steps, several thermal
power plants
pi MW during 1h with power plant #i:
productionCost
= C x pi
+ startUpCosti
+ C2 x pi2 ==> ok, looks simple: we just optimize the sum of
costs.


●

●

● Classical model
● Reserves constraint (for adaptivity
against uncertain load, faulty power
plants, renewable intermittencies) ==>
rules of thumb such as “short term PP>x
MW” etc
● Recourse
This simple model does not solve hydro
●

Alternate tools:

●

●
●
●

Real-time control ?
Games ?
Bilevel problems


●

●

● Classical model
● Recourse (==> when big change, reoptimize with new data ==> fast UC)
●

Alternate tools:

●

●
●
●

Real-time control ?
Games ?
Bilevel problems


●

●

● Classical model
● Recourse
● Network constraints (“N-k” style: should
be ok if k connections destroyed)
●

Alternate tools:

●

●
●
●

Real-time control ?
Games ?
Bilevel problems

Let us see why it is more
complicated than that
●

This optimization problem is termed “Unit
Commitment” (UC).

●

There is a problem with hydro-power.

●

Inflows are very stochastic

●

Water comes for free (nearly), but it is limited.
==> more subtle optimization here

At least this uncertainty must be tackled

How to handle hydro ?
●

●

For the moment, we have seen deterministic
optimization problems.
Solution 1:
●

●

●

●

Just optimize, assuming that inflows = average
inflow value given observed information.
This is so-called “Model predictive control” (many
variants, e.g. using a quantile of the inflow random
process).
Not satisfactory if high stochasticity

Solution 2: optimize a policy, not
decisions.

Bellman's solution
●
●

Let's work on time steps [1,....,365x6] (6 years)
Define V(x,t) = expected cost between time t
and T=365*6 if
●

water stock 1 = x1,

●

water stock 2 = x2,

●

…

●

Water stock d = xd

●

V(x,T) = easy to compute

●

Given V(x,t),

V(x,t-1) computed by UC

V(x,t) computed Bellman-style
●

V(x,t+1) is known

●

V(x,t) = optimal expected cost to go
= minimum of:
instantaneous cost Ct(a)
+ future cost

V(x',t+1)

where x'=future state when
choosing action a
It is stochastic! ==> expected value.

So far so good
●

●

●

But V(x,t) is not correctly defined.
It also depends on random processes (load,
inflows...) ==> V(random process,x,t)
It also depends on PP status (on / off) ==> we
need additional variables in x, not only the
stock levels.

More complete picture
●
●

Let's work on time steps {1,....,365x6} (6 years)
Define V(a, x,t) = expected cost between time t and
T=365*6 if
●

x = all state variables

●

random processes = a

●

V(a,x,T) = easy to compute

●

Given V(a,x,t),

V(a,x,t-1) computed by UC:

V(a,x,t-1) = min cost(x,decision) + V(a,x',t)
with x' = nextState(a,x,t,decision)

So, Bellman-style:
●

●

●

●

We have plenty of UC problems (one for each
V(a,x,t) to be computed).
Usually people rewrite the optimization
problems as linear problems or quadratic
problems, because otherwise solving is too
slow.
The real problem is usually simplified a lot so
that it can be solved.
The problem becomes more and more
complicated.

Standard tool.
This version is probably run by electricity
producers in your country.
We will now see phenomena which are poorly
handled by this algorithm.

Outline
●

●

● Classical model
● Recourse
● Future is (even) more complicated

Alternate tools:
● Games ?

●

New decision variables
●

Day-ahead adaptivity: some big companies
(loads) accept “adaptivity” contracts:
●

●

●

At 5pm, the producer says “tomorrow, I multiply
prices by 10” (he can do so 22 days per year)
The other days, prices are reduced

With renewable energy, uncertainty increases;
the principle above can be generalized:
●
●

Small time scales, smaller consumers
Other loads: heat networks, fridges, electric
vehicles

Summary: models
●

●

●

●

Simple UC problem = optimizing
deterministically.
Advanced version = (spinning) reserves (this is
also simplified).
Hydro-power (& other stocks!): in Bellman's
solution we must solve plenty of UC problems
and take into accounts stocks; this involves
computing V(a,x,t) backwards (traditionally
Bellman, but you might use your favorite game
algorithm).
Investment problem on top of all this.

Still too simple
●

●

●

Ok, we solve plenty of UC for computing the
hydro power or other stocks.
Theoretically, solves the problem of uncertainty
in inflows
(but very expensive)
Many other uncertainties are not negligible:
●

●

●

PP directly connected to distribution networks:
voltage issues introduce many constraints.
Wind power uncertainty: day-ahead planning often
wrong ==> real-time
Long term: how to get the probability that PV units
become cheaper than XXX in 2050 ?

Real-time control, #1

Some power plants have a real-time control:
Not enough power ==> implies more power output
● Too much power ==> implies less power output
==> “P” (proportional) control
●

This is necessary.
Power plants to not accept frequency change.
So we must regulate frequencies in real-time.

Real-time control #2
Some PPs have a real-time control #1:
●

Additional control for market constraints:
If flow “Spain → France” too high,
then increase power output in Spain (PI control).

Real-time control
Some PPs have a real-time control #1:

●

Additional control for market constraints:
If flow “Spain → France” too high,
then increase power output in Spain (PI control).

Not very satisfactory.
When day-ahead market based on wrong wind forecasts,
the resources management is very bad:
- by law, grid owners (sometimes) must buy wind power
- day ahead market orders already decided
- the grid owner must find people for buying excess
electricity (or destruction!) ==> negative prices

Games ?

●

Looks like a game

●

Uncertainties = weather, load, wind...

●

Actions = switching on/off, which power output

●

Many players; for us, one-player game ==>
maximum social welfare policy
(politicians and
economist will have to find rules for the
political/economical/ecological solution to be
implemented...)

●

Winning = low cost + low pollution

●

●

●

●

Games ?
Consequence
Of
Uncertainties = weather, load, wind...
deregulation
Looks like a game

Actions = switching on/off, which power output
Many players; for us, one-player game ==>
maximum social welfare policy
(politicians and
economist will have to find rules for the
political/economical/ecological solution to be
implemented...)

●

Winning = low cost + low pollution

So the game model for us
Just one player
Who controls all the Pps, all the networks.
He optimizes the social welfare.
At each time step:
- observes the state
(inflows history, stock levels, PP status...)
- makes decisions (switch on/off, produce power)
- gets reward (economy / ecology)

Bilevel problems
Simple version:
I have K possible systems:
● Today's European grid
● The same + a big new HVDC connection
● The same + X MW of wind power
● ...
Which one is best ?
I spend T hours, each simulation of one of them
needs (e.g.) one hour, how do I choose which
system to simulate.
LOOKS LIKE A NICE BANDIT PROBLEM

Non-stationary
bilevel problems
I have K possible systems:
● Today's European grid
● The same + a big new HVDC connection
● The same + X MW of wind power
● ...
In fact, I have to optimize the management of
each system. If I optimize during simulations,
options become better ==> non-stationary
bandit problem.

Power systemsilablri

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (10)

Similar to Power systemsilablri

Similar to Power systemsilablri (20)

Recently uploaded

Recently uploaded (20)

Power systemsilablri