Stochastic Co-Optimization of Electric Vehicle Charging and Frequency Regulation
1. Optimal Decision Making for Electric Vehicles
Providing Electric Grid Frequency Regulation:
A Stochastic Dynamic Programming Approach
Jonathan Donadee
Ph.D. Student, ECE, Carnegie Mellon University
jdonadee@andrew.cmu.edu
Marija Ilic
IEEE Fellow, Professor of ECE and EPP, Carnegie Mellon University
milic@ece.cmu.edu
9th International Conference on Computational Management Science
Imperial College London, UK
April 20th, 2012
Support for this research was provided by
Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology)
through the Carnegie Mellon Portugal Program
3. Electrical Grid Frequency Regulation
Power Supply and Demand Balance
Electrical Grid AC frequency
must be maintained within
± 0.05 Hz
60Hz (USA)
50Hz (Europe)
Demand < Supply, f Regulation is on seconds to minutes timescale
Demand > Supply, f
Some generators follow “AGC”
or “Regulation” control signal
on second to minute basis
Generators bid capacity (MW)
into hourly markets
3
Graphic Source: PJM ISO Training
4. One Day’s AGC Signal
1 Day of PJM AGC Signal
1
0.5
Normalized Signal
0
-0.5
-1
-1.5
0 5 10 15 20 25
Hours
4
5. Motivation
In the future, greater need for fast responding
electrical grid resources
Compensate renewable resource forecast errors
Mitigate trend of decreasing system inertia
smaller imbalances causing larger frequency deviations
Integration of Electric Vehicles
Minimize EV charging cost
Deterministic models and methods are not
well suited for managing uncertain resources
5
6. Electric Vehicle Participation in
Frequency Regulation
EVs can adjust charge rate to follow AGC signal
We’ll focus on charging only strategies, no discharging
Specify preferred charge rate, Pavg
Specify capacity for regulating, B
Adjust according to negative of AGC signal scaled by B
0 ≤ 𝐵 ≤ 𝑃𝑎𝑣𝑔
0 ≤ 𝑃𝑎𝑣𝑔 ≤ 𝑃 𝑚𝑎𝑥
0 ≤ 𝐵 ≤ 𝑃 𝑚𝑎𝑥 − 𝑃𝑎𝑣𝑔
6
7. Smart Charging Scenario
Driver arrives at home and plugs in vehicle
Inputs time of departure
Inputs inconvenience cost for not finishing on time ($/hr)
Smart charger optimizes Pavg, B decisions for each hour
Smart charger can participate in markets directly, without
delay
7
Picture Source: Ford.com
8. A Stochastic Model is Needed
Ebatt
Emax
t1 t2 t3 tf State of charge can take
any non-decreasing path
e0
t
Average charge rate to hit Emax at tf
Bound of Possible State of Charge
8
9. A Stochastic Model is Needed
Ebatt Regulation Bid Violated
Emax
t1 t2 t3 tf State of charge can take
any non-decreasing path
Regulation contract is
e0 violated if charging finishes
early
t
Average charge rate to hit Emax at tf
Bound of Possible State of Charge
9
10. A Stochastic Model is Needed
Ebatt Regulation Bid Violated
Emax
t1 t2 t3 tf State of charge can take
any non-decreasing path
Regulation contract is
e0 Driver violated if charging finishes
Inconvenienced
early
t
Average charge rate to hit Emax at tf Driver inconvenienced if
Bound of Possible State of Charge
vehicle is not charged on
time
10
11. A Stochastic Model is Needed
Ebatt Regulation Bid Violated
Histogram of July 2011 PJM AGC Signal Energy
18
t1 t2 t3 tf
Emax 16
14
12
Counts
10
8
e0 Driver
6
Inconvenienced
4
2
t
Average charge rate to hit Emax at tf 0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Integrated Hourly Energy
Bound of Possible State of Charge
11
12. A Stochastic Model is Needed
Ebatt Regulation Bid Violated
Histogram of July 2011 PJM AGC Signal Energy
18
t1 t2 t3 tf
Emax 16
14
12
Counts
10
8
e0 Driver
6
Inconvenienced
4
2
t
Average charge rate to hit Emax at tf 0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Integrated Hourly Energy
Bound of Possible State of Charge
Providing regulation makes future battery state of charge uncertain
Literature ignores effect of regulation or optimizes considering expected value
Stochastic Model needed to:
value risk of regulation contract violation (pro-rated by time)
value risk of inconveniencing EV driver
Optimize choice of average charge rates and regulation contracts size under
uncertainty 12
13. Dynamic Programming Solution
Solve many hour long optimization problems in a backwards
recursion
𝑉ℎ 𝐸 𝑖,ℎ = min 𝔼 𝜔 𝐽ℎ 𝐸 𝑖,ℎ , 𝑃𝑖,ℎ , 𝐵 𝑖,ℎ , 𝑅ℎ𝜔 + 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓
𝑃 𝑖,ℎ ,𝐵 𝑖,ℎ
Minimize Expected Future Cost given current time and state
of charge
Find a single decision to minimize average cost over all future
outcomes 𝜔 𝑖,ℎ ∈ Ω ℎ
𝑉ℎ 𝐸 𝑖,ℎ is a Stochastic Deterministic Equivalent Problem
13
14. DEP and Optimal Value Function
Solving for V5(16.8) Using 30 Sample Regulation Signals
0.25
Optimal Value Function Cost ($)
0.2
0.15
0.1 𝑉6
0.05 Energy at Pavg
0
Hour 6
Path Bounds
(from Regulation Bid)
Time Energy in sample ω
24
22
20
Hour 5 18
16
Energy (kWh)
14
15. Sample Path Generation
Each DEP uses 30, hour long, AGC signals, 𝑅ℎ𝜔
Sample historical data using crude monte carlo
𝜔 𝑖,ℎ ∈ Ω ℎ
Integrate signal over 5 minutes
Becomes normalized energy for discrete state
equations
𝜔
𝑅 𝐻 is a correlated 12 dimensional vector
Assume AGC independent across hours
15
17. Regulation Contract Risk
If the battery reaches full state of charge, cannot provide
regulation
Payment is pro-rated, time-based
Regulation contract violation indicator, 𝑢
Allows penalty to be a function of time, not energy
𝑢 is a binary variable Ebatt
𝑢1 = 0 𝑢2 = 0 𝑢3 = 1
Emax
𝑢 𝑡𝜔 ∙ 𝑃 𝑚𝑎𝑥 ∙ ∆𝑡 ≥ 𝑠 𝑡 𝜔 , 𝑡 ≠ 𝑡 𝑓 , ∀𝜔
𝜔
𝑒 𝑡+1 ≥ 𝑢 𝑡𝜔 ∙ 𝐸 𝑚𝑎𝑥 , 𝑡 ≠ 𝑡 𝑓 , , ∀𝜔
𝜔
𝑢 𝑡𝜔 ≥ 𝑢 𝑡−1 , 𝑡 ≠ 𝑡 𝑓 , ∀𝜔 e0
t
Average charge rate to hit Emax at tf
Bound of Possible State of Charge
17
18. Driver Inconvenience Risk
When not fully charged by
unplug time
Charge at 𝑃 𝑚𝑎𝑥 until battery is Ebatt 𝜔
𝑇
full Emax
t1 t2 t3 tf
𝐸 𝑚𝑎𝑥 −𝑒 𝑡 𝜔
𝑓
𝑇𝜔= time late on
𝑃 𝑚𝑎𝑥
sample path ω
e0 Driver
𝐿 Driver’s inconvenience cost Inconvenienced
($/hr)
t
𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) Average charge rate to hit Emax at tf
Bound of Possible State of Charge
18
36. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
Find state, cost points on the convex hull of all
points
Andrews Monotone Chain Algorithm
Basically compares slopes
𝑉5 𝐸5
E5 36
37. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
Find state, cost points on the convex hull of all
points
Andrews Monotone Chain Algorithm
Basically compares slopes
𝑉5 𝐸5
Not on the hull
On the hull
E5 37
38. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
Create inequalities from points on the convex hull
Add new inequality constraints to DEP 𝑉4 𝐸 𝑖,4
𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 ≥ 𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 𝑗 − 𝑆𝑙𝑜𝑝𝑒 𝑗 ∗ 𝑒 𝑡 𝜔𝑓 , ∀𝑗
𝑉5 𝐸5
Cut j
E5 38
39. Stochastic Dynamic Programming
Optimal Value Function Cost ($) 0.25
0.2
𝑉ℎ+1 𝑒 𝑡 𝜔𝑓
0.15
0.1
0.05
𝑒1𝑓
𝑡
0
Hour 6
5
Time
24
22
20
Hour 5
4 18
16
Energy (kWh)
39
40. Stochastic Dynamic Programming
Repeat backwards recursion until the current
state is reached
Unplug Time
Ebatt
Emax
E1
h1 h2 h3 h4 h5 Time (Hours)
40
41. Stochastic Dynamic Programming
Repeat backwards recursion until the current
state is reached
Unplug Time
Ebatt
Emax
E1
h1 h2 h3 h4 h5 Time (Hours)
41
42. Stochastic Dynamic Programming
Repeat backwards recursion until the current
state is reached
Unplug Time
Ebatt
Emax
E1
h1 h2 h3 h4 h5 Time (Hours)
42
43. Stochastic Dynamic Programming
Repeat backwards recursion until the current
state is reached
Unplug Time
Ebatt
Emax
E1
h1 h2 h3 h4 h5 Time (Hours)
43
44. Implementation
Calculate Optimal Value
Functions, Vh 26
Simulation Results
At initial state, time 24
Solve one DEP for P1, B1
Battery State of Charge (kWh)
22
Implement decision and 20 Energy Bounds
wait 1hr, see what happens (from Regulation Bid)
18
Given new state, 16 Energy at Pavg
Optimize decision, Actual Energy
14
implement, wait State
At unplug time 12
1 2 3 4 5 6 7 8
If not full, charge at Pmax Simulation Timestep (hr)
Else, Done!
44
45. Forward Simulation for Comparison
Simulate 150 different, 7 hour long
realizations of AGC Signal
Each trial uses the same
Optimal Value Functions
Initial state
Deterministic prices
Set of samples in DEPs
Compare with an expected value formulation
1 sample, using expected value of AGC signal
45
46. Results- 150 forward Simulations
Histogram of Expected Value Formulation Costs
$20/hr Inconvenience cost 120
Stochastic Expected Value 100
Model Model 80
μ $ 0.23 $ 0.44
Counts
60
Σ2 5.0 E-4 0.35 40
Late trials 0% 29% 20
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
Cost($)
$200/hr Inconvenience cost
Histogram of Stochastic Formulation Costs
35
Stochastic Expected Value
Model Model 30
Μ $ 0.23 $ 2.29 25
Σ2 5.0 E-4 35.58 20
Counts
Late trials 0% 29% 15
10
5
0
0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3
Cost($)
46
47. Observations
Expected Value formulation often inconveniences
driver, while Stochastic formulation is robust
For final decision, P,B are chosen such that driver is not
inconvenienced on any sample path
Cost of uncharged energy ÷ 30 > All hourly Energy Prices
Vast majority of DEP solutions are on the CH
good approximation of 𝑉ℎ
If Charging, regulation contract size, B is on upper
bound , but decisions are dependent
47
48. Future Work
Method Improvement/Evaluation
Bias Estimation and Correction
Number of AGC Samples
Number of Discretizations
Parallelize
Model Expansion
Investigate AGC signal properties
Uncertain Prices- ARIMA or GARCH
Form CH in 4 dimensions with QuickHull
Fleet Aggregation
Apply method to other technologies (flywheels)
Integrate into broader Smart Distribution Network model
48
49. Conclusions
Stochastic models are necessary for demand
side frequency regulation
We have accurately modeled risks of
providing frequency regulation
Our method is tractable and parallelizable
49
50. Thank you!
Histogram of Stochastic Formulation Costs
35
30
25
20
Counts
15
10
5
0
0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3
Cost($)
Simulation Results
26
24
Battery State of Charge (kWh)
22
20
18
16
14
12
1 2 3 4 5 6 7 8
Simulation Timestep (hr)
Support for this research was provided by
Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology)
through the Carnegie Mellon Portugal Program
50