SlideShare a Scribd company logo
1 of 50
Download to read offline
Optimal Decision Making for Electric Vehicles
Providing Electric Grid Frequency Regulation:
       A Stochastic Dynamic Programming Approach


                                Jonathan Donadee
                      Ph.D. Student, ECE, Carnegie Mellon University
                               jdonadee@andrew.cmu.edu

                                      Marija Ilic
            IEEE Fellow, Professor of ECE and EPP, Carnegie Mellon University
                                   milic@ece.cmu.edu

        9th International Conference on Computational Management Science
                             Imperial College London, UK
                                   April 20th, 2012

                           Support for this research was provided by
  Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology)
                        through the Carnegie Mellon Portugal Program
Outline
Problem Introduction
Deterministic Equivalent Problem Model
Stochastic Dynamic Programming Algorithm
Simulation Results
Conclusions
Questions


                                            2
Electrical Grid Frequency Regulation
                                        Power Supply and Demand Balance

 Electrical Grid AC frequency
  must be maintained within
  ± 0.05 Hz
    60Hz (USA)
    50Hz (Europe)
 Demand < Supply, f               Regulation is on seconds to minutes timescale

 Demand > Supply, f
 Some generators follow “AGC”
  or “Regulation” control signal
  on second to minute basis
 Generators bid capacity (MW)
  into hourly markets
                                                                          3
                                       Graphic Source: PJM ISO Training
One Day’s AGC Signal
                                     1 Day of PJM AGC Signal
                           1


                         0.5
     Normalized Signal




                           0


                         -0.5


                          -1


                         -1.5
                             0   5        10           15      20   25
                                               Hours



                                                                         4
Motivation
In the future, greater need for fast responding
 electrical grid resources
   Compensate renewable resource forecast errors
   Mitigate trend of decreasing system inertia
     smaller imbalances causing larger frequency deviations
Integration of Electric Vehicles
   Minimize EV charging cost
Deterministic models and methods are not
 well suited for managing uncertain resources
                                                           5
Electric Vehicle Participation in
Frequency Regulation
 EVs can adjust charge rate to follow AGC signal
    We’ll focus on charging only strategies, no discharging
 Specify preferred charge rate, Pavg
 Specify capacity for regulating, B
 Adjust according to negative of AGC signal scaled by B



                                          0 ≤ 𝐵 ≤ 𝑃𝑎𝑣𝑔
                                          0 ≤ 𝑃𝑎𝑣𝑔 ≤ 𝑃 𝑚𝑎𝑥
                                          0 ≤ 𝐵 ≤ 𝑃 𝑚𝑎𝑥 − 𝑃𝑎𝑣𝑔




                                                                 6
Smart Charging Scenario
 Driver arrives at home and plugs in vehicle
      Inputs time of departure
      Inputs inconvenience cost for not finishing on time ($/hr)
      Smart charger optimizes Pavg, B decisions for each hour
      Smart charger can participate in markets directly, without
       delay




                                                                    7
   Picture Source: Ford.com
A Stochastic Model is Needed
Ebatt

Emax
        t1           t2             t3               tf        State of charge can take
                                                                any non-decreasing path

e0


                                                          t
             Average charge rate to hit Emax at tf
             Bound of Possible State of Charge




                                                                                           8
A Stochastic Model is Needed
Ebatt                       Regulation Bid Violated

Emax
        t1           t2             t3               tf        State of charge can take
                                                                any non-decreasing path
                                                               Regulation contract is
e0                                                              violated if charging finishes
                                                                early
                                                          t
             Average charge rate to hit Emax at tf
             Bound of Possible State of Charge




                                                                                            9
A Stochastic Model is Needed
Ebatt                       Regulation Bid Violated

Emax
        t1           t2             t3               tf        State of charge can take
                                                                any non-decreasing path
                                                               Regulation contract is
e0                        Driver                                violated if charging finishes
                          Inconvenienced
                                                                early
                                                          t
             Average charge rate to hit Emax at tf             Driver inconvenienced if
             Bound of Possible State of Charge
                                                                vehicle is not charged on
                                                                time




                                                                                           10
A Stochastic Model is Needed
Ebatt                       Regulation Bid Violated
                                                                                    Histogram of July 2011 PJM AGC Signal Energy
                                                                       18
        t1           t2             t3               tf
Emax                                                                   16

                                                                       14

                                                                       12




                                                              Counts
                                                                       10

                                                                        8
e0                        Driver
                                                                        6
                          Inconvenienced
                                                                        4

                                                                        2
                                                          t
             Average charge rate to hit Emax at tf                      0
                                                                        -1   -0.8    -0.6   -0.4    -0.2      0      0.2   0.4     0.6   0.8
                                                                                              Integrated Hourly Energy
             Bound of Possible State of Charge




                                                                                                                                         11
A Stochastic Model is Needed
  Ebatt                       Regulation Bid Violated
                                                                                      Histogram of July 2011 PJM AGC Signal Energy
                                                                         18
          t1           t2             t3               tf
  Emax                                                                   16

                                                                         14

                                                                         12




                                                                Counts
                                                                         10

                                                                          8
  e0                        Driver
                                                                          6
                            Inconvenienced
                                                                          4

                                                                          2
                                                            t
               Average charge rate to hit Emax at tf                      0
                                                                          -1   -0.8    -0.6   -0.4    -0.2      0      0.2   0.4     0.6   0.8
                                                                                                Integrated Hourly Energy
               Bound of Possible State of Charge

 Providing regulation makes future battery state of charge uncertain
 Literature ignores effect of regulation or optimizes considering expected value
 Stochastic Model needed to:
      value risk of regulation contract violation (pro-rated by time)
      value risk of inconveniencing EV driver
      Optimize choice of average charge rates and regulation contracts size under
       uncertainty                                                                                                                         12
Dynamic Programming Solution
 Solve many hour long optimization problems in a backwards
  recursion
      𝑉ℎ 𝐸 𝑖,ℎ = min           𝔼   𝜔   𝐽ℎ 𝐸 𝑖,ℎ , 𝑃𝑖,ℎ , 𝐵 𝑖,ℎ , 𝑅ℎ𝜔 + 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓
                𝑃 𝑖,ℎ ,𝐵 𝑖,ℎ



 Minimize Expected Future Cost given current time and state
  of charge
    Find a single decision to minimize average cost over all future
     outcomes 𝜔 𝑖,ℎ ∈ Ω ℎ
 𝑉ℎ 𝐸 𝑖,ℎ is a Stochastic Deterministic Equivalent Problem


                                                                                     13
DEP and Optimal Value Function
                   Solving for V5(16.8) Using 30 Sample Regulation Signals

                                        0.25
      Optimal Value Function Cost ($)
                                         0.2

                                        0.15

                                         0.1                             𝑉6
                                        0.05      Energy at Pavg

                                          0
      Hour 6
                                                                                       Path Bounds
                                                                                   (from Regulation Bid)

                                               Time                           Energy in sample ω


                                                                                                           24
                                                                                               22
                                                                                   20
                                                      Hour 5        18
                                                               16
                                                                              Energy (kWh)


                                                                                                                14
Sample Path Generation
Each DEP uses 30, hour long, AGC signals, 𝑅ℎ𝜔
   Sample historical data using crude monte carlo
   𝜔 𝑖,ℎ ∈ Ω ℎ
Integrate signal over 5 minutes
   Becomes normalized energy for discrete state
    equations
      𝜔
   𝑅 𝐻 is a correlated 12 dimensional vector
   Assume AGC independent across hours
                                                     15
State Equations
                                                                                      Energy Actually
          𝝎                         𝝎       𝝎                                           Consumed
𝒆 𝒕𝝎 = 𝒆 𝒕−𝟏 + ∆𝒕 ∙ 𝑷 − 𝑩 ∙ ∆𝒕 ∙ 𝑹 𝒕−𝟏 − 𝒔 𝒕−𝟏 ,                 𝒕 ≥ 𝟐, ∀𝝎                   ≠
                                                                                           𝑃 ∙ ∆𝑡
𝑒 𝑡 𝜔 = 𝐸 𝑖,ℎ ,   𝑡 = 1, ∀𝜔

           𝜔
𝑒 𝑡 𝜔 ≥ 𝑒 𝑡−1 ,   ∀𝑡, ∀𝜔                           24
                                                             Example State Dynamics


                                                   23
  𝜔
𝑒 𝑡 ≤ 𝐸 𝑚𝑎𝑥 ,     ∀𝑡, ∀𝜔                           22
                                    Energy (kWh)
                                                   21

𝑠 𝑡 𝜔 ≥ 0,        ∀𝑡, ∀𝜔                           20

                                                   19

                                                   18

                                                   17

                                                   16
                                                        T0                                 1Hr
                                                                     Time
                                                                                                    16
Regulation Contract Risk
 If the battery reaches full state of charge, cannot provide
  regulation
 Payment is pro-rated, time-based
 Regulation contract violation indicator, 𝑢
    Allows penalty to be a function of time, not energy
    𝑢 is a binary variable         Ebatt
                                                          𝑢1 = 0      𝑢2 = 0        𝑢3 = 1
                                                   Emax
    𝑢 𝑡𝜔 ∙ 𝑃 𝑚𝑎𝑥 ∙ ∆𝑡 ≥ 𝑠 𝑡 𝜔 ,   𝑡 ≠ 𝑡 𝑓 , ∀𝜔
      𝜔
   𝑒 𝑡+1 ≥ 𝑢 𝑡𝜔 ∙ 𝐸 𝑚𝑎𝑥 ,         𝑡 ≠ 𝑡 𝑓 , , ∀𝜔
              𝜔
    𝑢 𝑡𝜔 ≥ 𝑢 𝑡−1 ,                𝑡 ≠ 𝑡 𝑓 , ∀𝜔     e0


                                                                                                     t
                                                            Average charge rate to hit Emax at tf
                                                            Bound of Possible State of Charge
                                                                                                17
Driver Inconvenience Risk
 When not fully charged by
  unplug time
    Charge at 𝑃 𝑚𝑎𝑥 until battery is      Ebatt                                                𝜔
                                                                                           𝑇
     full                                  Emax
                                                   t1          t2            t3       tf

             𝐸 𝑚𝑎𝑥 −𝑒 𝑡 𝜔
                        𝑓
    𝑇𝜔=                    time late on
                𝑃 𝑚𝑎𝑥
     sample path ω
                                           e0                       Driver
    𝐿 Driver’s inconvenience cost                                  Inconvenienced
     ($/hr)
                                                                                                    t
    𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )             Average charge rate to hit Emax at tf
                                                   Bound of Possible State of Charge




                                                                                           18
Objective Function
(if final decision)


               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡




                                                                                                                        19
Objective Function
(if final decision)
                                            Baseline
                                           Energy Cost

               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡




                                                                                                                        20
Objective Function
(if final decision)
                                            Baseline             Regulation Contract
                                           Energy Cost               Revenue

               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡




                                                                                                                        21
Objective Function
(if final decision)
                                            Baseline             Regulation Contract
                                           Energy Cost               Revenue

               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡


                               Energy Cost
                               Adjustement




                                                                                                                        22
Objective Function
(if final decision)
                                            Baseline             Regulation Contract
                                           Energy Cost               Revenue

               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡


                               Energy Cost                      Contract
                               Adjustement                   Violation Cost




                                                                                                                        23
Objective Function
(if final decision)
                                            Baseline             Regulation Contract
                                           Energy Cost               Revenue

               𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃



                      𝑡 𝑓 −1
   1
𝜃=               𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡


                               Energy Cost                      Contract                          Driver
                               Adjustement                   Violation Cost                 Inconvenience Cost




                                                                                                                        24
Objective Function
(if final decision)
                                           Baseline             Regulation Contract
                                          Energy Cost               Revenue

               𝑉ℎ 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃
                                                                                                    Future Energy
                                                                                                     Purchases
                     𝑡 𝑓 −1
   1
𝜃=              𝑐𝐻            −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁         𝑡=1                                                      𝑡


                              Energy Cost                      Contract                          Driver
                              Adjustement                   Violation Cost                 Inconvenience Cost




                                                                                                                       25
Objective Function
(not final decision)
                                            Baseline             Regulation Contract
                                           Energy Cost               Revenue

               𝑉ℎ 𝐸 𝑖,ℎ = min 𝑃 𝑖,ℎ ,𝐵 𝑖,ℎ 𝑐ℎ 𝑃𝑖,ℎ − 𝑟ℎ 𝐵 𝑖,ℎ + 𝜃
                                                                                   Future Optimal Value Function


                      𝑡 𝑓 −1
                                                                                          + 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓
   1
𝜃=               𝑐ℎ            −𝐵 𝑖,ℎ ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,ℎ ∙       𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐ℎ+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 )
   𝑁
       𝜔∈Ω 𝑁          𝑡=1                                                      𝑡


                            Energy Cost                         Contract
                            Adjustement                      Violation Cost




                                                                                                                        26
Exact Linearization of Contract
Violation Cost
𝑄 ∙ ∆𝑡 ∙ 𝐵 ∙      𝑡   𝑢 𝑡𝜔 is nonlinear

Replace with 𝑄 ∙ ∆𝑡 ∙             𝑡   𝑥 𝑡𝜔
Add constraints
      𝑥 𝑡𝜔 ≤ 𝐵
         𝜔
             𝑃 𝑚𝑎𝑥 𝜔
      𝑥𝑡 ≤          𝑢𝑡
               2
         𝜔
                  𝑃 𝑚𝑎𝑥
      𝑥𝑡 ≥ 𝐵 −          1 − 𝑢 𝑡𝜔
                    2
    When 𝑢 = 0, 𝑥=0
    When 𝑢 = 1, 𝑥= 𝐵
                                              27
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      28
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      29
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      30
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      31
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      32
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      33
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      34
Stochastic Dynamic Programming
                    𝑉5 𝐸5




                                                Unplug Time

                                      E5
       Ebatt
Emax




 E1
          h1   h2           h3   h4        h5   Time (Hours)

                                                      35
Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
  Find state, cost points on the convex hull of all
   points
         Andrews Monotone Chain Algorithm
         Basically compares slopes
𝑉5 𝐸5




                                             E5        36
Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
  Find state, cost points on the convex hull of all
   points
         Andrews Monotone Chain Algorithm
         Basically compares slopes
𝑉5 𝐸5
            Not on the hull


               On the hull




                                             E5        37
Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓
   Create inequalities from points on the convex hull
   Add new inequality constraints to DEP 𝑉4 𝐸 𝑖,4

         𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 ≥ 𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 𝑗 − 𝑆𝑙𝑜𝑝𝑒 𝑗 ∗ 𝑒 𝑡 𝜔𝑓 ,   ∀𝑗


𝑉5 𝐸5




                                                          Cut j

                                                                  E5   38
Stochastic Dynamic Programming
     Optimal Value Function Cost ($)   0.25

                                        0.2
                                                                        𝑉ℎ+1 𝑒 𝑡 𝜔𝑓
                                       0.15

                                        0.1

                                       0.05
                                                     𝑒1𝑓
                                                      𝑡
                                         0
     Hour 6
          5




                                              Time



                                                                                            24
                                                                                       22
                                                                             20
                                                     Hour 5
                                                          4        18
                                                              16
                                                                        Energy (kWh)



                                                                                                 39
Stochastic Dynamic Programming
 Repeat backwards recursion until the current
  state is reached
                                          Unplug Time

       Ebatt
Emax




 E1
          h1   h2   h3       h4      h5   Time (Hours)

                                                 40
Stochastic Dynamic Programming
 Repeat backwards recursion until the current
  state is reached
                                          Unplug Time

       Ebatt
Emax




 E1
          h1   h2   h3       h4      h5   Time (Hours)

                                                 41
Stochastic Dynamic Programming
 Repeat backwards recursion until the current
  state is reached
                                          Unplug Time

       Ebatt
Emax




 E1
          h1   h2   h3       h4      h5   Time (Hours)

                                                 42
Stochastic Dynamic Programming
 Repeat backwards recursion until the current
  state is reached
                                          Unplug Time

       Ebatt
Emax




 E1
          h1   h2   h3       h4      h5   Time (Hours)

                                                 43
Implementation
 Calculate Optimal Value
  Functions, Vh                                                    26
                                                                                            Simulation Results


 At initial state, time                                           24

    Solve one DEP for P1, B1




                                   Battery State of Charge (kWh)
                                                                   22

 Implement decision and                                           20           Energy Bounds
  wait 1hr, see what happens                                                (from Regulation Bid)
                                                                   18
 Given new state,                                                 16                                          Energy at Pavg
    Optimize decision,                                                     Actual Energy
                                                                   14
     implement, wait                                                            State


 At unplug time                                                   12

                                                                        1       2     3          4       5         6    7        8
    If not full, charge at Pmax                                                          Simulation Timestep (hr)

    Else, Done!

                                                                                                                            44
Forward Simulation for Comparison
Simulate 150 different, 7 hour long
 realizations of AGC Signal
Each trial uses the same
     Optimal Value Functions
     Initial state
     Deterministic prices
     Set of samples in DEPs
Compare with an expected value formulation
   1 sample, using expected value of AGC signal
                                                   45
Results- 150 forward Simulations
                                                                             Histogram of Expected Value Formulation Costs
$20/hr Inconvenience cost                                 120




                Stochastic   Expected Value               100


                Model        Model                        80


      μ         $ 0.23       $ 0.44




                                              Counts
                                                          60


      Σ2        5.0 E-4      0.35                         40


  Late trials   0%           29%                          20



                                                           0
                                                                0      0.5      1     1.5      2      2.5      3      3.5      4    4.5
                                                                                                Cost($)
$200/hr Inconvenience cost
                                                                               Histogram of Stochastic Formulation Costs
                                                          35
                Stochastic   Expected Value
                Model        Model                        30



      Μ         $ 0.23       $ 2.29                       25



      Σ2        5.0 E-4      35.58                        20




                                                 Counts
  Late trials   0%           29%                          15


                                                          10


                                                           5


                                                           0
                                                                    0.16     0.18     0.2      0.22     0.24       0.26      0.28   0.3
                                                                                                Cost($)
                                                                                                                                          46
Observations
 Expected Value formulation often inconveniences
  driver, while Stochastic formulation is robust
    For final decision, P,B are chosen such that driver is not
     inconvenienced on any sample path
    Cost of uncharged energy ÷ 30 > All hourly Energy Prices
 Vast majority of DEP solutions are on the CH
    good approximation of 𝑉ℎ
 If Charging, regulation contract size, B is on upper
  bound , but decisions are dependent

                                                                  47
Future Work
 Method Improvement/Evaluation
      Bias Estimation and Correction
      Number of AGC Samples
      Number of Discretizations
      Parallelize
 Model Expansion
    Investigate AGC signal properties
    Uncertain Prices- ARIMA or GARCH
    Form CH in 4 dimensions with QuickHull
 Fleet Aggregation
 Apply method to other technologies (flywheels)
 Integrate into broader Smart Distribution Network model
                                                            48
Conclusions
Stochastic models are necessary for demand
 side frequency regulation
We have accurately modeled risks of
 providing frequency regulation
Our method is tractable and parallelizable




                                              49
Thank you!
                                                                                                                                        Histogram of Stochastic Formulation Costs
                                                                                                                          35


                                                                                                                          30


                                                                                                                          25


                                                                                                                          20




                                                                                                                 Counts
                                                                                                                          15


                                                                                                                          10


                                                                                                                          5


                                                                                                                          0
                                                                                                                               0.16   0.18     0.2      0.22     0.24      0.26     0.28    0.3
                                                                                                                                                         Cost($)



                                                                                                         Simulation Results
                                                                                       26

                                                                                       24




                                                       Battery State of Charge (kWh)
                                                                                       22

                                                                                       20

                                                                                       18

                                                                                       16

                                                                                       14

                                                                                       12

                                                                                            1   2   3          4       5         6     7      8
                                                                                                        Simulation Timestep (hr)




                         Support for this research was provided by
Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology)
                      through the Carnegie Mellon Portugal Program
                                                                                                                                                                                           50

More Related Content

Similar to Stochastic Co-Optimization of Electric Vehicle Charging and Frequency Regulation

Automotive emc-compliance
Automotive emc-complianceAutomotive emc-compliance
Automotive emc-compliance
Intertek CE
 
Autosar 5 saalfeld_lecture
Autosar 5 saalfeld_lectureAutosar 5 saalfeld_lecture
Autosar 5 saalfeld_lecture
samerammoun
 
Ertss2010 multicore scheduling
Ertss2010 multicore schedulingErtss2010 multicore scheduling
Ertss2010 multicore scheduling
Nicolas Navet
 
Multicore scheduling in automotive ECUs
Multicore scheduling in automotive ECUsMulticore scheduling in automotive ECUs
Multicore scheduling in automotive ECUs
RealTime-at-Work (RTaW)
 
Power transformer maintenance
Power transformer maintenancePower transformer maintenance
Power transformer maintenance
Lucian Lazar
 

Similar to Stochastic Co-Optimization of Electric Vehicle Charging and Frequency Regulation (9)

Internship presentation
Internship presentationInternship presentation
Internship presentation
 
Automotive emc-compliance
Automotive emc-complianceAutomotive emc-compliance
Automotive emc-compliance
 
Autosar 5 saalfeld_lecture
Autosar 5 saalfeld_lectureAutosar 5 saalfeld_lecture
Autosar 5 saalfeld_lecture
 
ISLANDING DETECTION TECHNIQUE FOR DISTRIBUTED GENERATION SYSTEM
ISLANDING DETECTION TECHNIQUE FOR DISTRIBUTED GENERATION SYSTEMISLANDING DETECTION TECHNIQUE FOR DISTRIBUTED GENERATION SYSTEM
ISLANDING DETECTION TECHNIQUE FOR DISTRIBUTED GENERATION SYSTEM
 
Ertss2010 multicore scheduling
Ertss2010 multicore schedulingErtss2010 multicore scheduling
Ertss2010 multicore scheduling
 
Multicore scheduling in automotive ECUs
Multicore scheduling in automotive ECUsMulticore scheduling in automotive ECUs
Multicore scheduling in automotive ECUs
 
Gsm based transformer fault detection system
Gsm based transformer fault detection systemGsm based transformer fault detection system
Gsm based transformer fault detection system
 
Power transformer maintenance
Power transformer maintenancePower transformer maintenance
Power transformer maintenance
 
Power transformer maintenance
Power transformer maintenancePower transformer maintenance
Power transformer maintenance
 

Stochastic Co-Optimization of Electric Vehicle Charging and Frequency Regulation

  • 1. Optimal Decision Making for Electric Vehicles Providing Electric Grid Frequency Regulation: A Stochastic Dynamic Programming Approach Jonathan Donadee Ph.D. Student, ECE, Carnegie Mellon University jdonadee@andrew.cmu.edu Marija Ilic IEEE Fellow, Professor of ECE and EPP, Carnegie Mellon University milic@ece.cmu.edu 9th International Conference on Computational Management Science Imperial College London, UK April 20th, 2012 Support for this research was provided by Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) through the Carnegie Mellon Portugal Program
  • 2. Outline Problem Introduction Deterministic Equivalent Problem Model Stochastic Dynamic Programming Algorithm Simulation Results Conclusions Questions 2
  • 3. Electrical Grid Frequency Regulation Power Supply and Demand Balance  Electrical Grid AC frequency must be maintained within ± 0.05 Hz  60Hz (USA)  50Hz (Europe)  Demand < Supply, f Regulation is on seconds to minutes timescale  Demand > Supply, f  Some generators follow “AGC” or “Regulation” control signal on second to minute basis  Generators bid capacity (MW) into hourly markets 3 Graphic Source: PJM ISO Training
  • 4. One Day’s AGC Signal 1 Day of PJM AGC Signal 1 0.5 Normalized Signal 0 -0.5 -1 -1.5 0 5 10 15 20 25 Hours 4
  • 5. Motivation In the future, greater need for fast responding electrical grid resources  Compensate renewable resource forecast errors  Mitigate trend of decreasing system inertia smaller imbalances causing larger frequency deviations Integration of Electric Vehicles  Minimize EV charging cost Deterministic models and methods are not well suited for managing uncertain resources 5
  • 6. Electric Vehicle Participation in Frequency Regulation  EVs can adjust charge rate to follow AGC signal  We’ll focus on charging only strategies, no discharging  Specify preferred charge rate, Pavg  Specify capacity for regulating, B  Adjust according to negative of AGC signal scaled by B 0 ≤ 𝐵 ≤ 𝑃𝑎𝑣𝑔 0 ≤ 𝑃𝑎𝑣𝑔 ≤ 𝑃 𝑚𝑎𝑥 0 ≤ 𝐵 ≤ 𝑃 𝑚𝑎𝑥 − 𝑃𝑎𝑣𝑔 6
  • 7. Smart Charging Scenario  Driver arrives at home and plugs in vehicle  Inputs time of departure  Inputs inconvenience cost for not finishing on time ($/hr)  Smart charger optimizes Pavg, B decisions for each hour  Smart charger can participate in markets directly, without delay 7 Picture Source: Ford.com
  • 8. A Stochastic Model is Needed Ebatt Emax t1 t2 t3 tf  State of charge can take any non-decreasing path e0 t Average charge rate to hit Emax at tf Bound of Possible State of Charge 8
  • 9. A Stochastic Model is Needed Ebatt Regulation Bid Violated Emax t1 t2 t3 tf  State of charge can take any non-decreasing path  Regulation contract is e0 violated if charging finishes early t Average charge rate to hit Emax at tf Bound of Possible State of Charge 9
  • 10. A Stochastic Model is Needed Ebatt Regulation Bid Violated Emax t1 t2 t3 tf  State of charge can take any non-decreasing path  Regulation contract is e0 Driver violated if charging finishes Inconvenienced early t Average charge rate to hit Emax at tf  Driver inconvenienced if Bound of Possible State of Charge vehicle is not charged on time 10
  • 11. A Stochastic Model is Needed Ebatt Regulation Bid Violated Histogram of July 2011 PJM AGC Signal Energy 18 t1 t2 t3 tf Emax 16 14 12 Counts 10 8 e0 Driver 6 Inconvenienced 4 2 t Average charge rate to hit Emax at tf 0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 Integrated Hourly Energy Bound of Possible State of Charge 11
  • 12. A Stochastic Model is Needed Ebatt Regulation Bid Violated Histogram of July 2011 PJM AGC Signal Energy 18 t1 t2 t3 tf Emax 16 14 12 Counts 10 8 e0 Driver 6 Inconvenienced 4 2 t Average charge rate to hit Emax at tf 0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 Integrated Hourly Energy Bound of Possible State of Charge  Providing regulation makes future battery state of charge uncertain  Literature ignores effect of regulation or optimizes considering expected value  Stochastic Model needed to:  value risk of regulation contract violation (pro-rated by time)  value risk of inconveniencing EV driver  Optimize choice of average charge rates and regulation contracts size under uncertainty 12
  • 13. Dynamic Programming Solution  Solve many hour long optimization problems in a backwards recursion 𝑉ℎ 𝐸 𝑖,ℎ = min 𝔼 𝜔 𝐽ℎ 𝐸 𝑖,ℎ , 𝑃𝑖,ℎ , 𝐵 𝑖,ℎ , 𝑅ℎ𝜔 + 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 𝑃 𝑖,ℎ ,𝐵 𝑖,ℎ  Minimize Expected Future Cost given current time and state of charge  Find a single decision to minimize average cost over all future outcomes 𝜔 𝑖,ℎ ∈ Ω ℎ  𝑉ℎ 𝐸 𝑖,ℎ is a Stochastic Deterministic Equivalent Problem 13
  • 14. DEP and Optimal Value Function Solving for V5(16.8) Using 30 Sample Regulation Signals 0.25 Optimal Value Function Cost ($) 0.2 0.15 0.1 𝑉6 0.05 Energy at Pavg 0 Hour 6 Path Bounds (from Regulation Bid) Time Energy in sample ω 24 22 20 Hour 5 18 16 Energy (kWh) 14
  • 15. Sample Path Generation Each DEP uses 30, hour long, AGC signals, 𝑅ℎ𝜔  Sample historical data using crude monte carlo  𝜔 𝑖,ℎ ∈ Ω ℎ Integrate signal over 5 minutes  Becomes normalized energy for discrete state equations 𝜔  𝑅 𝐻 is a correlated 12 dimensional vector  Assume AGC independent across hours 15
  • 16. State Equations Energy Actually 𝝎 𝝎 𝝎 Consumed 𝒆 𝒕𝝎 = 𝒆 𝒕−𝟏 + ∆𝒕 ∙ 𝑷 − 𝑩 ∙ ∆𝒕 ∙ 𝑹 𝒕−𝟏 − 𝒔 𝒕−𝟏 , 𝒕 ≥ 𝟐, ∀𝝎 ≠ 𝑃 ∙ ∆𝑡 𝑒 𝑡 𝜔 = 𝐸 𝑖,ℎ , 𝑡 = 1, ∀𝜔 𝜔 𝑒 𝑡 𝜔 ≥ 𝑒 𝑡−1 , ∀𝑡, ∀𝜔 24 Example State Dynamics 23 𝜔 𝑒 𝑡 ≤ 𝐸 𝑚𝑎𝑥 , ∀𝑡, ∀𝜔 22 Energy (kWh) 21 𝑠 𝑡 𝜔 ≥ 0, ∀𝑡, ∀𝜔 20 19 18 17 16 T0 1Hr Time 16
  • 17. Regulation Contract Risk  If the battery reaches full state of charge, cannot provide regulation  Payment is pro-rated, time-based  Regulation contract violation indicator, 𝑢  Allows penalty to be a function of time, not energy  𝑢 is a binary variable Ebatt 𝑢1 = 0 𝑢2 = 0 𝑢3 = 1 Emax 𝑢 𝑡𝜔 ∙ 𝑃 𝑚𝑎𝑥 ∙ ∆𝑡 ≥ 𝑠 𝑡 𝜔 , 𝑡 ≠ 𝑡 𝑓 , ∀𝜔 𝜔 𝑒 𝑡+1 ≥ 𝑢 𝑡𝜔 ∙ 𝐸 𝑚𝑎𝑥 , 𝑡 ≠ 𝑡 𝑓 , , ∀𝜔 𝜔 𝑢 𝑡𝜔 ≥ 𝑢 𝑡−1 , 𝑡 ≠ 𝑡 𝑓 , ∀𝜔 e0 t Average charge rate to hit Emax at tf Bound of Possible State of Charge 17
  • 18. Driver Inconvenience Risk  When not fully charged by unplug time  Charge at 𝑃 𝑚𝑎𝑥 until battery is Ebatt 𝜔 𝑇 full Emax t1 t2 t3 tf 𝐸 𝑚𝑎𝑥 −𝑒 𝑡 𝜔 𝑓  𝑇𝜔= time late on 𝑃 𝑚𝑎𝑥 sample path ω e0 Driver  𝐿 Driver’s inconvenience cost Inconvenienced ($/hr) t  𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) Average charge rate to hit Emax at tf Bound of Possible State of Charge 18
  • 19. Objective Function (if final decision) 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 19
  • 20. Objective Function (if final decision) Baseline Energy Cost 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 20
  • 21. Objective Function (if final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 21
  • 22. Objective Function (if final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 Energy Cost Adjustement 22
  • 23. Objective Function (if final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 Energy Cost Contract Adjustement Violation Cost 23
  • 24. Objective Function (if final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉 𝐻 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 Energy Cost Contract Driver Adjustement Violation Cost Inconvenience Cost 24
  • 25. Objective Function (if final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉ℎ 𝐸 𝑖,𝐻 = min 𝑃 𝑖,𝐻 ,𝐵 𝑖,𝐻 𝑐 𝐻 𝑃𝑖,𝐻 − 𝑟 𝐻 𝐵 𝑖,𝐻 + 𝜃 Future Energy Purchases 𝑡 𝑓 −1 1 𝜃= 𝑐𝐻 −𝐵 𝑖,𝐻 ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,𝐻 ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐 𝐻+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 Energy Cost Contract Driver Adjustement Violation Cost Inconvenience Cost 25
  • 26. Objective Function (not final decision) Baseline Regulation Contract Energy Cost Revenue 𝑉ℎ 𝐸 𝑖,ℎ = min 𝑃 𝑖,ℎ ,𝐵 𝑖,ℎ 𝑐ℎ 𝑃𝑖,ℎ − 𝑟ℎ 𝐵 𝑖,ℎ + 𝜃 Future Optimal Value Function 𝑡 𝑓 −1 + 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 1 𝜃= 𝑐ℎ −𝐵 𝑖,ℎ ∙ 𝑅 𝑡𝜔 ∙ ∆𝑡 − 𝑠 𝑡 𝜔 + 𝑄 ∙ ∆𝑡 ∙ 𝐵 𝑖,ℎ ∙ 𝑢 𝑡𝜔 + 𝐿 ∙ 𝑇 𝜔 + 𝑐ℎ+1 (𝐸 𝑚𝑎𝑥 − 𝑒 𝑡 𝜔𝑓 ) 𝑁 𝜔∈Ω 𝑁 𝑡=1 𝑡 Energy Cost Contract Adjustement Violation Cost 26
  • 27. Exact Linearization of Contract Violation Cost 𝑄 ∙ ∆𝑡 ∙ 𝐵 ∙ 𝑡 𝑢 𝑡𝜔 is nonlinear Replace with 𝑄 ∙ ∆𝑡 ∙ 𝑡 𝑥 𝑡𝜔 Add constraints 𝑥 𝑡𝜔 ≤ 𝐵 𝜔 𝑃 𝑚𝑎𝑥 𝜔 𝑥𝑡 ≤ 𝑢𝑡 2 𝜔 𝑃 𝑚𝑎𝑥 𝑥𝑡 ≥ 𝐵 − 1 − 𝑢 𝑡𝜔 2  When 𝑢 = 0, 𝑥=0  When 𝑢 = 1, 𝑥= 𝐵 27
  • 28. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 28
  • 29. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 29
  • 30. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 30
  • 31. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 31
  • 32. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 32
  • 33. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 33
  • 34. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 34
  • 35. Stochastic Dynamic Programming 𝑉5 𝐸5 Unplug Time E5 Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 35
  • 36. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓 Find state, cost points on the convex hull of all points  Andrews Monotone Chain Algorithm  Basically compares slopes 𝑉5 𝐸5 E5 36
  • 37. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓 Find state, cost points on the convex hull of all points  Andrews Monotone Chain Algorithm  Basically compares slopes 𝑉5 𝐸5 Not on the hull On the hull E5 37
  • 38. Future Cost - 𝑉 𝟓 𝑒 𝑡 𝜔𝑓  Create inequalities from points on the convex hull  Add new inequality constraints to DEP 𝑉4 𝐸 𝑖,4 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 ≥ 𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 𝑗 − 𝑆𝑙𝑜𝑝𝑒 𝑗 ∗ 𝑒 𝑡 𝜔𝑓 , ∀𝑗 𝑉5 𝐸5 Cut j E5 38
  • 39. Stochastic Dynamic Programming Optimal Value Function Cost ($) 0.25 0.2 𝑉ℎ+1 𝑒 𝑡 𝜔𝑓 0.15 0.1 0.05 𝑒1𝑓 𝑡 0 Hour 6 5 Time 24 22 20 Hour 5 4 18 16 Energy (kWh) 39
  • 40. Stochastic Dynamic Programming Repeat backwards recursion until the current state is reached Unplug Time Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 40
  • 41. Stochastic Dynamic Programming Repeat backwards recursion until the current state is reached Unplug Time Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 41
  • 42. Stochastic Dynamic Programming Repeat backwards recursion until the current state is reached Unplug Time Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 42
  • 43. Stochastic Dynamic Programming Repeat backwards recursion until the current state is reached Unplug Time Ebatt Emax E1 h1 h2 h3 h4 h5 Time (Hours) 43
  • 44. Implementation  Calculate Optimal Value Functions, Vh 26 Simulation Results  At initial state, time 24  Solve one DEP for P1, B1 Battery State of Charge (kWh) 22  Implement decision and 20 Energy Bounds wait 1hr, see what happens (from Regulation Bid) 18  Given new state, 16 Energy at Pavg  Optimize decision, Actual Energy 14 implement, wait State  At unplug time 12 1 2 3 4 5 6 7 8  If not full, charge at Pmax Simulation Timestep (hr)  Else, Done! 44
  • 45. Forward Simulation for Comparison Simulate 150 different, 7 hour long realizations of AGC Signal Each trial uses the same  Optimal Value Functions  Initial state  Deterministic prices  Set of samples in DEPs Compare with an expected value formulation  1 sample, using expected value of AGC signal 45
  • 46. Results- 150 forward Simulations Histogram of Expected Value Formulation Costs $20/hr Inconvenience cost 120 Stochastic Expected Value 100 Model Model 80 μ $ 0.23 $ 0.44 Counts 60 Σ2 5.0 E-4 0.35 40 Late trials 0% 29% 20 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Cost($) $200/hr Inconvenience cost Histogram of Stochastic Formulation Costs 35 Stochastic Expected Value Model Model 30 Μ $ 0.23 $ 2.29 25 Σ2 5.0 E-4 35.58 20 Counts Late trials 0% 29% 15 10 5 0 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3 Cost($) 46
  • 47. Observations  Expected Value formulation often inconveniences driver, while Stochastic formulation is robust  For final decision, P,B are chosen such that driver is not inconvenienced on any sample path  Cost of uncharged energy ÷ 30 > All hourly Energy Prices  Vast majority of DEP solutions are on the CH  good approximation of 𝑉ℎ  If Charging, regulation contract size, B is on upper bound , but decisions are dependent 47
  • 48. Future Work  Method Improvement/Evaluation  Bias Estimation and Correction  Number of AGC Samples  Number of Discretizations  Parallelize  Model Expansion  Investigate AGC signal properties  Uncertain Prices- ARIMA or GARCH  Form CH in 4 dimensions with QuickHull  Fleet Aggregation  Apply method to other technologies (flywheels)  Integrate into broader Smart Distribution Network model 48
  • 49. Conclusions Stochastic models are necessary for demand side frequency regulation We have accurately modeled risks of providing frequency regulation Our method is tractable and parallelizable 49
  • 50. Thank you! Histogram of Stochastic Formulation Costs 35 30 25 20 Counts 15 10 5 0 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3 Cost($) Simulation Results 26 24 Battery State of Charge (kWh) 22 20 18 16 14 12 1 2 3 4 5 6 7 8 Simulation Timestep (hr) Support for this research was provided by Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) through the Carnegie Mellon Portugal Program 50