Dynamics of Prices in Electric Power Networks

Sean Meyn                                                                  ...
OREGON

 California’s 25,000
 Mile Electron Highway                            COAL
                                      ...
Meeting Calendar   |   OASIS   |   Employment   |   Site Map   |   Contact Us

                                     HOME  ...
Meeting Calendar   |   OASIS   |   Employment   |   Site Map   |   Contact Us

                  HOME   |                 ...
One Hot Week in Urbana ...



                          Purchase Price $/MWh
 5000


 4000


 3000


 2000


 1000


   0
...
Southern California, July 8-15, 1998 ...


             Spinning Reserve Prices                        PX Prices
         ...
First Impressions:



 July 1998: first signs of "serious market dysfunction" in California
                              ...
APX Europe, June 2003



      Prices (Eur/MWh)                               Week 25
400                                 ...
APX Europe, October 2005


4000

3000                                                 Previous week
2000                  ...
Ontario, November 2005




                         Ancillary service
                         contract clause:
          ...
Ontario, November 2005




                         Ancillary service
                         contract clause:
          ...
Ontario, November 2005




                         Ancillary service
                         contract clause:
          ...
Australia January 16 2007



                            Victoria




                                       0




       ...
Australia January 16 2007

                        Victoria
 Volume (MWh)



                8,000


                6,000...
16
                  3




         I
Centralized Control
Dynamic model
                               On-line capacity
                               Forecast
Reserve options for ...
Dynamic Single-Commodity Model

Stochastic model: G Goods available at time t
                  D Normalized demand

Exces...
Dynamic Single-Commodity Model

Stochastic model: G Goods available at time t
                  D Normalized demand

Exces...
Dynamic Single-Commodity Model

Average cost: pQ density when q ∗ = 0
                              ¯

       E[c(Q)] =   ...
Dynamic Single-Commodity Model

Average cost: pQ density when q ∗ = 0
                              ¯                   RB...
Ancillary service
                        G(t)




                                                         a
            ...
Ancillary service
                        G(t)




                                                         a
            ...
Ancillary service
                        G(t)




                                                         a
            ...
Ancillary service
                         a        G(t )
                        G (t )
                                 ...
Ancillary service
                             a        G(t )
                            G (t )
                         ...
Q(t)
Diffusion model & control                X(t) =
                                                   G a(t)


Relaxatio...
Diffusion model & control

 Markov model:            Hedging-point policy:

         Q(t)
X(t) =
         G a(t)



      ...
Markov model & control

 Markov model:             Hedging-point policy:

         Q(t)              Optimal parameters:
X...
Simulation

Discrete Markov model:                 Optimal hedging-points
                                       for RBM:
...
Simulation

Discrete Markov model:                            Optimal hedging-points
                                     ...
II
Relaxations



              (skip to market)
Texas model

                D1            E1             Gp
                                              1




         ...
Aggregate model
                                                    Line 1            Line 2




                         ...
Effective cost ¯(xA, d)
               c

                                                    Line 1            Line 2



...
Effective cost ¯(xA, d)
                      c

                                                                         ...
Effective cost ¯(xA, d)
                      c

                                                                         ...
Inventory model




                                                         demand 1




demand 2




    Controlled work...
Inventory model:
Workload relaxation


 w2
      Resource 1                +
            is idle         W

              ...
III
Decentralized Control
Supply & Demand


Cost of generation depends on source

     Price

                                                      ...
Supply & Demand


Demand for power is not flexible

               High Priority Customers $5,000/MW?
     Price          ...
Supply & Demand


Equilibrium Price & Quantity: Intersection of supply & demand curves

                 High Priority Cus...
Supply & Demand


Equilibrium Price & Quantity: Intersection of supply & demand curves

                      High Priorit...
www.apx.nl

          Main Page                      Market Results




Welcome to APX!
APX is the first electronic energy...
Second Welfare Theorem


Each player independently optimizes ...

Consumer: value of consumption
  5000
             minus...
Second Welfare Theorem


Is there an equilibrium price functional?

Is the equilibrium efficient??
  5000


  4000


  300...
Second Welfare Theorem


Is there an equilibrium price functional?

Is the equilibrium efficient??
  5000


  4000
       ...
Efficient Equilibrium
                        Prices
                        Normalized demand
                        Res...
Southern California, July 8-15, 1998 ...


             Spinning Reserve Prices                        PX Prices
         ...
Conclusions
                                                       Spinning Reserve Prices                        PX Price...
Extensions and future work        10000


                                   8000


                                   600...
Extensions and future work                10000


                                           8000


                      ...
Epilogue                           10000


                                    8000


                                    ...
Epilogue                           10000


                                    8000


                                    ...
Epilogue                             10000


                                      8000


                                ...
Epilogue                             10000


                                      8000


                                ...
References
                                                                q1 − q2
                                       ...
Poisson’s Equation


First reflection times,

  τp :=inf{t ≥ 0 : Q(t) = q p },
                          ¯           τa :=...
Poisson’s Equation


First reflection times,

  τp :=inf{t ≥ 0 : Q(t) = q p },
                          ¯                ...
Poisson’s Equation

         Q(t)
X(t) =
         Ga(t)
                      h(X(τa ))




                              ...
Derivative Representations



                               1
                      ∇h(x),   1   = 0,        x = X(τa )

...
Derivative Representations                                  1
                                   λa (x) = ∇c(x),          ...
Derivative Representations


                           τa
           1
  ∇h(x),   1   = Ex             λa (X(t)) dt
     ...
Dynamic Programming Equations


If q p = q p∗ and ¯ a = ¯ a∗
   ¯     ¯        q     q

Then h solves the dynamic programm...
Upcoming SlideShare
Loading in …5
×

Efficiency and Marginal Cost Pricing in Dynamic Competitive Markets with Friction

1,514 views
1,451 views

Published on

https://netfiles.uiuc.edu/meyn/www/spm_files/Market06/Market06.html

Abstract:
This paper examines a dynamic general equilibrium model with supply friction. With or without friction, the competitive equilibrium is efficient. Without friction, the market price is completely determined by the marginal production cost and the consumers gain positive surplus from trading. If friction is present, no matter how small, then the market price fluctuates between zero and the ``choke-up'' price, without any tendency to converge to the marginal production cost, exhibiting considerable volatility. The gains from trading can deviate significantly from the prediction of the static model in the efficient market outcome. Also considered is a monopolistic market model in which a single firm determines market prices as a function of time. The market outcome is identical in the case of a continuum of consumers. In a model with a single consumer the market prices increase, and the supplier extracts the entire gain from trading.

Published in: Education, Technology, Business
1 Comment
0 Likes
Statistics
Notes
  • Lots of new work recently - see tutorial for 'Energy Systems Week' at Cambridge,
    http://www.newton.ac.uk/programmes/SCS/esw.html

    and the new applications to wind integration,
    https://netfiles.uiuc.edu/meyn/www/spm_files/wind2010/Wind10.html
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
1,514
On SlideShare
0
From Embeds
0
Number of Embeds
327
Actions
Shares
0
Downloads
58
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Efficiency and Marginal Cost Pricing in Dynamic Competitive Markets with Friction

  1. 1. Dynamics of Prices in Electric Power Networks Sean Meyn Prices Normalized demand Reserve Department of ECE and the Coordinated Science Laboratory University of Illinois Joint work with M. Chen and I-K. Cho NSF support: ECS 02-17836 & 05-23620 Control Techniques for Complex Networks DOE Support: http://www.sc.doe.gov/grants/FAPN08-13.html Extending the Realm of Optimization for Complex Systems: Uncertainty, Competition and Dynamics PIs: Uday V. Shanbhag, Tamer Basar, Sean P. Meyn and Prashant G. Mehta
  2. 2. OREGON California’s 25,000 Mile Electron Highway COAL LEGEND GEOTHERMAL HYDROELECTRIC NUCLEAR OIL/GAS BIOMASS MUNICIPAL SOLID WASTE (MSW) SOLAR WIND AREAS SAN FRANCISCO What is the value of improved transmission? NEVADA More responsive ancillary service? How does a centralized planner optimize capacity? MEXICO Is there an efficient decentralized solution?
  3. 3. Meeting Calendar | OASIS | Employment | Site Map | Contact Us HOME | Search http://www.caiso.com/outlook/outlook.html September 28, 2008 40,000 MW 4,000 Megawatts Available Resources The current forecast of generating and import resources available to serve the demand for energy 30,000 MW within the California ISO service area Forecast Demand Forecast of the demand expected today. The procurement of energy resources for the day is based on this forecast 20,000 MW Day Ahead Demand Forecast Revised Demand Forecast Actual Demand Hour Beginning Available Resources Forecast Actual Demand Today's actual system demand Revised Demand Forecast The current forecast of the system demand expected throughout the remainder of the day. This forecast is updated hourly.
  4. 4. Meeting Calendar | OASIS | Employment | Site Map | Contact Us HOME | Search http://www.caiso.com Emergency Notices Generating Reserves 7.0% Stage 1 Stage 1 Generating reserves less than requirements Emergency Emergency (Continuously recalculated. Between 6.0% & 7.0%) 6.0% 5.0% Stage 2 Stage 2 Generating reserves less than 5.0% 4.0% Emergency Emergency 3.0% Stage 3 Stage 3 Generating reserves less than largest contingency 2.0% Emergency Emergency (Continuously recalculated. Between 1.5% & 3.0%) 1.0% 0.0%
  5. 5. One Hot Week in Urbana ... Purchase Price $/MWh 5000 4000 3000 2000 1000 0 13 17 21 13 17 21 13 17 21 13 17 21 13 17 21 1 5 9 1 5 9 1 5 9 1 5 9 1 5 9 Monday Tuesday Wednesday Thursday Friday
  6. 6. Southern California, July 8-15, 1998 ... Spinning Reserve Prices PX Prices 70 250 60 200 50 150 40 30 100 20 50 10 0 Weds Thurs Fri Sat Sun Mon Tues Weds
  7. 7. First Impressions: July 1998: first signs of "serious market dysfunction" in California Lessons From the California “Apocalypse:” Jurisdiction Over Electric Utilities Nicholas W. Fels and Frank R. Lindh Energy Law Journal, Vol 22, No. 1, 2001 FERC ....authorized the ISO to "[reject]...bids in excess of whatever price levels it believes are appropriate ... file additional market-monitoring reports".
  8. 8. APX Europe, June 2003 Prices (Eur/MWh) Week 25 400 Week 26 350 300 250 200 150 100 50 0 Mon Tues Weds Thurs Fri Sat Sun
  9. 9. APX Europe, October 2005 4000 3000 Previous week 2000 Current week (10/24/05) 1000 Volume (MWh) 0 800 Price (Euro) 600 400 200 0 vr za zo ma di wo do
  10. 10. Ontario, November 2005 Ancillary service contract clause: Minimum overall ramp rate of 50 MW/min. Projected power prices reached $2000/MWh
  11. 11. Ontario, November 2005 Ancillary service contract clause: Minimum overall ramp rate of 50 MW/min. Projected power prices reached $2000/MWh
  12. 12. Ontario, November 2005 Ancillary service contract clause: Minimum overall ramp rate of 50 MW/min. Projected power prices reached $2000/MWh
  13. 13. Australia January 16 2007 Victoria 0 Tasmania
  14. 14. Australia January 16 2007 Victoria Volume (MWh) 8,000 6,000 + 10,000 Price (Aus $/MWh) 4,000 0 - 500 1,400 Volume (MWh) Tasmania 1,200 1,000 Price (Aus $/MWh) + 500 800 0 - 500 - 1,000
  15. 15. 16 3 I Centralized Control
  16. 16. Dynamic model On-line capacity Forecast Reserve options for services Actual demand based on forecast statistics Revised forecast Centered demand: D(t) = demand - forecast t
  17. 17. Dynamic Single-Commodity Model Stochastic model: G Goods available at time t D Normalized demand Excess/shortfall: Q(t) = G(t) − D(t) Normalized cost as a function of Q: c− Excess production c+ Shortfall q Scarf, Arrow, Bellman, ...
  18. 18. Dynamic Single-Commodity Model Stochastic model: G Goods available at time t D Normalized demand Excess/shortfall: Q(t) = G(t) − D(t) Generation is rate-constrained: -ζ- q Q(t) ζ+ t High cost
  19. 19. Dynamic Single-Commodity Model Average cost: pQ density when q ∗ = 0 ¯ E[c(Q)] = c( q − ¯ ) pQ (dq) q q∗ ¯ c− pQ (q − q ∗ ) ¯ c+ q Optimal hedging-point: q ∗ solves ¯ − c− P{ Q ≤ 0} + c+P{ Q ≥ 0} = 0
  20. 20. Dynamic Single-Commodity Model Average cost: pQ density when q ∗ = 0 ¯ RBM model: E[c(Q)] = c( q − ¯ ) pQ (dq) q pQ exponential q∗ ¯ c− pQ (q − q ∗ ) ¯ c+ q Optimal hedging-point: q ∗ solves ¯ ∗ = 1 σ2 c− q ¯ 2 ζ + log c+ − c−P{ Q ≤ 0} + c+P{ Q ≥ 0} = 0 Wein '92
  21. 21. Ancillary service G(t) a G (t) The two goods are substitutable, but 1. primary service is available at a lower price 2. ancillary service can be ramped up more rapidly
  22. 22. Ancillary service G(t) a G (t) The two goods are substitutable, but 1. primary service is available at a lower price 2. ancillary service can be ramped up more rapidly
  23. 23. Ancillary service G(t) a G (t) The two goods are substitutable, but 1. primary service is available at a lower price 2. ancillary service can be ramped up more rapidly
  24. 24. Ancillary service a G(t ) G (t ) K Excess capacity: Q(t) = G(t) + G a(t) − D (t), t≥ 0. Power flow subject to peak and rate constraints: d a d −ζ a− ≤ G (t) ≤ ζ a+ −ζ − ≤ G(t) ≤ ζ + dt dt
  25. 25. Ancillary service a G(t ) G (t ) K Policy: hedging policy with multiple thresholds Q(t) = G(t) + G a(t) − D (t) -ζ- Downward trend: Ga(t) = 0 d G (t) = - ζ - dt q ζ+ q ζ ++ ζ a+ t Blackout
  26. 26. Q(t) Diffusion model & control X(t) = G a(t) Relaxations: instantaneous ramp-down rates: d d a −∞ ≤ G (t) ≤ ζ +, −∞ ≤ G (t) ≤ ζ a+. dt dt Cost structure: c(X(t)) = c1G (t) + c2G a(t) + c3|Q(t)|1{Q(t) < 0} Control: design hedging points to minimize average-cost, min Eπ [c(Q(t))] .
  27. 27. Diffusion model & control Markov model: Hedging-point policy: Q(t) X(t) = G a(t) Ancillary service is ramped-up when excess capacity falls below q2 ¯ q2 ¯ q1 ¯
  28. 28. Markov model & control Markov model: Hedging-point policy: Q(t) Optimal parameters: X(t) = (proof) G a(t) ∗ − q ∗ = 1 log c2 q1 ¯2 ¯ γ1 c1 ∗ = 1 log c3 q2 ¯ γ0 c2 ζ + +ζ a+ ζ+ γ0 = 2 , γ1 = 2 2 . σD 2 σD q2 ¯ q1 ¯
  29. 29. Simulation Discrete Markov model: Optimal hedging-points for RBM: Q(k + 1) − Q(k) q1 − q2 = 14.978 ¯ ¯ = ζ(k) + ζ a(k) + E(k + 1) q2 = 2.996 ¯ E(k) i.i.d. Bernoulli. ζ(k), ζ a(k) allocation increments.
  30. 30. Simulation Discrete Markov model: Optimal hedging-points for RBM: Q(k + 1) − Q(k) q1 − q2 = 14.978 ¯ ¯ = ζ(k) + ζ a(k) + E(k + 1) ¯ q2 = 2.996 Average cost: CRW Average cost: Diffusion model 24 24 22 22 20 20 18 18 7 16 ¯ ¯ q1 − q2 ¯ ¯ q1 − q2 3 ¯ q2 ¯ q2
  31. 31. II Relaxations (skip to market)
  32. 32. Texas model D1 E1 Gp 1 Line 1 Ga 2 Line 2 Ga 3 D2 D3 Line 3 E2 E3 Resource pooling from San Antonio to Houston?
  33. 33. Aggregate model Line 1 Line 2 Line 3 QA(t) = extraction - demand = Ei(t) − Di(t) a a GA(t) = aggregate ancillary = Gi (t) Assume Brownian demand, rate constraints as before Provided there are no transmission constraints, a XA = (QA ,GA ) ≡ single producer/consumer model
  34. 34. Effective cost ¯(xA, d) c Line 1 Line 2 Given demand and aggregate state Line 3 find the cheapest consistent network configuration subject to transmission constraints p p − min (ci gi + cagi + cboqi ) i a i s.t. qA = (ei − di) consistency a gA = a gi p a 0 = (gi + gi − ei ) extraction = generation q = e−d vector reserves f = ∆p power flow equations f ∈F transmission constraints
  35. 35. Effective cost ¯(xA, d) c Line 1 Line 2 Line 3 a gA 50 40 xA 1 What do these 30 xA 2 + aggregate states say X R about the network? 20 10 xA 3 xA 4 qA 0 - 50 - 40 - 30 - 20 - 10 0 10 20 30 40 50
  36. 36. Effective cost ¯(xA, d) c Line 1 Line 2 Line 3 a gA 50 City 1 in blackout: q1 = −46, 46, q2 = 3.0564, 3.0564, q3 = 12.9436 12 40 xA 1 Insufficient primary generation: a a g1 = −71, g2 = 33.0564, 33 0564, g3 = 6.9436 6. 30 Transmission constraints binding: 20 f1 = 13, 13, f2 = −5, f 3 = −8 10 qA 0 - 50 - 40 - 30 - 20 - 10 0 10 20 30 40 50
  37. 37. Inventory model demand 1 demand 2 Controlled work-release, controlled routing, uncertain demand.
  38. 38. Inventory model: Workload relaxation w2 Resource 1 + is idle W −(1 − ρ) Asymptotes: 1 σ2 c− Resource 2 q ∗ = 2 ζ + log c+ ¯ is idle w1
  39. 39. III Decentralized Control
  40. 40. Supply & Demand Cost of generation depends on source Price Supply Curve Gas Turbine ($20-$30) Coal ($10 -$15) Nuclear ($6) Quantity
  41. 41. Supply & Demand Demand for power is not flexible High Priority Customers $5,000/MW? Price Customers with interruptible services Demand Curve Quantity
  42. 42. Supply & Demand Equilibrium Price & Quantity: Intersection of supply & demand curves High Priority Customers Price Customers with interruptible services Supply Curve Consumer Surplus Equilibrium Price $20 - $30 Supplier Surplus Gas Turbine ($20-$30) Coal ($10 -$15) Nuclear ($6) Demand Curve Equilibrium Quantity 45,000MW Quantity
  43. 43. Supply & Demand Equilibrium Price & Quantity: Intersection of supply & demand curves High Priority Customers Price 5000 Customers with interruptible services Where do ramp constraints appear? 4000 Supply Curve Variability? Surplus Equilibrium Consumer Where is hedging? 3000 Price $20 - $30 Supplier Surplus Gas Turbine ($20-$30) 2000 Coal ($10 -$15) 1000 Nuclear ($6) Demand Curve Equilibrium Quantity 13 17 21 13 17 21 13 17 21 13 17 21 13 17 21 1 5 9 1 5 9 1 5 9 1 5 9 1 5 9 Quantity 45,000MW
  44. 44. www.apx.nl Main Page Market Results Welcome to APX! APX is the first electronic energy trading Prices (Eur/MWh) 400 platform in continental Europe. The daily spot 350 Week 25 market has been operational since May 1999. 300 Week 26 250 The spot market enables distributors, 200 producers, traders, brokers and industrial 150 end-users to buy and sell electricity on a 100 50 day-ahead basis. 0 Volumes (MWh) The APX-index will be published daily around 2500 MWh 12h00 (GMT +01:00) to provide transparency 2000 MWh in the market. Prices can be used as a 1500 MWh benchmark. 1000 MWh 500 MWh Mon Tue Wed Thu Fri Sat Sun
  45. 45. Second Welfare Theorem Each player independently optimizes ... Consumer: value of consumption 5000 minus prices paid 4000 minus disaster WD (t) := v min D(t),G p (t) + Ga (t) − pp Gp (t) + pa Ga (t) + cbo Q− (t) 3000 2000 Supplier: profits from two sources of generation 1000 WS (t) := pp − cp Gp (t) + pa − ca )Ga (t) 13 17 21 13 17 21 13 17 21 13 17 21 13 17 21 1 5 9 1 5 9 1 5 9 1 5 9 1 5 9
  46. 46. Second Welfare Theorem Is there an equilibrium price functional? Is the equilibrium efficient?? 5000 4000 3000 2000 1000 13 17 21 13 17 21 13 17 21 13 17 21 13 17 21 1 5 9 1 5 9 1 5 9 1 5 9 1 5 9
  47. 47. Second Welfare Theorem Is there an equilibrium price functional? Is the equilibrium efficient?? 5000 4000 Yes to all ! 3000 2000 p (r , d) = (v + c )I{r < 0} e e bo e 1000 13 17 21 13 17 21 13 17 21 13 17 21 13 17 21 1 5 9 1 5 9 1 5 9 1 5 9 1 5 9 cbo cost of insufficient service reserve Q(t) = q v value of consumption demand D(t) = d
  48. 48. Efficient Equilibrium Prices Normalized demand Reserve 10000 8000 6000 4000 2000 0
  49. 49. Southern California, July 8-15, 1998 ... Spinning Reserve Prices PX Prices 70 250 60 200 50 150 40 30 100 20 50 10 0 Weds Thurs Fri Sat Sun Mon Tues Weds
  50. 50. Conclusions Spinning Reserve Prices PX Prices 70 250 60 200 50 150 40 30 100 20 50 10 0 Weds Thurs Fri Sat Sun Mon Tues Weds The hedging point (affine) policy is average cost optimal Amazing solidarity between CRW and CBM models Deregulation is a disaster! Future work?
  51. 51. Extensions and future work 10000 8000 6000 4000 2000 0 Complex models: Workload or aggregate relaxations Price caps: No! Responsive demand: Yes! Is ENRON off the hook: ?
  52. 52. Extensions and future work 10000 8000 6000 4000 2000 0 Complex models: Workload or aggregate relaxations Price caps: No! Responsive demand: Yes! Is ENRON off the hook: ? What kind of society isn't structured on greed? The problem of social organization is how to set up an arrangement under which greed will do the least harm; capitalism is that kind of system. -M. Friedman
  53. 53. Epilogue 10000 8000 6000 4000 2000 0 Fundamentally, there are only two ways of coordinating the economic activities of millions. One is central direction involving the use of coercion - the technique of the army and of the modern totalitarian state. The other is voluntary cooperation of individuals - the technique of the marketplace. -Milton Friedman
  54. 54. Epilogue 10000 8000 6000 4000 2000 0 Justification: 1. Economic systems are complex 2. Regulators cannot be trusted
  55. 55. Epilogue 10000 8000 6000 4000 2000 0 Justification: 1. Economic systems are complex 2. Regulators cannot be trusted Airplanes, highway networks, cell phones... all complex
  56. 56. Epilogue 10000 8000 6000 4000 2000 0 Justification: 1. Economic systems are complex 2. Regulators cannot be trusted Airplanes, highway networks, cell phones... all complex Complexity is only inherent in the uncontrolled system: In each of these examples, the behavior of the closed loop system is very simple, provided appropriate rules of use, and appropriate feeback mechanisms are adopted.
  57. 57. References q1 − q2 ¯ ¯ 16 3 q2 ¯ 1 c ¯∗ ¯∗ q1 − q2 = log 2 γ1 c1 1 c ¯∗ q2 = log 3 γ0 c2 • M. Chen, I.-K. Cho, and S. Meyn. Reliability by design in a distributed power transmission network. Automatica 2006 (invited) • I.-K. Cho and S. P. Meyn. The dynamics of the ancillary service prices in power networks. 42nd IEEE Conference on Decision and Control. De- cember 2003 • I.-K. Cho and S. P. Meyn. Efficiency and marginal cost pricing in dy- namic competitive markets. Under revision for J. Theo. Economics. 46th IEEE Conference on Decision and Control 2006 • P. Ruiz. Reserve Valuation in Electric Power Systems. PhD disserta- tion, ECE UIUC 2008
  58. 58. Poisson’s Equation First reflection times, τp :=inf{t ≥ 0 : Q(t) = q p }, ¯ τa :=inf{t ≥ 0 : Q(t) ≥ q a } ¯ τp h(x) = Ex c(X(s)) − φ ds 0 return
  59. 59. Poisson’s Equation First reflection times, τp :=inf{t ≥ 0 : Q(t) = q p }, ¯ τa :=inf{t ≥ 0 : Q(t) ≥ q a } ¯ τp h(x) = Ex c(X(s)) − φ ds 0 Solves martingale problem, t M (t) = h(X(t)) + c(X(s)) − φ ds 0
  60. 60. Poisson’s Equation Q(t) X(t) = Ga(t) h(X(τa )) τa h(x) = Ex h(X(τa ))+ c(X(s))−φ ds 0 qa ¯ qp ¯
  61. 61. Derivative Representations 1 ∇h(x), 1 = 0, x = X(τa ) τa h(x) = Ex h(X(τa ))+ c(X(s))−φ ds 0 qa ¯ qp ¯
  62. 62. Derivative Representations 1 λa (x) = ∇c(x), 1 = ca − I{q ≤ 0}cbo 1 ∇h(x), 1 = 0, x = X(τa ) τa h(x) = Ex h(X(τa ))+ c(X(s))−φ ds 0 qa ¯ qp ¯
  63. 63. Derivative Representations τa 1 ∇h(x), 1 = Ex λa (X(t)) dt 0 τa = ca E[τa ] − cbo Ex I{Q(t) ≤ 0} dt 0 Computable based on one-dimensional height (ladder) process, H a (t) = q a − Q(t) ¯
  64. 64. Dynamic Programming Equations If q p = q p∗ and ¯ a = ¯ a∗ ¯ ¯ q q Then h solves the dynamic programming equations, 1. Poisson's equation 1 2. ∇h(x), 1 < 0, x ∈ Ra ¯ a∗ q q p∗ 1 3. ∇h(x), 0 < 0, x ∈ Rp return

×