SlideShare a Scribd company logo

Stochastic Control of Optimal Trade Order Execution

How to sell a large block in a fixed period of time, to maximize sale proceeds, managing Price Impact on the Order Book.

1 of 16
Download to read offline
Stochastic Control of Optimal Trade Order Execution
Ashwin Rao
ICME, Stanford University
December 14, 2018
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 1 / 16
Overview
1 Order Book and Price Impact
2 Definition of Optimal Trade Order Execution Problem
3 Simple Models, leading to Analytical Solutions
4 Real-World Considerations, Extensions, Reinforcement Learning
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 2 / 16
Trading Order Book
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 3 / 16
Basics of Trading Order Book
Buyers/Sellers express their intent to trade by submitting bids/asks
These are Limit Orders (LO) with a size and price
Order Book (OB) aggregates order sizes for each unique price
So we can represent with two sorted lists of (Price, Size) pairs
Bids: [(P
(b)
i ,N
(b)
i ) 1 ≤ i ≤ m],P
(b)
i > P
(b)
j for i < j
Asks: [(P
(a)
i ,N
(a)
i ) 1 ≤ i ≤ n],P
(a)
i < P
(a)
j for i < j
We call P
(b)
1 as simply Bid, P
(a)
1 as Ask,
P
(a)
1 +P
(b)
1
2 as Mid
We call P
(a)
1 − P
(b)
1 as Spread, P
(a)
n − P
(b)
m as Market Depth
A new Sell LO (P,N) such that P = P
(b)
i for some 1 ≤ i ≤ n will
remove min(N,N
(b)
i ) from the existing LO (likewise for new Buy LO)
A new LO not matching any current LO Price (of the opposite side)
will simply be added to the OB
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 4 / 16
Price Impact and Order Book Dynamics
A Sell Market Order N will remove the best bid prices on the OB
Removal: [(P
(b)
i ,min(N
(b)
i ,max(0,N −
i−1
∑
j=1
N
(b)
j ))) 1 ≤ i ≤ m]
A Buy Market Order N will remove the best ask prices on the OB
Removal: [(P
(a)
i ,min(N
(a)
i ,max(0,N −
i−1
∑
j=1
N
(a)
j ))) 1 ≤ i ≤ n]
A large-sized MO often results in a big Spread which could soon be
replenished by new LOs, potentially from either side
So a large-sized MO moves the Bid/Ask/Mid (Price Impact of MO)
Subsequent Replenishment activity is part of Order Book Dynamics
Models for Order Book Dynamics can be quite complex
We will cover a few simple Models in this lecture
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 5 / 16
Optimal Trade Order Execution Problem
The task is to sell a large number N of shares
We are allowed to trade in T discrete time steps
We are only allowed to submit Market Orders
We consider both Temporary and Permanent Price Impact
For simplicity, we consider a model of just the Bid Price Dynamics
Goal is to maximize Expected Total Utility of Sales Proceeds
By breaking N into appropriate chunks (timed appropriately)
If we sell too fast, we are likely to get poor prices
If we sell too slow, we risk running out of time
Selling slowly also leads to more uncertain proceeds (lower Utility)
This is a Dynamic Optimization problem
We can model this problem as a Markov Decision Process (MDP)
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 6 / 16
Problem Notation
Time steps indexed by t = 1,...,T
Pt denotes Bid Price at start of time step t
Nt denotes number of shares sold in time step t
Rt = N − ∑t−1
i=1 Ni = shares remaining to be sold at start of time step t
Note that R1 = N,NT = RT
Price Dynamics given by:
Pt+1 = ft(Pt,Nt, t)
where ft(⋅) is an arbitrary function incorporating:
Permanent Price Impact of selling Nt shares
Impact-independent market-movement of Bid Price over time step t
t denotes source of randomness in Bid Price market-movement
Sales Proceeds in time step t defined as:
Nt ⋅ Qt = Nt ⋅ (Pt − gt(Pt,Nt))
where gt(⋅) is an arbitrary func representing Temporary Price Impact
Utility of Sales Proceeds function denoted as U(⋅)
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 7 / 16
Markov Decision Process (MDP) Formulation
Order of MDP activity in each time step 1 ≤ t ≤ T:
Observe State = (t,Pt,Rt)
Perform Action = Nt
Receive Reward = U(Nt ⋅ Qt) = U(Nt ⋅ (Pt − gt(Pt,Nt)))
Experience Price Dynamics Pt+1 = ft(Pt,Nt, t)
Goal is to find a Policy π∗
(t,Pt,Rt) = Nt that maximizes:
E[
T
∑
t=1
γt
⋅ U(Nt ⋅ Qt)] where γ is MDP discount factor
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 8 / 16
A Simple Linear Impact Model with No Risk-Aversion
We consider a simple model with Linear Price Impact
N,Nt,Pt are all continuous-valued (∈ R)
In particular, we allow Nt to be possibly negative (unconstrained)
Price Dynamics: Pt+1 = Pt − αNt + t where α ∈ R+
t is i.i.d. with E[ t Nt,Pt] = 0
So, Permanent Price Impact is αNt
Temporary Price Impact given by βNt, so Qt = Pt − βNt (β ∈ R+
)
Utility function U(⋅) is the identity function, i.e., no Risk-Aversion
MDP Discount factor γ = 1
This is an unrealistic model, but solving this gives plenty of intuition
Approach: Define Optimal Value Function & invoke Bellman Equation
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 9 / 16
Optimal Value Function and Bellman Equation
Denote Value Function for policy π as:
V π
(t,Pt,Rt) = Eπ[
T
∑
t=1
Nt(Pt − βNt) (t,Pt,Rt)]
Denote Optimal Value Function as V ∗
(t,Pt,Rt) = maxπV π
(t,Pt,Rt)
Optimal Value Function satisfies the Bellman Equation (∀1 ≤ t < T):
V ∗
(t,Pt,Rt) = max
Nt
(Nt(Pt − βNt) + E[V ∗
(t + 1,Pt+1,Rt+1)])
Note: V ∗
(T,PT ,RT ) = RT (PT − βRT )
From the above, we can infer V ∗
(T − 1,PT−1,RT−1) as:
max
NT−1
{NT−1(PT−1 − βNT−1) + E[RT (PT − βRT )]}
= max
NT−1
{NT−1(PT−1 − βNT−1) + E[(RT−1 − NT−1)(PT − β(RT−1 − NT−1))}
= max
NT−1
{NT−1(PT−1−βNT−1)+(RT−1−NT−1)(PT−1−αNT−1−β(RT−1−NT−1))}
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 10 / 16
Optimal Policy and Optimal Value Function
Differentiating this expression w.r.t. NT−1 and setting to 0 gives:
2N∗
T−1(α − 2β) − RT−1(α − 2β) = 0 ⇒ N∗
T−1 =
RT−1
2
Substitute N∗
T−1 in the expression for V ∗
(T − 1,PT−1,RT−1):
V ∗
(T − 1,PT−1,RT−1) = RT−1PT−1 − R2
T−1(
α + 2β
4
)
Continuing backwards in time in this manner gives:
N∗
t =
Rt
T − t + 1
V ∗
(t,Pt,Rt) = RtPt −
R2
t
2
(
2β + (T − t)α
T − t + 1
)
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 11 / 16
Interpreting the solution
Rolling forward in time, we see that N∗
t = N
T , i.e., uniformly split
Hence, Optimal Policy is a constant (independent of State)
Uniform split makes intuitive sense because Price Impact and Market
Movement are both linear and additive, and don’t interact
Essentially equivalent to maximizing ∑T
t=1 N2
t with ∑T
t=1 Nt = N
Optimal Expected Total Sale Proceeds = NP1 − N2
2 (α + 2β−α
T )
So, Implementation Shortfall from Price Impact is N2
2 (α + 2β−α
T )
Note that Implementation Shortfall is non-zero (αN2
2 ) when T → ∞
This is because we assumed non-zero Permanent Price Impact (α ≠ 0)
If Price Impact were purely temporary (i.e., Price fully snapped back),
Implementation Shortfall would be zero when T → ∞
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 12 / 16
Models in Bertsimas-Lo paper
Bertsimas-Lo was the first paper on Optimal Trade Order Execution
They assumed no risk-aversion, i.e. identity Utility function
The first model in their paper is a special case of our simple Linear
Impact model, with fully Permanent Impact (i.e., α = β)
Next, Betsimas-Lo extended the Linear Permanent Impact model
To include dependence on Serially-Correlated Variable Xt
Pt+1 = Pt − (αNt + θXt) + t,Xt+1 = ρXt + ηt,Qt = Pt − (αNt + θXt)
t and ηt are i.i.d. (and mutually independent) with mean zero
Xt can be thought of as market factor affecting Pt linearly
Bellman Equation on Optimal VF and same approach as before yields:
N∗
t =
Rt
T − t + 1
+ h(t,α,θ,ρ)Xt
V ∗
(t,Pt,Rt,Xt) = RtPt + quadratic in (Rt,Xt) + constant
Seral-correlation predictability (ρ ≠ 0) alters uniform-split strategy
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 13 / 16
A more Realistic Model: LPT Price Impact
Next, Bertsimas-Lo present a more realistic model called “LPT”
Linear-Percentage Temporary Price Impact model features:
Geometric random walk: consistent with real data, & avoids prices ≤ 0
% Price Impact gt (Pt ,Nt )
Pt
doesn’t depend on Pt (validated by real data)
Purely Temporary Price Impact
Pt+1 = PteZt
,Xt+1 = ρXt + ηt,Qt = Pt(1 − αNt − θXt)
Zt is a random variable with mean µZ and variance σ2
Z
With the same derivation as before, we get the solution:
S∗
t = c
(1)
t + c
(2)
t Rt + c
(3)
t Xt
V ∗
(t,Pt,Rt,Xt) = eµZ +
σ2
Z
2 ⋅ Pt ⋅ (c
(4)
t + c
(5)
t Rt + c
(6)
t Xt
+ c
(7)
t R2
t + c
(8)
t X2
t + c
(9)
t RtXt)
c
(k)
t ,1 ≤ k ≤ 9 are independent of Pt,Rt,Xt
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 14 / 16
Incorporating Risk-Aversion/Utility of Proceeds
For analytical tractability, Bertsimas-Lo ignored Risk-Aversion
But one is typically wary of Risk of Uncertain Proceeds
We’d trade some (Expected) Proceeds for lower Variance of Proceeds
Almgren-Chriss work in this Risk-Aversion framework
They consider our simple linear model maximizing E[Y ] − λVar[Y ]
Where Y is the total (uncertain) proceeds ∑T
i=1 Ni Qi
λ controls the degree of risk-aversion and hence, the trajectory of N∗
t
λ = 0 leads to uniform split strategy N∗
t = N
T
The other extreme is to minimize Var[Y ] which yields N∗
1 = N
Almgren-Chriss derive Efficient Frontier and solutions for specific U(⋅)
Much like classical Portfolio Optimization problems
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 15 / 16
Real-world Optimal Trade Order Execution (& Extensions)
Arbitrary Price Dynamics ft(⋅) and Temporary Price Impact gt(⋅)
Non-stationarity/non-linear dynamics/impact require (Numerical) DP
Frictions: Discrete Prices/Sizes, Constraints on Prices/Sizes, Fees
Large State space to incorporate various external factors in the State
Need to utilize methods in Approximate Dynamic Programming
And if model is unknown/to be learnt, Reinforcement Learning
This problem can be extended in two important ways:
State is Complete Order Book (Model of Order Book Dynamics)
Optimal Execution of a Portfolio (Cross-Asset Impact Modeling)
Can this be combined with Portfolio Optimization problem?
Can we exploit recent advances in Deep Reinforcement Learning?
Exciting area for Future Research as well as Engineering Design
Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 16 / 16

Recommended

Fundamental Theorems of Asset Pricing
Fundamental Theorems of Asset PricingFundamental Theorems of Asset Pricing
Fundamental Theorems of Asset PricingAshwin Rao
 
Stochastic Control/Reinforcement Learning for Optimal Market Making
Stochastic Control/Reinforcement Learning for Optimal Market MakingStochastic Control/Reinforcement Learning for Optimal Market Making
Stochastic Control/Reinforcement Learning for Optimal Market MakingAshwin Rao
 
HJB Equation and Merton's Portfolio Problem
HJB Equation and Merton's Portfolio ProblemHJB Equation and Merton's Portfolio Problem
HJB Equation and Merton's Portfolio ProblemAshwin Rao
 
Overview of Stochastic Calculus Foundations
Overview of Stochastic Calculus FoundationsOverview of Stochastic Calculus Foundations
Overview of Stochastic Calculus FoundationsAshwin Rao
 
Adaptive Multistage Sampling Algorithm: The Origins of Monte Carlo Tree Search
Adaptive Multistage Sampling Algorithm: The Origins of Monte Carlo Tree SearchAdaptive Multistage Sampling Algorithm: The Origins of Monte Carlo Tree Search
Adaptive Multistage Sampling Algorithm: The Origins of Monte Carlo Tree SearchAshwin Rao
 
Policy Gradient Theorem
Policy Gradient TheoremPolicy Gradient Theorem
Policy Gradient TheoremAshwin Rao
 
Risk-Aversion, Risk-Premium and Utility Theory
Risk-Aversion, Risk-Premium and Utility TheoryRisk-Aversion, Risk-Premium and Utility Theory
Risk-Aversion, Risk-Premium and Utility TheoryAshwin Rao
 

More Related Content

What's hot

Pricing Exotics using Change of Numeraire
Pricing Exotics using Change of NumerairePricing Exotics using Change of Numeraire
Pricing Exotics using Change of NumeraireSwati Mital
 
CS229 Machine Learning Lecture Notes
CS229 Machine Learning Lecture NotesCS229 Machine Learning Lecture Notes
CS229 Machine Learning Lecture NotesEric Conner
 
Machine learning (1)
Machine learning (1)Machine learning (1)
Machine learning (1)NYversity
 
Ada boost brown boost performance with noisy data
Ada boost brown boost performance with noisy dataAda boost brown boost performance with noisy data
Ada boost brown boost performance with noisy dataShadhin Rahman
 
Black scholes pricing concept
Black scholes pricing conceptBlack scholes pricing concept
Black scholes pricing conceptIlya Gikhman
 
Black scholes pricing concept
Black scholes pricing conceptBlack scholes pricing concept
Black scholes pricing conceptIlya Gikhman
 
Expanding further the universe of exotic options closed pricing formulas in t...
Expanding further the universe of exotic options closed pricing formulas in t...Expanding further the universe of exotic options closed pricing formulas in t...
Expanding further the universe of exotic options closed pricing formulas in t...caplogic-ltd
 
Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Asma Ben Slimene
 
Cash Flow Valuation Mode Lin Discrete Time
Cash Flow Valuation Mode Lin Discrete TimeCash Flow Valuation Mode Lin Discrete Time
Cash Flow Valuation Mode Lin Discrete TimeIOSR Journals
 
A Maximum Entropy Approach to the Loss Data Aggregation Problem
A Maximum Entropy Approach to the Loss Data Aggregation ProblemA Maximum Entropy Approach to the Loss Data Aggregation Problem
A Maximum Entropy Approach to the Loss Data Aggregation ProblemErika G. G.
 
Fin econometricslecture
Fin econometricslectureFin econometricslecture
Fin econometricslectureNBER
 
Pages from fin econometrics brandt_1
Pages from fin econometrics brandt_1Pages from fin econometrics brandt_1
Pages from fin econometrics brandt_1NBER
 
Max Entropy
Max EntropyMax Entropy
Max Entropyjianingy
 
A Tutorial of the EM-algorithm and Its Application to Outlier Detection
A Tutorial of the EM-algorithm and Its Application to Outlier DetectionA Tutorial of the EM-algorithm and Its Application to Outlier Detection
A Tutorial of the EM-algorithm and Its Application to Outlier DetectionKonkuk University, Korea
 
Interactive Visualization in Human Time -StampedeCon 2015
Interactive Visualization in Human Time -StampedeCon 2015Interactive Visualization in Human Time -StampedeCon 2015
Interactive Visualization in Human Time -StampedeCon 2015StampedeCon
 

What's hot (20)

Pricing Exotics using Change of Numeraire
Pricing Exotics using Change of NumerairePricing Exotics using Change of Numeraire
Pricing Exotics using Change of Numeraire
 
CS229 Machine Learning Lecture Notes
CS229 Machine Learning Lecture NotesCS229 Machine Learning Lecture Notes
CS229 Machine Learning Lecture Notes
 
Machine learning (1)
Machine learning (1)Machine learning (1)
Machine learning (1)
 
Basic concepts and how to measure price volatility
Basic concepts and how to measure price volatility Basic concepts and how to measure price volatility
Basic concepts and how to measure price volatility
 
Ada boost brown boost performance with noisy data
Ada boost brown boost performance with noisy dataAda boost brown boost performance with noisy data
Ada boost brown boost performance with noisy data
 
Black scholes pricing concept
Black scholes pricing conceptBlack scholes pricing concept
Black scholes pricing concept
 
Black scholes pricing concept
Black scholes pricing conceptBlack scholes pricing concept
Black scholes pricing concept
 
Expanding further the universe of exotic options closed pricing formulas in t...
Expanding further the universe of exotic options closed pricing formulas in t...Expanding further the universe of exotic options closed pricing formulas in t...
Expanding further the universe of exotic options closed pricing formulas in t...
 
Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...
 
Cash Flow Valuation Mode Lin Discrete Time
Cash Flow Valuation Mode Lin Discrete TimeCash Flow Valuation Mode Lin Discrete Time
Cash Flow Valuation Mode Lin Discrete Time
 
A Maximum Entropy Approach to the Loss Data Aggregation Problem
A Maximum Entropy Approach to the Loss Data Aggregation ProblemA Maximum Entropy Approach to the Loss Data Aggregation Problem
A Maximum Entropy Approach to the Loss Data Aggregation Problem
 
Vidyasagar rocond09
Vidyasagar rocond09Vidyasagar rocond09
Vidyasagar rocond09
 
Fin econometricslecture
Fin econometricslectureFin econometricslecture
Fin econometricslecture
 
Pages from fin econometrics brandt_1
Pages from fin econometrics brandt_1Pages from fin econometrics brandt_1
Pages from fin econometrics brandt_1
 
Max Entropy
Max EntropyMax Entropy
Max Entropy
 
Two Curves Upfront
Two Curves UpfrontTwo Curves Upfront
Two Curves Upfront
 
Ceske budevice
Ceske budeviceCeske budevice
Ceske budevice
 
18.1 combining models
18.1 combining models18.1 combining models
18.1 combining models
 
A Tutorial of the EM-algorithm and Its Application to Outlier Detection
A Tutorial of the EM-algorithm and Its Application to Outlier DetectionA Tutorial of the EM-algorithm and Its Application to Outlier Detection
A Tutorial of the EM-algorithm and Its Application to Outlier Detection
 
Interactive Visualization in Human Time -StampedeCon 2015
Interactive Visualization in Human Time -StampedeCon 2015Interactive Visualization in Human Time -StampedeCon 2015
Interactive Visualization in Human Time -StampedeCon 2015
 

Similar to Stochastic Control of Optimal Trade Order Execution

Market risk and liquidity of the risky bonds
Market risk and liquidity of the risky bondsMarket risk and liquidity of the risky bonds
Market risk and liquidity of the risky bondsIlya Gikhman
 
Hull White model presentation
Hull White model presentationHull White model presentation
Hull White model presentationStephan Chang
 
lecture 1
lecture 1lecture 1
lecture 1sajinsc
 
Financial Trading as a Game: A Deep Reinforcement Learning Approach
Financial Trading as a Game: A Deep Reinforcement Learning ApproachFinancial Trading as a Game: A Deep Reinforcement Learning Approach
Financial Trading as a Game: A Deep Reinforcement Learning Approach謙益 黃
 
Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Asma Ben Slimene
 
Intro to Quantitative Investment (Lecture 1 of 6)
Intro to Quantitative Investment (Lecture 1 of 6)Intro to Quantitative Investment (Lecture 1 of 6)
Intro to Quantitative Investment (Lecture 1 of 6)Adrian Aley
 
Planning Under Uncertainty With Markov Decision Processes
Planning Under Uncertainty With Markov Decision ProcessesPlanning Under Uncertainty With Markov Decision Processes
Planning Under Uncertainty With Markov Decision Processesahmad bassiouny
 
Pricing average price advertising options when underlying spot market prices ...
Pricing average price advertising options when underlying spot market prices ...Pricing average price advertising options when underlying spot market prices ...
Pricing average price advertising options when underlying spot market prices ...Bowei Chen
 
Bond Pricing and CVA
Bond Pricing and CVABond Pricing and CVA
Bond Pricing and CVAIlya Gikhman
 
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...Lake Como School of Advanced Studies
 
Skiena algorithm 2007 lecture09 linear sorting
Skiena algorithm 2007 lecture09 linear sortingSkiena algorithm 2007 lecture09 linear sorting
Skiena algorithm 2007 lecture09 linear sortingzukun
 
MM framework for RL
MM framework for RLMM framework for RL
MM framework for RLSung Yub Kim
 
Scalable inference for a full multivariate stochastic volatility
Scalable inference for a full multivariate stochastic volatilityScalable inference for a full multivariate stochastic volatility
Scalable inference for a full multivariate stochastic volatilitySYRTO Project
 
Optimization Methods in Finance
Optimization Methods in FinanceOptimization Methods in Finance
Optimization Methods in Financethilankm
 
Anlysis and design of algorithms part 1
Anlysis and design of algorithms part 1Anlysis and design of algorithms part 1
Anlysis and design of algorithms part 1Deepak John
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Vai Jayanthi
 

Similar to Stochastic Control of Optimal Trade Order Execution (20)

Market risk and liquidity of the risky bonds
Market risk and liquidity of the risky bondsMarket risk and liquidity of the risky bonds
Market risk and liquidity of the risky bonds
 
Hull White model presentation
Hull White model presentationHull White model presentation
Hull White model presentation
 
lecture 1
lecture 1lecture 1
lecture 1
 
Financial Trading as a Game: A Deep Reinforcement Learning Approach
Financial Trading as a Game: A Deep Reinforcement Learning ApproachFinancial Trading as a Game: A Deep Reinforcement Learning Approach
Financial Trading as a Game: A Deep Reinforcement Learning Approach
 
Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...
 
Intro to Quantitative Investment (Lecture 1 of 6)
Intro to Quantitative Investment (Lecture 1 of 6)Intro to Quantitative Investment (Lecture 1 of 6)
Intro to Quantitative Investment (Lecture 1 of 6)
 
Planning Under Uncertainty With Markov Decision Processes
Planning Under Uncertainty With Markov Decision ProcessesPlanning Under Uncertainty With Markov Decision Processes
Planning Under Uncertainty With Markov Decision Processes
 
Pricing average price advertising options when underlying spot market prices ...
Pricing average price advertising options when underlying spot market prices ...Pricing average price advertising options when underlying spot market prices ...
Pricing average price advertising options when underlying spot market prices ...
 
American option
American optionAmerican option
American option
 
Bond Pricing and CVA
Bond Pricing and CVABond Pricing and CVA
Bond Pricing and CVA
 
Randomized algorithms ver 1.0
Randomized algorithms ver 1.0Randomized algorithms ver 1.0
Randomized algorithms ver 1.0
 
analysis.ppt
analysis.pptanalysis.ppt
analysis.ppt
 
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...
Econophysics VI: Price Cross-Responses in Correlated Financial Markets - Thom...
 
Skiena algorithm 2007 lecture09 linear sorting
Skiena algorithm 2007 lecture09 linear sortingSkiena algorithm 2007 lecture09 linear sorting
Skiena algorithm 2007 lecture09 linear sorting
 
MM framework for RL
MM framework for RLMM framework for RL
MM framework for RL
 
Scalable inference for a full multivariate stochastic volatility
Scalable inference for a full multivariate stochastic volatilityScalable inference for a full multivariate stochastic volatility
Scalable inference for a full multivariate stochastic volatility
 
Optimization Methods in Finance
Optimization Methods in FinanceOptimization Methods in Finance
Optimization Methods in Finance
 
Anlysis and design of algorithms part 1
Anlysis and design of algorithms part 1Anlysis and design of algorithms part 1
Anlysis and design of algorithms part 1
 
Cambridge
CambridgeCambridge
Cambridge
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1
 

More from Ashwin Rao

Evolutionary Strategies as an alternative to Reinforcement Learning
Evolutionary Strategies as an alternative to Reinforcement LearningEvolutionary Strategies as an alternative to Reinforcement Learning
Evolutionary Strategies as an alternative to Reinforcement LearningAshwin Rao
 
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...Ashwin Rao
 
Understanding Dynamic Programming through Bellman Operators
Understanding Dynamic Programming through Bellman OperatorsUnderstanding Dynamic Programming through Bellman Operators
Understanding Dynamic Programming through Bellman OperatorsAshwin Rao
 
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...Ashwin Rao
 
Value Function Geometry and Gradient TD
Value Function Geometry and Gradient TDValue Function Geometry and Gradient TD
Value Function Geometry and Gradient TDAshwin Rao
 
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...Ashwin Rao
 
A Quick and Terse Introduction to Efficient Frontier Mathematics
A Quick and Terse Introduction to Efficient Frontier MathematicsA Quick and Terse Introduction to Efficient Frontier Mathematics
A Quick and Terse Introduction to Efficient Frontier MathematicsAshwin Rao
 
Towards Improved Pricing and Hedging of Agency Mortgage-backed Securities
Towards Improved Pricing and Hedging of Agency Mortgage-backed SecuritiesTowards Improved Pricing and Hedging of Agency Mortgage-backed Securities
Towards Improved Pricing and Hedging of Agency Mortgage-backed SecuritiesAshwin Rao
 
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural Network
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural NetworkRecursive Formulation of Gradient in a Dense Feed-Forward Deep Neural Network
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural NetworkAshwin Rao
 
Demystifying the Bias-Variance Tradeoff
Demystifying the Bias-Variance TradeoffDemystifying the Bias-Variance Tradeoff
Demystifying the Bias-Variance TradeoffAshwin Rao
 
Category Theory made easy with (ugly) pictures
Category Theory made easy with (ugly) picturesCategory Theory made easy with (ugly) pictures
Category Theory made easy with (ugly) picturesAshwin Rao
 
Risk Pooling sensitivity to Correlation
Risk Pooling sensitivity to CorrelationRisk Pooling sensitivity to Correlation
Risk Pooling sensitivity to CorrelationAshwin Rao
 
Abstract Algebra in 3 Hours
Abstract Algebra in 3 HoursAbstract Algebra in 3 Hours
Abstract Algebra in 3 HoursAshwin Rao
 
OmniChannelNewsvendor
OmniChannelNewsvendorOmniChannelNewsvendor
OmniChannelNewsvendorAshwin Rao
 
The Newsvendor meets the Options Trader
The Newsvendor meets the Options TraderThe Newsvendor meets the Options Trader
The Newsvendor meets the Options TraderAshwin Rao
 
The Fuss about || Haskell | Scala | F# ||
The Fuss about || Haskell | Scala | F# ||The Fuss about || Haskell | Scala | F# ||
The Fuss about || Haskell | Scala | F# ||Ashwin Rao
 
Career Advice at University College of London, Mathematics Department.
Career Advice at University College of London, Mathematics Department.Career Advice at University College of London, Mathematics Department.
Career Advice at University College of London, Mathematics Department.Ashwin Rao
 
Careers outside Academia - USC Computer Science Masters and Ph.D. Students
Careers outside Academia - USC Computer Science Masters and Ph.D. StudentsCareers outside Academia - USC Computer Science Masters and Ph.D. Students
Careers outside Academia - USC Computer Science Masters and Ph.D. StudentsAshwin Rao
 
IIT Bombay - First Steps to a Successful Career
IIT Bombay - First Steps to a Successful CareerIIT Bombay - First Steps to a Successful Career
IIT Bombay - First Steps to a Successful CareerAshwin Rao
 
Stanford FinMath - Careers in Quant Finance
Stanford FinMath - Careers in Quant FinanceStanford FinMath - Careers in Quant Finance
Stanford FinMath - Careers in Quant FinanceAshwin Rao
 

More from Ashwin Rao (20)

Evolutionary Strategies as an alternative to Reinforcement Learning
Evolutionary Strategies as an alternative to Reinforcement LearningEvolutionary Strategies as an alternative to Reinforcement Learning
Evolutionary Strategies as an alternative to Reinforcement Learning
 
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...
Principles of Mathematical Economics applied to a Physical-Stores Retail Busi...
 
Understanding Dynamic Programming through Bellman Operators
Understanding Dynamic Programming through Bellman OperatorsUnderstanding Dynamic Programming through Bellman Operators
Understanding Dynamic Programming through Bellman Operators
 
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...
A.I. for Dynamic Decisioning under Uncertainty (for real-world problems in Re...
 
Value Function Geometry and Gradient TD
Value Function Geometry and Gradient TDValue Function Geometry and Gradient TD
Value Function Geometry and Gradient TD
 
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...
Stanford CME 241 - Reinforcement Learning for Stochastic Control Problems in ...
 
A Quick and Terse Introduction to Efficient Frontier Mathematics
A Quick and Terse Introduction to Efficient Frontier MathematicsA Quick and Terse Introduction to Efficient Frontier Mathematics
A Quick and Terse Introduction to Efficient Frontier Mathematics
 
Towards Improved Pricing and Hedging of Agency Mortgage-backed Securities
Towards Improved Pricing and Hedging of Agency Mortgage-backed SecuritiesTowards Improved Pricing and Hedging of Agency Mortgage-backed Securities
Towards Improved Pricing and Hedging of Agency Mortgage-backed Securities
 
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural Network
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural NetworkRecursive Formulation of Gradient in a Dense Feed-Forward Deep Neural Network
Recursive Formulation of Gradient in a Dense Feed-Forward Deep Neural Network
 
Demystifying the Bias-Variance Tradeoff
Demystifying the Bias-Variance TradeoffDemystifying the Bias-Variance Tradeoff
Demystifying the Bias-Variance Tradeoff
 
Category Theory made easy with (ugly) pictures
Category Theory made easy with (ugly) picturesCategory Theory made easy with (ugly) pictures
Category Theory made easy with (ugly) pictures
 
Risk Pooling sensitivity to Correlation
Risk Pooling sensitivity to CorrelationRisk Pooling sensitivity to Correlation
Risk Pooling sensitivity to Correlation
 
Abstract Algebra in 3 Hours
Abstract Algebra in 3 HoursAbstract Algebra in 3 Hours
Abstract Algebra in 3 Hours
 
OmniChannelNewsvendor
OmniChannelNewsvendorOmniChannelNewsvendor
OmniChannelNewsvendor
 
The Newsvendor meets the Options Trader
The Newsvendor meets the Options TraderThe Newsvendor meets the Options Trader
The Newsvendor meets the Options Trader
 
The Fuss about || Haskell | Scala | F# ||
The Fuss about || Haskell | Scala | F# ||The Fuss about || Haskell | Scala | F# ||
The Fuss about || Haskell | Scala | F# ||
 
Career Advice at University College of London, Mathematics Department.
Career Advice at University College of London, Mathematics Department.Career Advice at University College of London, Mathematics Department.
Career Advice at University College of London, Mathematics Department.
 
Careers outside Academia - USC Computer Science Masters and Ph.D. Students
Careers outside Academia - USC Computer Science Masters and Ph.D. StudentsCareers outside Academia - USC Computer Science Masters and Ph.D. Students
Careers outside Academia - USC Computer Science Masters and Ph.D. Students
 
IIT Bombay - First Steps to a Successful Career
IIT Bombay - First Steps to a Successful CareerIIT Bombay - First Steps to a Successful Career
IIT Bombay - First Steps to a Successful Career
 
Stanford FinMath - Careers in Quant Finance
Stanford FinMath - Careers in Quant FinanceStanford FinMath - Careers in Quant Finance
Stanford FinMath - Careers in Quant Finance
 

Recently uploaded

VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlue
VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlueVM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlue
VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlueShapeBlue
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceVijayananda Mohire
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerSaiLinnThu2
 
software-quality-assurance question paper 2023
software-quality-assurance question paper 2023software-quality-assurance question paper 2023
software-quality-assurance question paper 2023RohanMistry15
 
iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingerssuser9354ce
 
eXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionseXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionsElanthirayan Madhavan
 
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Chris Bingham
 
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGBoosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGRick Ossendrijver
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueShapeBlue
 
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...James Anderson
 
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureMuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureManik S Magar
 
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...BookNet Canada
 
Trading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfTrading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfLucas Lagone
 
Python For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emPython For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emNho Vĩnh
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Why Disability Justice should be at the core of your digital accessibility jo...
Why Disability Justice should be at the core of your digital accessibility jo...Why Disability Justice should be at the core of your digital accessibility jo...
Why Disability Justice should be at the core of your digital accessibility jo...Modality Co
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueShapeBlue
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationAppsthentic Technology
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Jay Zhao
 

Recently uploaded (20)

VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlue
VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlueVM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlue
VM Migration from VMware to CloudStack and KVM – Suresh Anaparti, ShapeBlue
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial Intelligence
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
 
software-quality-assurance question paper 2023
software-quality-assurance question paper 2023software-quality-assurance question paper 2023
software-quality-assurance question paper 2023
 
iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostinger
 
eXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionseXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductions
 
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
 
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGBoosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
 
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
 
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureMuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
 
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
 
Trading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfTrading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdf
 
Python For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ emPython For Kids - Sách Lập trình cho trẻ em
Python For Kids - Sách Lập trình cho trẻ em
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficient
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Why Disability Justice should be at the core of your digital accessibility jo...
Why Disability Justice should be at the core of your digital accessibility jo...Why Disability Justice should be at the core of your digital accessibility jo...
Why Disability Justice should be at the core of your digital accessibility jo...
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and Optimization
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
 

Stochastic Control of Optimal Trade Order Execution

  • 1. Stochastic Control of Optimal Trade Order Execution Ashwin Rao ICME, Stanford University December 14, 2018 Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 1 / 16
  • 2. Overview 1 Order Book and Price Impact 2 Definition of Optimal Trade Order Execution Problem 3 Simple Models, leading to Analytical Solutions 4 Real-World Considerations, Extensions, Reinforcement Learning Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 2 / 16
  • 3. Trading Order Book Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 3 / 16
  • 4. Basics of Trading Order Book Buyers/Sellers express their intent to trade by submitting bids/asks These are Limit Orders (LO) with a size and price Order Book (OB) aggregates order sizes for each unique price So we can represent with two sorted lists of (Price, Size) pairs Bids: [(P (b) i ,N (b) i ) 1 ≤ i ≤ m],P (b) i > P (b) j for i < j Asks: [(P (a) i ,N (a) i ) 1 ≤ i ≤ n],P (a) i < P (a) j for i < j We call P (b) 1 as simply Bid, P (a) 1 as Ask, P (a) 1 +P (b) 1 2 as Mid We call P (a) 1 − P (b) 1 as Spread, P (a) n − P (b) m as Market Depth A new Sell LO (P,N) such that P = P (b) i for some 1 ≤ i ≤ n will remove min(N,N (b) i ) from the existing LO (likewise for new Buy LO) A new LO not matching any current LO Price (of the opposite side) will simply be added to the OB Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 4 / 16
  • 5. Price Impact and Order Book Dynamics A Sell Market Order N will remove the best bid prices on the OB Removal: [(P (b) i ,min(N (b) i ,max(0,N − i−1 ∑ j=1 N (b) j ))) 1 ≤ i ≤ m] A Buy Market Order N will remove the best ask prices on the OB Removal: [(P (a) i ,min(N (a) i ,max(0,N − i−1 ∑ j=1 N (a) j ))) 1 ≤ i ≤ n] A large-sized MO often results in a big Spread which could soon be replenished by new LOs, potentially from either side So a large-sized MO moves the Bid/Ask/Mid (Price Impact of MO) Subsequent Replenishment activity is part of Order Book Dynamics Models for Order Book Dynamics can be quite complex We will cover a few simple Models in this lecture Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 5 / 16
  • 6. Optimal Trade Order Execution Problem The task is to sell a large number N of shares We are allowed to trade in T discrete time steps We are only allowed to submit Market Orders We consider both Temporary and Permanent Price Impact For simplicity, we consider a model of just the Bid Price Dynamics Goal is to maximize Expected Total Utility of Sales Proceeds By breaking N into appropriate chunks (timed appropriately) If we sell too fast, we are likely to get poor prices If we sell too slow, we risk running out of time Selling slowly also leads to more uncertain proceeds (lower Utility) This is a Dynamic Optimization problem We can model this problem as a Markov Decision Process (MDP) Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 6 / 16
  • 7. Problem Notation Time steps indexed by t = 1,...,T Pt denotes Bid Price at start of time step t Nt denotes number of shares sold in time step t Rt = N − ∑t−1 i=1 Ni = shares remaining to be sold at start of time step t Note that R1 = N,NT = RT Price Dynamics given by: Pt+1 = ft(Pt,Nt, t) where ft(⋅) is an arbitrary function incorporating: Permanent Price Impact of selling Nt shares Impact-independent market-movement of Bid Price over time step t t denotes source of randomness in Bid Price market-movement Sales Proceeds in time step t defined as: Nt ⋅ Qt = Nt ⋅ (Pt − gt(Pt,Nt)) where gt(⋅) is an arbitrary func representing Temporary Price Impact Utility of Sales Proceeds function denoted as U(⋅) Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 7 / 16
  • 8. Markov Decision Process (MDP) Formulation Order of MDP activity in each time step 1 ≤ t ≤ T: Observe State = (t,Pt,Rt) Perform Action = Nt Receive Reward = U(Nt ⋅ Qt) = U(Nt ⋅ (Pt − gt(Pt,Nt))) Experience Price Dynamics Pt+1 = ft(Pt,Nt, t) Goal is to find a Policy π∗ (t,Pt,Rt) = Nt that maximizes: E[ T ∑ t=1 γt ⋅ U(Nt ⋅ Qt)] where γ is MDP discount factor Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 8 / 16
  • 9. A Simple Linear Impact Model with No Risk-Aversion We consider a simple model with Linear Price Impact N,Nt,Pt are all continuous-valued (∈ R) In particular, we allow Nt to be possibly negative (unconstrained) Price Dynamics: Pt+1 = Pt − αNt + t where α ∈ R+ t is i.i.d. with E[ t Nt,Pt] = 0 So, Permanent Price Impact is αNt Temporary Price Impact given by βNt, so Qt = Pt − βNt (β ∈ R+ ) Utility function U(⋅) is the identity function, i.e., no Risk-Aversion MDP Discount factor γ = 1 This is an unrealistic model, but solving this gives plenty of intuition Approach: Define Optimal Value Function & invoke Bellman Equation Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 9 / 16
  • 10. Optimal Value Function and Bellman Equation Denote Value Function for policy π as: V π (t,Pt,Rt) = Eπ[ T ∑ t=1 Nt(Pt − βNt) (t,Pt,Rt)] Denote Optimal Value Function as V ∗ (t,Pt,Rt) = maxπV π (t,Pt,Rt) Optimal Value Function satisfies the Bellman Equation (∀1 ≤ t < T): V ∗ (t,Pt,Rt) = max Nt (Nt(Pt − βNt) + E[V ∗ (t + 1,Pt+1,Rt+1)]) Note: V ∗ (T,PT ,RT ) = RT (PT − βRT ) From the above, we can infer V ∗ (T − 1,PT−1,RT−1) as: max NT−1 {NT−1(PT−1 − βNT−1) + E[RT (PT − βRT )]} = max NT−1 {NT−1(PT−1 − βNT−1) + E[(RT−1 − NT−1)(PT − β(RT−1 − NT−1))} = max NT−1 {NT−1(PT−1−βNT−1)+(RT−1−NT−1)(PT−1−αNT−1−β(RT−1−NT−1))} Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 10 / 16
  • 11. Optimal Policy and Optimal Value Function Differentiating this expression w.r.t. NT−1 and setting to 0 gives: 2N∗ T−1(α − 2β) − RT−1(α − 2β) = 0 ⇒ N∗ T−1 = RT−1 2 Substitute N∗ T−1 in the expression for V ∗ (T − 1,PT−1,RT−1): V ∗ (T − 1,PT−1,RT−1) = RT−1PT−1 − R2 T−1( α + 2β 4 ) Continuing backwards in time in this manner gives: N∗ t = Rt T − t + 1 V ∗ (t,Pt,Rt) = RtPt − R2 t 2 ( 2β + (T − t)α T − t + 1 ) Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 11 / 16
  • 12. Interpreting the solution Rolling forward in time, we see that N∗ t = N T , i.e., uniformly split Hence, Optimal Policy is a constant (independent of State) Uniform split makes intuitive sense because Price Impact and Market Movement are both linear and additive, and don’t interact Essentially equivalent to maximizing ∑T t=1 N2 t with ∑T t=1 Nt = N Optimal Expected Total Sale Proceeds = NP1 − N2 2 (α + 2β−α T ) So, Implementation Shortfall from Price Impact is N2 2 (α + 2β−α T ) Note that Implementation Shortfall is non-zero (αN2 2 ) when T → ∞ This is because we assumed non-zero Permanent Price Impact (α ≠ 0) If Price Impact were purely temporary (i.e., Price fully snapped back), Implementation Shortfall would be zero when T → ∞ Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 12 / 16
  • 13. Models in Bertsimas-Lo paper Bertsimas-Lo was the first paper on Optimal Trade Order Execution They assumed no risk-aversion, i.e. identity Utility function The first model in their paper is a special case of our simple Linear Impact model, with fully Permanent Impact (i.e., α = β) Next, Betsimas-Lo extended the Linear Permanent Impact model To include dependence on Serially-Correlated Variable Xt Pt+1 = Pt − (αNt + θXt) + t,Xt+1 = ρXt + ηt,Qt = Pt − (αNt + θXt) t and ηt are i.i.d. (and mutually independent) with mean zero Xt can be thought of as market factor affecting Pt linearly Bellman Equation on Optimal VF and same approach as before yields: N∗ t = Rt T − t + 1 + h(t,α,θ,ρ)Xt V ∗ (t,Pt,Rt,Xt) = RtPt + quadratic in (Rt,Xt) + constant Seral-correlation predictability (ρ ≠ 0) alters uniform-split strategy Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 13 / 16
  • 14. A more Realistic Model: LPT Price Impact Next, Bertsimas-Lo present a more realistic model called “LPT” Linear-Percentage Temporary Price Impact model features: Geometric random walk: consistent with real data, & avoids prices ≤ 0 % Price Impact gt (Pt ,Nt ) Pt doesn’t depend on Pt (validated by real data) Purely Temporary Price Impact Pt+1 = PteZt ,Xt+1 = ρXt + ηt,Qt = Pt(1 − αNt − θXt) Zt is a random variable with mean µZ and variance σ2 Z With the same derivation as before, we get the solution: S∗ t = c (1) t + c (2) t Rt + c (3) t Xt V ∗ (t,Pt,Rt,Xt) = eµZ + σ2 Z 2 ⋅ Pt ⋅ (c (4) t + c (5) t Rt + c (6) t Xt + c (7) t R2 t + c (8) t X2 t + c (9) t RtXt) c (k) t ,1 ≤ k ≤ 9 are independent of Pt,Rt,Xt Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 14 / 16
  • 15. Incorporating Risk-Aversion/Utility of Proceeds For analytical tractability, Bertsimas-Lo ignored Risk-Aversion But one is typically wary of Risk of Uncertain Proceeds We’d trade some (Expected) Proceeds for lower Variance of Proceeds Almgren-Chriss work in this Risk-Aversion framework They consider our simple linear model maximizing E[Y ] − λVar[Y ] Where Y is the total (uncertain) proceeds ∑T i=1 Ni Qi λ controls the degree of risk-aversion and hence, the trajectory of N∗ t λ = 0 leads to uniform split strategy N∗ t = N T The other extreme is to minimize Var[Y ] which yields N∗ 1 = N Almgren-Chriss derive Efficient Frontier and solutions for specific U(⋅) Much like classical Portfolio Optimization problems Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 15 / 16
  • 16. Real-world Optimal Trade Order Execution (& Extensions) Arbitrary Price Dynamics ft(⋅) and Temporary Price Impact gt(⋅) Non-stationarity/non-linear dynamics/impact require (Numerical) DP Frictions: Discrete Prices/Sizes, Constraints on Prices/Sizes, Fees Large State space to incorporate various external factors in the State Need to utilize methods in Approximate Dynamic Programming And if model is unknown/to be learnt, Reinforcement Learning This problem can be extended in two important ways: State is Complete Order Book (Model of Order Book Dynamics) Optimal Execution of a Portfolio (Cross-Asset Impact Modeling) Can this be combined with Portfolio Optimization problem? Can we exploit recent advances in Deep Reinforcement Learning? Exciting area for Future Research as well as Engineering Design Ashwin Rao (Stanford) Optimal Order Execution December 14, 2018 16 / 16