This is an introduction to computational economics by dynamic programming. MATLAB program is provided to understand how it is solved by using statistical computing.

1. Introdunction to
Computational Economics
ver.1
27th Feb 2014
Toshifumi Kuga, CEO of TOSHI STATS SDN. BHD.

2. Theme
1. What is computational economics ?
2. What is dynamic programming (DP) ?
3. How can DP be applied to economics ?
2

3. 1. What is computational economics ?
What is computational economics?
•
Computational economics explores the intersection of
economics and computation. These areas include agent-based
computational modeling, computational econometrics and
statistics, computational finance, computational modeling of
dynamic macroeconomic systems, computational tools for the
design of automated Internet markets, programming tools
specifically designed for computational economics, and
pedagogical tools for the teaching of computational economics.
from the website of Society of Computational Economics
3

4. 1. What is computational economics ?
What are algorithms in computational economics?
•
Dynamic programming (explained in this presentation)
•
Agent Based models
•
Dynamic stochastic general equilibrium models
•
Artificial intelligence
•
Genetic algorithms
•
There are a lot of other algorithms
4

5. 1. What is computational economics ?
What are tools in computational economics?
•
MATLAB (propriety, used in this presentation)
•
R language
•
Python
•
Mathematica (propriety)
•
Maple (propriety)
•
There are a lot of other tools
5

6. 2. What is dynamic programming ?
What is dynamic programming (DP) ?
1. It was developed by Richard Bellman (1957)
2. the mathematical theory of multi-stage decision
processes
3. An optimal policy has the property that, what the initial
state and decision are, the remaining decisions must
constitute an optimal policy with regard to the state
resulting from the first decision.
6

7. 2. What is dynamic programming ?
What is Bellman equation ?
1. The principle of optimality is expressed in the form
of the Bellman equation
•
v(s) =max { f(s,x)+δ p(s /s,x)v(s ) }
2. The need of optimally balance an immediately
reward against expected future reward
7

8. 2. What is dynamic programming ?
Structure of Bellman equation
Current Value
=
Immediate
Reward
+
Future Value
maximize
Keep the balance between Reward and Future Value
by optimizing of policy path
8

9. 3. How can DP be applied to economics?
How can DP be applied to economics?
1. state matrix
2. policy matrix
3. reward matrix
4. transition matrix (deterministic or stochastic)
5. value function(path of optimal policy should be determined)
9

10. 3. How can DP be applied to economics?
Problem : Mine management
1. A mine operator decides how much ore to extract from a mine
2. The price of extracted ore is p=1 (JPY) per ton
3. The total cost of extracting x tons of ore in any year, given that the mine
contains s tons at the beginning of the year, is (x^2)/(1+s) (JPY)
4. The mine current contains S=5 tons of ore
5. Discount factor is 0.9 per a year
6. Assuming the amount of ore extracted in any year must be an integer number
of tons, what extraction schedule maximizes profits?
10

11. 3. How can DP be applied to economics?
Bellman equation of this problem
•
v(s) = max { price*x-(x^2)/(1+s)+δv(s-x) }
immediate
reward
11
future
value

12. 3. How can DP be applied to economics?
How to solve the Bellman equation
Value function iteration
1. Given the current value v, update the value vnew
•
vnew ← max { f(x)+δp(x)v }
2. If Δv
0, stop the process, If not, return step1
12

13. 3. How can DP be applied to economics?
(1) state matrix : S
0
final state
1
2
state
3
initial state
4
5
13

14. 3. How can DP be applied to economics?
(2) policy matrix : X
policy
0
1
2
3
4
Any policy can be taken
to maximize the value function
14
5

15. 3. How can DP be applied to economics?
(3) transition matrix : g
policy
0
2
3
4
5
0
1
-
-
-
-
-
1
state
1
2
1
-
-
-
-
2
3
2
1
-
-
-
3
4
3
2
1
-
-
4
5
4
3
2
1
-
5
6
5
4
3
2
1
15
index of
state matrix

16. 3. How can DP be applied to economics?
(4) reward matrix : f
policy
0
2
3
4
5
0
0
-
-
-
-
-
1
state
1
0
0.5
-
-
-
-
2
0
0.666 0.666
3
0
0.75
4
0
0.8
5
0
-
-
-
1.0
0.75
-
-
1.2
1.2
0.8
-
0.833 1.333 1.5 1.333 0.833
16
reward can be
obtained by
revenue and cost

17. 3. How can DP be applied to economics?
(5) value function : v
value
0
1
state
?
?
2
?
3
?
4
?
5
obtain value matrix
by value function iteration
?
17

18. 3. How can DP be applied to economics?
(6) optimal state path
optimal state path
0
1
time
?
?
2
?
3
?
4
?
5
obtain optimal state path
so that the value function can be maximized
?
18

19. 3. How can DP be applied to economics?
Result : value function v
value
state
0
1
3
0.5
2 1.1167
2.25
value
0
Value
1.5
3 1.7550
0.75
4 2.3795
5 2.9749
0
0
1
2
3
state
19
4
5

20. 3. How can DP be applied to economics?
Result : optimal state path
optimal state path
1
time
2
3
5
5
4
4
3
2
state
0
optimal state path
3
2
4
1
1
5
0
0
0
1
2
3
time
20
4
5

21. 3. How can DP be applied to economics?
MATLAB program : reward matrix f
function f=reward(S,X,p)!
n=length (S);!
m=length (X);!
f=zeros(n,m);!
for i=1:n!
for k=1:m!
if X(k)<=S(i)!
f(i,k)=p*X(k)-(X(k)^2)./(1+S(i));!
else!
f(i,k)=-inf;!
end!
end!
end!
21

22. 3. How can DP be applied to economics?
MATLAB program : transition matrix g
function g=transition(S,X)!
n=length(S);!
m=length(X);!
g=zeros(n,m);!
for i=1:n!
for k=1:m!
snext=S(i)-X(k);!
if snext<=0;!
snext=0;!
end
!
g(i,k)=find(S==snext);!
end!
end!
22

23. 3. How can DP be applied to economics?
MATLAB program: function iteration
function dp(f,g,v)!
[n,m]=size(f);!
for i=1:10!
p=sparse(1:n*m, g(:),1,n*m,n)*v;!
pp=reshape(p,n,m);!
ppp=f+0.9*pp!
[vnew,xstar]=max(ppp,[],2);!
if abs(vnew(n)-v(n))<=10^(-6),break,end !
v=vnew !
xstar!
end!
v=vnew !
xstar
23

24. lecture 4 How to learn PA in R channel
MATLAB ®
•
MATLAB® is a high-level language
and interactive environment for
numerical computation,
visualization, and programming
•
http://www.mathworks.com
24

25. lecture 4 How to learn PA in R channel
reference
1. Applied Computational Economics and Finance
•
MIT Press (2002)
•
Mario J. Miranda、 Paul L. Fackler
2. DYNAMIC PROGRAMMING
•
Dover Publications, Inc. (2003)
•
Richard Bellman
25

26. Thanks for your attentions
!
• TOSHI STATS SDN. BHD
statistical computing
• CEO : Toshifumi Kuga
• http://www.toshistats.jimdo.co
• https://www.facebook.com/toshistatsco
Please do not hesitate to send your opinion and
•
massages about our courses
26

27. Disclaimer
•
TOSHI STATS SDN. BHD. and I do not accept any responsibility or
liability for loss or damage occasioned to any person or property
through using materials, instructions, methods or ideas contained
herein, or acting or refraining from acting as a result of such use.
TOSHI STATS SDN. BHD. and I expressly disclaim all implied
warranties, including merchantability or fitness for any particular
purpose. There will be no duty on TOSHI STATS SDN. BHD. and
me to correct any errors or defects in the codes and the
software.
© 2014 TOSHI STATS SDN. BHD. All rights reserved
27