The document discusses optimal control and agent-based economic models. It summarizes key concepts from neoclassical production theory including the production function, Cobb-Douglas production function, and capital dynamics. It also discusses the utility function, consumption, savings and investment. The document then provides an overview of concepts in optimal control theory including the Hamiltonian, Pontryagin's maximum principle, and infinite horizon problems. It concludes with an introduction to the Ramsey-Cass-Koopmans model for determining optimal savings.
Optimal Control in Agent-based Economics: A Survey
1. āThe laws of history are as absolute at the laws of physics, and if the probabilities of error are greater, it is only because history does not deal with as many humans as physics does atoms, so that individual variations count for more.ā
Isaac Asimov, Foundation and Empire (1952)
2. Optimal Control in Agent-based Economic Models: A Survey
James Matthew B. Miraflor | CS 296 - Seminar
3. āThis is what I mean by the āmechanicsā of economic development - the construction of a mechanical, artificial world, populated by the interacting robots that economics typically studies, that is capable of exhibiting behavior the gross features of which resemble those of the actual world.ā
Robert Lucas, Jr., āOn the Mechanics of Economic Developmentā. Journal of Monetary Economics. 1988.
4. The āRobotsā = āRepresentative Agentsā
ā¢Firms. Production function
ā¢Consumers. Utility function
ā¢Government
āUsually is a constraint rather than an objective function
āBehavior ā Efficient or Inefficient? Benevolent or Corrupt?
ā¢Optimization
ā¢Equilibrium
āWhen does an economy stop growing? At what point do firms stop producing, consumers stop consuming, government stop spending?
6. The Neoclassical Production Function
ā¢Production function: ķ=ķ¹(ķ¾,ķæ,ķ“)
āwhere ķ is the output, ķ“ is the total factor productivity/technology, ķæ is the labor input, ķ¾ is the capital input
āsatisfies conditions below:
1.Constant returns to scale:
ķ¹ķķ¾,ķķæ,ķķ“=ķķ¹(ķ¾,ķæ,ķ“),āķ>0
āReplication argument
2.Positive and diminishing returns
o ķķ¹ ķķ¾ >0, ķ2ķ¹ ķķ¾2<0, ķķ¹ ķķæ >0, ķ2ķ¹ ķķæ2<0
9. Capital Dynamics
ā¢Change in capital is produced good not consumed.
ā¢ķ =ķķāķā(ķ+ķæ)ķ, where:
āķ is the per capita capital (ķ¾/ķæ), ķ = ķķ ķķ” is the change in capital, or the investment, due to savings.
āķæ is the depreciation rate of capital
āķ is the population rate
ā¢Notice that ķ behaves like a depreciation rate since it represents the fraction of resources to be given to the next generation.
10. Savings/Investment and Consumption
Capital (k)
Output (y)
Gross
Product
f(k)
Some level of Capital (k)
Actual GDP (y)
Actual
Savings
(s*y)
Gross Savings s*f(k)
consumption per worker
investment per worker
Borrowed/derived from DE201 lecture slides of Prof. Emmanuel de Dios
11. Gross
Product
f(k)
k*
Gross Savings s*f(k)
Population Growth
k1
k2
Investment is greater than population growth; capital per person increases
Population growth is greater than investment; capital per person decreases.
Savings/Investment and Consumption
Capital
(k)
Output (y)
Borrowed/derived from DE201 lecture slides of Prof. Emmanuel de Dios
12. Savings and Economic Growth
Gross Product f(k)
Gross Savings s*f(k)
High economic growth
Zero growth
Negative economic growth
Population Growth
Capital
(k)
Output (y)
Borrowed/derived from DE201 lecture slides of Prof. Emmanuel de Dios
13. Cobb-Douglas Production Function
ķ=ķ¹ķ¾,ķæ=ķ“ķæķ¼ķ¾ķ½
ā¢ķ¼ and ķ½ are the output elasticities of capital & labor
āmeasures the responsiveness of output to a change in either labor or capital, ceteris paribus.
ā¢If we want production per capita, we divide the function buy ķæ to get ķ ķæ =ķ“ķæķ¼ā1ķ¾ķ½=ķ“ķæķ¼āķ½ā1ķ¾ ķæ ķ½ āķ¦=ķ“ķæķ¼āķ½ā1ķķ½
ā¢where ķ¦ and ķ are per capita production and per capita capital respectively.
ā¢Conventionally, we set ķ¼+ķ½=1 so that ķ½=1āķ¼
15. The Utility Function
ā¢ķķ”=ķķ” represents the consumption at time ķ”
ā¢ķ¢ķ”= ķ¢ķķ”=ķ¢(ķķ”) represents the utility of consumers from consuming ķķ”.
ā¢ķ0 is the total, accumulated utility over infinite time of the consumer, i.e. ķ0= ķāķķ” ķ¢ķķ” ķķ” ā 0
oIf ķæķ” is the population level at time ķ”, we have: ķ0= ķāķķ” ķ¢ķķ”ķæķ” ķķ” ā 0
16. Constant Elasticity of Substitution
ā¢If production is Cobb-Douglas, then the necessary and sufficient conditions for optimal savings (Kurz, 1968) :
1.A ķ¢(ķķ”) must be Constant Elasticity of Intertemporal Substitution (CEIS)
ķ¢ķķ”= ķķ” 1āķā11āķ
āwhere ķ= 1 ķ and ķ is the savings rate
āConstant aversion to fluctuations in consumption
āOne doesnāt get more or less risk averse as one gets richer (or poorer).
2.Discount rate ķ must be related to the parameters of ķ=ķ½āķ , where ķ½ is share of capital to production.
17. Optimal Cumulative Consumption
ā¢A consumer agent will want to maximize consumption over time, i.e. maxķ0= ķā(ķāķ)ķ” ķķ” 1āķā11āķ ķķ” ā 0
ķ .ķ”. ķ =ķķāķāķķāķæķ, ķ0>0 ķķķ£ķķ
ā¢The solution to this optimal control problem will then govern the dynamics of savings across time.
ā¢How do we solve?
19. The Lagrangean (Static Optimization)
mķķ„ {ķ(ķ„)|ķķ =0,ļ¢ķļķ¾} ā¢Given the function ķ¦=ķ(ķ„1,ā¦,ķ„ķ) subject to a constraint ķķ(ķ„1,ā¦,ķ„ķ)=0, ķ=1,ā¦,ķ the Lagrangean is
ķæķ„;ķ=ķķ„+ ķķķķ(ķ„) ķ ķ=1
ā¢where the ļ¬ is the vector of Lagrangean multipliers.
ā¢Let ķ¦=ķķ„1,ķ„2, ķķ„1,ķ„2=0
ķæķ„1,ķ„2,ķ=ķķ„1,ķ„2+ķķ(ķ„1,ķ„2)
ā¢To determine ķ„1ā,ķ„2ā,ķā: ķķæ ķķ„1= ķķķ„1,ķ„2 ķķ„1+ ķķķ„1,ķ„2 ķķ„1=0 ķķæ ķķ„2= ķķķ„1,ķ„2 ķķ„2+ ķķķ„1,ķ„2 ķķ„2=0
20. The Hamiltonian (Dynamic Optimization)
ā¢Definition. Given the problem
ķķķ„ ķ½= ķķ„,ķ¢,ķ”ķķ” ķ 0
ķ.ķ”. ķ„ā²ķ”=ķ(ķ„,ķ¢,ķ”)
ķ„0=ķ„0,ķ„ķ free,ķ fixed
ķ¢ķ”ķ ā
ā¢The Hamiltonian of the problem is the function
ķ»ķ„,ķ¢,ķ,ķ”=ķķ„,ķ¢,ķ”+ķķ(ķ„,ķ¢,ķ”)
ā¢ķ(ķ”) is called the costate variable.ā”
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
21. Problem of Dynamic Optimization
ā¢From among functions x ķC1[0,T] starting at (0,x0) and ending at (T,xT), choose an x* such that J(x*) ā„ J(x).
ā¢Note that in Optimal Control terms, we are actually selecting a u (governing x) that optimizes x
0 T
x
x0
xT
x*(t)
Figure 1
ā”
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
22. The Pontryagin Maximum Principle in Terms of the Hamiltonian
ā¢Theorem. Given the problem
ā¢ķķķ„ ķ½= ķķ„,ķ¢,ķ”ķķ” ķ 0
ā¢ķ.ķ”. ķ„ā²ķ”=ķ(ķ„,ķ¢,ķ”)
ā¢ ķ„0=ķ„0,ķ„ķ free,ķ fixed
ā¢ u(t) ķ ā
ā¢ķ»ķ„,ķ¢,ķ,ķ”=ķķ„,ķ¢,ķ”+ķķ(ķ„,ķ¢,ķ”)
ā¢If the pair (u*(t), x*(t)) is optimal, then there is a continuously differentiable function ķ(ķ”) such that:
a. ķķ» ķķ = x*ā² (HĪ» = x*ā²)
b. ķķ» ķķ„ =āķā² (Hx = āĪ»ā²)
c. ķķ» ķķ¢ =0 (Hu = 0)
d.x*(0) = x0
e.Ī»(T) = 0
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
23. Infinite Horizon Problems
ā¢The Infinite Horizon Optimal Control Problem
ā¢ķķķ„ ķķ„,ķ¢,ķ”ķāķķ”ķķ” ā 0
ā¢ķ.ķ”. ķ„ā²ķ”=ķ(ķ„,ķ¢,ķ”)
ā¢ ķ„0=ķ„0,ķ¢(ķ”)āķ
ķ»ķ„,ķ¢,ķ,ķ”=ķķ„,ķ¢,ķ”ķāķķ”+ķķ(ķ„,ķ¢,ķ”)
ā¢The objective function is sensible only if, for all admissible pairs (ķ„,ķ¢,ķ”), the integral converges.
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
24. Infinite Horizon Problems
Transversality Conditions. ķ ķ„ā ķķķķ
lim ķ”āā ķķ”=0 ķ ķ„āā„ķ„ķķķ
lim ķ”āā ķ(ķ”)ā„0 lim ķ”āā ķķ”=0 ķķ ķ„ā>ķ„ķķķ ķ ķ„ā ķķķ„ķķ
lim ķ”āā ķ»=0
TVC is a description of how the optimal path crosses a terminal line in variable endpoint problems.
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
26. The Ramsey-Cass-Koopmans Model
ā¢Question: How much should a nation save?
ā¢maxķ0= ķā(ķāķ)ķ” ķķ” 1āķā11āķ ķķ” ā 0
ā¢ķ .ķ”. ķ =ķķāķāķķāķæķ, ķ0>0 ķķķ£ķķ
ā¢The solution to this optimal control problem will then govern the dynamics of savings across time.
ā¢Note that lim ķ”āā ķā(ķāķ)ķ” ķķ” 1āķā11āķ =0
oOne can verify that for this to happen, ķ>ķ
27. Solving the Model
ā¢Setting up the Hamiltonian
ķ»=ķā(ķāķ)ķ” ķķ” 1āķā11āķ +ķ(ķķāķāķķāķæķ)
ā¢Where ķ is the dynamic Lagrange multiplier
oķ can also be interpreted as the shadow price of investment
ā¢The First Order Conditions (FOCs) are:
oķ»ķ=0
oķ»ķ=āķ
oTransversality condition (TVC): lim ķ”āā ķķ”ķ£ķ”=0
28. ķ»=ķā(ķāķ)ķ” ķķ” 1āķā11āķ +ķ(ķķāķāķķāķæķ)
ā¢ķ»ķ=0āķā(ķāķ)ķ”ķķ” āķāķ=0 (1)
ā¢ķ»ķ=āķ āķķā²ķāķāķæ=āķ
oā ķ ķ =ķā²ķāķāķæ (2)
ā¢Take logs and time derivative of (1):
oāķāķķ”āķlnķķ”=lnķ
oāķāķāķ ķ ķ = ķ ķ ā ķ ķ =ķā1(āķ+ķā ķ ķ ) (3)
ā¢Plug (3) to (2) to get:
oķ¾ķ= ķ ķ =ķā1ķā²(ķ)āķāķæ
oIn the Cobb-Douglas case: ķ¾ķ=ķā1ķ½ķā(1āķ½)āķāķæ
29. Equilibrium in Consumption
ķ¾ķ= ķ ķ =ķā1ķā²(ķ)āķāķæ =ķā1ķ½ķā(1āķ½)āķāķæ
ā¢In equilibrium, i.e. ķ ķ =0, ķā²ķ=ķ+ķæ
oIf consumption is to remain at its current level, marginal return to capital must at least reach the level of the combined future discounting and capital depreciation
oAt this level, an individual is indifferent between consuming and spending.
30. Transitional Dynamics
From our constraint: ķ =ķķāķāķķāķæķ
ā¢ ķ ķ = ķķāķ ķ āķāķæ
ā¢ ķ ķ =ķā1ķā²(ķ)āķāķæ
ā¢In equilibrium
o ķ ķ =0āķ=ķķāķ+ķæķ
o ķ ķ =0āķā²(ķ)=ķ+ķæ
31. ķ =0 curve: ķ=ķ(ķ)ā(ķ+ķæ)ķ ķ =0 curve: ķā²(ķ)=ķ+ķæ
(ķ+ķæ)ķ
ķ(ķ)
ķ
ķ
ķ
ķā
ķ =0
ķ =0
ķø
ķø = steady state
ķ0
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
32. ķā
ķā
ķø
ķ =0
ķ =0
ķ
ķ
+
ā
+
ā
The Ramsey Model
Stable branch
Unstable branch
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
33. The Ramsey Model
ā¢The steady state E = (k*,c*) is a saddle-point equilibrium. It is unique.
ā¢At equilibrium, k* is a constant; hence, y* = f(k*) is a constant. Since k ā” K/L and y ā” Y/L, then at E, the variables Y, K, and L all grow at the same rate.
ā¢The only way for the economy to move toward the steady state is to hitch onto a stable branch.
ā¢Given an initial capital-labor ratio k0, it must choose an initial per capita consumption level c0 such that the pair (k0,c0) lies on the stable branch.
34. ķā
ķā
ķø
ķ =0
ķ =0
ķ
ķ
+
ā
+
ā
The Ramsey Model
Stable branch
Unstable branch
If the economy is not on a stable branch, ever-increasing k accompanied by ever-decreasing c leads per capita consumption towards starvation.
35. ķā
ķā
ķø
ķ =0
ķ =0
ķ
ķ
+
ā
+
ā
The Ramsey Model
Stable branch
Unstable branch
If economy is not on a stable branch: Ever-increasing c accompanied by ever-decreasing k leads to capital exhaustion.
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao
36. The Ramsey Model
ā¢If the economy is not on a stable branch, the dynamic forces lead to:
āEver-increasing ķ accompanied by ever- decreasing ķ (streamlines pointing to the southeast) leading per capita consumption towards starvation.
āEver-increasing c accompanied by ever- decreasing ķ (streamlines pointing to the northwest) implying overindulgence leading to capital exhaustion.
ā¢The only viable long-run alternative for the economy is the steady state at ķø.
37. The Ramsey Model
ā¢At steady state, per-capita consumption becomes constant and its level cannot be further raised over time.
ā¢Thatās because the production function does not include technological progress.
ā¢To make a possible rising per-capita consumption, technological progress must be introduced.
38. ROLE OF THE GOVERNMENT
The Barro Model of Public Spending
Leyte Vice Gov. Carlo Loreto
Tacloban City Vice Mayor Jerry Yaokasin
Gov. Dom Petilla
Coca-Cola FEMSA CEO Juan Ramon Felix
Coca-Cola Corporate Affairs director Jose Dominguez
Reopening of Plant at Tacloban City
39. Barro Model of Public Spending
ā¢Economist Robert J. Barro (1990) builds on the Ramsey-Cass-Koopmans model to propose a āpublic spendingā model:
ķ¦=ķķ,ķ=ķ“ķ(1āķ¼)ķķ¼
oķ is the governmentās input to production (infrastructures, highways, public works, etc.)
oķ¼ is the elasticity of governmentās share to production
oķ is financed entirely from tax revenues.
oportion of the economy is taxed so the government can spend.
40. Effect of Taxes
oDefining ķ as the constant average and marginal income tax rate, we then have
ķ= ķķ¦=ķķ“ķ(1āķ¼)ķķ¼.
ā¢Note also, that disposable savings is less than that earlier, since aside from consumption, a portion (ķ) of production is taxed. In this case, our constraint is transformed into: ķ =1āķķ“ķ1āķ¼ķķ¼āķ, ķ0>0 ķķķ£ķķ
41. Barroās Optimal Control Problem
ā¢The optimal control problem in Barro (1990) then becomes:
ā¢maxķ0= ķāķķ” ķķ” 1āķā11āķ ķķ” ā 0
ā¢ķ .ķ”. ķ =1āķķ“ķ1āķ¼ķķ¼āķ, ķ0> 0 ķķķ£ķķ
ā¢where ķ= ķķ¦=ķķ“ķ(1āķ¼)ķķ¼
ā¢This is the ācompetitive caseā, wherein consumer agents take government spending as a given and then optimize.
42. Competitive Case
ā¢Setup the Hamiltonian
ķ»=ķāķķ” ķķ” 1āķā11āķ +ķ1āķķ“ķ1āķ¼ķķ¼āķ
ā¢The First Order Conditions (FOCs) are:
o(1) ķ»ķ=0āķāķķ”ķāķ=ķ
o(2) ķ»ķ=āķ āķ =āķ1āķķ“ķāķ¼ķķ¼= āķ£1āķķ“1āķ¼ ķ ķ ķ¼
o(3) TVC
43. Competitive Case
ā¢For (1), taking log of both sides and differentiating, we get:
o ķāķķ”ķāķ=ķāāķķ”āķlnķ=lnķ
o ķ ķķ” āķķ”āķlnķ= ķ ķķ” lnķāāķāķ ķ ķ = ķ ķ (3)
oSubstitute (3) into (2) to get:
o ķ ķ =ā1āķķ“1āķ¼ ķ ķ ķ¼
oāķāķ ķ ķ =ā1āķķ“1āķ¼ ķ ķ ķ¼
o ķ ķ =ķā11āķķ“1āķ¼ ķ ķ ķ¼ āķ (4)
44. Size of the Government
ķ= ķķ¦=ķķ“ķ(1āķ¼)ķķ¼
ā¢We can get the size of the government ķ by
oķ= ķ ķ¦ = ķ ķ“ķ1āķ¼ķķ¼= ķ ķ 1āķ¼ ķ“ā1ā
o ķ ķ =(ķķ“)1/(1āķ¼)
ā¢Plug it into (4) to get:
o ķ ķ =ķā11āķķ“1āķ¼ ķ ķ ķ¼ āķ
ā¢ķ¾1= ķ ķ =ķā1ķ“1āāķ
owhere ķ“1ā=1āķ¼ķ“ 11āķ¼1āķķ ķ¼ 1āķ¼
45. Command Economy
ā¢In a command economy, the government will take into account that private output affects public income and (through the production function) other peopleās marginal product of capital.
ā¢To solve, subsitute ķ“1ā=1āķ¼ķ“ 11āķ¼1āķķ ķ¼ 1āķ¼ in the A part of the Hamiltonian
ā¢ķ»= ķāķķ” ķķ” 1āķā11āķ +ķ1āķķ“ķ1āķ¼ķķ¼āķ
46. Command Economy
ā¢The FOCs are:
oķāķķ”ķāķ=ķ
o ķ ķ =ā1āķķ“ 11āķ¼ķ ķ¼ 1āķ¼āķ
ā¢Substituting the usual way we obtain:
ā¢ķ¾2= ķ ķ =ķā1ķ“2āāķ
ā¢Where ķ“2ā=1āķķ“ 11āķ¼ķ ķ¼ 1āķ¼
47. Efficiency of the Command Economy
ā¢ķ¾1= ķ ķ =ķā1ķ“1āāķ
owhere ķ“1ā=1āķ¼ķ“ 11āķ¼1āķķ ķ¼ 1āķ¼
ā¢ķ¾2= ķ ķ =ķā1ķ“2āāķ
āWhere ķ“2ā=1āķķ“ 11āķ¼ķ ķ¼ 1āķ¼
ā¢Note that since for all values of ķ, ķ“1ā<ķ“2āāķ¾1<ķ¾2
ā¢Government is forced to provide one more unit of public input for every unit of savings by individuals.
ā¢The assumption is that the government is a ābenevolent dictatorā.
ā¢What if it is not?
49. Ellis & Fender (2006) Model
ā¢Takes off from a Ramsey type model of economic growth in which the āengine of growthā is public capital accumulation.
ā¢Public capital financed by taxes on private output.
ā¢Government can either use taxes to fund public capital accumulation or engage in corruption.
ā¢Ellis & Fender defines output as:
ķ¦ķ”=ķķķ”,ķķ”=ķķ”ķ¼ķ(ķ”)ķ½,0<ķ¼,ķ½<1
ā¢where ķ(ķ”) is the effective labor and ķ(ķ”) is the public capital.
50. Ellis & Fender (2006) Model
ā¢Public capital then moves according to: ķ =ķķ”āķāķķ”āķāķķ(ķ”)
ā¢where:
oķķ”āķ represents the taxes paid at interval of length ķ in the past,
oķķ”āķ is portion of past tax payments that were corrupted by the government
oķ is capital depreciation.
ā¢Interval ķ is production lag
āBut their subsequent results demonstrate that it can also be seen as ātransparencyā
51. Optimal Consumption
ā¢Goal of consumer-citizens - maximize accumulated consumption
ā¢Let ķķ” be the consumption and āķķ” be the decision to pursue leisure.
ķ¢ķ”=ķ¢ķķ”,ķķ”=[ķķ”+ķķ”]ķ
ā¢where 0<ķ<1 is the intertemporal substitution parameter (similar to 1āķ earlier), and r is the discount rate (similar to ķ earlier).
āinstantaneous budget constraint ķķ”=ķ¦ķ”āķķ” must be satisfied.
ā¢Optimization problem max [ķķ”+ķķ”]ķķāķķ” ķķ” ā 0
52. Optimal Corruption
ā¢Goal of the government - maximize its accumulated corruption.
ā¢Having defined ķ earlier: max ķķ”ķāķķ” ķķ” ā 0
ā¢We set up the second goal as a constraint together with the equation of motion ķ
ā¢Reduces to an isoperimetric (due to the integral term in the constraint) Ramsey-type optimization problem
54. āThe relevant question to ask about the āassumptionsā of a theory is not whether they are descriptively ārealistic,ā for they never are, but whether they are sufficiently good approximations for the purpose in hand. And this question can be answered only by seeing whether the theory works, which means whether it yields sufficiently accurate predictions.ā
Milton Friedman, āThe Methodology of Positive Economicsā (1966)
Thank you for listening!
55. Sources
ā¢Ellis, Christopher James & John Fender (2006, May). āCorruption and Transparency in a Growth Modelā. International Tax and Public Finance. Volume 13, Issue 2-3: 115-149.
ā¢Sala-i-Martin, Xavier (1990a, December). āLecture Notes on Economic Growth(I): Introduction to the Literature and Neoclassical Modelsā. NBER Working Paper No. 3563.
ā¢Sala-i-Martin, Xavier (1990b, December). āLecture Notes on Economic Growth(II): Five Prototype Models of Endogenous Growthā. NBER Working Paper No. 3564.