Discrete Dynamic Optimization:Six ExamplesDr.Tai-kuang HoAssociate Professor.Department of Quantitative Finance,National Tsing Hua University,No. 101,Section2,Kuang-Fu Road,Hsinchu,Taiwan30013,Tel:+886-3-571-5131,ext.62136,Fax: +886-3-562-1823,E-mail:tkho@.tw.Example1:the neoclassical growth model Uhlig(1999),section4Model principlesSpecify the environment explicitly:1.Preferences2.Technologies3.EndowmentsrmationState the object of study:1.The social planner’s problem2.The competition equilibrium3.The gameThe environment:Preferences:the representative agent experiences utility according to35U=E t241X t=0 t C1 t 110< <1>0Absolute risk aversion: u00(c)u0(c)Relative risk aversion: cu00(c)u0(c)is the coe¢cient of relative risk aversionTechnologies:we assume a Cobb-Douglas production functionY t=Z t K t N1tBudget constraint:K t+1=I t+(1 )K tC t+I t=Y tor equivalently+(1 )K tC t+K t+1=Z t K t N1t0< <10< <1Z t,the total factor productivity,is exogenously evolving according tolog Z t=(1 )log Z+log Z t 1+ tt i:i:d:N 0; 20<<1Endowment:N t=1K0Information:C t,N t,and K t+1need to be chosen based on all information I t up to time t.The social planner’s problemThe objective of the social planner is to maximize the utility of the representative agent subject to feasibility,max (C t;K t)1t=0E t241X t=0 t C1 t 1135s:t:K0;Z0C t+K t+1=Z t K t+(1 )K tlog Z t=(1 )log Z+log Z t 1+ tt i:i:d:N 0; 2To solve it,form the Lagrangian:L=max(C t;K t+1)1t=0E t1X t=024 t C1 t 11+ t t Z t K t+(1 )K t C t K t+135The…rst order conditions are:@L:0=C t+K t+1 Z t K t (1 )K t@ t@L:0=C t t@C t@L:0= t+ E t h t+1 Z t+1K 1t+1+(1 ) i @K t+1Notice that K t+1appears in period t and period t+1.The…rst order conditions are often also called Euler equations. One also obtains the transversality condition.0=limE0h T C T K T+1iT!1The transversality condition is a limiting Kuhn-Tucker condition.It means that in net present value terms,the agent should not have capital left over at in…nity.Rewrite the necessary conditions:C t=Z t K t+(1 )K t K t+1(1)+(1 )(2)R t= Z t K 1t1=E t" C t C t+1! R t+1#(3) log Z t=(1 )log Z+log Z t 1+ t; t i:i:d:N 0; 2 (4) Equation accounting.To solve for the steady state,dropping the time indices and yieldC= Z K +(1 ) K KR= Z K 1+(1 )1= RFrom the welfare theorems,the solution to the competitive equilibrium yields the same allocation as the solution to the social planner’s problem.The case for competitive equilibrium is provided in Uhlig(1999)section4.4.Example2:Hansen’s real business cycle model Uhlig(1999),section4Hansen’s real business cycle modelThe model is an extension of the stochastic neoclassical growth model. The main di¤erence is to endogenize the labour supply.The social planner solves the problem of the representative agent:max E t 1X t=1 t0@C1 t 11AN t1As:t:C t+I t=Y tK t+1=I t+(1 )K tY t=Z t K t N1tlog Z t=(1 )log Z+log Z t 1+ t; t i:i:d:N 0; 2 Hansen only consider the case =1,so that the objective function is:1X t=1 t(log C t AN t)E tProof:lim !1C1t 1 1=lim!1e(1 )ln C t 11=lim!1@h e(1 )ln C t 1i=@@[1 ]=@=lim!1e(1 )ln C t ( ln C t)1=e0 ( ln C t)1=ln C tTo solve it,form the Lagrangian:L=max(C t;N t;K t+1)1t=0E t1X t=02664 t C1 t 11 AN t!+ t t Z t K t N1 t+(1 )K t C t K t+13775The…rst order conditions are:@L@C t=C t t=0@L@N t= A+ t(1 )Z t K t N t=0@L@K t+1= t+ E t t+1 Z t+1K 1t+1N1 t+1+(1 ) =0After arrangementA=C t(1 )Y t Nt(1)R t= Y tK t+1 (2)1= E t" C t C t+1! R t+1#(3) The steady state for the real business cycle model above is obtained by droppingthe time subscripts and stochastic shocks in the equations above.A= C (1 ) Y N1= RR= YK+1Exercise:Equation accounting.Example 3:Brock-Mirman growth model IChow (1997),chapter 2Consider the Brock-Mirman growth model:max f c t gE t1Xt =0t ln c ts:t:k t+1=k t z t c tThe Lagrangian is:1X t=0h t ln c t+ t t(k t z t c t k t+1)i L=E tThe…rst order conditions are:@L @c t = t1c ttt!=0@L@k t+1= t t+ t+1E t t+1 k 1t+1z t+1 =0 The…rst order conditions can be rearranged as:1c t= tt= E t t+1k 1t+1z t+1 Solve the problem explicitly.k T+1=0c T=k T z Tc t=d k t z t(guess a solution)E t t +1k 1t +1z t +1=E t1c t +1k 1t +1z t +1!=E t1d k t +1z t +1k 1t +1z t +1!=E t1d 1k t +1!=1d 1k t z t c tt =E t t +1k 1t +1z t +1,1c t = 1d 1kt z t c tc t =d(k t z t c t )d k t z t=d(k t z t d k t z t)1=1(1 d)d=1)c t=d k t z t=(1 )k t z tExample 4:Brock-Mirman growth model II Chow (1997),chapter 2We shall use dynamic programming to solve the Brock-Mirman growth model. Consider the Brock-Mirman growth model:max f c t g E t 1X t =0 t ln c ts:t:k t+1=k t z t c t Control variable:c tState variables:k t;z tThe Bellman equation is:[ln c t+ E t V(k t+1;z t+1)]V(k t;z t)=maxc tWe conjecture value function takes the form:V(k t;z t)=a+b ln k t+c ln z ta,b,c are coe¢cients to be determined.E t V(k t+1;z t+1)V(k t+1;z t+1)=a+b ln k t+1+c ln z t+1=a+b ln(k t z t c t)+c ln z t+1*E t ln z t+1=0)E t V(k t+1;z t+1)=a+b ln(k t z t c t) It followsV (k t ;z t )=max c t [ln c t + E t V (k t +1;z t +1)]=max c t f ln c t + [a +b ln (k t z t c t )]g max c t fgFirst order condition.@fg @c t =1c t bk t z t c t =0)c t=k t z t1+ bSubstitute this optimal c t=k t z t1+ b into maxc t fg and equate the expression to thevalue function,we can solve the coe¢cients a,b,c,which after some algebraic manipulation are:a=(1 ) 1h ln(1 )+ (1 ) 1ln( )ib= (1 ) 1c=(1 ) 1Example5:the money-in-the-utility function(MIU) modelWalsh(2003),chapter2Utility function,labour supply,and production function.u c t;m t;l t =C1 t1 + l1 t1C t h ac1 b t+(1 a)m1 b t i11 bn t=1 l t=f(k t;n t;z t) y t=e z t k t n1tz t= z t 1+e tTwo state variables.a t t+(1+i t 1)b t 11+ t +m t 11+ tk t The objective function is:1X t=0 t u c t;m t;1 n tE ts:t:f(k t;n t;z t)+(1 )k t+a t=c t+k t+1+m t+b t The Bellman equation is:V(a t;k t)=max h u c t;m t;1 n t + E t V(a t+1;k t+1)i Use the de…nition of a t to substitute for a t+1.a t+1 t+1+(1+i t)b t1+ t+1+m t1+ t+1Use the budget constraint to eliminate k t+1.k t +1=f (k t ;n t ;z t )+(1 )k t +a t c t m t b tRewrite the Bellman equation as:V (a t ;k t )=max h u c t ;m t;1 n t+ E t V (a t +1;k t +1)i=max 26664u c t ;m t;1 n t+ E t V 0@ t +1+(1+i t )b t 1+ t +1+m t 1+ t +1;f (k t ;n t ;z t )+(1 )k t +a t c t m t b t1A 37775=maxc t ;b t ;m t ;n tfgThe…rst order conditions are:@fg@c t:u c c t;m t;1 n t E t V k(a t+1;k t+1)=0@fg @b t : E t1+i t1+ t+1!V a(a t+1;k t+1) E t V k(a t+1;k t+1)=0@fg@m t:u m c t;m t;1 n t + E t11+ t+1!V a(a t+1;k t+1) E t V k(a t+1;k t+1)=0@fg@n t: u n c t;m t;1 n t + E t V k(a t+1;k t+1)f n(k t;n t;z t)=0@V(a t;k t)@a t =@fg@a t:V a(a t;k t)= E t V k(a t+1;k t+1)@V(a t;k t)@k t =@fg@k t:V k(a t;k t)= E t V k(a t+1;k t+1)[f k(k t;n t;z t)+1 ]Resources constrains.y t+(1 )k t=c t+k t+1We can eliminate the partial di¤erentiations of value function in the F.O.C. and then simplify the equilibrium conditions into9equations with9endogenous variables to be determined.Equilibrium conditions include F.O.C.,de…nitions,and resources constraints. y t;c t;k t+1;m t;n t;R t; t;z t;u tu c c t;m t;1 n t =u m c t;m t;1 n t + E t 24u c c t+1;m t+1;1 n t+11+ t+135(1)u l c t;m t;1 n t =u c c t;m t;1 n t f n(k t;n t;z t)(2) u c c t;m t;1 n t = E t R t u c c t+1;m t+1;1 n t+1 (3) R t=1 +f k(k t;n t;z t)(4)y t+(1 )k t=c t+k t+1(5)y t=f(k t;n t;z t)(6)m t=1+ t1+ t!m t 1(7)z t= z t 1+e t(8) u t= u t 1+ z t 1+'t(9)Exercises:use Lagrangian to solve the MIU model.L =1Xt =0t uc t ;m t;1 n t+1Xt =0 t +1E t t +124f (k t ;n t ;z t )+(1 )k t + t +(1+i t 1)b t 11+ t +m t 11+ t c t k t +1 m t b t 35@L @ct ;@L @m t ;@L @n t ;@L @k t +1;@L @b tShow that combing the F.O.C.of the Lagrangian method gives us the same equations as by using the Bellman equation.Example6:the cash-in-advance(CIA)model Walsh(2003),chapter3,a certainty caseUtility function:1X t=0 t u(c t)CIA constraint:。