Your SlideShare is downloading. ×
0
5th International Summer School
Achievements and Applications of Contemporary Informatics,
Mathematics and Physics
Nationa...
Outline

•   Stochastic Differential Equations

•   Parameter Estimation

•   Various Statistical Models

•   C-MARS

•   ...
Stock Markets
Stochastic Differential Equations



       dX t   a( X t , t )dt b( X t , t )dWt
                 drift   and   diffusion...
Stochastic Differential Equations



       dX t    a( X t , t )dt b( X t , t )dWt
                   drift    and      di...
Regression

                                                 T
Input vector   X           X 1 , X 2 ,..., X m       and ou...
Generalized Additive Models

                                             m
     E Yi xi1 , xi 2 ,..., xi m       0       ...
Generalized Additive Models


 •   Given data ( yi , xi ) (i = 1,2,...,N ),



 •   penalized residual sum of squares
    ...
Generalized Additive Models

          min             t
          t , β0 , f
                                            ...
Generalized Additive Models

          min             t
          t , β0 , f
                                            ...
MARS




y                                           y




    c-(x, )=[ (x )]   c+(x, )=[ (x )]           c-(x, )=[ (x )]...
C-MARS



                         N                               M max        2
                                        ...
C-MARS

Tikhonov regularization:

                                             2                2
              PRSS      ...
Stochastic Differential Equations Revisited



       dX t    a( X t , t )dt b( X t , t )dWt
                   drift    a...
Stochastic Differential Equations Revisited



       dX t   a( X t , t )dt b( X t , t )dWt
                 drift   and  ...
Stochastic Differential Equations Revisited



       dX t    a( X t , t )dt b( X t , t )dWt
                   drift    a...
Stochastic Differential Equations


Milstein Scheme :




ˆ        ˆ        ˆ                              ˆ              ...
Stochastic Differential Equations


 •   step length    hj         tj   1    tj :     tj
                                 ...
Stochastic Differential Equations


 •   More simple form:

                                Xj         Gj    H j cj      (...
Stochastic Differential Equations
                                                                                        ...
Stochastic Differential Equations

 •    penalized sum of squares PRRS
                                                   ...
Stochastic Differential Equations
                                     T
             T       T           T
              ...
Stochastic Differential Equations

                                                      2       2                      2 ...
Stochastic Differential Equations


                       2
                                      2
     min      X    A ...
Stochastic Differential Equations

min    t
 t,

                         0N         A           t                 X
subje...
Stochastic Differential Equations


 (t , , , , 1 ,   2   ) is a primal dual optimal solution if and only if



          ...
Stochastic Differential Equations

Ex.:


              dVt      t
                        T
                            (...
Nonlinear Regression


                          N                      2
         min f                    dj   g xj ,
  ...
Nonlinear Regression

                                                             k 1   :   k   qk
 • Gauss-Newton method...
Nonlinear Regression


alternative solution



 min    t,
  t,q

                               T
 subject to         F( )...
Portfolio Optimization

     max utility !   or

     min costs !


              martingale method:




                 ...
Portfolio Optimization

     max utility !   or

     min costs !


              martingale method:

                    ...
Portfolio Optimization

     max utility !   or

     min costs !


              martingale method:




                 ...
Portfolio Optimization

     max utility !   or

     min costs !


              martingale method:




                 ...
References
Aster, A., Borchers, B., and Thurber, C., Parameter Estimation and Inverse Problems, Academic Press, 2004.
Boyd...
References
Nemirovski, A., Modern Convex Optimization, lecture notes, Israel Institute of Technology (2005).
Nesterov, Y.E...
Appendix

Generalized Additive Models


              a                                                        b
         ...
Appendix

C-MARS


           cluster




           cluster




                     robust optimization
Appendix

Nonlinear Regression

alternative solution


min    t,
 t,q

                                T
subject to       ...
Upcoming SlideShare
Loading in...5
×

Parameter Estimation in Stochastic Differential Equations by Continuous Optimization

1,423

Published on

AACIMP 2010 Summer School lecture by Gerhard Wilhelm Weber. "Applied Mathematics" stream. "Modern Operational Research and Its Mathematical Methods with a Focus on Financial Mathematics" course. Part 8.
More info at http://summerschool.ssa.org.ua

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,423
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Parameter Estimation in Stochastic Differential Equations by Continuous Optimization"

  1. 1. 5th International Summer School Achievements and Applications of Contemporary Informatics, Mathematics and Physics National University of Technology of the Ukraine Kiev, Ukraine, August 3-15, 2010 Parameter Estimation in Stochastic Differential Equations by Continuous Optimization Gerhard-Wilhelm Weber * Nüket Erbil, Ceren Can, Vefa Gafarova, Azer Kerimov Institute of Applied Mathematics Middle East Technical University, Ankara, Turkey Pakize Taylan Dept. Mathematics, Dicle University, Diyarbakır, Turkey * Faculty of Economics, Management and Law, University of Siegen, Germany Center for Research on Optimization and Control, University of Aveiro, Portugal Universiti Teknologi Malaysia, Skudai, Malaysia
  2. 2. Outline • Stochastic Differential Equations • Parameter Estimation • Various Statistical Models • C-MARS • Accuracy vs. Stability • Tikhonov Regularization • Conic Quadratic Programming • Nonlinear Regression • Portfolio Optimization • Outlook and Conclusion
  3. 3. Stock Markets
  4. 4. Stochastic Differential Equations dX t a( X t , t )dt b( X t , t )dWt drift and diffusion term Wt N (0, t ) (t [0, T ]) Wiener process
  5. 5. Stochastic Differential Equations dX t a( X t , t )dt b( X t , t )dWt drift and diffusion term Ex.: price, wealth, interest rate, volatility processes Wt N (0, t ) (t [0, T ]) Wiener process
  6. 6. Regression T Input vector X X 1 , X 2 ,..., X m and output variable Y ; linear regression : m Y E (Y X 1 ,..., X m ) 0 Xj j , j 1 T 0, 1 ,..., m which minimizes N 2 T RSS : yi x i i 1 ˆ X X T 1 XT y, ˆ Cov( β ) XTX 1 2
  7. 7. Generalized Additive Models m E Yi xi1 , xi 2 ,..., xi m 0 f j xi j j 1 f j are estimated by a smoothing on a single coordinate. Standard convention : E f j xij 0. • Backfitting algorithm (Gauss-Seidel) ri j yi ˆ f k xik , 0 k j it “cycles” and iterates.
  8. 8. Generalized Additive Models • Given data ( yi , xi ) (i = 1,2,...,N ), • penalized residual sum of squares 2 b N m m 2 PRSS ( 0 , f1 ,..., f m ) : yi 0 f j ( xij ) μj '' f (t j ) dt j j i 1 j 1 j 1 a j 0. • New estimation methods for additive model with CQP :
  9. 9. Generalized Additive Models min t t , β0 , f 2 N m subject to yi β0 f j ( xij ) t2, t 0, i=1 j 1 2 '' f (t j ) dt j j Mj (j 1, 2,..., m), dj j splines: f j ( x) l hl j ( x). l 1 By discretizing, we get min t t , β0 , f 2 subject to W( 0, ) 2 t2, t 0, 2 Vj ( 0, ) Mj (j 1,..., m). 2
  10. 10. Generalized Additive Models min t t , β0 , f 2 N m subject to yi β0 f j ( xij ) t2, t 0, i=1 j 1 2 '' f (t j ) dt j j Mj (j 1, 2,..., m), dj j splines: f j ( x) l hl j ( x). l 1 By discretizing, we get min t t , β0 , f 2 subject to W( 0, ) 2 t2, t 0, 2 Vj ( 0, ) Mj (j 1,..., m). 2
  11. 11. MARS y y c-(x, )=[ (x )] c+(x, )=[ (x )] c-(x, )=[ (x )] c+(x,egressionx ith)] r )=[ ( w x x
  12. 12. C-MARS N M max 2 2 2 PRSS : yi f ( xi ) μm 2 m Dr ,s m (t ) d t m m i 1 m 1 1 r s ( 1, 2 ) r ,s V ( m) Tradeoff between both accuracy and complexity. V ( m) : m j | j 1, 2,..., K m Dr ,s m (t m ) : m 1 trm 2 tsm (t m ) t m := (tm1 , tm2 ,..., tm K )T m ( 1, 2 ) : 1 2 , where 1 , 2 0,1
  13. 13. C-MARS Tikhonov regularization: 2 2 PRSS y (d ) L 2 2 L 2 Conic quadratic programming: y (d ) 2 min t, t, subject to (d ) y t, 2 L 2 M
  14. 14. Stochastic Differential Equations Revisited dX t a( X t , t )dt b( X t , t )dWt drift and diffusion term Ex.: price, wealth, interest rate, volatility, processes Wt N (0, t ) (t [0, T ]) Wiener process
  15. 15. Stochastic Differential Equations Revisited dX t a( X t , t )dt b( X t , t )dWt drift and diffusion term Ex.: bioinformatics, biotechnology (fermentation, population dynamics) Universiti Teknologi Malaysia Wt N (0, t ) (t [0, T ]) Wiener process
  16. 16. Stochastic Differential Equations Revisited dX t a( X t , t )dt b( X t , t )dWt drift and diffusion term Ex.: price, wealth, interest rate, volatility, processes Wt N (0, t ) (t [0, T ]) Wiener process
  17. 17. Stochastic Differential Equations Milstein Scheme : ˆ ˆ ˆ ˆ 1 ˆ Xj 1 Xj a ( X j , t j )(t j 1 t j ) b( X j , t j )(W j 1 Wj ) (b b)( X j , t j ) (W j 1 W j ) 2 (t j 1 tj) 2 and, based on our finitely many data: Wj ( W j )2 Xj a ( X j , t j ) b( X j , t j ) 1 2(b b)( X j , t j ) 1 . hj hj
  18. 18. Stochastic Differential Equations • step length hj tj 1 tj : tj Xj 1 Xj , if j 1, 2,..., N 1 hj Xj : XN XN 1 , if j N hN • Wt N (0, t ), W j (independent), Var( W j ) tj • Wj Zj tj , Zj N (0,1) Zj 1 Xj a ( X j , t j ) b( X j , t j ) (b b)( X j , t j ) Z j2 1 hj 2
  19. 19. Stochastic Differential Equations • More simple form: Xj Gj H j cj ( H j H j )d j , where G j : a( X j , t j ), H j : b( X j , t j ), cj : Z j hj , d j : 1 2 Z j2 1 . • Our problem: N 2 min X j (G j H jc j ( H j H j )d j ) y 2 j 1 y is a vector which comprises a subset of all the parameters.
  20. 20. Stochastic Differential Equations g 2 2 dp l l Gj a( X j , t j ) 0 f p (U j , p ) 0 p B p (U j , p ) p 1 p 1 l 1 2 2 d rh m H jc j b ( X j , t j )c j 0 g r (U j ,r ) 0 r Crm (U j ,r ) r 1 r 1 m 1 2 2 d sf n Fj d j b b( X j , t j )d j 0 hs (U j ,s ) 0 s Dsn (U j ,s ) s 1 s 1 n 1 where Uj U j ,1 ,U j ,2 : X j , tj ; • k th order base spline B ,k : a polynomial of degree k − 1, with knots, say x , 1, x x x 1 B ,1 ( x) 0, otherwise x x x k x B ,k ( x) B ,k 1 ( x) B 1, k 1 ( x) x k 1 x x k x 1
  21. 21. Stochastic Differential Equations • penalized sum of squares PRRS N 2 2 2 PRSS ( f , g , h) : Xj Gj H jc j Fj d j p f p (U p ) dU p j 1 p 1 2 2 2 2 r g r (U r ) dU r s hs (U s ) dU s r 1 s 1 b • p , r , s 0 (smoothing parameters), ( p, r , s ) a • large values of p , r , s yield smoother curves, smaller ones allow more fluctuation N 2 Xj Gj H jc j Fj d j j 1 2 N 2 dh p 2 d rg 2 d sf l Xj 0 p B lp (U j , p ) 0 r m Crm (U j ,r ) 0 n s Dsn (U j ,s ) j 1 p 1 l 1 r 1 m 1 s 1 n 1
  22. 22. Stochastic Differential Equations T T T T , , , T g T T T 1 2 dp 0 , 1 , 2 , p p , p ,..., p ( p 1, 2), T T T T 1 2 d rh 0 , 1 , 2 , r r , r ,..., r ( r 1, 2), T T T T 1 2 d sf 0, 1 , 2 , s s , s ,..., s ( s 1, 2). N 2 2 T • A A1T , A2 ,..., AN T T Then, Xj Aj X A . j 1 2 T X X 1 , X 2 ,..., X N • Furthermore, b 2 N 1 2 f p (U p ) dU p f p (U jp ) (U j 1, p U jp ) a j 1 g 2 N 1 dp l p B lp (U jP )u j . j 1 l 1
  23. 23. Stochastic Differential Equations 2 2 2 2 2 2 2 B C PRSS ( f , g , h) X A p A p p 2 r A r r 2 s AsD s 2 2 p 1 r 1 s 1 2 • If p r s : : 2 2 2 PRSS ( f , g , h) X A L 2 , 2 where 0 A1B 0 0 0 0 0 0 0 0 0 A2B 0 0 0 0 0 0 0 0 0 0 A1C 0 0 0 0 T T T T L: , , , . 0 0 0 0 0 A2C 0 0 0 0 0 0 0 0 0 0 A1D 0 0 0 0 0 0 0 0 0 A2D
  24. 24. Stochastic Differential Equations 2 2 min X A μ L 2 Tikhonov regularization 2 min t, Conic quadratic programming t, subject to A X t, 2 L 2 M Interior Point Methods
  25. 25. Stochastic Differential Equations min t t, 0N A t X subject to : , 1 0T m 0 primal problem 06( N 1) L t 06( N 1) : , 0 0T m M LN 1 , L6( N 1) 1 LN 1 : x ( x1 , x2 ,..., xN )T R N 1 | xN+1 x12 2 2 x2 ... xN max ( X T , 0) 1 0T N 1) , 6( M 2 0T N 1 0T N 6( 1) 0 1 subject to 1 2 , dual problem AT 0m LT 0m 0m 1 LN 1 , 2 L6 ( N 1) 1
  26. 26. Stochastic Differential Equations (t , , , , 1 , 2 ) is a primal dual optimal solution if and only if 0N A t X : , 1 0T m 0 06( N 1) L t 06( N 1) : 0 0T m M 0T N 1 0T N 6( 1) 0 1 1 2 AT 0m LT 0m 0m T T 1 0, 2 0 1 LN 1 , 2 L6( N 1) 1 LN 1 , L6( N 1) 1 .
  27. 27. Stochastic Differential Equations Ex.: dVt t T ( μ rt ) + rt Vt dt ct dt t T σVt dWt , drt α R rt dt t rt τ dWt , dX t t , X t , Z t dt t , X t , Z t dWt . nonlinear regression
  28. 28. Nonlinear Regression N 2 min f dj g xj , j 1 N : f j2 j 1 T F( ) : f1 ( ),..., f N ( ) min f ( ) F T ( )F ( )
  29. 29. Nonlinear Regression k 1 : k qk • Gauss-Newton method : T F( ) F ( )q F ( )F ( ) • Levenberg-Marquardt method : 0 T F( ) F( ) Ip q F ( )F ( )
  30. 30. Nonlinear Regression alternative solution min t, t,q T subject to F( ) F( ) Ip q F ( )F ( ) t, t 0, 2 || Lq || 2 M conic quadratic programming interior point methods
  31. 31. Portfolio Optimization max utility ! or min costs ! martingale method: Optimization Problem Representation Problem or stochastic control
  32. 32. Portfolio Optimization max utility ! or min costs ! martingale method: Parameter Estimation Optimization Problem Representation Problem or stochastic control
  33. 33. Portfolio Optimization max utility ! or min costs ! martingale method: Optimization Problem Representation Problem Parameter Estimation or stochastic control
  34. 34. Portfolio Optimization max utility ! or min costs ! martingale method: Optimization Problem Representation Problem Parameter Estimation or stochastic control
  35. 35. References Aster, A., Borchers, B., and Thurber, C., Parameter Estimation and Inverse Problems, Academic Press, 2004. Boyd, S., and Vandenberghe, L., Convex Optimization, Cambridge University Press, 2004. Buja, A., Hastie, T., and Tibshirani, R., Linear smoothers and additive models, The Ann. Stat. 17, 2 (1989) 453-510. Fox, J., Nonparametric regression, Appendix to an R and S-Plus Companion to Applied Regression, Sage Publications, 2002. Friedman, J.H., Multivariate adaptive regression splines, Annals of Statistics 19, 1 (1991) 1-141. Friedman, J.H., and Stuetzle, W., Projection pursuit regression, J. Amer. Statist Assoc. 76 (1981) 817-823. Hastie, T., and Tibshirani, R., Generalized additive models, Statist. Science 1, 3 (1986) 297-310. Hastie, T., and Tibshirani, R., Generalized additive models: some applications, J. Amer. Statist. Assoc. 82, 398 (1987) 371-386. Hastie, T., Tibshirani, R., and Friedman, J.H., The Element of Statistical Learning, Springer, 2001. Hastie, T.J., and Tibshirani, R.J., Generalized Additive Models, New York, Chapman and Hall, 1990. Kloeden, P.E, Platen, E., and Schurz, H., Numerical Solution of SDE Through Computer Experiments, Springer Verlag, New York, 1994. Korn, R., and Korn, E., Options Pricing and Portfolio Optimization: Modern Methods of Financial Mathematics, Oxford University Press, 2001. Nash, G., and Sofer, A., Linear and Nonlinear Programming, McGraw-Hill, New York, 1996. Nemirovski, A., Lectures on modern convex optimization, Israel Institute of Technology (2002).
  36. 36. References Nemirovski, A., Modern Convex Optimization, lecture notes, Israel Institute of Technology (2005). Nesterov, Y.E , and Nemirovskii, A.S., Interior Point Methods in Convex Programming, SIAM, 1993. Önalan, Ö., Martingale measures for NIG Lévy processes with applications to mathematical finance, presentation in: Advanced Mathematical Methods for Finance, Side, Antalya, Turkey, April 26-29, 2006. Taylan, P., Weber G.-W., and Kropat, E., Approximation of stochastic differential equations by additive models using splines and conic programming, International Journal of Computing Anticipatory Systems 21 (2008) 341-352. Taylan, P., Weber, G.-W., and A. Beck, New approaches to regression by generalized additive models and continuous optimization for modern applications in finance, science and techology, in the special issue in honour of Prof. Dr. Alexander Rubinov, of Optimization 56, 5-6 (2007) 1-24. Taylan, P., Weber, G.-W., and Yerlikaya, F., A new approach to multivariate adaptive regression spline by using Tikhonov regularization and continuous optimization, to appear in TOP, Selected Papers at the Occasion of 20th EURO Mini Conference (Neringa, Lithuania, May 20-23, 2008) 317- 322. Seydel, R., Tools for Computational Finance, Springer, Universitext, 2004. Stone, C.J., Additive regression and other nonparametric models, Annals of Statistics 13, 2 (1985) 689-705. Weber, G.-W., Taylan, P., Akteke-Öztürk, B., and Uğur, Ö., Mathematical and data mining contributions dynamics and optimization of gene-environment networks, in the special issue Organization in Matter from Quarks to Proteins of Electronic Journal of Theoretical Physics. Weber, G.-W., Taylan, P., Yıldırak, K., and Görgülü, Z.K., Financial regression and organization, to appear in the Special Issue on Optimization in Finance, of DCDIS-B (Dynamics of Continuous, Discrete and Impulsive Systems (Series B)).
  37. 37. Appendix Generalized Additive Models a b I1 (3a) a b I1 I2 I3 I4 I5 I6 I7 I8 (3b) a b I1 I2 I3 I4 I5 I6 (3c) . . . . . . ... . .. . . . .. . . . . . . .. . Ind j : = d j ( D j ) v j (V j ) a b
  38. 38. Appendix C-MARS cluster cluster robust optimization
  39. 39. Appendix Nonlinear Regression alternative solution min t, t,q T subject to F( ) F( ) Ip q F ( )F ( ) t, t 0, 2 || Lq || 2 M 1 min Q(q) := f ( ) + qT F ( ) F ( ) + qT F ( ) T F ( )q q 2 subject to q2 trust region
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×