Ball and beam, Navier-Stokes, stability of an airplane wrt to pilot commands. nonlinear systems, number of variables = number of equations The solution means determining some values of variables/parameters that strictly satisfy the given relationships (equations). Impossible to solve them by hand even if they are homogeneous.
Direct solve – reuse the factorization (quasi-implicit Newton, chord method)
Computational fluid dynamics – millions of variables (chesapeake bay)
Examples of cost functions Talk about differentiability
Convergence rates - LINEAR We want Bk to mimic the behavior of newton => symmetric positive definite (at least in a neighborhood)
Trust regions determine a region around the current iterate where the approximate model of the objective function is accurate. The step is taken to be the minimizer on the trust region.