Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Algebra unit 10.7 by Mark Ryder 671 views
- 10 examples for if statement by fyjordan9 341 views
- Algebra unit 10.1 by Mark Ryder 427 views
- Study Electronic And Mechanical Pro... by IOSR Journals 116 views
- Grds international conference on pu... by Global R & D Serv... 174 views
- Higher-Order Conjugate Gradient Met... by iosrjce 126 views

740 views

Published on

Performance Optimization

Published in:
Education

No Downloads

Total views

740

On SlideShare

0

From Embeds

0

Number of Embeds

3

Shares

0

Downloads

18

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Performance Optimization
- 2. Basic Optimization Algorithm p k - Search Direction k - Learning Rate or
- 3. Steepest Descent Choose the next step so that the function decreases: For small changes in x we can approximate F ( x ): where If we want the function to decrease: We can maximize the decrease by choosing:
- 4. Example
- 5. Plot
- 6. Stable Learning Rates (Quadratic) Stability is determined by the eigenvalues of this matrix. Eigenvalues of [ I - A ]. Stability Requirement: ( i - eigenvalue of A )
- 7. Example
- 8. Minimizing Along a Line Choose k to minimize where
- 9. Example
- 10. Plot Successive steps are orthogonal. F x T x x k 1 + = p k g k 1 + T p k = =
- 11. Newton’s Method Take the gradient of this second-order approximation and set it equal to zero to find the stationary point:
- 12. Example
- 13. Plot
- 14. Non-Quadratic Example Stationary Points: F ( x ) F 2 ( x )
- 15. Different Initial Conditions F ( x ) F 2 ( x )
- 16. Conjugate Vectors A set of vectors is mutually conjugate with respect to a positive definite Hessian matrix A if One set of conjugate vectors consists of the eigenvectors of A . (The eigenvectors of symmetric matrices are orthogonal.)
- 17. For Quadratic Functions The change in the gradient at iteration k is where The conjugacy conditions can be rewritten This does not require knowledge of the Hessian matrix.
- 18. Forming Conjugate Directions Choose the initial search direction as the negative of the gradient. Choose subsequent search directions to be conjugate. where or or
- 19. Conjugate Gradient algorithm <ul><li>The first search direction is the negative of the gradient. </li></ul><ul><li>Select the learning rate to minimize along the line. </li></ul><ul><li>Select the next search direction using </li></ul><ul><li>If the algorithm has not converged, return to second step. </li></ul><ul><li>A quadratic function will be minimized in n steps. </li></ul>(For quadratic functions.)
- 20. Example
- 21. Example
- 22. Plots Conjugate Gradient Steepest Descent

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment