Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

# Applied numerical methods lec8

511 views

Published on

Curve Fitting and Interpolation (Least-Squares Regression)

Published in: Engineering
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

• Be the first to like this

### Applied numerical methods lec8

1. 1. 1 Curve Fitting and Interpolation Least-Squares Regression
2. 2. 2 • Fit the best curve to a discrete data set and obtain estimates for other data points • Two general approaches: – Data exhibit a significant degree of scatter Find a single curve that represents the general trend of the data. – Data is very precise. Pass a curve(s) exactly through each of the points. • Two common applications in engineering: Trend analysis. Predicting values of dependent variable: extrapolation beyond data points or interpolation between data points. Hypothesis testing. Comparing existing mathematical model with measured data. Curve Fitting
3. 3. 3 In sciences, if several measurements are made of a particular quantity, additional insight can be gained by summarizing the data in one or more well chosen statistics: Arithmetic mean - The sum of the individual data points (yi) divided by the number of points. Standard deviation – a common measure of spread for a sample or variance Coefficient of variation – quantifies the spread of data (similar to relative error) ni n y y i ,,1   1 2     n yy S i y )( 1 2 2     n yy S i y )( Simple Statistics %100.. y S vc y 
4. 4. 4 yi : measured value e : error yi = a0 + a1 xi + e e = yi - a0 - a1 xi a1 : slope a0 : intercept Linear Regression e Error Line equation y = a0 + a1 x Given: n points (x1, y1), (x2, y2), …, (xn, yn) Find: a line y = a0 + a1x that best fits the n points.
5. 5. 5 • Best strategy is to minimize the sum of the squares of the residuals between the measured-y and the y calculated with the linear model: • Yields a unique line for a given set of data • Need to compute a0 and a1 such that Sr is minimized!          n i iir n i modelimeasuredi n i ir xaayS yy eS 1 2 10 1 2 1 2 )( )( ,, e Error Minimize the sum of the residual errors for all available data?
6. 6.   00)(2 00)(2 2 101 1 101           iiiiiioi r iiioi o r xaxaxyxxaay a S xaayxaay a S Normal equations which can be solved simultaneously       iiii ii xyaxax yaxna naa       1 2 0 10 00 (2) (1) Since    n i ii n i ir xaayeS 1 2 10 1 2 )(:errorMinimize Least-Squares Fit of a Straight Line
7. 7. 7 Least-Squares Fit of a Line      02 10 0 ii r xaay a S      0][2 10 1 iii r xxaay a S    ii yaxna 10 To minimize Sr: where and        221 ii iiii xxn yxyxn a      iiii yxaxax 1 2 0 xaya 10  n y y i  n x x i  y = a0 + a1x Mean values
8. 8. 8 where and        221 ii iiii xxn yxyxn a xaya 10  n y y i  n x x i  y = a0 + a1x Mean values
9. 9. 9 Is our prediction reliable? Once an equation is found for the least square line, we need to have some way of judging just how good the equation is for predictive purposes. In order to have a quantitative basis for confidence in our predictions, we need to calculate coefficient of correlation, denoted r. It may be calculated using the following formula: The value of r that is close to 1 or -1 (r2 = 1 ) indicates that our formula will give us a reliable prediction
10. 10. 10
11. 11. 11 Example (1) of Least-Squares Fit of a Line (Linear Regression)
12. 12. 12 7n 5119. ii yx 140 2  ix   28ix 4 7 28 x 24 iy 4285713 7 24 .y         83928570 281407 242851197 21 . .    a   07142857048392857042857130 ... a y = a0 + a1x
13. 13. 13
14. 14. 14
15. 15. 15
16. 16. 16
17. 17. Example (2): 17 A sales manager noticed that the annual sales of his employees increase with years of experience. To estimate the annual sales for his potential new sales person he collected data concerning annual sales and years of experience of his current employees: use his data to create a formula that will help him estimate annual sales based on years of experience.
18. 18. 18 where and        221 ii iiii xxn yxyxn a xaya 10  n y y i  n x x i  y = a0 + a1x Mean values
19. 19. 19
20. 20. 20
21. 21. 21 Linearization of Nonlinear Relationships Nonlinear regression Linear transformation (if possible) Data that don’t fit linear form
22. 22. 22 Example (3) of Linearization Linear regression on (log x, log y) b2 = 1.75 x y log x log y 1 0.5 0 -0.301 2 1.7 0.301 0. 226 3 3.4 0.477 0.534 4 5.7 0.602 0.753 5 8.4 0.699 0.922 log y = 1.75 log x – 0.300 log a2 = – 0.300 a2 = 10-0.3 = 0.5 y = 0.5x1.75
23. 23. 23 Polynomial Regression )1(   mn S s r xy /   2 1 2 210   n i m imiiir xaxaxaayS ...   2 1 2 210   n i iiir xaxaayS Given: n points (x1, y1), (x2, y2), …, (xn, yn) Find: a polynomial y = a0 + a1x + a2x2 + … amxm that minimizes Example: 2nd-order polynomial y = a0 + a1x + a2x2      02 2 210 0 iii r xaxaay a S      0][2 2 210 1 iiii r xxaxaay a S      0][2 22 210 2 iiii r xxaxaay a S      iii yaxaxna 2 2 10        iiiii yxaxaxax 2 3 1 2 0        iiiii yxaxaxax 2 2 4 1 3 0 2 Standard error:
24. 24. 24 Example (4) of 2nd-order Polynomial Regression
25. 25. 25 m = 2 ∑xi = 15 ∑xi 4 = 979 n = 6 ∑yi = 152.6 ∑xiyi = 585.6 ∑xi 2= 55 ∑xi 2yi = 2488.9 ∑xi 3= 225                                82488 6585 6152 97922555 2255515 55156 2 1 0 . . . a a a 52.x y = 2.47857 + 2.35929x + 1.86071x2 121 36 746573 . . /   xys 998510 392513 7465733925132 . . ..      t rt S SS r 43325.y      iii yaxaxna 2 2 10        iiiii yxaxaxax 2 3 1 2 0        iiiii yxaxaxax 2 2 4 1 3 0 2 2nd-order polynomial y = a0 + a1x + a2x2
26. 26. 26y = 2.47857 + 2.35929x + 1.86071x2
27. 27. Example (5): 27 Fit a second-order polynomial to the data in the following table      iii yaxaxna 2 2 10        iiiii yxaxaxax 2 3 1 2 0        iiiii yxaxaxax 2 2 4 1 3 0 2 2nd-order polynomial y = a0 + a1x + a2x2
28. 28. 28      iii yaxaxna 2 2 10        iiiii yxaxaxax 2 3 1 2 0        iiiii yxaxaxax 2 2 4 1 3 0 2 2nd-order polynomial y = a0 + a1x + a2x2
29. 29. 29 Multiple Linear Regression      0][2 222110 2 iiii r xxaxaay a S   2 1 22110   n i iiir xaxaayS Given: n points 3D (y1, x11, x12) (y2, x12, x22), …, (yn, x1n, x2n) Find: a plane y = a0 + a1x1 + a2x2 that minimizes      02 22110 0 iii r xaxaay a S      0][2 122110 1 iiii r xxaxaay a S      iii yaxaxna 22110        iiiiii yxaxxaxax 12211 2 101        iiiiii yxaxaxxax 22 2 212102 Generation to m dimensions: hyper plane y = a0 + a1x1 + a2x2 + … + amxm
30. 30. 30 General Linear Least Squares                                                   nmmnnn m m n e e e a a a zzz zzz zzz y y y       2 1 1 0 10 21202 11101 2 1 Linear least squares: y = a0 + a1x1 Multi-linear least squares: y = a0 + a1x1 + a2x2 + … + amxm Polynomial least squares: y = a0 + a1x + a2x2 + … amxm 2 1 01 2             n i m j jiji n i ir zayeS           YZAZZ TT  y = a0z0 + a1z1 + a2z2 + … + amzm {Y} = [Z] {A} + {E} [C] {A} = {D} ([C] is symmetric, e.g. linear and polynomial)