Lecture slides based on Autar Kaw's Numerical Methods Section on Linear Regression, Ch 6.03, http://nm.mathforcollege.com

1. St. John's University of Tanzania
MAT210 NUMERICAL ANALYSIS
2013/14 Semester II
REGRESSION
Linear Regression
Kaw, Chapter 6.03

2. MAT210 2013/14 Sem II 2 of 19
●
Linear regression = most popular model
●
A measure of goodness of fit, that is, how well
the line predicts the response variable y at
each of the n data points
●
Ideally, if all the residuals are 0, then all points
are on the line. Therefore
●
Minimization of the residual is an objective of
obtaining the best regression coefficients
Introduction
y=a0
+a1
x
Residual ,Ei
=yi
−(a0
+a1
xi)

3. MAT210 2013/14 Sem II 3 of 19
Least Squares Fit
● Most popular method to minimize residual
●
Least squares methods
– Estimates of the constants of the models are chosen
such that the sum of squared residuals is minimized
● To find Sr, minimize it relative to a0 and a1
Sr
=∑
i=1
i=n
Ei
2
=∑
i=1
i=n
(yi
−(a0
+a1
xi
))
2

4. MAT210 2013/14 Sem II 4 of 19
Lots of sums
∂Sr
∂a0
=2∑
i=1
i=n
(yi
−(a0
+a1
xi
))(−1)=0
∂Sr
∂a1
=2∑
i=1
i=n
(yi
−(a0
+a1
xi
))(−xi
)=0
∑
i=1
i=n
yi
−∑
i=1
i=n
a0
+∑
i=1
i=n
a1
xi
=0
∑
i=1
i=n
yi
xi
−∑
i=1
i=n
a0
xi
+∑
i=1
i=n
a1
xi
2
=0
which leads to

5. MAT210 2013/14 Sem II 5 of 19
Not as bad as it looks
∑
i=1
i=n
a0
is simply na0
a0
n+a1∑
i=1
i=n
xi
=∑
i=1
i=n
yi
a0∑
i=1
i=n
xi
+a1∑
i=1
i=n
xi
2
=∑
i=1
i=n
yi
xi
which leads to
now we are getting somewhere

6. MAT210 2013/14 Sem II 6 of 19
Two equations, two unknowns
a1
=
n∑
i=1
i=n
xi
yi
−∑
i=1
i=n
xi ∑
i=1
i=n
yi
n∑
i=1
i=n
xi
2
−
(∑
i=1
i=n
xi)
2
a0
=
∑
i=1
i=n
xi
2
∑
i=1
i=n
yi
−∑
i=1
i=n
xi ∑
i=1
i=n
xi
yi
n∑
i=1
i=n
xi
2
−
(∑
i=1
i=n
xi)
2
Solving the two equations yields

7. MAT210 2013/14 Sem II 7 of 19
Clean-up with means, variances

8. MAT210 2013/14 Sem II 8 of 19
Example: Torque (T)
The torque T needed to turn the torsional
spring of a mousetrap through an angle θ

9. MAT210 2013/14 Sem II 9 of 19
Tabulating data
Not too hard with a spreadsheet

10. MAT210 2013/14 Sem II 10 of 19
Calculating k2

11. MAT210 2013/14 Sem II 11 of 19
Calculating k1

12. MAT210 2013/14 Sem II 12 of 19
End Result
0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Angle, radians
Torque,N-m

13. MAT210 2013/14 Sem II 13 of 19
Example: Stress vs Strain
Longitudinal modulus of a composite material

14. MAT210 2013/14 Sem II 14 of 19
Using your head
● Note that stress of 0 produces 0 strain
Force the line to go through 0,0

15. MAT210 2013/14 Sem II 15 of 19
Simplified result

16. MAT210 2013/14 Sem II 16 of 19
Tabulated Data

17. MAT210 2013/14 Sem II 17 of 19
Finding the Elastic Modulus, E

18. MAT210 2013/14 Sem II 18 of 19

19. MAT210 2013/14 Sem II 19 of 19
The Modern World
● Spreadsheets and other data analysis tools,
including GnuPlot and Scilab, eliminate the
need for doing hand calculations
●
That is good news, because it means we
focus less on mechanics and more on
intellect
● Understanding the model
– Must the line go through (0,0)?
– Is the right model a line?
– What about data errors? Invalid points?