3. Nonlinear Regression
)
( bx
ae
y
)
( b
ax
y
x
b
ax
y
Some popular nonlinear regression models:
1. Exponential model:
2. Power model:
3. Saturation growth model:
4. Polynomial model: )
( 1
0
m
mx
a
...
x
a
a
y
http://numericalmethods.eng.usf.edu
3
4. Nonlinear Regression
Given n data points )
,
(
,
...
),
,
(
),
,
( 2
2
1
1 n
n y
x
y
x
y
x best fit )
(x
f
y
to the data, where )
(x
f is a nonlinear function of x .
Figure. Nonlinear regression model for discrete y vs. x data
)
(x
f
y
)
,
(
n
n
y
x
)
,
( 1
1
y
x
)
,
(
2
2
y
x
)
,
(
i
i
y
x
)
(
i
i
x
f
y
http://numericalmethods.eng.usf.edu
4
6. Exponential Model
)
,
(
,
...
),
,
(
),
,
( 2
2
1
1 n
n y
x
y
x
y
x
Given best fit
bx
ae
y to the data.
Figure. Exponential model of nonlinear regression for y vs. x data
bx
ae
y
)
,
(
n
n
y
x
)
,
( 1
1
y
x
)
,
(
2
2
y
x
)
,
(
i
i
y
x
i
bx
i ae
y
http://numericalmethods.eng.usf.edu
6
7. Finding Constants of Exponential Model
n
i
bx
i
r
i
ae
y
S
1
2
The sum of the square of the residuals is defined as
Differentiate with respect to a and b
0
2
1
i
i bx
n
i
bx
i
r
e
ae
y
a
S
0
2
1
i
i bx
i
n
i
bx
i
r
e
ax
ae
y
b
S
http://numericalmethods.eng.usf.edu
7
8. Finding Constants of Exponential Model
Rewriting the equations, we obtain
0
1
2
1
n
i
bx
n
i
bx
i
i
i e
a
e
y
0
1
2
1
n
i
bx
i
n
i
bx
i
i
i
i e
x
a
e
x
y
http://numericalmethods.eng.usf.edu
8
9. Finding constants of Exponential Model
Substituting a back into the previous equation
0
1
2
1
2
1
1
n
i
bx
i
n
i
bx
bx
n
i
i
bx
i
n
i
i
i
i
i
i e
x
e
e
y
e
x
y
The constant b can be found through numerical
methods such as bisection method.
n
i
bx
n
i
bx
i
i
i
e
e
y
a
1
2
1
Solving the first equation for a yields
http://numericalmethods.eng.usf.edu
9
10. Example 1-Exponential Model
t(hrs) 0 1 3 5 7 9
1.000 0.891 0.708 0.562 0.447 0.355
Many patients get concerned when a test involves injection of a
radioactive material. For example for scanning a gallbladder, a
few drops of Technetium-99m isotope is used. Half of the
Technetium-99m would be gone in about 6 hours. It, however,
takes about 24 hours for the radiation levels to reach what we
are exposed to in day-to-day activities. Below is given the
relative intensity of radiation as a function of time.
Table. Relative intensity of radiation as a function of time.
http://numericalmethods.eng.usf.edu
10
11. Example 1-Exponential Model cont.
Find:
a) The value of the regression constants A
and
b) The half-life of Technetium-99m
c) Radiation intensity after 24 hours
The relative intensity is related to time by the equation
t
Ae
http://numericalmethods.eng.usf.edu
11
13. Constants of the Model
The value of λ is found by solving the nonlinear equation
0
1
2
1
2
1
1
n
i
t
i
n
i
t
n
i
t
i
t
i
n
i
i
i
i
i
i e
t
e
e
e
t
f
n
i
t
n
i
t
i
i
i
e
e
A
1
2
1
t
Ae
http://numericalmethods.eng.usf.edu
13
14. Setting up the Equation in MATLAB
0
1
2
1
2
1
1
n
i
t
i
n
i
t
n
i
t
i
t
i
n
i
i
i
i
i
i e
t
e
e
e
t
f
t (hrs) 0 1 3 5 7 9
γ 1.000 0.891 0.708 0.562 0.447 0.355
http://numericalmethods.eng.usf.edu
14
15. Setting up the Equation in MATLAB
0
1
2
1
2
1
1
n
i
t
i
n
i
t
n
i
t
i
t
i
n
i
i
i
i
i
i e
t
e
e
e
t
f
t=[0 1 3 5 7 9]
gamma=[1 0.891 0.708 0.562 0.447 0.355]
syms lamda
sum1=sum(gamma.*t.*exp(lamda*t));
sum2=sum(gamma.*exp(lamda*t));
sum3=sum(exp(2*lamda*t));
sum4=sum(t.*exp(2*lamda*t));
f=sum1-sum2/sum3*sum4;
1151
.
0
http://numericalmethods.eng.usf.edu
15
16. Calculating the Other Constant
The value of A can now be calculated
6
1
2
6
1
i
t
i
t
i
i
i
e
e
A
9998
.
0
The exponential regression model then is
t
e 1151
.
0
9998
.
0
http://numericalmethods.eng.usf.edu
16
17. Plot of data and regression curve
t
e 1151
.
0
9998
.
0
http://numericalmethods.eng.usf.edu
17
18. Relative Intensity After 24 hrs
The relative intensity of radiation after 24 hours
24
1151
.
0
9998
.
0
e
2
10
3160
.
6
This result implies that only
%
317
.
6
100
9998
.
0
10
316
.
6 2
radioactive intensity is left after 24 hours.
http://numericalmethods.eng.usf.edu
18
19. Homework
• What is the half-life of Technetium-99m
isotope?
• Write a program in the language of your
choice to find the constants of the
model.
• Compare the constants of this regression
model with the one where the data is
transformed.
• What if the model was ?
http://numericalmethods.eng.usf.edu
19
t
e
21. Polynomial Model
)
,
(
,
...
),
,
(
),
,
( 2
2
1
1 n
n y
x
y
x
y
x
Given best fit
m
m
x
a
...
x
a
a
y
1
0
)
2
(
n
m to a given data set.
Figure. Polynomial model for nonlinear regression of y vs. x data
m
m
x
a
x
a
a
y
1
0
)
,
(
n
n
y
x
)
,
( 1
1
y
x
)
,
(
2
2
y
x
)
,
(
i
i
y
x
)
(
i
i
x
f
y
http://numericalmethods.eng.usf.edu
21
22. Polynomial Model cont.
The residual at each data point is given by
m
i
m
i
i
i x
a
x
a
a
y
E
.
.
.
1
0
The sum of the square of the residuals then is
n
i
m
i
m
i
i
n
i
i
r
x
a
x
a
a
y
E
S
1
2
1
0
1
2
.
.
.
http://numericalmethods.eng.usf.edu
22
23. Polynomial Model cont.
To find the constants of the polynomial model, we set the derivatives
with respect to i
a where
0
)
(
.
.
.
.
2
0
)
(
.
.
.
.
2
0
)
1
(
.
.
.
.
2
1
1
0
1
1
0
1
1
1
0
0
m
i
n
i
m
i
m
i
i
m
r
i
n
i
m
i
m
i
i
r
n
i
m
i
m
i
i
r
x
x
a
x
a
a
y
a
S
x
x
a
x
a
a
y
a
S
x
a
x
a
a
y
a
S
,
,
1 m
i
equal to zero.
http://numericalmethods.eng.usf.edu
23
24. Polynomial Model cont.
These equations in matrix form are given by
n
i
i
m
i
n
i
i
i
n
i
i
m
n
i
m
i
n
i
m
i
n
i
m
i
n
i
m
i
n
i
i
n
i
i
n
i
m
i
n
i
i
y
x
y
x
y
a
a
a
x
x
x
x
x
x
x
x
n
1
1
1
1
0
1
2
1
1
1
1
1
1
2
1
1
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The above equations are then solved for m
a
a
a ,
,
, 1
0
http://numericalmethods.eng.usf.edu
24
25. Example 2-Polynomial Model
Temperature, T
(oF)
Coefficient of
thermal
expansion, α
(in/in/oF)
80 6.47×10−6
40 6.24×10−6
−40 5.72×10−6
−120 5.09×10−6
−200 4.30×10−6
−280 3.33×10−6
−340 2.45×10−6
Regress the thermal expansion coefficient vs. temperature data to
a second order polynomial.
1.00E-06
2.00E-06
3.00E-06
4.00E-06
5.00E-06
6.00E-06
7.00E-06
-400 -300 -200 -100 0 100 200
Temperature, o
F
Thermal
expansion
coefficient,
α
(in/in/
o
F)
Table. Data points for
temperature vs
Figure. Data points for thermal expansion coefficient vs
temperature.
α
http://numericalmethods.eng.usf.edu
25
26. Example 2-Polynomial Model cont.
2
2
1
0 T
a
T
a
a
α
We are to fit the data to the polynomial regression model
n
i
i
i
n
i
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
T
T
a
a
a
T
T
T
T
T
T
T
T
n
1
2
1
1
2
1
0
1
4
1
3
1
2
1
3
1
2
1
1
2
1
The coefficients 2
1
0 , a
,a
a are found by differentiating the sum of the
square of the residuals with respect to each variable and setting the
values equal to zero to obtain
http://numericalmethods.eng.usf.edu
26
27. Example 2-Polynomial Model cont.
The necessary summations are as follows
Temperature, T
(oF)
Coefficient of
thermal expansion,
α (in/in/oF)
80 6.47×10−6
40 6.24×10−6
−40 5.72×10−6
−120 5.09×10−6
−200 4.30×10−6
−280 3.33×10−6
−340 2.45×10−6
Table. Data points for temperature vs. α 5
7
1
2
10
5580
.
2
i
i
T
7
7
1
3
10
0472
.
7
i
i
T
10
7
1
4
10
1363
.
2
i
i
T
5
7
1
10
3600
.
3
i
i
3
7
1
10
6978
.
2
i
i
i
T
1
7
1
2
10
5013
.
8
i
i
i
T
http://numericalmethods.eng.usf.edu
27
28. Example 2-Polynomial Model cont.
1
3
5
2
1
0
10
7
5
7
5
2
5
2
10
5013
.
8
10
6978
.
2
10
3600
.
3
10
1363
.
2
10
0472
.
7
10
5800
.
2
10
0472
.
7
10
5800
.
2
10
600
.
8
10
5800
.
2
10
6000
.
8
0000
.
7
a
a
a
Using these summations, we can now calculate 2
1
0 , a
,a
a
Solving the above system of simultaneous linear equations we have
11
9
6
2
1
0
10
2218
.
1
10
2782
.
6
10
0217
.
6
a
a
a
The polynomial regression model is then
2
11
9
6
2
2
1
0
T
10
1.2218
T
10
6.2782
10
6.0217
α
T
a
T
a
a
http://numericalmethods.eng.usf.edu
28
29. Transformation of Data
To find the constants of many nonlinear models, it results in solving
simultaneous nonlinear equations. For mathematical convenience,
some of the data for such models can be transformed. For example,
the data for an exponential model can be transformed.
As shown in the previous example, many chemical and physical processes
are governed by the equation,
bx
ae
y
Taking the natural log of both sides yields,
bx
a
y
ln
ln
Let y
z ln
and a
a ln
0
(implying) o
a
e
a with b
a
1
We now have a linear regression model where x
a
a
z 1
0
http://numericalmethods.eng.usf.edu
29
30. Transformation of data cont.
Using linear model regression methods,
_
1
_
0
1
2
1
2
1
1 1
1
x
a
z
a
x
x
n
z
x
z
x
n
a
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
Once 1
,a
ao are found, the original constants of the model are found as
0
1
a
e
a
a
b
http://numericalmethods.eng.usf.edu
30
32. Example 3-Transformation of
data
t(hrs) 0 1 3 5 7 9
1.000 0.891 0.708 0.562 0.447 0.355
Many patients get concerned when a test involves injection of a radioactive
material. For example for scanning a gallbladder, a few drops of Technetium-
99m isotope is used. Half of the Technetium-99m would be gone in about 6
hours. It, however, takes about 24 hours for the radiation levels to reach what
we are exposed to in day-to-day activities. Below is given the relative intensity
of radiation as a function of time.
Table. Relative intensity of radiation as a function
of time
0
0.5
1
0 5 10
Relative
intensity
of
radiation,
γ
Time t, (hours)
Figure. Data points of relative radiation intensity
vs. time
http://numericalmethods.eng.usf.edu
32
33. Example 3-Transformation of data
cont.
Find:
a) The value of the regression constants A
and
b) The half-life of Technetium-99m
c) Radiation intensity after 24 hours
The relative intensity is related to time by the equation
t
Ae
http://numericalmethods.eng.usf.edu
33
34. Example 3-Transformation of data
cont.
t
Ae
Exponential model given as,
t
A
ln
ln
Assuming
ln
z ,
A
ao ln
and
1
a we obtain
t
a
a
z
1
0
This is a linear relationship between z and t
http://numericalmethods.eng.usf.edu
34
35. Example 3-Transformation of data cont.
Using this linear relationship, we can calculate 1
0 ,a
a
n
i
n
i
i
n
i
i
n
i
n
i
i
i
i
t
t
n
z
t
z
t
n
a
1
2
1
2
1
1
1 1
1
and
t
a
z
a 1
0
where
1
a
0
a
e
A
http://numericalmethods.eng.usf.edu
35
36. Example 3-Transformation of Data
cont.
1
2
3
4
5
6
0
1
3
5
7
9
1
0.891
0.708
0.562
0.447
0.355
0.00000
−0.11541
−0.34531
−0.57625
−0.80520
−1.0356
0.0000
−0.11541
−1.0359
−2.8813
−5.6364
−9.3207
0.0000
1.0000
9.0000
25.000
49.000
81.000
25.000 −2.8778 −18.990 165.00
Summations for data transformation are as follows
Table. Summation data for Transformation of data
model
i i
t i
i
i
z
ln
i
i
z
t 2
i
t
With 6
n
000
.
25
6
1
i
i
t
6
1
8778
.
2
i
i
z
6
1
990
.
18
i
i
i
z
t
00
.
165
6
1
2
i
i
t
http://numericalmethods.eng.usf.edu
36
37. Example 3-Transformation of Data
cont.
Calculating 1
0 ,a
a
2
1
25
00
.
165
6
8778
.
2
25
990
.
18
6
a 11505
.
0
6
25
11505
.
0
6
8778
.
2
0
a
4
10
6150
.
2
Since
A
a ln
0
0
a
e
A
4
10
6150
.
2
e 99974
.
0
11505
.
0
1
a
also
http://numericalmethods.eng.usf.edu
37
38. Example 3-Transformation of Data
cont.
Resulting model is
t
e 11505
.
0
99974
.
0
0
0.5
1
0 5 10
Time, t (hrs)
Relative
Intensity
of
Radiation,
t
e 11505
.
0
99974
.
0
Figure. Relative intensity of radiation as a function of
temperature using transformation of data model.
http://numericalmethods.eng.usf.edu
38
39. Example 3-Transformation of Data
cont.
The regression formula is then
t
e 11505
.
0
99974
.
0
b) Half life of Technetium-99m is when
0
2
1
t
hours
.
t
.
t
.
.
e
e
.
e
.
t
.
.
t
.
0248
6
5
0
ln
11505
0
5
0
99974
0
2
1
99974
0
11508
0
0
11505
0
11505
0
http://numericalmethods.eng.usf.edu
39
40. Example 3-Transformation of Data
cont.
c) The relative intensity of radiation after 24 hours is then
24
11505
.
0
99974
.
0
e
063200
.
0
This implies that only %
3216
.
6
100
99983
.
0
10
3200
.
6 2
of the radioactive
material is left after 24 hours.
http://numericalmethods.eng.usf.edu
40
41. Comparison
Comparison of exponential model with and without data
Transformation:
With data
Transformation
(Example 3)
Without data
Transformation
(Example 1)
A 0.99974 0.99983
λ −0.11505 −0.11508
Half-Life (hrs) 6.0248 6.0232
Relative intensity
after 24 hrs.
6.3200×10−2 6.3160×10−2
Table. Comparison for exponential model with and without data
Transformation.
http://numericalmethods.eng.usf.edu
41
42. Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/nonlinear_r
egression.html