The term regression was originally employed by Francis Galton while describing the laws of inheritance. Galton believed that these laws caused population extremes to “regress toward the mean.” By this he meant that children of individuals having extreme values of a certain characteristic would tend to have less extreme values of this characteristic than their parent.

1. REGRESSION

2. Meaning
• In statistics, regression analysis is a statistical process
for estimating the relationships among variables.
• Regression analysis:There are several types of
regression:
– Linear regression
– Simple linear regression
– Logistic regression
– Nonlinear regression
– Nonparametric regression
– Robust regression
– Stepwise regression

3. Meaning: regression
Noun
1.the act of going back to a previous place or state;
return or reversion.
2.retrogradation; retrogression.
3.Biology. reversion to an earlier or less advanced
state or form or to a common or general type.
4.Psychoanalysis. the reversion to a chronologically
earlier or less
adapted pattern of behaviour and feeling.
5.a subsidence of a disease or its manifestations:
a regression of symptoms.

4. INTRODUCTION
• Many engineering and scientific problems
– concerned with determining a relationship between a
set of variables.
• In a chemical process, might be interested in
– the relationship between
• the output of the process, the temperature at which it
occurs, and
• the amount of catalyst employed.
• Knowledge of such a relationship would enable us
– to predict the output for various values of
temperature and amount of catalyst.

5. INTRODUCTION
• Situation-
– there is a single response variable Y , also called
the dependent variable
– depends on the value of a set of input, also called
independent, variables x1, . . . , xr
• The simplest type of relationship these
varibles is a linear relationship

6. INTRODUCTION
• If this was the linear relationship between Y and
the xi, i = 1, . . . , r, then it would be possible
– once the βi were learned
– to exactly predict the response for any set of input
values.
• In practice, such precision is almost never
attainable
• The most that one can expect is that Equation
would be valid subject to random error

7. INTRODUCTION

8. Introduction

10. • Suppose that the responses Yi corresponding to the input
values xi, i = 1, . . . , n are to be observed and used to
estimate α and β in a simple linear regression model.
• To determine estimators of α and β we reason as follows:
– If A is the estimator of α and B of β, then the estimator of the
response corresponding to the input variable xi would be A + Bxi.
• Since the actual response is Yi ,
– the squared difference is (Yi − A − Bxi)2, and so if A and B are the
estimators of α and β,
– then the sum of the squared differences between the estimated
responses
– the actual response values—call it SS — is given by

16. DISTRIBUTION OF THE ESTIMATORS
• To specify the distribution of the estimators A
and B, it is necessary to make additional
assumptions about the random errors aside
from just assuming that their mean is 0.
• The usual approach is to assume that the
random errors are independent normal
random variables having mean 0 and variance
σ2.

17. DISTRIBUTION OF THE ESTIMATORS

18. DISTRIBUTION OF THE ESTIMATORS

19. DISTRIBUTION OF THE ESTIMATORS

20. DISTRIBUTION OF THE ESTIMATORS

21. DISTRIBUTION OF THE ESTIMATORS

22. DISTRIBUTION OF THE ESTIMATORS
• Remarks

23. DISTRIBUTION OF THE ESTIMATORS

24. DISTRIBUTION OF THE ESTIMATORS

25. DISTRIBUTION OF THE ESTIMATORS
• Notation:

26. DISTRIBUTION OF THE ESTIMATORS

27. EXAMPLE
• The following data relate
– X: the moisture of a wet mix of a certain product
– Y: the density of the finished product
• Fit a linear curve to these data. Also determine SSR.

28. EXAMPLE

29. EXAMPLE

30. DISTRIBUTION OF THE ESTIMATORS

32. Distributions of the
Regression parameters

33. How to use these for β ??

34. How to use these for α??

35. How to use these for α+ βx0??

36. Regression
The term regression was originally employed by Francis
Galton while describing the laws
of inheritance.
Galton believed that these laws caused population
extremes to “regress toward the mean.”
By this he meant that children of individuals having
extreme values of a certain characteristic would tend to
have less extreme values of this characteristic than
their parent.

37. STATISTICAL INFERENCES ABOUT THE
REGRESSION PARAMETERS
• Inferences Concerning β

38. Inferences Concerning β

39. Inferences Concerning β

40. Inferences Concerning β

41. EXAMPLE
• An individual claims that the fuel consumption of
his automobile does not depend on how fast the
car is driven.
• To test the plausibility of this hypothesis, the car
was tested at various speeds between 45 and 70
miles per hour. The miles per gallon attained at
each of these speeds was determined, with the
following data resulting is given.
• Do these data refute the claim that the mileage
per gallon of gas is unaffected by the speed
at which the car is being driven?

42. the mileage per gallon of gas is unaffected
by
the speed at which the car is being driven

44. x Y x*x Y*Y x*Y
45.00 24.20 2,025.00 585.64 1,089.00
50.00 25.00 2,500.00 625.00 1,250.00
55.00 23.30 3,025.00 542.89 1,281.50
60.00 22.00 3,600.00 484.00 1,320.00
65.00 21.50 4,225.00 462.25 1,397.50
70.00 20.60 4,900.00 424.36 1,442.00
75.00 19.80 5,625.00 392.04 1,485.00
Sum 420.0 156.4 25,900 3,516.18 9,265
x-mean 60.00
y-mean 22.34
Sxx 700.00
SYY 21.76
SxY -119.00
SSR 1.53

45. EXAMPLE

47. EXAMPLE

49. Inferences Concerning α

55. Summary of Distributional Results

56. Summary of Distributional Results

57. THE COEFFICIENT OF DETERMINATION AND THE
SAMPLE CORRELATION COEFFICIENT

58. THE COEFFICIENT OF DETERMINATION AND THE
SAMPLE CORRELATION COEFFICIENT

59. THE COEFFICIENT OF DETERMINATION AND THE
SAMPLE CORRELATION COEFFICIENT

60. ANALYSIS OF RESIDUALS: ASSESSING
THE MODEL
The figure shows that, as indicated both by its scatter diagram and the random
nature of its standardized residuals, appears to fit the straight-line model quite
well.

61. The figure of the residual plot shows a discernible pattern, in that the residuals
appear to be first decreasing and then increasing as the input level increases. This
often means that higher-order (than just linear) terms are needed to describe the
relationship between the input and response. Indeed, this is also indicated by the
scatter diagram in this case.

62. The standardized residual plot shows a pattern, in that the absolute value
of the residuals, and thus their squares, appear to be increasing, as the
input level increases. This often indicates that the variance of the response
is not constant but, rather, increases with the input level.

63. TRANSFORMING TO LINEARITY
• The mean response is not a linear function
• In such cases, if the form of the relationship
can be determined it is sometimes
possible, by a change of variables, to
transform it into a linear form.
• For instance, in certain applications it is known that
W(t), the amplitude of a signal a time t after its
origination, is approximately related to t by the
functional form

64. TRANSFORMING TO LINEARITY

65. EXAMPLE
• The following table gives the percentages of a chemical that
were used up when an experiment was run at various
temperatures (in degrees celsius).
• Use it to estimate the percentage of the chemical that would
be used up if the experiment were to be run at 350 degrees.

66. EXAMPLE
• Let P(x) be the percentage of the chemical that is used up when the
experiment is run at 10x degrees.
• Even though a plot of P(x) looks roughly linear, we can improve upon the
fit by considering a nonlinear relationship between x and P(x).
• Specifically, let us consider a relationship of the form : 1 − P(x) ≈ c(1 − d)x

67. EXAMPLE

68. EXAMPLE

69. POLYNOMIAL REGRESSION

70. EXAMPLE
• Fit a polynomial to the following data.

71. EXAMPLE

72. EXAMPLE

73. MULTIPLE LINEAR REGRESSION

74. MULTIPLE LINEAR REGRESSION

75. MULTIPLE LINEAR REGRESSION

76. MULTIPLE LINEAR REGRESSION