Regression analysis is a statistical technique used to model the relationship between a dependent or outcome variable and one or more independent variables, and can be used to establish the nature and extent of the influence of the independent variables on the dependent variable. Simple regression involves one independent variable, while multiple regression involves using several independent variables to model their joint influence on the dependent variable. The goal of regression analysis is to fit a mathematical function or "regression model" to the data to help understand and predict the outcome variable.

1. Quantitative Methods
Varsha Varde

2. Quantitative Methods
Models for Data Analysis & Interpretation:
Regression Analysis

3. Varsha Varde 3
Cause and Effect
The Present Contains Nothing
More Than The Past, and
What Is Found In The Effect
Was Already In The Cause.
- Henri Bergson
(19th
Century French Philosopher)

4. Varsha Varde 4
Regression Model
• A Statistical Model which Depicts the
Influence of One Cardinal Variable (The
Cause) on Another Cardinal Variable (The
Effect).
• These Models Have a Wide Variety of
Forms and Degrees of Complexity.

5. Varsha Varde 5
Regression
• The Step Logically Next To Correlation.
• Situation: Usually, Correlation Between
Two Variables Is Not Mere Benign
Association. But, It Is In Fact Causation.
• It Is a Cause and Effect Relationship,
Where X Influences Y.
• X is the Cause Variable.
• Y is the Effect Variable.

6. Varsha Varde 6
Some Examples
Cause Effect
Movie Ticket Price Multiplex Occupancy
Machine Downtime Production
Rainfall at Night Absenteeism Next Day
R&D Expenditure Gross Profit
? ?

7. Varsha Varde 7
Regression
• Dictionary Says: The Act of Returning or
Stepping Back to a Previous Stage.
• Query: Do Quantitative Methods Force Us
to Regress instead of Progress?
• Or, Is It Back to the Future?
• Answer: Statistics, Like Any Other Field,
Adopts Crazy Names Arising from Some
Important Historical Events.
• Soap Opera.

8. Varsha Varde 8
Story of Regression
• Sir Francis Galton Studied the Heights of
the Sons in Relation to the Heights of
Their Fathers.
• His Conclusion: Sons of Tall Fathers Were
Not So Tall and Sons of Short Fathers
Were Not So Short as their Fathers.
• Path Breaking Finding: Human Heights
Tend To Regress Back To Normalcy.

9. Varsha Varde 9
Evolution of the Term
‘Regression’
• Since Then (1880), Similar Studies on
Nature and Extent of Influence of One or
More Variables on Some Other Variable
Acquired the Name ‘Regression Analysis’.
• In Quantitative Methods, Regression
Means a ‘Cause and Effect Relationship’.
• Cause Variable = Independent Variable
• Effect Variable = Dependent Variable

10. Varsha Varde 10
Scatter Plot
Horizontal Axis: Reasoning Scores
Vertical Axis: Creativity Scores

11. Varsha Varde 11
Scatter Plot
Horizontal Axis: Cause Variable: Reasoning Scores
Vertical Axis: Effect Variable: Creativity Scores

12. Varsha Varde 12
Regression Curve
Horizontal Axis: Cause Variable: Reasoning Scores
Vertical Axis: Effect Variable: Creativity Scores

13. Varsha Varde 13
Regression Analysis
• A Quantitative Method which Tries to
Estimate the Value of a Cardinal Variable
(the Effect) by Studying Its Relationship
with Other Cardinal Variables (the Cause).
• This Relationship is Expressed by a
Custom-Designed Statistical Formula
Called Regression Equation.

14. Varsha Varde 14
Purpose of Regression Analysis
1. To Establish Exact Nature of Influence of
Cause Variable on Effect Variable.
2. To Determine the Quantum of Influence.
3. To Estimate an Unknown Value of Effect
Variable from Value of Cause Variable.
4. To Forecast Future Values of Effect
Variable from Info about Cause Variable

15. Varsha Varde 15
Patterns of Regression Curves
• Pattern: Upward Sloping Straight Line
• Mathematical Model: Y = a + bX (b > 0)

16. Varsha Varde 16
Estimating Regression
Parameters a & b
• Formula for Regression Coefficient b :
Mean of Products of Values – Product of the Two Means
= --------------------------------------------------------------------------
Variance of Cause Variable
• Formula for Regression Constant a :
a = Mean of Effect Variable Minus b times
Mean of Cause Variable
• Don’t Worry. This is the Job of SPSS.

17. Varsha Varde 17
Estimating
Correlation Coefficient
• Recall the Formula for Correlation Coeff.
• Pearson’s Correlation Coefficient
• Formula:
Mean of Products of Values – Product of the Two Means
= --------------------------------------------------------------------------
Product of the Two Standard Deviations
• Spot the Similarity and the Difference.

18. Varsha Varde 18
A Simple Example
Empl. No. Yrs in Co. Salary (‘000) Product
1 2 25 50
2 3 30 90
3 5 37 185
4 7 38 266
5 8 40 320
Total 25 170 911
Arith Mean 5 34
Std. Dev. 2.3 5.6

19. Varsha Varde 19
Regression Model
• Formula for Regression Coefficient b :
Mean of Products of Values – Product of the Two Means
= --------------------------------------------------------------------------
Variance of Cause Variable
(911 / 5) – (5 x 34) 182.2 – 170 12.2
= ----------------------- = ------------- = ----------- = 2.30
2.3 x 2.3 5.3 5.3
• Formula for Regression Constant a :
a = Mean of Effect Variable Minus b times Mean of
Cause Variable = 34 – 2.3 x 5 = 22.5
• Regression Model: Y = 22.5 + 2.3 X

20. Varsha Varde 20
Check Goodness of the Model
Empl. No. Yrs in Co. Salary (‘000) Estimate
1 2 25
2 3 30
3 5 37
4 7 38
5 8 40
Total 25 170
Arith Mean 5 34
Std. Dev. 2.3 5.6

21. Varsha Varde 21
Check Goodness of the Model
Empl. No. Yrs in Co. Salary (‘000) Estimate
1 2 25 27.1
2 3 30 29.4
3 5 37 34.0
4 7 38 38.6
5 8 40 40.9
Total 25 170 170
Arith Mean 5 34
Std. Dev. 2.3 5.6

22. Varsha Varde 22
Concept: Error of Estimation
• Note the Difference Between the Actual
Values of Effect Variable (Salary) and the
Values Estimated by the Regression
Model
• This is the Error of Estimation
• Less the Error, Better the Model. Ideally 0.
• Statistical Model: Y = a + b X + e
• If Correlation is Perfect (+1 or -1), e = 0.

23. Given below are five observations collected in a
regression study on two variables, x (independent
variable) and y (dependent variable).
x y
2 4
3 4
4 3
5 2
6 1
a. Develop the least squares estimated regression
equation.
b. Estimate value of y for x=7.Varsha Varde 23
Q5.

24. Varsha Varde 24
Exercise: Fit a Regression Model to
Reasoning & Creativity Scores
Apl No, RsnSc CrvSc Apl No, RsnSc CrvSc
01 15.2 11.9 11 8.1 6.8
02 9.9 13.1 12 15.2 13.0
03 7.1 8.9 13 10.9 13.9
04 17.9 17.4 14 17.2 19.1
05 5.1 6.9 15 8.2 10.1
06 10.0 8.8 16 10.8 15.9
07 7.2 14.0 17 12.0 12.1
08 17.1 15.8 18 13.1 16.0
09 15.2 9.7 19 17.9 19.2
10 9.2 12.1 20 7.1 11.9

25. Varsha Varde 25
Exercise
• Does Your Model Look Like What I Got?:
Creativity Scores = 5.23 + 0.65 x Reasoning Scores + e
• Test the Goodness of Your Regression
Model
• How Bad are the Errors?

26. Varsha Varde 26
Other Patterns of Regression Curves
• Pattern: Downward Sloping Straight Line
• Statistical Model: Y = a - bX + e (b > 0)

27. Varsha Varde 27
Other Patterns of Regression Curves
• Pattern: Simple Exponential
Model: Log Y = a + bX + e (b > 0)
• Pattern: Negative Exponential
Model: Log (1/Y) = a + bX + e (b > 0)
• Pattern: Upward Curvilinear
Model: Y = a + b Log X + e (b > 0)
• Pattern: Downward Curvilinear
• Pattern: Logistic or S Curve

28. Varsha Varde 28
Your Role as a Manager
• Grasp the Situation Thoroughly. (Qualitative)
• Identify Related Cardinal Variables. (DIY)
• Obtain Quantitative Data on Them.
• Draw Scatter Plot. Your Asstt Will Do It For You
• If It Shows a Pattern, Compute Correlation
Coefficient. (Use SPSS or YAWDIFY)
• If It Is High (+ or -), Draw a Free Hand Curve
and Identify the Pattern of Regression Curve.
• Compute Regression Parameters for the Pattern
and Fit Regression Model. (SPSS or YAWDIFY)

29. Varsha Varde 29
A Word of Caution
• Undertake Regression Analysis Only For
Cardinal Variables.
• Select the Variables Only If You Logically
Suspect Influence of One Over the Other.
• Carry Out Regression Analysis Only After
Completing Correlation Analysis AND
Only If The Selected Cause and Effect
Variables Are Highly Correlated.

30. Varsha Varde 30
Simple and Multiple Regression
• Simple Regression: One Cause Variable
Influences the Effect Variable.
• This is What We Focused On So Far.
• Regression Models Have a Wide Variety
of Forms and Degrees of Complexity.
• Multiple Regression: Several Cause
Variables Jointly Influence Effect Variable.

31. Varsha Varde 31
Multiple Regression
• Multiple Regression Analysis is a Method
to Analyze the Effect of Joint Influence of
Many Cause Variables on Effect Variable.
• Multiple Regression Model:
Y = a + b1X1 + b2X2 + - - - - +bnXn + e
• Caution: Cause Variables X1, X2, - - - -, Xn
Should Not Be Inter-Correlated.
• Otherwise You Face Multicollinearity.

32. Varsha Varde 32
Exercise: Select Cause Variables
Cause # 1
X1
Cause # 2
X2
Cause # 3
X3
Effect
Y
Machine
Downtime
Labour
Absenteeism
Power
Outage
Monthly
Production
EPS ? ? BASF
Share Price
? ? ? MRP
? ? ? Manpower
Requiremt