Multivariate OLS regression analysis. With interaction terms.

1. 1
What is covered in this presentation and the topics that
precede it
• Descriptive (summary) statistics
• Binary linear regression
• Multiple (multivariate) regression
– Two (or more) scale independent variables
– Dummy variable as independent variable
– Interaction terms

2. 2
Review
Ordinary Least Square (OLS) Linear Regression
• No measurement error
• All relevant Xs are included
• No irrelevant X is included
• E(ei) = 0
• Var(ei) = σ2 (constant)
• Cov(ei, Xi) = 0
• Variations in X
• k < n
When several conditions are met, OLS is B.L.U.E.
(Best, Linear, Unbiased Estimator).

3. 30.00 5.00 10.00 15.00 20.00 25.00
Percent workers who are union members
30.0
40.0
50.0
60.0PercentvotingforKerry2004
Review
Linear Regression. Find the line that best fits the data

4. 4
Y
constant
0 X
Y = α + β X
Y = αhat+ βhatXi + εi
..
.
. The OLS regression line
minimizes the sum of
squared errors. Error =
Review
Linear Regression

5. 5
Review
Linear Regression
Hillary Clinton’s
vote share as a
function of %black.
Each dot
represents a state
in the United
States.

6. 6
Review
Multiple regression with two scale/interval level
independent variables
X1
X2
Y
We can’t visualize
multiple regression
with more than two
independent variables
for obvious reason.
X1i
X2j
Yi
Y = α+ β1X1 +β2X2
α

7. 7
Y
X
Two parallel lines with
different intercepts and common slope.
Review
Multiple regression with one interval level
independent variable and one dummy variable

8. 8
Weekend
Weekday
-1.3 X
(temperature)
-29.7
28.3
Review
Multiple regression with dummy variable
Example (1)
Ice
cream
sales
Ice cream sales = α + βtemp*Temperature +
βweekend*Weekend

9. 9
Review
Multiple Regression with dummy variable
Example (2)
Pay
($)
Experience
Pay = α + βexperience*Experience + βmale*Male
Male-female
difference

10. A sidenote
A nominal/categorical variable with more
than two value categories
10
NEW DUMMY VARIABLES
Frequency Percent
Valid
Percent
Cumulativ
e Percent white black other hispanic
-9 -9. Missing 33 .8 .8 .8
1 1. White, non-Hispanic
3038 71.1 71.1 71.9
1 0 0 0
2 2. Black, non-Hispanic
398 9.3 9.3 81.2
0 1 0 0
3 3. Asian, native Hawaiian
or other Pacif Islr,non-
Hispanic
148 3.5 3.5 84.7
0 0 1 0
4 4. Native American or
Alaska Native, non-Hispanic 27 .6 .6 85.3
0 0 1 0
5 5. Hispanic
450 10.5 10.5 95.9
0 0 0 1
6 6. Other non-Hispanic incl
multiple races [WEB: blank
'Other' counted as a race]
177 4.1 4.1 100.0
0 0 1 0
Total 4271 100.0 100.0
V161310x PRE: SUMMARY - R self-identified race
Valid
Can be transformed into a series of dummy
variables. These multiple dummy variables can be
used as independent variables.

11. 11
Weekend
Weekday
-1.3 X
(temperature)
-29.7
28.3
Ice
cream
sales
Ice cream sales = α + βtemp*Temperature +
βweekend*Weekend
Ice cream sales cannot take
negative value…but
temperature in F virtually never
hits 0 in nature.
The ice cream example. Does this
represent actual data well? Maybe…….

12. 12
Maybe not!
Interaction example (1)
Ice
cream
sales
Weekend
Weekday
Weekend
Weekday
Temperature
Temperature
Ice
cream
sales

13. 13
Interaction example (2)
Pay
($)
Experience
Pay
($)
Experience

14. 14
Interaction example (2) continued
Pay
($)
Experience
Pay
($)
Experience
Both the slopes
and the constants
can vary.

15. 15
Interaction example (3) a. and b.
Bone
density
Cola Consumption
Alcohol
tolerance
Body weight
Male
Female
Male
Female

16. 16
How Interaction Works (1) The simple model without
interaction
Pay = α + βexperience*Experience + βmale*Male
Pay
($)
Experience
α
βexperience
βmale
βexperience>0, βmale>0
The two groups, men and women, have different
intercepts and common slope

17. 17
How Interaction Works (2) Adding an interaction
Pay = α + βexperience*Experience + βmale*Male +
βinteraction*Interaction
Pay
($)
Experience
α
βexperience
βmale
βinteraction
βexperience>0, βmale>0, βinteraction>0
The two groups, men and women, have different
intercepts and different slopes

18. 18
How Interaction Works (3)
Pay = α + βmale+ (βexperience+ βinteraction)*Experience
Pay = α + βexperience*Experience
Pay
($)
Experience
α
βexperience
βmale
βinteraction

19. 19
How Interaction Works (4)
Both cases (male, female) can be summarized by the
following single equation.
Pay = α + βexperience*Experience + βmale*Male +
βinteraction*Interaction
Interaction : Male*Experience
Pay
($)
Experience
α
βexperience
βmale
βinteraction

20. 20
Employee# Salary Gender Experience Interaction (Male*Experience)
1 35.6 Male 14 14
2 16.4 Female 5 0
3 15.2 Male 3 3
4 24.7 Male 8 8
5 23.1 Female 9 0
(continued)
How Interaction Works (5) Example dataset
Creating an interaction
Interaction = Male*Experience
In SPSS, use Transform – Compute variables
In STATA, “gen male_exp=male*experience”

21. 21
How Interaction Works (6) Estimation
Hypothetical results
Pay = 10 + 2*Experience + 1.5*Male + 0.2*Interaction
Pay = 10 + 1.5 + (2+0.2)*Experience
Pay = 10 + (2)*Experience

22. 22
Summry
Multiple regression with Interaction
• Interaction term is a variable
• It allows slopes to diverge between Group 1 (1) and
Group 2 (0)
• It measures the difference in slope b/w Group 1 (1) and
Group 2 (0)
• It is created by multiplying dummy variable and the
interval level variable of interest
• When βinteraction > 0, the effect of X on Group 1 is greater
than its effect on Group 2. The slope will be steeper.
• When βinteraction < 0, the effect of X on Group 1 is smaller
than its effect on Group 2. The slope will be less steep.

23. 23
Multiple Regression with Interaction
Steps in using interaction term
1. Choose a dependent variable (must be scale for OLS)
2. Create a dummy variable from one of the independent
variables. OR use R’s gender
3. Choose a scale variable for which an interaction may be
at work
4. Create an interaction by multiplying the two variables:
dummy and the scale
5. Estimate multiple regression with dummy, scale, and
the interaction b/w the two.