Regression analysis is a statistical technique for predicting the value of a variable based on the value of one or more other variables. Linear regression involves using a linear equation to model the relationship between a dependent or criterion variable (Y) and one or more independent or predictor variables (X). Multiple regression expands this to model the relationship between a dependent variable and more than one independent variable. Key aspects of regression covered in the document include the regression equation, slope, intercept, correlation, prediction, and use of regression in research.

1. REGRESSION
Desmond Ayim-Aboagye, Ph.D.
LINEAR AND MULTIVARIATE

2. LINEAR REGRESSION
• Regression is a technique for predicting a
score on variable Y based on what we already
know to be true about the value of some
variable X.
• Use one variable (i.e., mid term grade) to
predict the value of another variable (i.e., final
course grade)
• If correlation is equal to association, then,
regression is equal to prediction

3. Bivariate or 2 Variable Regression
• Regression analysis is based on correlational
analysis, and it involves examining changes in
the level of Y relative to changes in the level of
X
• Variable Y is the dependent measure–
criterion variable
• Variable X is independent measure– predictor
variable

4. Z-Score Approach to Regression
• A variable Y can be predicted from X using the
Z score regression equation, which is,
• Zŷ = rxyZx
• Whre Zŷ is a predicted score variable Y. Here ŷ
(“Y caret“ or "Y hat") will be used to indicate
a predicted or estimated value for Y. The
correlation between variables X and Y is
denoted rxy and Zx is an actual z score
based on variable X.

5. Importance of Z score Equation
• 1. When rxy is positive in value, zx will be multiplied by
a positive number, thus Zŷ will be positive when Zx is
positive and it will be negative when Zx is negative.
• The characteristic above is important. When rxy is
positive, then Zŷ will have the same sign as Zx, so that a
high score will covary with high scores and low scores
will do so with low scores. When rxy is negative,
however, the sign of Zŷ will be opposite of Zx; low
scores will be associated with high scores and high
scores with low scores.
• 2. rxy = ±1.00, Zŷ will have the same score as Zx
• When < ± 1.00, Zŷ will be closer to 0.0 than Zx

6. The role of the Mean
• When two variables are uncorrelated with one
another, the best predictor of any individual
score on one of the variables is the mean. The
mean is the predicted value of X or Y when the
correlation between these variables is 0.

7. Computational Approaches to
Regression
• Linear relationships between variables X and Y
• Y = a + b (X),
• Y = criterion variable (trying to predict)
• a and b constants fixed values
• X = predictor variable

8. Slope of the line
• B is also called the slope of the line
• B = change in Y
• change in X
• A = the intercept of the line or y intercept
• The intercept is the point in a regression of Y
and X where the line crosses the Y axis

9. Regression Line
• A regression line is a straight line projecting
through a given set of data, one designed to
represent the best fitting linear relationship
between variables X and Y

10. Regression toward the mean
• Regression toward the mean refers to
situations where initially high or low
observations are found to move closer to or
"regress toward" their mean after subsequent
measurement.

11. Research tool
• In practice, we forget that there is really little to be gained
from thinking about regression as a way to predict Y from X
when we have all the actual values of Y. Regression is really
for predicting the behaviour of individuals in samples
beyond the original sample.
• 1. Economists: income as a predictor variable and criterion
variables like consumption and savings
• 2. Management professionals rely on regression to link
skills, effort, responsibility, and job conditions to wages.
• 3. Verify personnel decision
• 4. Computer science instructor: how doing homework
actually predicts their exams performance.

12. Multivariate Regression
• Multiple regression is a statistical technique
for exploring the relationship between one
dependent variable (Y) and more than one
independent variable (X₁, X₂, … Xn).
• Used in behavioral and natural sciences
• Y = a + b₁ (X₁) + b₂ (X₂).

13. Multiple Regression Analysis
• A. multiple correlation coefficient
• B. Symbolized by letter R
• C. Indicates the relationship between a given
criterion variable (Y) and a set of predictor
variables (X).
• D. As R increases in magnitude, the multiple
regression equation is said to perform a better
job of predicting the dependent measure from
the independent variables.
• R ² percentage of variance in Y that is accounted
for by the set of predictors, that is, X variables.

14. Regression Class TEST
• 1. What is the nature of the relationship between correlation and
regression? (2 marks)
• 2. Define each of the variables and constants in the formula Y= a + bX. (3
marks)
• 3. A student wonders if birth order (first born, second born, and so on)
predicts shyness, such that first or only children tend to be shyer than
later born children are. The student gives a standardized measure of
shyness to 60 participants (30 males, 30 females), asking each one to
indicate their number of siblings and their birth orders. Is a regression
analysis appropriate? (3 marks)
•
• 4. Describe the importance of SPSS to behavioural science research. (2
marks)