Measures of Association and Correlation Coefficients
•Download as PPT, PDF•
1 like•612 views
This chapter discusses various measures of association that can be used to study relationships between two or more variables. It introduces correlation analysis and regression analysis, which are used to measure the strength and direction of relationships between continuous variables. Nonparametric measures are also presented as alternatives when key assumptions of parametric techniques cannot be met. Examples are provided to illustrate concepts like correlation coefficients, lines of best fit, and testing goodness of fit. Key terms related to correlation, regression, and other measures of association are defined.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Measures of Association and Correlation Coefficients
Similar to Measures of Association and Correlation Coefficients (20)
More from Dhamo daran
More from Dhamo daran (20)
Recently uploaded
Recently uploaded (17)
Measures of Association and Correlation Coefficients
- 1. Chapter 18Chapter 18 Measures ofMeasures of AssociationAssociation McGraw-Hill/Irwin Copyright © 2011 by The McGraw-Hill Companies, Inc. All Rights Reserved.
- 2. 18-2 Learning ObjectivesLearning Objectives Understand . . . • How correlation analysis may be applied to study relationships between two or more variables • The uses, requirements, and interpretation of the product moment correlation coefficient. • How predictions are made with regression analysis using the method of least squares to minimize errors in drawing a line of best fit.
- 3. 18-3 Learning ObjectivesLearning Objectives Understand . . . • How to test regression models for linearity and whether the equation is effective in fitting the data. • Nonparametric measures of association and the alternatives they offer when key assumptions and requirements for parametric techniques cannot be met.
- 4. 18-4 Invalid AssumptionsInvalid Assumptions “The invalid assumption that correlation implies cause is probably among the two or three most serious and common errors of human reasoning.” Stephen Jay Gould paleontologist and science writer
- 5. 18-5 PulsePoint:PulsePoint: Research RevelationResearch Revelation 25 The percent of students using a credit card for college costs due to convenience.
- 6. 18-6 Measures of Association:Measures of Association: Interval/Ratio DataInterval/Ratio Data Pearson correlation coefficient For continuous linearly related variables Correlation ratio (eta) For nonlinear data or relating a main effect to a continuous dependent variable Biserial One continuous and one dichotomous variable with an underlying normal distribution Partial correlation Three variables; relating two with the third’s effect taken out Multiple correlation Three variables; relating one variable with two others Bivariate linear regression Predicting one variable from another’s scores
- 7. 18-7 Measures of Association:Measures of Association: Ordinal DataOrdinal Data Gamma Based on concordant-discordant pairs; proportional reduction in error (PRE) interpretation Kendall’s tau b P-Q based; adjustment for tied ranks Kendall’s tau c P-Q based; adjustment for table dimensions Somers’s d P-Q based; asymmetrical extension of gamma Spearman’s rho Product moment correlation for ranked data
- 8. 18-8 Measures of Association:Measures of Association: Nominal DataNominal Data Phi Chi-square based for 2*2 tables Cramer’s V CS based; adjustment when one table dimension >2 Contingency coefficient C CS based; flexible data and distribution assumptions Lambda PRE based interpretation Goodman & Kruskal’s tau PRE based with table marginals emphasis Uncertainty coefficient Useful for multidimensional tables Kappa Agreement measure
- 9. 18-9 Researchers Search for InsightsResearchers Search for Insights Burke, one of the world’s leading research companies, claims researchers add the most value to a project when they look beyond the raw numbers to the shades of gray… what the data really mean.
- 10. 18-10 Pearson’s Product MomentPearson’s Product Moment CorrelationCorrelation rr Is there a relationship between X and Y? What is the magnitude of the relationship? What is the direction of the relationship?
- 11. 18-11 Connections andConnections and DisconnectionsDisconnections “To truly understand consumers’ motives and actions, you must determine relationships between what they think and feel and what they actually do.” David Singleton, vp of insights Zyman Marketing Group
- 12. 18-12 Scatterplots of RelationshipsScatterplots of Relationships
- 14. 18-14 Diagram of Common VarianceDiagram of Common Variance
- 15. 18-15 Interpretation of CorrelationsInterpretation of Correlations X causes YX causes Y Y causes XY causes X X and Y are activated by one or more other variables X and Y are activated by one or more other variables X and Y influence each other reciprocally X and Y influence each other reciprocally
- 17. 18-17 Interpretation of CoefficientsInterpretation of Coefficients A coefficient is not remarkable simply because it is statistically significant! It must be practically meaningful.
- 18. 18-18 Comparison of Bivariate LinearComparison of Bivariate Linear Correlation and RegressionCorrelation and Regression
- 19. 18-19 Examples of Different SlopesExamples of Different Slopes
- 20. 18-20 Concept ApplicationConcept Application X Average Temperature (Celsius) Y Price per Case (FF) 12 2,000 16 3,000 20 4,000 24 5,000 Mean =18 Mean = 3,500
- 21. 18-21 Plot of Wine Price by AveragePlot of Wine Price by Average TemperatureTemperature
- 22. 18-22 Distribution of Y forDistribution of Y for Observation of XObservation of X
- 23. 18-23 Wine Price Study ExampleWine Price Study Example
- 24. 18-24 Least Squares Line: Wine PriceLeast Squares Line: Wine Price StudyStudy
- 25. 18-25 Plot of Standardized ResidualsPlot of Standardized Residuals
- 26. 18-26 Prediction andPrediction and Confidence BandsConfidence Bands
- 27. 18-27 Testing Goodness of FitTesting Goodness of Fit Y is completely unrelated to X and no systematic pattern is evident There are constant values of Y for every value of X The data are related but represented by a nonlinear function
- 28. 18-28 Components of VariationComponents of Variation
- 29. 18-29 FF Ratio in RegressionRatio in Regression
- 30. 18-30 Coefficient of Determination:Coefficient of Determination: rr22 Total proportion of variance in Y explained by X Desired r2 : 80% or more
- 31. 18-31 Chi-Square Based MeasuresChi-Square Based Measures
- 32. 18-32 Proportional Reduction ofProportional Reduction of Error MeasuresError Measures
- 33. 18-33 Statistical AlternativesStatistical Alternatives for Ordinal Measuresfor Ordinal Measures
- 34. 18-34 Calculation of Concordant (Calculation of Concordant (PP),), Discordant (Discordant (QQ), Tied (), Tied (Tx,TyTx,Ty), and), and Total Paired Observations:Total Paired Observations: KeyDesign ExampleKeyDesign Example
- 35. 18-35 KDL Data for Spearman’s RhoKDL Data for Spearman’s Rho _______ _____ Rank By_____ _____ _____ Applicant Panel x Psychologist y d d2 1 2 3 4 5 6 7 8 9 10 3.5 10.0 6.5 2.0 1.0 9.0 3.5 6.5 8.0 5.0 6.0 5.0 8.0 1.5 3.0 7.0 1.5 9.0 10.0 4.0 -2.5 5.0 -1.5 .05 -2 2.0 2.0 -2.5 -2 1.0 6.25 25.00 2.52 0.25 4.00 4.00 4.00 6.25 4.00 _1.00_ 57.00 .
- 36. 18-36 Key TermsKey Terms • Artifact correlations • Bivariate correlation analysis • Bivariate normal distribution • Chi-square-based measures • Contingency coefficient C • Cramer’s V • Phi • Coefficient of determination (r2) • Concordant • Correlation matrix • Discordant • Error term • Goodness of fit • lambda
- 37. 18-37 Key TermsKey Terms • Linearity • Method of least squares • Ordinal measures • Gamma • Somers’s d • Spearman’s rho • tau b • tau c • Pearson correlation coefficient • Prediction and confidence bands • Proportional reduction in error (PRE) • Regression analysis • Regression coefficients
- 38. 18-38 Key TermsKey Terms • Intercept • Slope • Residual • Scatterplot • Simple prediction • tau
Editor's Notes
- This chapter explains the use of several techniques including correlation analysis and regression analysis.
- See the text Instructors Manual (downloadable from the text website) for ideas for using this research-generated statistic.
- Exhibit 18-1 in the text presents a list of commonly used measures of association. These measures of association will be discussed further in this chapter. This slide shows those measures of association relevant for interval and ratio scaled data. The measures used for ordinal and nominal data are presented on the following slides.
- Exhibit 18-1 This slide presents the commonly used measures of association for ordinal data.
- This final component of Exhibit 18-1 presents the measures of association used for nominal data.
- The Pearson correlation coefficient varies over a range of +1 to -1. The designation r symbolizes the coefficient’s estimate of linear association based on sampling data. The coefficient p represents the population correlation. Correlation coefficients reveal the magnitude and direction of relationships. The magnitude is the degree to which variables move in unison or opposition. The closer the coefficient is to 1 (regardless of sign) the stronger the relationship. The sign indicates the direction of the relationship. A positive sign indicates a direct relationship and a negative sign indicates an inverse relationship.
- Exhibit 18-2 Scatterplots are essential for understanding the relationships between variables. They provide a means for visual inspection of data that a list for two variables cannot. Both the direction and shape of a relationship are conveyed in a plot. Exhibit 18-2 contains a series of scatterplots that depict some relationships across the range r. The three plots on the left side of the figure have their points sloping from the upper left to the lower right of each x-y plot. They represent different magnitudes of negative relationships. On the right side of the figure, the three plots have opposite directional patterns and show positive relationships. When stronger relationships are apparent, the points cluster close to an imaginary line passing through the data. For a weak relationship or no relationship (r=0), data points are randomly scattered with little or no directional pattern.
- Exhibit 18-4 The need for data visualization is illustrated with four small data sets possessing identical summary statistics but displaying different patterns. Exhibit 18-3 contains the data and the exhibit shown in the slide plots them. In Plot 1, the variables are positively related. In Plot 2, the data are curvilinear and r is an inappropriate measure of their relationship. In Plot 3, an influential point that changed the coefficient is shown. In Plot 4, the values of x are constant. Correlations are based on linear relationships and bivariate normal distributions.
- Exhibit 18-7 The amount of common variance in X and Y may be summarized by r2, the coefficient of determination. As shown in the exhibit in the slide, the overlap between the two variables is the proportion of their common or shared variance. The area of overlap represents the percentage of the total relationship accounted for by one variable or the other. In the slide, 86% of the variance in X is explained by Y and vice versa.
- A correlation coefficient of any magnitude or sign, regardless of its statistical significance, does not imply causation. Correlation provides no evidence of cause and effect. There are several alternate explanations that may be provided for correlation results and these are listed in the slide. Note that causation may actually exist, but the correlation only shows that a relationship exists. Therefore, x may cause y or y may cause x. However, it may also be that z causes x and y or that y and x interact.
- Exhibit 18-8 Exhibit 18-8 Artifact correlations occur where distinct groups combine to give the impression of one. Artifact correlations should be avoided because they appear as one group when, in fact, there are distinct groups present. The left panel shows data from two business sectors. If all the data points for x and y variables are aggregated and a correlation is computed for a single group, a positive correlation results. Separate calculations for each sector reveal no relationship. In the right panel, the companies in the financial sector score high on assets and low in sales but they are all banks. When the data for banks are removed, the correlation is nearly perfect. When banks are kept in, the overall relationship drops considerably.
- In many relationships, other factors combine to make the coefficient’s meaning misleading. For example, there is a relationship between the results of presidential elections and the results of the Washington Redskins’ game before the election. This does not indicate a meaningful relationship.
- Exhibit 18-9 Relationships also serve as a basis for estimation and prediction. Simple and multiple predictions are made with a technique called regression analysis. When we take the observed values of X to estimate or predict corresponding Y values, the process is called simple prediction. When more than one X variable is used, the outcome is a function of multiple predictors. The similarities and differences of correlation and regression are summarized in Exhibit 18-9. Their relatedness suggests that beneath many correlation problems is a regression analysis that could provide further insight about the relationship of Y with X.
- Exhibit 18-10 A straight line is fundamentally the best way to model the relationship between two continuous variables. The bivariate linear regression may be expressed as Use formula from page 545 Where the value of the dependent variable Y is a linear function of the corresponding value of the independent variable Xi in the ith observation. The slope and the Y intercept are known as regression coefficients. The slope, 1, is the change in Y for a 1-unit change in X. It is sometimes called the “rise over run.” This is defined by the formula Use formula from page 545 This is the ratio of change in the rise of the line relative to the run or travel along the X axis. Exhibit 18-10 shows a few of the many possible slopes. The intercept is the value for the linear function when it crosses the Y axis; it is the estimate of Y when X = 0. A formula for the intercept based on the mean scores of the X and Y variables is Use formula from page 545
- In this example, X represents the average growing-season temperature in degrees Celsius and Y the price of a 12-bottle case in French francs (6.8 French francs = 1 euro). The plotted data in Exhibit 18-11 is shown on the next slide.
- Exhibit 18-11 The plot shows a linear relationship between the pairs of points and a perfect positive correlation, ryx = 1.0. The slope of the line is calculated: Use formula from page 546 Where the XiYi values are the data points (20,4000) and XjYj are points (16, 3000). The intercept is -1,000, the point at which X = 0 in this plot. This area is off the graph and appears in an insert on the figure. Substituting into the formula, we have the simple regression equation We can now predict that a warm growing season with 25.5C temperature would bring a case price of 5,375 French francs.
- Exhibit 18-12 It is more likely that we will collect data where the values of Y vary for each X value. Considering Exhibit 18-12, we should expect a distribution of price values for the temperature X = 16, another for X = 20, and another for each value of X. The means of these Y distributions will also vary in some systematic way with X. These variations lead us to construct a probabilistic model that also uses a linear function. This function is written Use formula from page 547 As shown in the exhibit, the actual values of Y may be found above or below the regression line represented by the mean value of Y (o + 1Xi) for a particular value of X. These deviations are the error in fitting the line and are often called the error term.
- Exhibit 18-14 Exhibit 18-14 contains a new data set for the wine price example. Our prediction of Y from X must now account for the fact that the X and Y pairs do not fall neatly along the line. Exhibit 18-14, shown in the slide, suggests two alternatives based on visual inspection. The method of least squares allows us to find a regression line, or line of best fit, which will keep these errors to a minimum. It uses the criterion of minimizing the total squared errors of estimate. When we predict values of Y for each Xi, the difference between the actual Yi and the predicted Y is the error. This error is squared and then summed. The line of best fit is the one that minimizes the total squared errors of prediction. Use formula from page 549 Regression coefficients 0 and 1 are used to find the least-squares solution. They are computed as follows:
- Exhibit 18-15 Substituting data from Exhibit 18-14 into both formulas, we get Use formula from page 524 Before drawing the regression line, we select two values of X to compute. Using values 13 and 24 for Xi, the points are Use formula from page 000
- Exhibit 18-16 A residual is what remains after the line is fit. When standardized, residuals are comparable to Z scores with a mean of 0 and a standard deviation of 1. In this plot, the standardized residuals should fall between 2 and -2, be randomly distributed about zero, and show no discernible pattern. All these conditions say the model is applied appropriately.
- Exhibit 18-17 Prediction and confidence bands are bow-tie shaped confidence interval around a predictor. Predictors farther from the mean have larger bandwidths. If we wanted to predict the price of a case of investment-grade red wine for a growing season that averages 21 degrees Celsius, our prediction would be Use formula from page 552 This is a point prediction Y and should be corrected for greater precision. As with other confidence estimates, we establish the degree of confidence desired and substitute into the formula Use formula from page 552
- With the regression line plotted, next we evaluate the goodness of fit. Goodness of fit is a measure of how well the regression model is able to predict Y. The most important test in bivariate linear regression is whether the slope, 1 , is equal to zero. Zero slopes result from the conditions listed in the slide. To test whether the slope is equal to zero, we use a two-tailed test. The test follows the t distribution for n-2 degrees of freedom. In this case, we reject the null hypothesis because the calculated t is greater than any t value for 8 degrees of freedom and an alpha of .01.
- Exhibit 18-18 Computer printouts generally contain an analysis of variance (ANOVA) table with an F test of the regression model. In bivariate regression, t and F tests produce the same results since t2 is equal to F. In multiple regression, the F test has an overall role for the model and each of the independent variables is evaluated in a separate t-test. For regression, the ANOVA comprises explained deviations and unexplained deviations. Together, they constitute total deviation. This is shown graphically in the exhibit in the slide. These sources of deviation are squared for all observations and summed across the data points.
- Exhibit 18-19 In Exhibit 18-19, we develop this concept sequentially, concluding with the F test of the regression model for the wine data. Based on the results presented in that table, we find evidence of a linear relationship. The null hypothesis is rejected.
- Exhibit 18-20 Nominal measures are used to assess the strength of relationships in cross-classification tables. They are often used with chi-square. Exhibit 18-20 reports a 2 x 2 table showing the test of an advertising campaign involving 66 people The variables are success of the campaign and whether direct mail was used. In this example, the observed significance level is less than the testing level (.05) and the null hypothesis is rejected. A correction to chi-square is provided. The exhibit also provides an approximate significance of the coefficient based on the chi-square distribution. This is a test of the null hypothesis that no relationship exists between the variables of direct mail and campaign success. Phi is used with chi-square and is a measure of association for nominal, nonparametric variables. Phi ranges from 0 to +1. It is calculated as Cramer’s V is also used with chi-square and is a measure of association for nominal variables with larger than 2 x 2 tables. It is calculated as The contingency coefficient C is a not comparable to other measures and has a different upper limit for various table sizes. C is calculated as
- Exhibit 18-21Proportional reduction in error (PRE) statistics are the second type used with contingency tables. Lambda and tau are the examples discussed in the text. Lambda is a measure of how well the frequencies of one nominal variable predict the frequencies of another variable. Exhibit 18-21 shows result from an opinion survey with a sample of 400 (see highlighted portion of exhibit) shareholders in publicly traded advertising firms. If asked to predict the opinions of an individual in the sample, we would achieve the best prediction by choosing the mode. However, by doing so, we would be wrong 180 out of 400 times. Lambda shows us that we improve the prediction by 39% if we include information about a respondent’s occupational status. The formula for Lambda is This figure is also highlighted. Goodman and Kruskal’s tau uses table marginals to reduce prediction errors. In predicting opinion in the example without knowledge of occupational class, we would expect a 50.5% correct classification and a 49.5% probability of error. When additional knowledge of occupational class is used, information for correct classification of the opinion variable is improved to 62.7% with a 37.3% probability of error. Tau is computed as
- Exhibit 18-22 When data require ordinal measures, we may use Gamma, Kendall’s tau b and c, Somers’s d, and Spearman’s rho. All but Spearman’s rank-order correlation are based on the concept of concordant and discordant pairs, which are discussed further on the following slide. Exhibit 18-22 presents data for 70 employees of KeyDesign who have been evaluated for coronary risk. The management levels are ranked as are the fitness assessments by the physicians. The information in the exhibit has been arranged so that the number of concordant and discordant pairs of individual observations may be calculated. When a subject that ranks higher on one variable also ranks higher on the other, the pairs of observations are said to be concordant. If a higher ranking on one is accompanied by a lower ranking on the other variable, the pairs of observations are discordant. Exhibit 18-23 summarizes the procedure for calculating the summary terms needed. Gamma is a statistic that compares concordant and discordant pairs and then standardizes the outcome by maximizing the value of the denominator. Kendall’s tau b is a refinement of gamma for ordinal data that considers “tied” pairs for square tables. Kendall’s tau c is a refinement of gamma for ordinal data that considers “tied” pairs for any-size table. Somers’s d is a measure of association for ordinal data that compensates for “tied” ranks and adjusts for direction of the independent variable.
- Exhibit 18-23Exhibit 18-23
- Exhibit 18-24 Spearman’s rho correlates ranks been two ordinal variables. Spearman’s rho has many advantages. When data are transformed by logs or squaring, rho remains unaffected. Outliers or extreme scores are not a threat. It is easy to compute. In the example in the exhibit in the slide KDL, a media firm, is recruiting account executive trainees. Assume the field has been narrowed to 10 applicants for final evaluation. They go through a battery of tests and interviews. The results are evaluated by an industrial psychologist who then ranks the 10 candidates. The executives produce a composite ranking based on the interviews. Now we need to determine how well these two sets of rankings agree. Using the following equation, we can see that there is a strong relationship between the rankings and reject the null. The relationship between the panel’s and the psychologist’s rankings is moderately high, suggesting agreement between the two measures. The test of the null hypothesis that there is no relationship between the measures is rejected at the .05 level with n-2 degrees of freedom.