The document discusses factor analysis as an exploratory and confirmatory multivariate technique. It explains that factor analysis is commonly used for data reduction, scale development, and evaluating the dimensionality of variables. Factor analysis determines underlying factors or dimensions from a set of interrelated variables. It reduces a large number of variables to a smaller number of factors. The key steps in factor analysis include computing a correlation matrix, extracting factors, rotating factors, and making decisions on the number of factors.

1. Factor Analysis SPSS for Windows ® Intermediate & Advanced Applied Statistics Zayed University Office of Research SPSS for Windows ® Workshop Series Presented by Dr. Maher Khelifa Associate Professor Department of Humanities and Social Sciences College of Arts and Sciences © Dr. Maher Khelifa

2. Understanding Factor Analysis This workshop discusses factor analysis as an exploratory and confirmatory multivariate technique. © Dr. Maher Khelifa

3. Understanding Factor Analysis Factor analysis is commonly used in: Data reduction Scale development The evaluation of the psychometric quality of a measure, and The assessment of the dimensionality of a set of variables. © Dr. Maher Khelifa

4. Understanding Factor Analysis Regardless of purpose, factor analysis is used in: the determination of a small number of factors based on a particular number of inter-related quantitative variables. Unlike variables directly measured such as speed, height, weight, etc., some variables such as egoism, creativity, happiness, religiosity, comfort are not a single measurable entity . They are constructs that are derived from the measurement of other, directly observable variables . © Dr. Maher Khelifa

5. Understanding Factor Analysis Constructs are usually defined as unobservable latent variables. E.g.: motivation/love/hate/care/altruism/anxiety/worry/stress/product quality/physical aptitude/democracy /reliability/power. Example: the construct of teaching effectiveness. Several variables are used to allow the measurement of such construct (usually several scale items are used) because the construct may include several dimensions . Factor analysis measures not directly observable constructs by measuring several of its underlying dimensions. The identification of such underlying dimensions (factors) simplifies the understanding and description of complex constructs. © Dr. Maher Khelifa

6. Understanding Factor Analysis Generally, the number of factors is much smaller than the number of measures. Therefore, the expectation is that a factor represents a set of measures. From this angle, factor analysis is viewed as a data-reduction technique as it reduces a large number of overlapping variables to a smaller set of factors that reflect construct(s) or different dimensions of contruct(s). © Dr. Maher Khelifa

7. Understanding Factor Analysis The assumption of factor analysis is that underlying dimensions (factors) can be used to explain complex phenomena. Observed correlations between variables result from their sharing of factors. Example: Correlations between a person’s test scores might be linked to shared factors such as general intelligence, critical thinking and reasoning skills, reading comprehension etc. © Dr. Maher Khelifa

8. Ingredients of a Good Factor Analysis Solution A major goal of factor analysis is to represent relationships among sets of variables parsimoniously yet keeping factors meaningful. A good factor solution is both simple and interpretable. When factors can be interpreted, new insights are possible. © Dr. Maher Khelifa

9. Application of Factor Analysis This workshop will examine three common applications of factor analysis: Defining indicators of constructs Defining dimensions for an existing measure Selecting items or scales to be included in a measure. © Dr. Maher Khelifa

10. Application of Factor Analysis Defining indicators of constructs: Ideally 4 or more measures should be chosen to represent each construct of interest. The choice of measures should, as much as possible, be guided by theory, previous research, and logic. © Dr. Maher Khelifa

11. Application of Factor Analysis Defining dimensions for an existing measure: In this case the variables to be analyzed are chosen by the initial researcher and not the person conducting the analysis. Factor analysis is performed on a predetermined set of items/scales. Results of factor analysis may not always be satisfactory: The items or scales may be poor indicators of the construct or constructs. There may be too few items or scales to represent each underlying dimension. © Dr. Maher Khelifa

12. Application of Factor Analysis Selecting items or scales to be included in a measure. Factor analysis may be conducted to determine what items or scales should be included and excluded from a measure. Results of the analysis should not be used alone in making decisions of inclusions or exclusions. Decisions should be taken in conjunction with the theory and what is known about the construct(s) that the items or scales assess. © Dr. Maher Khelifa

13. Steps in Factor Analysis Factor analysis usually proceeds in four steps: 1 st Step : the correlation matrix for all variables is computed 2 nd Step : Factor extraction 3 rd Step : Factor rotation 4 th Step : Make final decisions about the number of underlying factors © Dr. Maher Khelifa

14. Steps in Factor Analysis: The Correlation Matrix 1 st Step : the correlation matrix Generate a correlation matrix for all variables Identify variables not related to other variables If the correlation between variables are small, it is unlikely that they share common factors (variables must be related to each other for the factor model to be appropriate). Think of correlations in absolute value. Correlation coefficients greater than 0.3 in absolute value are indicative of acceptable correlations. Examine visually the appropriateness of the factor model. © Dr. Maher Khelifa

15. Steps in Factor Analysis: The Correlation Matrix Bartlett Test of Sphericity: used to test the hypothesis the correlation matrix is an identity matrix (all diagonal terms are 1 and all off-diagonal terms are 0). If the value of the test statistic for sphericity is large and the associated significance level is small , it is unlikely that the population correlation matrix is an identity. If the hypothesis that the population correlation matrix is an identity cannot be rejected because the observed significance level is large, the use of the factor model should be reconsidered. © Dr. Maher Khelifa

16. Steps in Factor Analysis: The Correlation Matrix The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy: is an index for comparing the magnitude of the observed correlation coefficients to the magnitude of the partial correlation coefficients. The closer the KMO measure to 1 indicate a sizeable sampling adequacy (.8 and higher are great, .7 is acceptable, .6 is mediocre, less than .5 is unaccaptable ). Reasonably large values are needed for a good factor analysis. Small KMO values indicate that a factor analysis of the variables may not be a good idea. © Dr. Maher Khelifa

17. Steps in Factor Analysis: Factor Extraction 2 nd Step : Factor extraction The primary objective of this stage is to determine the factors. Initial decisions can be made here about the number of factors underlying a set of measured variables. Estimates of initial factors are obtained using Principal components analysis . The principal components analysis is the most commonly used extraction method . Other factor extraction methods include: Maximum likelihood method Principal axis factoring Alpha method Unweighted lease squares method Generalized least square method Image factoring. © Dr. Maher Khelifa

18. Steps in Factor Analysis: Factor Extraction In principal components analysis, linear combinations of the observed variables are formed. The 1 st principal component is the combination that accounts for the largest amount of variance in the sample (1 st extracted factor). The 2 nd principle component accounts for the next largest amount of variance and is uncorrelated with the first (2 nd extracted factor). Successive components explain progressively smaller portions of the total sample variance, and all are uncorrelated with each other. © Dr. Maher Khelifa

19. Steps in Factor Analysis: Factor Extraction To decide on how many factors we need to represent the data, we use 2 statistical criteria: Eigen Values, and The Scree Plot . The determination of the number of factors is usually done by considering only factors with Eigen values greater than 1. Factors with a variance less than 1 are no better than a single variable, since each variable is expected to have a variance of 1. © Dr. Maher Khelifa Total Variance Explained Component Initial Eigenvalues Extraction Sums of Squared Loadings Total % of Variance Cumulative % Total % of Variance Cumulative % 1 3.046 30.465 30.465 3.046 30.465 30.465 2 1.801 18.011 48.476 1.801 18.011 48.476 3 1.009 10.091 58.566 1.009 10.091 58.566 4 .934 9.336 67.902 5 .840 8.404 76.307 6 .711 7.107 83.414 7 .574 5.737 89.151 8 .440 4.396 93.547 9 .337 3.368 96.915 10 .308 3.085 100.000 Extraction Method: Principal Component Analysis.

20. Steps in Factor Analysis: Factor Extraction The examination of the Scree plot provides a visual of the total variance associated with each factor. The steep slope shows the large factors. The gradual trailing off (scree) shows the rest of the factors usually lower than an Eigen value of 1. In choosing the number of factors, in addition to the statistical criteria, one should make initial decisions based on conceptual and theoretical grounds. At this stage, the decision about the number of factors is not final. © Dr. Maher Khelifa

21. Steps in Factor Analysis: Factor Extraction © Dr. Maher Khelifa Component Matrix using Principle Component Analysis Component Matrix a Component 1 2 3 I discussed my frustrations and feelings with person(s) in school .771 -.271 .121 I tried to develop a step-by-step plan of action to remedy the problems .545 .530 .264 I expressed my emotions to my family and close friends .580 -.311 .265 I read, attended workshops, or sought someother educational approach to correct the problem .398 .356 -.374 I tried to be emotionally honest with my self about the problems .436 .441 -.368 I sought advice from others on how I should solve the problems .705 -.362 .117 I explored the emotions caused by the problems .594 .184 -.537 I took direct action to try to correct the problems .074 .640 .443 I told someone I could trust about how I felt about the problems .752 -.351 .081 I put aside other activities so that I could work to solve the problems .225 .576 .272 Extraction Method: Principal Component Analysis. a. 3 components extracted.

22. Steps in Factor Analysis: Factor Rotation 3 rd Step: Factor rotation. In this step, factors are rotated. Un-rotated factors are typically not very interpretable (most factors are correlated with may variables). Factors are rotated to make them more meaningful and easier to interpret (each variable is associated with a minimal number of factors). Different rotation methods may result in the identification of somewhat different factors. © Dr. Maher Khelifa

23. Steps in Factor Analysis: Factor Rotation The most popular rotational method is Varimax rotations. Varimax use orthogonal rotations yielding uncorrelated factors/components. Varimax attempts to minimize the number of variables that have high loadings on a factor. This enhances the interpretability of the factors. © Dr. Maher Khelifa

24. Steps in Factor Analysis: Factor Rotation Other common rotational method used include Oblique rotations which yield correlated factors. Oblique rotations are less frequently used because their results are more difficult to summarize. Other rotational methods include: Quartimax (Orthogonal) Equamax (Orthogonal) Promax (oblique) © Dr. Maher Khelifa

25. Steps in Factor Analysis: Factor Rotation A factor is interpreted or named by examining the largest values linking the factor to the measured variables in the rotated factor matrix . © Dr. Maher Khelifa Rotated Component Matrix a Component 1 2 3 I discussed my frustrations and feelings with person(s) in school .803 .186 .050 I tried to develop a step-by-step plan of action to remedy the problems .270 .304 .694 I expressed my emotions to my family and close friends .706 -.036 .059 I read, attended workshops, or sought someother educational approach to correct the problem .050 .633 .145 I tried to be emotionally honest with my self about the problems .042 .685 .222 I sought advice from others on how I should solve the problems .792 .117 -.038 I explored the emotions caused by the problems .248 .782 -.037 I took direct action to try to correct the problems -.120 -.023 .772 I told someone I could trust about how I felt about the problems .815 .172 -.040 I put aside other activities so that I could work to solve the problems -.014 .155 .657 Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 5 iterations.

26. Steps in Factor Analysis: Making Final Decisions 4 th Step : Making final decisions The final decision about the number of factors to choose is the number of factors for the rotated solution that is most interpretable . To identify factors, group variables that have large loadings for the same factor. Plots of loadings provide a visual for variable clusters. Interpret factors according to the meaning of the variables This decision should be guided by: A priori conceptual beliefs about the number of factors from past research or theory Eigen values computed in step 2. The relative interpretability of rotated solutions computed in step 3. © Dr. Maher Khelifa

27. Assumptions Underlying Factor Analysis Assumption underlying factor analysis include. The measured variables are linearly related to the factors + errors. This assumption is likely to be violated if items limited response scales (two-point response scale like True/False, Right/Wrong items). The data should have a bi-variate normal distribution for each pair of variables. Observations are independent. The factor analysis model assumes that variables are determined by common factors and unique factors. All unique factors are assumed to be uncorrelated with each other and with the common factors. © Dr. Maher Khelifa

28. Obtaining a Factor Analysis Click: Analyze and select Dimension Reduction Factor A factor Analysis Box will appear © Dr. Maher Khelifa

29. Obtaining a Factor Analysis Move variables/scale items to Variable box © Dr. Maher Khelifa

30. Obtaining a Factor Analysis Factor extraction When variables are in variable box, select: Extraction © Dr. Maher Khelifa

31. Obtaining a Factor Analysis When the factor extraction Box appears, select: Scree Plot keep all default selections including: Principle component Analysis Based on Eigen Value of 1, and Un-rotated factor solution © Dr. Maher Khelifa

32. Obtaining a Factor Analysis During factor extraction keep factor rotation default of: None Press continue © Dr. Maher Khelifa

33. Obtaining a Factor Analysis During Factor Rotation: Decide on the number of factors based on actor extraction phase and enter the desired number of factors by choosing: Fixed number of factors and entering the desired number of factors to extract. Under Rotation Choose Varimax Press continue Then OK © Dr. Maher Khelifa

34. Bibliographical References Almar, E.C. (2000). Statistical Tricks and traps. Los Angeles, CA: Pyrczak Publishing. Bluman, A.G. (2008). Elemtary Statistics (6 th Ed.). New York, NY: McGraw Hill. Chatterjee, S., Hadi, A., & Price, B. (2000) Regression analysis by example. New York: Wiley. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2 nd Ed.). Hillsdale, NJ.: Lawrence Erlbaum. Darlington, R.B. (1990). Regression and linear models. New York: McGraw-Hill. Einspruch, E.L. (2005). An introductory Guide to SPSS for Windows (2 nd Ed.). Thousand Oak, CA: Sage Publications. Fox, J. (1997) Applied regression analysis, linear models, and related methods. Thousand Oaks, CA: Sage Publications. Glassnapp, D. R. (1984). Change scores and regression suppressor conditions. Educational and Psychological Measurement (44), 851-867. Glassnapp. D. R., & Poggio, J. (1985). Essentials of Statistical Analysis for the Behavioral Sciences. Columbus, OH: Charles E. Merril Publishing. Grimm, L.G., & Yarnold, P.R. (2000). Reading and understanding Multivariate statistics. Washington DC: American Psychological Association. Hamilton, L.C. (1992) Regression with graphics. Belmont, CA: Wadsworth. Hochberg, Y., & Tamhane, A.C. (1987). Multiple Comparisons Procedures. New York: John Wiley. Jaeger, R. M. Statistics: A spectator sport (2 nd Ed.). Newbury Park, London: Sage Publications. © Dr. Maher Khelifa

35. Bibliographical References Keppel, G. (1991). Design and Analysis: A researcher’s handbook (3 rd Ed.). Englwood Cliffs, NJ: Prentice Hall. Maracuilo, L.A., & Serlin, R.C. (1988). Statistical methods for the social and behavioral sciences. New York: Freeman and Company. Maxwell, S.E., & Delaney, H.D. (2000). Designing experiments and analyzing data: Amodel comparison perspective. Mahwah, NJ. : Lawrence Erlbaum. Norusis, J. M. (1993). SPSS for Windows Base System User’s Guide. Release 6.0. Chicago, IL: SPSS Inc. Norusis, J. M. (1993). SPSS for Windows Advanced Statistics. Release 6.0. Chicago, IL: SPSS Inc. Norusis, J. M. (2006). SPSS Statistics 15.0 Guide to Data Analysis . Upper Saddle River, NJ.: Prentice Hall. Norusis, J. M. (2008). SPSS Statistics 17.0 Guide to Data Analysis . Upper Saddle River, NJ.: Prentice Hall. Norusis, J. M. (2008). SPSS Statistics 17.0 Statistical Procedures Companion . Upper Saddle River, NJ.: Prentice Hall. Norusis, J. M. (2008). SPSS Statistics 17.0 Advanced Statistical Procedures Companion . Upper Saddle River, NJ.: Prentice Hall. Pedhazur, E.J. (1997). Multiple regression in behavioral research, third edition. New York: Harcourt Brace College Publishers. © Dr. Maher Khelifa

36. Bibliographical References SPSS Base 7.0 Application Guide (1996). Chicago, IL: SPSS Inc. SPSS Base 7.5 For Windows User’s Guide (1996). Chicago, IL: SPSS Inc. SPSS Base 8.0 Application Guide (1998). Chicago, IL: SPSS Inc. SPSS Base 8.0 Syntax Reference Guide (1998). Chicago, IL: SPSS Inc. SPSS Base 9.0 User’s Guide (1999). Chicago, IL: SPSS Inc. SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc. SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc. SPSS Interactive graphics (1999). Chicago, IL: SPSS Inc. SPSS Regression Models 11.0 (2001). Chicago, IL: SPSS Inc. SPSS Advanced Models 11.5 (2002) Chicago, IL: SPSS Inc. SPSS Base 11.5 User’s Guide (2002). Chicago, IL: SPSS Inc. SPSS Base 12.0 User’s Guide (2003). Chicago, IL: SPSS Inc. SPSS 13.0 Base User’s Guide (2004). Chicago, IL: SPSS Inc. SPSS Base 14.0 User’s Guide (2005). Chicago, IL: SPSS Inc.. SPSS Base 15.0 User’s Guide (2007). Chicago, IL: SPSS Inc. SPSS Base 16.0 User’s Guide (2007). Chicago, IL: SPSS Inc. SPSS Statistics Base 17.0 User’s Guide (2007). Chicago, IL: SPSS Inc. Tabachnik, B.G., & Fidell, L.S. (2001). Using multivariate statistics (4 th Ed). Boston, MA: Allyn and Bacon. © Dr. Maher Khelifa