FACTOR ANALYSIS
Wei-Jiun, Shen Ph. D.
Purpose
 To research for and define the fundamental
constructs or dimensions assumed to underlie
the original variables
 Data summarization
 Define a small numbers of factors that maximize the
explanation of the entire variable set
 Data reduction
 Identifying representative variables from a much larger set
of variables
 Exploratory and confirmatory
Kinds of research questions
 Defining the underlying structure
 Number of factor
 Nature of factor
 Importance of solutions and factors
 Testing theory in factor analysis
 Estimating scores on factors
Limitations to factor analysis
 Theoretical issues
 Factors
 Selection of observed variable
 Practical issues
 Sample size and missing data
 Normality
 Linearity
 Outlier
 Multicollinearity and singularity
 Factorability of R
What is factor?
 Linear combination of the original variables
Factor
Principle
component
Common factor
Total variance Shared variance
Variance
Case V1 V2 V3 V4
1 5 5 1 2
2 3 2 5 4
3 1 1 2 1
4 5 4 3 2
5 2 3 2 1
6 4 5 1 2
7 2 1 5 4
8 3 4 2 3
9 2 1 4 5
Variance 25 22.89 19.56 16
Total variance = 25+22.89+19.56+16=83.45
PCA and (C)FA
 Principle component analysis (PCA)
 Common factor analysis (FA)
Total variance
Shard (common)
variance
Unique variance
Specific and error
Variance extracted Variance excluded
Factor analysis
1
1
N
P
N*P
1
1
N
Q
N*Q
1
1
N
P
N*P
F
Q=P
F<P
PCA
FA
5 5 3 1
3 2 5 5
4 3 5 4
3 2 5 3
Factor analysis
V1 V2 V3 V4 V5 V6 V7 V8 V9
F1 F2 F3
Fundamental equation for factor analysis
 General form
 Matrix
 X: observed variables
 μ: means
 Λ: factors loadings
 F: factors
 E: errors matrix
 R: correlations
 D: unique variance
(R − D) = AA′
X = 𝜇 + ΛF + E X = 𝜇 + ΛF
R = AA′
PCAE=0
D=0
Fundamental equation for factor analysis
 Eigen formulation
 λ: eigenvalue
 K: eigenvector
R − λI K = 0
Eigenvalue and communalities
 Eigen formulation R − λI K = 0
m
5 5 3 1
3 2 5 5
4 3 5 4
3 2 5 3
1
1
P
F =
λ1 λ2 λ3 λ 𝑚
𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 =
𝑖=1
𝑝
𝑓𝑖𝑗
2
= 𝜆𝑗
𝑐11
𝑐22
𝑐33
𝑐 𝑝𝑝
𝑐𝑜𝑚𝑚𝑢𝑛𝑎𝑙𝑖𝑡𝑦 =
𝑖=1
𝑚
𝑓𝑖𝑗
2
= 𝐶𝑖𝑖
𝑓𝑖𝑗
facto
rvariable
Axis rotations
 Orthogonal rotation
 Non-correlations among factors are assumed
 Method
 Quartimax
 Simplify the rows of a factor matrix (largest variance on factor)
 Varimax
 Simplify the columns of a factor matrix (largest variance on
variable)
 Equimax
 Compromise quartimax and varimax
 Oblique rotation
 Correlations among factors are assumed
Orthogonal rotation
Unrotated F2
Unrotated F1
Rotated F1
Rotated F2
-1.0 -.50 0 +.50 +1.0
-.50
-1.0
+1.0
+.50
V1
V2
V3
V4
V5
Oblique rotation
Unrotated F2
Unrotated F1
Orthogonal
Rotated F1
Orthogonal rotated F2
-1.0 -.50 0 +.50 +1.0
-.50
-1.0
+1.0
+.50
V1
V2
V3
V4
V5
Oblique-rotated F2
Oblique-rotated F1
Data summarization
 Number of factor
 Kaiser rule
 Eigenvalue > 1 (Kaiser, 1970)
 Scree test
 Factors before inflection point (Carttelli, 1966)
Data reduction
 Factor loadings
 Correlation between item and factor
 Orthogonal rotation
 Factor matrix
 Oblique rotation
 Factor pattern matrix
 Unique contribution of item to factor (partial correlation)
 Factor structure matrix
 Unique variance and correlation among factors (correlation)
Data reduction
 Significant factor loadings based on sample size
Factor loadings Required sample size
.30 350
.35 250
.40 100
.45 150
.50 120
.55 100
.60 85
.65 70
.70 60
.75 50
Data reduction
 Preservation
 Variable with significant factor loading
 Variable with communality > .50
 Deletion
 Cross loading
Practice issue
 Metric variables
 Sample size
 200 (Comrey, 1973)
 10 observations per variable (Kline, 2005)
 Some degree of multicollinearity
 Bartlett test of sphericity
 Measure of sampling adequacy (MSA)
 Kaiser-Meyer-Olkin measurement of sampling
adequacy (KMO test)
Sampling adequacy test
 Bartlett test of sphericity (p < .05)
 Are correlations among observations big enough?
 MSA (> .05) and KMO (> .07) test
 Are partial correlations among observations small
enough?
V1
V3 V4
Comparison
PCA FA
Purpose
Maximize
explanation of total
variance
Maximize explanation
of shared variance
Term Principle component Common factor
Meaning Linear combination Construct
Relationship
among factor
0 R
Diagonal
matrix
1 Communality
Comparison
 PCA and FA reveal almost identical results
 High ratio: variable/factor
 High common variance
 Different results (Widaman, 1993)
 Low ratio: variable/factor (3:1)
 Low common variance (<.4)
 Which one is preferred?
 Psychological concept
 Satisfaction on product/consumer behavior
Latent variable
 Psychological concept
 Behavioral pattern/sampling behavior/item
V1 V2 V3 V4 V5 V6 V7 V8 V9
F1 F2 F3
u u u u u u u u u
Categorization
 Product/object/…
V1 V2 V3 V4 V5 V6 V7 V8 V9
F1 F2 F3
Procedure
1. Research question
2. Select the type of factor analysis
3. Check the assumptions
4. Derive factor analysis and assess overall fit
5. Interpret the factors
6. Validation of factor analysis
7. Additional use of factor analysis results
ITEM ANALYSIS
After EFA
 Item analysis
 Appropriateness of item
 Item discrimination
 Homogeneity of item
 Indices
 Critical ratio (CR) t > 3
 Item total correlation >.3
 Cronbach’s α ↑ if item deleted
PRACTICE
運動員沮喪反應量表的效化檢驗
 根據相關學理,沮喪這項心理特性會反應在三種向
度上,分別是:
 1.認知性
 2.身體性
 3.人際關係
 於是研究生迪蒲睿宣依各向度分別設計了8、9、5共
22題的問卷,他不太確定這份問卷是否能反應出三
種沮喪的向度,請幫他進行探索性因素分析。

Factor analysis

  • 1.
  • 2.
    Purpose  To researchfor and define the fundamental constructs or dimensions assumed to underlie the original variables  Data summarization  Define a small numbers of factors that maximize the explanation of the entire variable set  Data reduction  Identifying representative variables from a much larger set of variables  Exploratory and confirmatory
  • 3.
    Kinds of researchquestions  Defining the underlying structure  Number of factor  Nature of factor  Importance of solutions and factors  Testing theory in factor analysis  Estimating scores on factors
  • 4.
    Limitations to factoranalysis  Theoretical issues  Factors  Selection of observed variable  Practical issues  Sample size and missing data  Normality  Linearity  Outlier  Multicollinearity and singularity  Factorability of R
  • 5.
    What is factor? Linear combination of the original variables Factor Principle component Common factor Total variance Shared variance
  • 6.
    Variance Case V1 V2V3 V4 1 5 5 1 2 2 3 2 5 4 3 1 1 2 1 4 5 4 3 2 5 2 3 2 1 6 4 5 1 2 7 2 1 5 4 8 3 4 2 3 9 2 1 4 5 Variance 25 22.89 19.56 16 Total variance = 25+22.89+19.56+16=83.45
  • 7.
    PCA and (C)FA Principle component analysis (PCA)  Common factor analysis (FA) Total variance Shard (common) variance Unique variance Specific and error Variance extracted Variance excluded
  • 8.
  • 9.
    Factor analysis V1 V2V3 V4 V5 V6 V7 V8 V9 F1 F2 F3
  • 10.
    Fundamental equation forfactor analysis  General form  Matrix  X: observed variables  μ: means  Λ: factors loadings  F: factors  E: errors matrix  R: correlations  D: unique variance (R − D) = AA′ X = 𝜇 + ΛF + E X = 𝜇 + ΛF R = AA′ PCAE=0 D=0
  • 11.
    Fundamental equation forfactor analysis  Eigen formulation  λ: eigenvalue  K: eigenvector R − λI K = 0
  • 12.
    Eigenvalue and communalities Eigen formulation R − λI K = 0 m 5 5 3 1 3 2 5 5 4 3 5 4 3 2 5 3 1 1 P F = λ1 λ2 λ3 λ 𝑚 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 = 𝑖=1 𝑝 𝑓𝑖𝑗 2 = 𝜆𝑗 𝑐11 𝑐22 𝑐33 𝑐 𝑝𝑝 𝑐𝑜𝑚𝑚𝑢𝑛𝑎𝑙𝑖𝑡𝑦 = 𝑖=1 𝑚 𝑓𝑖𝑗 2 = 𝐶𝑖𝑖 𝑓𝑖𝑗 facto rvariable
  • 13.
    Axis rotations  Orthogonalrotation  Non-correlations among factors are assumed  Method  Quartimax  Simplify the rows of a factor matrix (largest variance on factor)  Varimax  Simplify the columns of a factor matrix (largest variance on variable)  Equimax  Compromise quartimax and varimax  Oblique rotation  Correlations among factors are assumed
  • 14.
    Orthogonal rotation Unrotated F2 UnrotatedF1 Rotated F1 Rotated F2 -1.0 -.50 0 +.50 +1.0 -.50 -1.0 +1.0 +.50 V1 V2 V3 V4 V5
  • 15.
    Oblique rotation Unrotated F2 UnrotatedF1 Orthogonal Rotated F1 Orthogonal rotated F2 -1.0 -.50 0 +.50 +1.0 -.50 -1.0 +1.0 +.50 V1 V2 V3 V4 V5 Oblique-rotated F2 Oblique-rotated F1
  • 16.
    Data summarization  Numberof factor  Kaiser rule  Eigenvalue > 1 (Kaiser, 1970)  Scree test  Factors before inflection point (Carttelli, 1966)
  • 17.
    Data reduction  Factorloadings  Correlation between item and factor  Orthogonal rotation  Factor matrix  Oblique rotation  Factor pattern matrix  Unique contribution of item to factor (partial correlation)  Factor structure matrix  Unique variance and correlation among factors (correlation)
  • 18.
    Data reduction  Significantfactor loadings based on sample size Factor loadings Required sample size .30 350 .35 250 .40 100 .45 150 .50 120 .55 100 .60 85 .65 70 .70 60 .75 50
  • 19.
    Data reduction  Preservation Variable with significant factor loading  Variable with communality > .50  Deletion  Cross loading
  • 20.
    Practice issue  Metricvariables  Sample size  200 (Comrey, 1973)  10 observations per variable (Kline, 2005)  Some degree of multicollinearity  Bartlett test of sphericity  Measure of sampling adequacy (MSA)  Kaiser-Meyer-Olkin measurement of sampling adequacy (KMO test)
  • 21.
    Sampling adequacy test Bartlett test of sphericity (p < .05)  Are correlations among observations big enough?  MSA (> .05) and KMO (> .07) test  Are partial correlations among observations small enough? V1 V3 V4
  • 22.
    Comparison PCA FA Purpose Maximize explanation oftotal variance Maximize explanation of shared variance Term Principle component Common factor Meaning Linear combination Construct Relationship among factor 0 R Diagonal matrix 1 Communality
  • 23.
    Comparison  PCA andFA reveal almost identical results  High ratio: variable/factor  High common variance  Different results (Widaman, 1993)  Low ratio: variable/factor (3:1)  Low common variance (<.4)  Which one is preferred?  Psychological concept  Satisfaction on product/consumer behavior
  • 24.
    Latent variable  Psychologicalconcept  Behavioral pattern/sampling behavior/item V1 V2 V3 V4 V5 V6 V7 V8 V9 F1 F2 F3 u u u u u u u u u
  • 25.
    Categorization  Product/object/… V1 V2V3 V4 V5 V6 V7 V8 V9 F1 F2 F3
  • 26.
    Procedure 1. Research question 2.Select the type of factor analysis 3. Check the assumptions 4. Derive factor analysis and assess overall fit 5. Interpret the factors 6. Validation of factor analysis 7. Additional use of factor analysis results
  • 27.
  • 28.
    After EFA  Itemanalysis  Appropriateness of item  Item discrimination  Homogeneity of item  Indices  Critical ratio (CR) t > 3  Item total correlation >.3  Cronbach’s α ↑ if item deleted
  • 29.
  • 30.
    運動員沮喪反應量表的效化檢驗  根據相關學理,沮喪這項心理特性會反應在三種向 度上,分別是:  1.認知性 2.身體性  3.人際關係  於是研究生迪蒲睿宣依各向度分別設計了8、9、5共 22題的問卷,他不太確定這份問卷是否能反應出三 種沮喪的向度,請幫他進行探索性因素分析。