CANONICAL ANALYSIS
Wei-Jiun, Shen Ph. D.
Purpose
 To analyze the relationship between 2 sets of
variables
 Multiple IVs
 Multiple DVs
Kinds of research questions
 It is considered a descriptive technique or a
screening procedure rather than hypothesis-
testing procedure
 Number of canonical variate pairs
 interpretation of canonical variates
 Importance of canonical variates
 Canonical variate scores
Limitations to factor analysis
 Theoretical issues
 Interpretability
 Linear relationship
 Sensitivity
 Causality
 Practical issues
 Ratio of cases to IVs 10:1
 Normality, linearity and homoscedasticity (not required)
 Missing data
 Absence of outliers
 Absence of multicollinearity and singularity
Fundamental equation for canonical analysis
 Multiple regression
 When Y is more than one…
ipipiii xxxy   2211
R
piiii xxxy  21
piiipiii xxxyyy   2121
Fundamental equation for canonical analysis
 Step 1: division of R
R = R 𝑦𝑦
−1
R 𝑦𝑥R 𝑥𝑥
−1
R 𝑥𝑦
Id TS TC BS BC
1 1.0 1.0 1.0 1.0
2 7.0 1.0 7.0 1.0
3 4.6 5.6 7.0 7.0
4 1.0 6.6 1.0 5.9
5 7.0 4.9 7.0 2.9
6 7.0 7.0 6.4 3.8
7 7.0 1.0 7.0 1.0
8 7.0 1.0 2.4 1.0
TS TC BS BC
TS 1.00
0
-.161 .758 -.341
TC -.161 1.00
0
.110 .857
BS .758 .110 1.00 .051
R 𝑥𝑥
R 𝑦𝑥
R 𝑥𝑦
R 𝑦𝑦
Fundamental equation for canonical analysis
 Step 2: eigenvalue and eigenvector
R = R 𝑦𝑦
−1
R 𝑦𝑥R 𝑥𝑥
−1
R 𝑥𝑦
R − λI K = 0
R 𝑦𝑦
−1R 𝑦𝑥R 𝑥𝑥
−1R 𝑥𝑦 − 𝑟𝑐𝑖
2
I K 𝑞 = 0
𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 = Λ =
𝑟𝑐1
2
⋯ ⋯
⋮ ⋱ ⋮
⋮ ⋯ 𝑟𝑐𝑖
2
𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟 = K = 𝑘1 ⋯ 𝑘 𝑞
Do you smell
something?
Fundamental equation for canonical analysis
 Step 1: division of R
1
1
N
P
N*P
X
1
1
N
Q
N*Q
Y
1
1
N
Q
N*P
X Y
P
1
1
Q
Q
(P+Q)*(P+Q)
P
P
R 𝑥𝑥
R 𝑦𝑥
R 𝑥𝑦
R 𝑦𝑦
Fundamental equation for canonical analysis
 Step 2: eigenvalue and eigenvector
1
N
N*n
Y
1 2 3 n…
1
N
N*m
X
1 2 3 m…
𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 = Λ
…
…
𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟 = K
Fundamental equation for canonical analysis
χ1
χ2
χ3
χ4
X1
X2
X3
X4
X5
η1
η2
η3
η4
Y1
Y2
Y3
Y4
𝑟𝑐1
𝑟𝑐2
𝑟𝑐3
𝑟𝑐4
0 0
Canonical
variate χ
Canonical
variate η
Canonical correlation
Number of set of canonical correlation
𝜒2 = − 𝑁 − 1 −
𝑘 𝑥 + 𝑘 𝑦 + 1
2
lnΛ 𝑚
Λ 𝑚 =
1
𝑚
1 − λ𝑖
F-test
 Wilk’s lambda
 Pillai’s trace
 Hotelling’s trace
 Roy’s gcr
Canonical weight
 Beta in regression
 Partialed out due to multicollinearity
 Instability
χn
X1
X2
X3
X4
X5
ηn
Y1
Y2
Y3
Y4
𝑟𝑐𝑛
λ 𝑤𝑥𝑛1
λ 𝑤𝑥𝑛2
λ 𝑤𝑥𝑛3
λ 𝑤𝑥𝑛4
λ 𝑤𝑥𝑛5
λ 𝑤𝑦𝑛1
λ 𝑤𝑦𝑛2
λ 𝑤𝑦𝑛3
λ 𝑤𝑦𝑛4
χ 𝑛 =
1
𝑖
𝑋𝑖 × λ 𝑤𝑥𝑛𝑖 η 𝑛 =
1
𝑖
𝑌𝑖 × λ 𝑤𝑦𝑛𝑖
Canonical loading
 Structure factor loading in FA
 Criterion: >.3
χn
X1
X2
X3
X4
X5
ηn
Y1
Y2
Y3
Y4
𝑟𝑐𝑛
λ 𝑥𝑛1
λ 𝑥𝑛2
λ 𝑥𝑛3
λ 𝑥𝑛4
λ 𝑥𝑛5
λ 𝑦𝑛1
λ 𝑦𝑛2
λ 𝑦𝑛3
λ 𝑦𝑛4
Canonical cross-loading
 Correlations of each variable and other canonical
variate
λ 𝑥𝑛𝑖:𝑦 = 𝑟𝑐𝑛 × λ 𝑥𝑛𝑖
λ 𝑦𝑛𝑖:𝑥 = 𝑟𝑐𝑛 × λ 𝑦𝑛𝑖
χn
X1
X2
X3
X4
X5
ηn
Y1
Y2
Y3
Y4
𝑟𝑐𝑛
λ 𝑥𝑛1
λ 𝑥𝑛2
λ 𝑥𝑛3
λ 𝑥𝑛4
λ 𝑥𝑛5
λ 𝑦𝑛1
λ 𝑦𝑛2
λ 𝑦𝑛3
λ 𝑦𝑛4
𝑟𝑐𝑛 × λ 𝑥𝑛1
𝑟𝑐𝑛 × λ 𝑥𝑛2
𝑟𝑐𝑛 × λ 𝑥𝑛3
𝑟𝑐𝑛 × λ 𝑥𝑛4
𝑟𝑐𝑛 × λ 𝑥𝑛5
𝑟𝑐𝑛 × λ 𝑦𝑛1
𝑟𝑐𝑛 × λ 𝑦𝑛2
𝑟𝑐𝑛 × λ 𝑦𝑛3
𝑟𝑐𝑛 × λ 𝑦𝑛4
Which interpretation approach to use
 Priority (Hair et al., 2010)
1. Canonical cross-loading
2. Canonical loading
3. Canonical weight
Redundancy (index)
 Variance the canonical variates from the IVs and
extract from the DVs, and vice versa
𝑝𝑣 𝑥𝑐 =
1
𝑖
λ 𝑥𝑛𝑖
2
𝑖
𝑝𝑣 𝑦𝑐 =
1
𝑖
λ 𝑦𝑛𝑖
2
𝑖
𝑟𝑑 = 𝑝𝑣 × 𝑟𝑐𝑛
2
Adequac
y
coefficien
t
Redundanc
y
index
Redundancy (index)
 Variance the canonical variates from the IVs and
extract from the DVs, and vice versa
χn
X1
X2
X3
X4
X5
ηn
Y1
Y2
Y3
Y4
𝑟𝑐𝑛
λ 𝑥𝑛1
λ 𝑥𝑛2
λ 𝑥𝑛3
λ 𝑥𝑛4
λ 𝑥𝑛5
λ 𝑦𝑛1
λ 𝑦𝑛2
λ 𝑦𝑛3
λ 𝑦𝑛4
𝑝𝑣 𝑥𝑐 𝑝𝑣 𝑦𝑐
𝑟𝑑η𝑛→X = 𝑝𝑣 𝑥𝑐 × 𝑟𝑐𝑛
2 𝑟𝑑χ𝑛→Y = 𝑝𝑣 𝑦𝑐 × 𝑟𝑐𝑛
2
Some important issue
 Importance of canonical variates
 Test for the significance
 Canonical correlation >.3
 Variate and its own variables
 Redundancy
 Interpretation of canonical variates
 Mathematical resolution of combining variables
 Loading >.3
Procedure
1. Research question
2. Designing a canonical analysis
3. Check the assumptions
4. Derive canonical analysis and assess overall fit
5. Interpret the canonical variate
6. Validation and diagnosis
PRACTICE
過去學業表現與現在學業表現
 研究生焦育布想瞭解過去學業表現與現在學業表
現之間的關係。他的研究問題是,大學生在高中
時期的學業表現是否與現階段的學業表現有關?
其中,高中學業表現包含國文、英文、三角函數
與線性代數等四個科目的評量分數,大學階段的
學業表現指標則包含國文、外語、微積分與統計
的評量分數。請以典型相關分析解答此問題。
Canonical correlation
χ1
HS_LAN
HS_ENG
HS_TRI
HS_LIA
η1
𝑟𝑐1=.994
-.99
-.99
-.61
-.30
CO_LAN
CO_ENG
CO_CAL
CO_STA
-.94
-.98
-.13
.15
χ1
HS_LAN
HS_ENG
HS_TRI
HS_LIA
η1
𝑟𝑐2=.965
-.01
-.06
.75
.65
CO_LAN
CO_ENG
CO_CAL
CO_STA
-.27
-.17
.73
.77
𝑟𝑑χ𝑛→Y = .58
𝑟𝑑χ𝑛→Y = .29
𝑟𝑑η𝑛→X = .60
𝑟𝑑η𝑛→X = .23
身體活動與智能
 研究生游志繪依想瞭解身體活動型態對於智力的
影響。他的研究問題是,青少年的身體活動與智
力之間是否有關?其中,身體活動包含坐式生活、
健走、中等強度以及高等強度活動量等四項指標,
智力的指標則包含語文、數學邏輯、空間、音樂、
肢體動覺、內省、人際與自然觀察的測驗表現。
請以典型相關分析解答此問題。
Canonical correlation
χ1
Strenuous
moderate
Walk
Sedentary
η1
Language
Math
Space
Music
𝑟𝑐1=.351
-.98
-.74
-.13
.15
Kinesthesis
Introspection
Interpersonal
Nature
science
-.43
-.06
.05
-.01
-.70
-.22
-.44
.01

Canonical analysis