Cairo, May 2015 Irina Vartanova (Institute for Futures Studies) Training on "Measurements And Analysis of Opinion Poll Data"

1. Factor Analysis:
Application using WVS data from selected Arab countries
Irina Vartanova
Institute for Futures Studies, Stockholm
ERF Workshop – May 10, 2015
1 / 28

2. Exploitative Factor Analysis: Understanding of Democracy
• V136. Civil rights protect people from state oppression.
• V133. People choose their leaders in free elections.
• V139 Women have the same rights as men.
• V132. Religious authorities ultimately interpret the laws.
• V135. The army takes over when government is incompetent.
• V138. People obey their rulers.
• V137. The state makes people’s incomes equal.
• V131. Governments tax the rich and subsidize the poor.
• V134. People receive state aid for unemployment.
2 / 28

3. Motivation
• Measure the underlying concept with smaller measurement
error.
• Dimensionality reduction to reduce the effect of
multi-collinearity.
3 / 28

4. Data
• WVS, 6th wave
• 57 countries (Bahrain excluded)
• Pool sample: 82289 cases
4 / 28

5. Understanding of democracy: the World
V139
V133
V136
V134
V131
V137
V138
V135
V132
V132 V135 V138 V137 V131 V134 V136 V133 V139
0.00
0.25
0.50
0.75
1.00
value
5 / 28

6. Understanding of democracy: the MENA countries
V139
V136
V137
V133
V134
V138
V131
V135
V132
V132 V135 V131 V138 V134 V133 V137 V136 V139
0.25
0.50
0.75
1.00
value
6 / 28

7. Understanding of democracy: the Western Countries
V137
V131
V134
V136
V139
V133
V135
V132
V138
V138 V132 V135 V133 V139 V136 V134 V131 V137
−0.25
0.00
0.25
0.50
0.75
1.00
value
7 / 28

8. Number of Factors: Scree plot
q
q
q
q
q
q q
q q
0.0
0.5
1.0
1.5
2.0
Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 Comp.6 Comp.7 Comp.8 Comp.9
x
Eigenvalues
8 / 28

9. Full Sample Results
Loadings:
Factor1 Factor2
V136 0.69
V133 0.72
V139 0.64
V132 0.71
V135 0.61
V138 0.44
V137 0.42
V131 0.32
V134 0.55
Factor1 Factor2
SS loadings 1.96 1.41
Proportion Var 0.22 0.16
Cumulative Var 0.22 0.37
Factor Correlations:
Factor1 Factor2
Factor1 1.00 -0.36
Factor2 -0.36 1.00
9 / 28

10. MENA Region Results
Loadings:
Factor1 Factor2
V136 0.73
V133 0.56
V139 0.57
V132 0.88
V135
V138
V137 0.49
V131
V134 0.51
Factor1 Factor2
SS loadings 1.87 0.98
Proportion Var 0.21 0.11
Cumulative Var 0.21 0.32
Factor Correlations:
Factor1 Factor2
Factor1 1.00 -0.55
Factor2 -0.55 1.00
10 / 28

11. Why economists do not like survey data
1
We can address the country-item-bias with measurement
invariance concept.
1
The figure is reproduced from: Stegmueller, D. (2011). Apples and
oranges? The problem of equivalence in comparative research. Political Analysis
11 / 28

12. Measuring Latent Variable:
Attitudes towards Gender Equality example
• V45. When jobs are scarce, men should have more right to a
job than women.
• V51. On the whole, men make better political leaders than
women do.
• V52. A university education is more important for a boy than
for a girl.
12 / 28

13. Data
• WVS, 6th wave
• 12 MENA countries: Algeria, Egypt, Iraq, Jordan, Kuwait,
Lebanon, Libya, Morocco, Palestine, Qatar, Yemen
• Bahrain excluded
• Pool sample: 13260 after listwize deletion of missing cases
13 / 28

14. Confirmatory Factor Analysis
GendEqAtt y2
y1
y3
e2
e1
e3
y2 = v2 + λ2GendEqAtt + e2
y1 = v1 + 1GendEqAtt + e1
y3 = v3 + λ3GendEqAtt + e3
14 / 28

15. CFA results: pooled sample
GendEqAtt y2
y1
y3
1.00
1.14
0.66
15 / 28

16. Degrees of Freedom and the Model Identification
• The power or CFA approach that it has clear criteria of model
fit. The estimated variance-covariance matrix is compared
with the one implied by the theoretical model. How big are
discrepancies is tested with the Pearson χ2.
• However, we need degrees of freedom to test the model fit,
meaning that the number of known parameters should be
larger than the number of parameters to be estimated.
• One factor with 3 indicators is just identified model and does
not provide us with fit indices.
• We can obtain additional degrees of freedom by fixing some
parameters.
16 / 28

17. CFA results: pooled sample with the second item loading
fixed to 1
GendEqAtt y2
y1
y3
1.00
1.00
0.62
Fit: χ2 = 10.160; df = 1; P-value = 0,001;
CFI = 0.998; RMSEA = 0.026; P-value RMSEA ≤ 0.05 = 0.995
17 / 28

18. Measurement Invariance
• To establish the cross-country comparability, we use the
concept of measurement invariance / equivalence.
• 3 level of invariance - structural, metric and scalar - provide
different levels of available model comparisons.
18 / 28

19. Structural invariance
GendEqAtt y2
y1
y3
y2g = v2g + λ2g GendEqAtt + e2g
y1g = v1g + 1 ∗ GendEqAtt + e1g
y3g = v3g + λ3g GendEqAtt + e3g
• The structural equivalence implies that used items measure
the same concept in all comparing countries.
19 / 28

20. Metric Invariance
GendEqAtt y2
y1
y3
y2g = v2g + λ2GendEqAtt + e2g
y1g = v1g + 1 ∗ GendEqAtt + e1g
y3g = v3g + λ3GendEqAtt + e3g
• The metric equivalence implies that the scale of the latent
latent variable has the same metric in groups, that is change
in one unit on the scale has the same meaning in different
groups.
• It is achieved by equality of item loadings across all the groups
20 / 28

21. Scalar Invariance
GendEqAtt y2
y1
y3
y2 = v2 + λ2GendEqAtt + e2g
y1 = v1 + 18GendEqAtt + e1g
y3 = v3 + λ3GendEqAtt + e3g
• Scalar equivalence determines correspondence of the absolute
value at the scale in different groups.
• It is justified by equal items intercepts.
21 / 28

22. Partial Invariance
GendEqAtt y2
y1
y3
y2 = v2 + λ2GendEqAtt + e2g
y1 = v1 + 1 ∗ GendEqAtt + e1g
y3g = v3 + λ3g GendEqAtt + e3g
• Partial invariance can be confirmed he at leas two items are
invariant.
22 / 28

23. Gender Equality: Test for Invariance
χ2
df p-value CFI RMSEA p-RMSEA
Structural 0 0 1 0
Metric 147.576 22 0 0.974 0.061 0.046
Partial in Morocco 97.745 21 0 0.984 0.047 0.648
Scalar 737.365 66 0 0.861 0.077 0.000
23 / 28

24. Model comparisons
• χ2 difference test, but it is sensitive to large sample sizes.
• Differences in their overall goodness-of-fit indices (Chen,
2007).
24 / 28

25. Existing alternatives
• Multilevel modelling approach (Stegmueller, 2011)
• ”EPC-interest” (Oberski, 2014)
• Alignment method (Muthen & Asparouhov, 2014)
25 / 28

26. Multilevel modelling approach
2
• Non-invariance or systematic country-item-bias can be
captured by allowing the country level latent variable ν
(3)
k to
directly affect the items’ variance
2
The figure is reproduced from: Stegmueller, D. (2011). Apples and
oranges? The problem of equivalence in comparative research. Political Analysis
26 / 28

27. EPC-interest approach
• Useful when the relationship between the latent variable and
some covariate vector ”structuralmodel” is of interest.
• Measurement invariance is usually tested by various fit
measures. However, violations not necessarily seriously affect
the fit measures, and substantial bias in the parameter of
interest may still remain.
• Expected Parameter Change approach accounts for the effect
that violations of measurement invariance assumption have on
the parameters of interest
• EPC-interest assesses impact on the parameter of interest if a
particular possible direct effect was freed.
27 / 28

28. Alignment method approach
• To define the metric system of a latent variable and make the
identifiable one of the two options is used: one of the item
loadings is fixed 1 or variance of the latent variable is fixed to
1. Other parameters are estimated base on the
variance-covariance matrix.
• The alignment approach can estimate the parameters by
incorporating the assumption that the number of non-invariant
measurement parameters can be held to minimum.
• For clarification, the authors relate to the analogy of using
rotation in EFA.
28 / 28