Structural_equation_modeling_SEM_worksho (2).pptx

Structural equation modeling
(SEM) workshop
Dr Siddhi Pittayachawan
BEng, GCTTL, MEng, PhD, MACS CP
School of Business IT and Logistics
RMIT University, Australia
Presented at Dhurakij Pundit University on 20th Jan 2013

About me
• Academic qualifications
– 1999: Bachelor of Engineering (Electronics),
Assumption University
– 2000: Master of Engineering (Telecommunication),
RMIT University
– 2008: Doctor of Philosophy (Business Information
Systems), RMIT University
– 2009: Graduate Certificate in Tertiary Teaching and
Learning, RMIT University
2

About me
• Professional qualifications
– 1999: Associate Electrical Communication Engineer,
Council of Engineers, Thailand
– 2009–2010: Certificates of Completion, Australian
Consortium for Social and Political Research Incorporated
• Practical measurement and multilevel analysis in the psychosocial
sciences
• Applied structural equation modelling
• Analysing categorical and continuous latent variables using Mplus
• Scale development, Rasch analysis and item response theory
• Introduction to social network research and network analysis
– 2009: Certified Professional, Australian Computer Society
3

About me
• Professional qualifications
– 2010–2012: Certificates of Completion, statistics.com
• Practical Rasch measurement—core topics
• Rasch applications in clinical assessment, survey research,
and educational measurement
• Practical Rasch measurement—further topics
• Introduction to Bayesian statistics
• Meta analysis
• Spatial statistics with geographic information systems
• Forecasting time series
• Introduction to Bayesian computing
• Bayesian regression modeling via MCMC techniques
• Bayesian hierarchical and multi-level modeling
4

About me
• Research projects
– 1998: A fluorescent dimmer with an PPM remote
controller
– 2000: A sample stock exchange—WAP application
– 2001: Fostering consumer trust and purchase
intention in B2C e-commerce
– 2007:
• Use of partial least squares regression and structural
equation modeling in information system research
• Evaluating the impact of information and communications
technology on the livelihood of rural communities
5

About me
• Research projects
– 2008:
• Use of e-government services by the urban community
• Green information technology
• Cross-country study to foster consumer trust in B2C e-commerce
– 2009: E-business assimilation and its effects on the growth and export
performance of Australian horticulture firms
– 2010:
• Piloting a hybrid work integrated learning model to enhance dual hub student
collaborations in an international work context
• Adoption of green IT best practices and their impact on the sustainability of
data centres
• Design and usability evaluation of mental healthcare information systems
– 2011:
• Classifying Australian PhD theses by research fields, courses and disciplines
(RFCD) codes
• Green IT organizational learning (GITOL)
6

About me
• Supervised projects
– Research projects
• Completed
– Fuzzy multicriteria analysis and its applications for decision making under uncertainty
• Current
– Managing quality issues along a complex supply chain management in fire truck
bodybuilder business: A Thai case study
– E-commerce adoption in marketing: Tourism in Saudi Arabia
– Supply network complexity and organizational performance: Investigating the
relationship between structural embeddedness and organizational social outcomes using
social network analysis
– Impact of agile manufacturing on Thailand automotive performance and competitive
advantage
– Information security behaviour for non-work activities
– Industrial projects
• 2011:
– AIS student chapter portal
– Botanical art website
– Couse moderation system
7

About me
• Current research interests:
– Trust in business and management information system
– E-commerce
– Green information system
• Expertise:
– Focus group
– Grounded theory
– Measurement
– Methodology
– Statistical modelling
– Survey
8

Overview
• Morning session (9AM–12PM)
• Lunch (12PM–1PM)
• Afternoon session (1PM–4PM)
9

Detailed overview
• 9AM
– Causality
– SEM vs regression
– Path analysis (w/ activity)
• 10AM
– GOF indices & model modification (w/ activity)
– Direct/indirect/total effects (w/ activity)
– Moderation effect (w/ activity)
10

Detailed overview
• 11AM
– Power analysis (w/ activity)
– Multiple-group analysis (w/ activity)
– Invariance analysis (w/ activity)
– Q&A
• 12PM
– Lunch
• 1PM
– Theory of measurement
– EFA
– One-factor congeneric model (w/ activity)
11

Detailed overview
• 2PM
– Two-factor congeneric model (w/ activity)
– Multi-factor congeneric model (w/ activity)
– Reliability analysis
• 3PM
– Reliability analysis (activity)
– Structural model (w/ activity)
– Q&A
12

“I WOULD RATHER DISCOVER ONE
CAUSAL LAW THAN BE KING OF
PERSIA.”
Democritus (460–370B.C.)
13

Causality
• The relationship between one event and
another when the former is the cause and the
latter is the effect.
14

Characteristics of causal relationships
• Artificial isolation
• First cause or infinite regress
• Continuity of action
• Ref: Bunge (2009)
15

Universe & Causality
René Descartes’ drawing
IN
OUT
Hand-eye model
Eye-hand model
16

Universe & Causality
• When the whole universe is considered, there is no causality.
Anything correlates with everything else.
• Causality requires:
– IN: the focus (i.e. scope) of the research
– OUT: the background or the boundary conditions of the research
• Specification of IN and OUT creates asymmetry in how we
perceive.
• In order to maintain the boundary of the research,
intervention (i.e. a controlled study) is required. If there is no
intervention, we have no clue whether the change in our
observation is due to things inside or outside the model.
17

Controlling study?
18
Confounding
variable
Cancelled out by
randomisation
/random
sampling
Model
Controlled study

Testing causality
• Experimental research is required in order to test
the existence of a relationship between an
independent variable and a dependent variable
while controlling external variables. However,
there are several reasons of why it cannot be
done:
– Too expensive
– Too time-consuming
– Randomisation and manipulation is impossible or
unethical
– Phenomenon is currently unobservable
19

Is there any other way?
• Pearl (1998) argues that structural equation
modeling (SEM) allows us to test our ideas with
non-experimental data under the assumption
that a causal model is true.
• The SEM results allow us to infer that, if we have
a physical mean to manipulation an independent
variable in a controlled experiment, when an
independent variable is fixed by 1 unit, a
dependent variable will be changed by x unit.
20

What is path analysis?
• “The method of path coefficients does not
furnish general formulae for deducing causal
relations from knowledge of correlations and
has never been claimed to do so. It does,
however, within certain limitations, give a
method of working out the logical
consequences of a hypothesis as to the causal
relations in a system of correlated variables.”
Wright (1923, p. 254)
21

What is SEM?
• SEM is a framework purposed by Karl Gustav
Jöreskog, James Ward Keesling, and David E.
Wiley in 1970s to integrate maximum
likelihood, a measurement model (i.e. factor
analysis), and a structural model (i.e. path
analysis). Bentler (1980) calls it the JKW
model. LISREL was the first software that
implemented this framework. Charles
Spearman is credited for factor analysis and
Sewall Wright for path analysis.
22

SEM software
• EQS
• LISREL
• Mplus
• Mx
• Ωnyx
• R: lavaan, OpenMx, sem2
• SAS: CALIS, TCALIS
• SPSS: AMOS
• Stata: GLLAMM, SEM
• STATISTICA: SEPATH
23

Variables
• There are 2 types of variables in SEM:
– Manifest variable (observable) representing in a rectangular
• Data
• Composite variable
– Latent variable (unobservable) representing in an eclipse
• Concept
• Construct
• Residual (unexplained variance for a latent variable and a dependent
variable)
• Error (unexplained variance for a manifest variable in a measurement
model)
• When a structural model contains only manifest variables,
it is called path analysis. When manifest and latent
variables are involved, it is called SEM.
24

Types of causal relationships
• Direct causal relationship
• Indirect causal relationship
25
A B
A C B
Ref: Jaccard, J., & Jacoby, J. (2010, p. 142)

• Spurious relationship
• Bidirectional causal relationship
26
A B
A
B
C

• Unanalysed relationship (correlation)
• Moderation effect
27
A B
C
A B

SEM VS REGRESSION
What is the difference?
28

Regression
• Purpose: To test the relationship between
multiple independent variables and a
dependent variable.
29
Type: interval
Samples: independent
Distribution: normal
No multicollinearity
IV2
IV1
DV
IV3
R
Type: nominal, ordinal,
interval, ratio
Residual
Variance : homoscedasticity
Distribution: normal

SEM
• Purpose: To test the structure and
measurement of the relationships between
multiple independent and dependent
variables
30
X1
IV1
e1
X2
e2
X3
IV2
e3
X4
e4
Y1
IV3
e5
Y2 e6
r2
r1
Type: nominal, ordinal,
interval, ratio Measurement model
Structural model
Type: interval

Aspect Regression SEM
Aim Maximise 𝑅2
of a
dependent variable
Replicate a sample variance–
covariance matrix
Model Simple Complex
Dependent Variable Single Multiple
Control Variable First block/variable All
Multicollinearity Not allowed Allowed
Data Fitting Just-identified (𝑑𝑓 = 0) Over-identified (𝑑𝑓 > 0)
Interpretation Change in observation Change from manipulation
Equation One at a time Simultaneously
Generalisability Adjusted 𝑅2 All results
Comparison
31

What is over-identification?
• It is a scenario when you have more data points (i.e. information) than you
need to estimate parameters (𝑑𝑓 > 0). Different combinations of data
points produce different solutions. This leads to multiple results. A higher
level of over-identification means a higher number of solutions are
supported by the conceptual model when the model fits the data.
Consequently, this brings about a higher level of generalisability,
compared to regression that produces a single result.
• Imagine that you have 3 equations (i.e. information): 𝑥 + 𝑦 = 0, 𝑥 − 𝑦 =
1, and 2𝑥 + 𝑦 = 5. There are 2 unknown variables. Different sets of 2
equations generate different answers.
32

Benefits of over-identification
33
Statistical power
Predictive
accuracy
Generalisability
Error
Sample-
dependent
Capitalisation on
chance

Why more generalisable?
• A model with parameters that are fully optimised from a specific sample
cannot be replicate in another sample because the results capture all
specificity of that sample.
• Statistics is a technique that aims in generalisation by discarding specificity
and retaining commonality across different samples.
• Since results from regression is fully optimised on a specific sample, its
generalisation can only be inferred. This is because regression uses 100%
of data. This is called just-identified, meaning that there is a single
solution in your model.
• In contrast, path analysis allows us to use data less than 100% (i.e.
depending on your model specification); consequently, the results are
over-identified, meaning that there are multiple solutions in your model.
• Basically, a model which can be applied in multiple solutions are better
than that which can be applied in a single solution. If it does not make
sense to you, substitute the word “solution” with either “scenario” or
“situation”.
34

Why less error?
• When you conduct multiple tests and assume that these
hypotheses are dependent, you inflates a Type I error. The
actual Type I error for multiple tests is called familywise error.
It can be calculated as 𝑓𝑎𝑚𝑖𝑙𝑦𝑤𝑖𝑠𝑒 𝑒𝑟𝑟𝑜𝑟 = 1 − 1 − 𝛼 𝑛
when n is the number of tests.
• For example, given 𝛼 = 5%, conducting a series of 4 multiple
regression analysis will have a Type I error of 19% rather than
5%.
• To maintain the same confidence level, we need to decrease α
to
5%
4
= 1.25% per test based on Bonferroni adjustment.
35

Familywise error
36
Test(s) Error Test(s) Error
1 5% 11 43%
2 10% 12 46%
3 14% 13 49%
4 19% 14 51%
5 23% 15 54%
6 26% 16 56%
7 30% 17 58%
8 34% 18 60%
9 37% 19 62%
10 40% 20 64%

Why less error?
• Although Bonferroni adjustment allows us to
use simple analysis to test multiple dependent
hypotheses, we are at risk of failing to reject
null hypotheses because α is too small,
especially when we have many hypotheses.
Remember that when we decreases α, β will
increase unless we increase n.
37

Benefits of SEM
• A Type I error is controlled
• Direct/indirect/total effects
• Causal inference
• Complex models
• Measurement reliability and validity
• Likert scale is not assumed as interval data
• Model fit
• Model misspecification
• Multiple-group analysis
• Invariance analysis
• Generalisability
• Combined with other advanced techniques
38

Aspect Experiment Non-experiment
Environment Controlled Uncontrolled
Causal assumptions Not required Required
Statistical assumptions Required Required
What to be tested It tests the existence of the
relationship between an
independent variable and
a dependent variable.
It tests to what extent an
independent variable
affects a dependent
variable under the
assumption that a causal
model is true.
What is tested in SEM?
39

Interpretation of 𝑦 = 𝑏𝑥 + 𝜀
Regression
• What would be the
difference in the expected
value of 𝑌 if we were to
observe 𝑋 at level 𝑥 + 1
instead of level 𝑥?
• 𝜀 is the deviation of 𝑌 from
its conditional expectation.
• The equality sign is
symmetrical.
SEM
• What would be the change
in the expected value of 𝑌 if
we were to intervene and
change the value of 𝑋 from
𝑥 to 𝑥 + 1?
• 𝜀 is the deviation of 𝑌 from
its controlled expectation.
• The equality sign is
asymmetrical.
40
Ref: Pearl (1998)

What is 𝜀?
• 𝜀 (epsilon) is an error variable. It represents
the influence of omitted variables, which are
background variables outside the conceptual
model.
• Note: go back to Descartes’ drawing to review
IN and OUT concepts.
41

Misunderstanding
• Myth: Regression produces a structural
equation
– Reality: Regression produces a regression
equation
• Myth: SEM produces a regression equation
– Reality: Regression equation is symmetrical, but
structural equation is asymmetrical (Pearl, 1998).
42

Misunderstanding
• Myth: Partial least squares (PLS) regression is a
branch of SEM
– Reality: Regression solve one equation at a time while
SEM solve all equations simultaneously; as a result,
regression cannot be categorised as part of SEM.
– Technically, regression produces the same estimate as
that of SEM when there is no confounding effect (e.g.
correlation between independent variables and/or
errors). This point has been mathematically proven by
Pearl (1995).
43

Misunderstanding
• Myth: PLS has a higher statistical power than
that of SEM when a sample size is small.
– Reality: Goodhue, Lewis, and Thompson (2006)
conducted a Monte Carlo study and found that
there is no difference in statistical power between
PLS and SEM.
44

PATH ANALYSIS
Start from the basic
45

AMOS’ capabilities
• Path analysis: A model that contains only manifest variables
• Multiple-group analysis: A model that simultaneously tests several groups of samples
• Invariance analysis: Multiple-group analysis with a constrained across groups of samples
• Non-recursive model: A model that contains a feedback loop
• Confirmatory factor analysis (CFA): A measurement model that contains several
manifest variables
• Specification search: An analysis that explores and compares multiple measurement
models to select the best one based on different combinations of sets of items
• Latent growth modeling: A model that tests a trend over time
• SEM: A model that contains both manifest and latent variables
• Latent mean structure analysis: A model that estimates and contains mean values of
latent variables across groups of samples
• Bayesian analysis: A model that generates new sets of samples for model validation
46

AMOS
• Note: AMOS and other SEM software can
permute missing values in your data set.
However, permuting data restricts several
features in an analysis. As a result, you are
recommended to address missing values in
SPSS first.
47

AMOS
• Step-by-step how to specify a conceptual
model on AMOS:
1. Open the data set
2. Draw and run your model
3. Evaluate your model
48

Step 1: Setting up data
Open Amos Graphics
If you see any model on your inferface,
click File  New:
(1) Click a Select data file(s) icon
(2) Click File Name; locate and open
your data file
(3) Click OK
49
2

Step 2: Model
specification
(1) Click a List variable in data set
icon to see the list of variables in
your data file
(2) If you see this, you have linked
your data file properly
(3) Drag and drop 2 variables from
the list to the empty drawing
space: EXPm and TRU2
(4) Click a Draw paths icon and draw
it from EXPm to TRU2
50
2
3

Step 2: Model
specification (cont)
(1) Click a unique variable icon
(2) Click on a variable TRU2 to
include a residual variable
(3) Click Plugins  Name
Unobserved Variables and you
will see that the residual variable
has been named
51
1
3
𝑇𝑅𝑈2𝑖 = 𝑏 ∙ 𝐸𝑋𝑃𝑚𝑖 + 𝑒1𝑖

Step 2: Model
specification (cont)
(1) Click an Analysis properties icon
(2) On Output tab, tick options as
shown in the figure; close the
Analysis Properties window
(3) Click save. You are recommended
to create a folder and save the
AMOS file since it will generate a
number of files.
(4) Click a Calculate estimates icon to
commence the analysis
52
2

Step 3: Model evaluation
(1) Click the Output button to show
the result visually
(2) Since Unstandardized estimates
are highlighted, the results are
unstandardised. Click
Standardized estimates to see
different results
(3) This figure shows the
standardised results
53

Step 3: Model evaluation (cont)
• Click a View Text icon to see numerical results
54

10-min Exercise: Path
analysis
Specify the model as shown on the left
hand side.
Also, in Analysis properties, Output
tab, tick Modification indices.
Run the model and notice that now
there is Modification Indices on the
output window.
55
File: Path analysis 1

Estimators
56
AMOS
• ML
• GLS
• ULS
• SLS
• ADF
• Bayesian
LISREL
• IV
• TSLS
• ULS
• GLS
• ML
• WLS
• DWLS
Mplus
• ML
• MLM
• MLMV
• MLR
• MLF
• MUML
• WLS
• WLSM
• WLSMV
• ULS
• ULSMV
• GLS
• Bayesian

AMOS Results
• Sample Moments: the sample variance–covariance matrix
• Estimates: the estimated results based on the conceptual
model. The results include:
– Unstandardised estimates
– Standardised estimates
– Variances
– Squared multiple correlations ( or coefficient of determination)
– Estimated variance–covariance matrix (implied covariances)
– Estimated correlation matrix (implied correlations)
– Residual variance–covariance matrix (residual covariances)
– Standardised residual variance–covariance matrix (standardized
residual covariances).
57

AMOS Results
• Assessment of Normality: univariate and multivariate
normality tests
• Observations farthest from the centroid:
– Mahalanobis distance (d2)
– p1: the probability of di
2 to exceed the centroid
• Large p1 value means that a case i is probably an outlier under the
assumption of multivariate normality.
– p2: the probability of the largest di
2 to exceed the centroid
• Large p2 value means that there are probably outliers under the
assumption of multivariate normality.
– Basically p1 looks at a specific case while p2 looks at all
cases.
58

What are we testing?
• In path analysis, software will estimate a sample
variance–covariance matrix based on our
conceptual model. This matrix is tested against
the real sample matrix to test the null hypothesis
that:
– H0: The estimated matrix and the sample matrix are
the same.
• This is done by using χ2 test which is a non-
parametric test equivalent to t-test.
• The χ2 test can be found at CMIN under Model
Fit.
59

Testing a model in SEM
Sample
Matrix
Estimated
Matrix
Model
Data
60

CMIN
• CMIN is a minimum discrepancy function. AMOS
supports the following functions:
– Maximum likelihood (ML)
– Generalized least squares (GLS)
– Unweighted least squares (ULS)
– Scale-free least squares (SLS)
– Asymptotically distribution-free (ADF)
• ML is generally robust against data which
moderately deviates from multivariate normality,
thereby being used by default.
61

Model types in results
• There are 3 types of models that you will find
in AMOS results:
– Default: the conceptual model
• This model is your hypothesis.
– Saturated: the conceptual model with df=0
• This model is equivalent to regression.
– Independence: the conceptual model with
maximum df
• This model assumes no relationship among variables.
62

Model fit: CMIN
• NPAR: number of parameters in the model
– More detail can be found under Parameter summary
• CMIN and P: χ2 test is a discrepancy function between
an estimated matrix and a sample matrix.
• DF: degrees of freedom
– df = total data points - free parameters
– When df<0, model is under-justified, meaning that it
cannot be solved because there are not enough data
points.
• CMIN/DF: normed χ2 test
𝜒2
𝑑𝑓
63

t-rule
• To enable an identifiable model, you must ensure
that you do not have the number of free
parameters higher than data points. This is a
necessary but not sufficient condition for model
identification. It can be calculated with the
formula below (Bollen, 1989):
• 𝑡 ≤
𝑘 𝑘+1
2
– t = the number of free parameters (i.e. parameters
that we estimate in a model)
– k = the number of observed variables
64

Model fit: RMR, GFI
• RMR: root mean square residual is the average
difference between the population matrix and the
sample matrix. However, in practice, standardized root
mean square residual (SRMR) is used.
– 𝑅𝑀𝑅 =
𝑖𝑗 𝑠𝑖𝑗−𝜎𝑖𝑗
2
𝑘
• GFI: goodness-of-fit index is the percentage of
variances that the model can reproduce.
– 𝐺𝐹𝐼 = 1 −
𝐹𝑀𝐼𝑁𝑚𝑜𝑑𝑒𝑙
𝐹𝑀𝐼𝑁𝑛𝑢𝑙𝑙
65

Model fit: RMR, GFI
• AGFI: adjusted goodness-of-fit index is the
adjusted value of GFI to account for model
complexity.
– 𝐴𝐺𝐹𝐼 = 1 − 1 − 𝐺𝐹𝐼
𝑑𝑓𝑚𝑜𝑑𝑒𝑙
𝑑𝑓𝑛𝑢𝑙𝑙
• PGFI: parsimonious goodness-of-fit index is
the adjusted value of GFI to account for model
parsimony
– 𝑃𝐺𝐹𝐼 = 𝐺𝐹𝐼
66

Model fit: Baseline comparisons
• NFI: normed fit index is a rescaled χ2 with a range of 0 and 1.
It is used to compared a conceptual model with an
independence model.
– 𝑁𝐹𝐼 = 1 −
𝜒𝑚𝑜𝑑𝑒𝑙
2
𝜒𝑛𝑢𝑙𝑙
2
• RFI: relative fit index, AKA BL86, is the adjusted value of NFI to
account for model complexity.
– 𝑅𝐹𝐼 = 1 −
2
2
67

• IFI: incremental fit index, AKA BL89, is derived from
NFI to account for complexity of an evaluated model.
– 𝐼𝐹𝐼 =
2
−𝜒𝑚𝑜𝑑𝑒𝑙
2
2 −𝑑𝑓𝑚𝑜𝑑𝑒𝑙
• TLI: Tucker–Lewis index, AKA NNFI (nonnormed fit
index), is an adjusted value of IFI to account for
model complexity.
– 𝑇𝐿𝐼 =
2
−
2
2
−1
68

• CFI: comparative fit index is a rescaled χ2 that
accounts for a noncentrality parameter.
– 𝐶𝐹𝐼 = 1 −
𝑁𝐶𝑃𝑚𝑜𝑑𝑒𝑙
𝑁𝐶𝑃𝑛𝑢𝑙𝑙
69

Model fit: Parsimony-adjusted
measures
• PRATIO: parsimony ratio is the ratio of the
degrees of freedom of the evaluated model
and of the baseline model.
– 𝑃𝑅𝐴𝑇𝐼𝑂 =
• PNFI: parsimonious normed fit index is the
adjusted value of NFI to account for model
parsimony.
– 𝑃𝑁𝐹𝐼 = 𝑁𝐹𝐼 × 𝑃𝑅𝐴𝑇𝐼𝑂
70

Model fit: Parsimony-adjusted
measures
• PCFI: parsimonious comparative fit index is
the adjusted value of CFI to account for model
parsimony.
– 𝑃𝑁𝐹𝐼 = 𝐶𝐹𝐼 × 𝑃𝑅𝐴𝑇𝐼𝑂
71

Model fit: NCP
• NCP: noncentrality parameter is an estimated
used for calculating CFI.
• LO 90: Lower limit of 90% confidence interval
of NCP.
• HI 90: Upper limit of 90% confidence interval
of NCP.
72

Model fit: FMIN
• FMIN: a minimum value of the discrepancy function
F, which derives from χ2 to account for the non-
centrality parameter
• F0: a discrepancy function between an estimated
matrix and the population matrix.
• LO 90: Lower limit of 90% confidence interval of F0.
• HI 90: Upper limit of 90% confidence interval of F0.
73

Model fit: RMSEA
• RMSEA: root mean square error of approximation is a
discrepancy function between an estimated matrix and the
population matrix while accounting for model complexity.
– 𝑅𝑀𝑆𝐸𝐴 =
𝐹0
• LO 90: Lower limit of 90% confidence interval of RMSEA.
• HI 90: Upper limit of 90% confidence interval of RMSEA.
• PCLOSE: a significance test of RMSEA to test the null
hypothesis:
• H0: The estimated matrix and the population matrix are the same.
74

Model fit: AIC
• AIC: Akaike information criterion is a discrepancy
function between an estimated sample matrix
and an estimated population matrix. It is used for
model selection process to penalise a complex
model without a substantive improvement.
• BCC: Browne–Cudeck criterion provides a penalty
greater than AIC.
• CAIC: Consistent Akaike information criterion
provides a penalty greater than BCC.
• BIC: Bayes information criterion provides a
penalty greater than CAIC.
75

Model fit: ECVI
• ECVI: Expected cross-validation index is
equivalent to AIC.
• MECVI: Modified expected cross-validation
index is equivalent to BCC.
76

Model fit: HOELTER
• HOELTER: a crude measurement of statistical power of
χ2. It is used to accept or reject a model when the
number of hypothetical observations is larger or
smaller than the number of actual observations.
HOELTER ≥ 200G (i.e. G is the number of groups)
signifies sufficient statistical power.
– When a model is accepted and HOELTER < 200, it means
that the model would be rejected when the number of
observations exceed HOELTER’s n.
– When a model is rejected while the number of
observations exceed HOELTER’s n, it means that the model
would be accepted when the number of observation is
equal or less than HOELTER’s n.
77

Index Value Meaning Reference
𝜒2 p ≥ 0.01 The sample matrix and the
estimated matrix are the same.
Yu (2002)
SRMR < 0.07 The average error in the model is
minimal.
RMSEA < 0.06 with
PCLOSE > 0.05
The population matrix and the
estimated matrix are the same, and
the average error in the model is
minimal.
IFI ≥ 0.95 The over-identification condition of
the model is at an acceptable level.
TLI
CFI ≥ 0.96
PCFI ≥ 0.85 The estimated parameter is robust
against other samples.
Mulaik (1998)
BIC Smallest The model is more generalisable. Pitt and Myung
(2002)
78

What if my model doesn’t fit the data?
• Then something must be wrong:
– Data
• Go back to your raw data and see what might have gone
wrong (e.g. typo)
– Assumptions
• Explore your data further whether any assumption is
extremely violated
– Model
• Misspecified model
• Your model is wrong
• Theory is wrong
79

Modification index
• Modification index (MI), or Lagrange multiplier (LM), is a
measure that indicates how to fit the model better by
estimating a new parameter. It provides two pieces of
information:
– χ2: This test tells you how much χ2 would reduce when a
parameter is modified.
– Par Change: This measure tells you how much a parameter
value will change when a parameter is modified.
• A positive value means the current model underestimates a
parameter.
• A negative value means the current model overestimates a parameter.
• MI provides three tables: covariances, variances, and
regression weights. When one or more parameters do not
fit the data, they will be listed here.
80

10-min Exercise: Fitting a model
• Use MI to make the model fits the data better.
• Hints:
– Modify one parameter at a time
– After modifying a parameter, run the analysis
– Make a note on what you have modified
81

Modified model
• MI must be used with caution. Mindless model
modification leads to the following issues:
– Theoretical nonsense
– Overfitting
– More error
– Less statistical power
– More capitalising on chance
– Less predictive accuracy
– More sample-dependent
– Reduced generalisability
82

Indirect & total effects
• In a complex model which contains
relationships among dependent variables, you
may wish to calculate indirect effects and total
effects, especially if they are also your
research interest.
83

10-min Exercise: Path
analysis #2
Specify the model as shown on the left
hand side.
In Analysis properties, Output tab, tick
Indirect, direct & total effects.
Run the model.
84

AMOS results: Scalar
• Unstandardised estimates (regression weights)
– Estimate
– S.E.
– z-test
– p-value (2-tailed)
• Standardised estimates
– Estimate
• Variances
– Estimate
– S.E.
– z-test
– p-value (2-tailed)
• Squared multiple correlations (𝑅2
or coefficient of determination): The percentage
that a dependent variable is explained by other variables
85

AMOS results: Matrix
• Estimated variance–covariance matrix (implied
covariances)
• Estimated correlation matrix (implied
correlations)
• Residual variance–covariance matrix (residual
covariances): 𝑆𝑎𝑚𝑝𝑙𝑒 𝑚𝑎𝑡𝑟𝑖𝑥 −
𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑚𝑎𝑡𝑟𝑖𝑥
• Standardised residual variance–covariance matrix
(standardized residual covariances):
𝑆𝑎𝑚𝑝𝑙𝑒 𝑚𝑎𝑡𝑟𝑖𝑥−𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑚𝑎𝑡𝑟𝑖𝑥
𝑆.𝐸.
86

AMOS results: Matrix
• Total effects
– 𝑇𝑜𝑡𝑎𝑙 𝑒𝑓𝑓𝑒𝑐𝑡𝑠 = 𝑑𝑖𝑟𝑒𝑐𝑡 𝑒𝑓𝑓𝑒𝑐𝑡 +
𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡 𝑒𝑓𝑓𝑒𝑐𝑡𝑠
• Standardised total effects
• Direct effects
• Standardised direct effects
• Indirect effects
• Standardised indirect effects
87

Direct effect
• The direct effect of X on Y is the increase that
we expect to see in Y by γ unit given a unit
increase in X.
88
X Y
γ

Indirect effect
• The indirect effect of X on Y is the increase
that we expect to see in Y by γβ unit while
leaving X untouched and increasing Z to
whatever value that Z would attain under a
unit increase of X.
89
X Z Y
γ β

Total effect
• The total effect of X on Y is the increase that
we expect to see in Y by γyx+γzxβ unit under a
unit increase of X.
90
X
Z
Y
γzx β
γyx

Interpretation of unstandardised
estimates
• Based on the results, if we were to change the
value of EXPm from expm to expm+1, the
value of TRU2 is expected to change from tru2
to tru2+0.166.
• Basically, it means that the difference of the
mean values of trust between inexperienced
online shoppers and experienced online
shoppers is 0.166.
91

Interpretation of standardised
estimates
• Based on the results, if we were to change the
value of EXPm from expm to expm+1σexpm, the
value of TRU2 is expected to change from tru2
to tru2+0.100σtru2.
• Since EXPm is dichotomous, the interpretation
in terms of standard deviation 𝜎 does not
make any sense! We can only interpret that
the effect size of EXPm on TRU2 is small.
92

Use of estimates
Unstandardised
• To create a mathematical
equation
• To use as a priori
parameters
• To simulate a model
• Stable across samples
Standardised
• To calculate an effect size
• To communicate with others
• To compare with other
studies
• Unstable across samples
since a parameter is
standardised using a
sample-specific standard
deviation
93

SEM procedure
Model
evaluation
Fit?
Model
rectification
No
Path
evaluation
Yes
94

Moderation effect
• An effect from one variable (e.g. z) impacting a
strength of a relationship between two other
variables (e.g. x & y).
• Mathematically, it can be written as:
𝑦 = 𝑏0 + 𝑏1𝑥 + 𝑏2𝑧 + 𝑏3𝑥𝑧 + 𝜀
x y
z
95

10-min exercise: Path analysis #3
• Specify model as show below:
96
File: Moderation effect

Residual centering
• Lance (1988) proposed use of residual
centering to address multicollinearity
problems from creating a moderating variable.
This is done by using variables, which are used
to create the moderator, to predict the
moderator.
97

Warning
• Having only 2 variables does not produce any
evidence about causality. You need at least 3
variables in a model. In fact, the third variable
may affect a relationship between 2 variables.
• Cum hoc ergo propter hoc (with this, therefore
because of this) is a logical fallacy stating that
correlation is causation.
• Correlation is a necessary but not sufficient
condition to be causation.
98

Importance of statistical power
• Significance test provides a p-value which tells us the
probability of observing H0, given that H0 is true.
• Effect size tells us to what extent a result is practical,
given that Ha is true.
• Statistical power tells us the probability of reproducing
Ha, given that Ha is true. For example, given 80% of
statistical power in your result, there is 80 × 80 =
64% chance of reproducing Ha in 2 consecutive
attempts.
• In summary, the higher the statistical power, the higher
reproducibility the result is.
• Ref: Schmidt & Hunter (1997)
100

Hypothesis & SEM
101
Conclusion
Sufficient power
(𝟏 − 𝜷 ≥ 𝟎. 𝟖)
Insufficient power
(𝟏 − 𝜷 < 𝟎. 𝟖)
Outcome
H0 is not rejected
(𝒑 ≥ 𝟎. 𝟎𝟏)
There is not sufficient
evidence to reject that the
model is correct.
Although there is not
sufficient evidence to
reject that the model is
correct, it may happen by
chance.
H0 is rejected
(𝒑 < 𝟎. 𝟎𝟏)
There is sufficient evidence
to reject that the model is
correct.
Although there is sufficient
evidence to reject that the
model is correct, it may
happen by chance.

MacCallum, Browne, Suguwara’s
power analysis
• They proposed the idea of testing hypotheses of exact fit,
close fit, and not close fit.
– Exact fit
• H0: RMSEA=.00, Ha: RMSEA=.06
– Close fit
– Not close fit
• These values are heuristic. MacCallum, Browne, & Suguwara
(1996) proposed to use RMSEA=.05, but later Yu (2002) found
that it is better to use RMSEA=.06.
102

RMSEA CI
103
.00
Exact fit
.03 .09
.06
Close fit
Not close fit

Outcomes of hypothesis testing
104
Reality
𝑹𝑴𝑺𝑬𝑨 ≤. 𝟎𝟔
(H0)
𝑹𝑴𝑺𝑬𝑨 >. 𝟎𝟔
(Ha)
Outcome
𝑹𝑴𝑺𝑬𝑨 ≤. 𝟎𝟔
Correct Outcome
[1-α]
False Negative
(Type II Error)
[β]
𝑹𝑴𝑺𝑬𝑨 >. 𝟎𝟔
False Positive
(Type I Error)
[α]
Correct Outcome
[1-β]

SPSS syntax for power analysis
• Provided by Timo Gnambs
• Step-by-step guide:
1. Look up at your RMSEA and its 90% confidence interval results (i.e.
LO90 and HI90).
2. Decide which hypothesis you want to test, preferably the value
outside 90% CI.
3. Change the value of df, α, n, RMSEA0, and RMSEAa accordingly.
4. You may set RMSEA0 to be the same value you get from your result.
5. Run the script Power analysis (beta).sps. Note that it does not run if
your data is blank.
6. If you want to further know what is the number of observations that
you need to have at least 80% of statistical power, run the script
Power analysis(n).sps. You may change the value of df, α, power (1-
β), RMSEA0, and RMSEAa accordingly.
105

5-min exercise: Power analysis
• Conduct power analysis to calculate statistical
power
106

Multiple-group analysis
• SEM allows you to test a model against
multiple groups of samples simultaneously.
This technique is useful when you
know/suspect/hypothesise that samples are
heterogeneous. This allows us to control a
Type I error.
107

10-min exercise: Multiple-group
analysis
1. From the path model #3, save it as a new file
2. From the menu Analyze, click Manage Group
3. Rename the current group to Experienced
4. Click New and rename the new group to Inexperienced
5. Click the icon Select data file(s)
6. Click the button Grouping Variable
7. Select the variable EXPm and click OK
8. Click the button Group Value
9. Select the group having the value 1, click OK
10. For the group Inexperienced, locate the same data file and
assign the group having the value 0
108
File: Multiple-group analysis

Invariance analysis
• SEM allows you to test a parameter to be
equal across groups of samples. This
technique is useful when, for example, you
hypothesise that the effect of one variable on
another variable is constant across groups of
samples.
109

10-min exercise: Invariance analysis
1. Identify which parameter is very similar across
both groups
2. Open Object Properties window for that
parameter
3. Name that parameter
4. Notice that All groups
are ticked, which means
that this parameter is
equal across groups of
samples
110
File: Invariance analysis

What types of parameters can be
constrained to be invariant?
• Mean
– AMOS automatically centred means of all parameters
in a model to 0. If you want to test mean values, first
you need to set AMOS to estimate means and
intercepts. After that, you must name all mean
parameters that you want AMOS to estimate through
Object Properties window.
• Slope (structural
parameters)
• Residual
111

“I call ‘em as I see ‘em,” said the first. The second
replied, “I call ‘em as they are.” The third said,
“what I call ‘em makes ‘em what they are.”
Theory of measurement
112

Measurement paradigm
• A 20th century philosophy of measurement called
representationalism saw numbers, not as properties inherent in an
object, but as the result of relationships between measurement
operations and the object (Chrisman, 1995, p. 272).
• Measurement of magnitudes is, in its most general sense, any
method by which a unique and reciprocal correspondence is
established between all or some of the magnitudes of a kind and all
or some of the numbers, integral, rational, or real, as the case may
be … In this general sense, measurement demands some one–one
relation between the numbers and magnitudes in question—a
relation which may be direct or indirect, important or trivial,
according to circumstances (Russell, 1903, p. 176).
• Different analyses require/support different levels of measurement.
113

114
𝑇𝑖𝑗 𝑋𝑖𝑗 𝐸𝑖𝑗
𝛼𝑗 𝛽𝑗 𝛾𝑗 𝜃𝑖
𝑅 = 𝑅𝑒, ≥
𝑂 = 𝐼 × 𝑃, ≽
Classical test theory
Operationalism
Error structure
Score
To describe
Latent variable theory
Realism
Explanatory structure
Data generation
To explain
Fundamental measurement theory
Constructivism
Representational structure
Scale
To represent

• “The classical test theory model is the theory of
psychological testing that is most often used in
empirical applications. The central concept in classical
test theory is the true score. The true scores are related
to the observations through the use of the expectation
operator: the true score is the expected value of the
observed score.” (Borsboom, 2005, p. 3)
• The true score of any person 𝑖 on an item 𝑗 (𝑡𝑖𝑗) is the
expected value of the observed score (𝜀 𝑋𝑖𝑗 ). The
difference between the observed score and the true
score is the error score (𝐸𝑖𝑗 = 𝑋𝑖𝑗 − 𝑡𝑖𝑗).
115

• “The latent variable model has been proposed as an
alternative to classical test theory, and is especially popular
in psychometric circles. The central idea of latent variable
theory is to conceptualize theoretical attributes as latent
variables. Latent variables are viewed as the observed
determinants of a set of observed scores; specifically, latent
variables are considered to be the common cause of the
observed variables.” (Borsboom, 2005, p. 4)
• Depending on a priori hypothetical model, a true score may
be caused by a person’s ability (𝜃𝑖), an item difficulty or a
precision (an intercept 𝛽𝑗), an item discrimination or a scale
(a slope 𝛼𝑗), and a guessing effect (𝛾𝑗).
116

• “The representational measurement model—also known as
‘abstract’, ‘axiomatic’, or ‘fundamental’ measurement
theory—offers a third line of thinking about psychological
measurement. The central concept in representationalism is
the scale. A scale is a mathematical representation of
empirically observable relations between the people
measured.” (Borsboom, 2005, p. 4)
• 𝑂 = 𝐼 × 𝑃, ≽ is an empirical relational system which
represents an observation of the product from an item and
a person which can be used to order items and persons
independently. When measurement is additive, 𝑂 can be
mapped into a numerical relational system: 𝑅 = 𝑅𝑒, ≥ .
117

In research ...
• Measurement
development
Fundamental
measurement
theory
• Measurement
validation
Latent variable
theory
• Measurement
reliability
Classical test
theory
118

Measurement assembly
x1 x2 x3 x4
Empirical plane
Construct
Ontological plane
Theoretical plane
Instrumental plane
X1: This course is useful.
X2: I learned a lot of things in this course.
X3: Content in this course is applicable in real-world situations.
X4: All lecturers in this course are hot!
-Theories
-Models
-Scales
-Law statements
119

TO REPRESENT
Fundamental measurement theory
120

Content validity
1. Construct specification
– Domain
• What is included
• What is excluded
– Facets (substrata)
• Some researchers refer to facets as dimensions.
– Dimensions (e.g. rate, duration, magnitude)
– Modes (e.g. thought, behaviour)
– Temporal parameters (response interval, duration of time-
sampling)
– Situations
2. Function of instrument (e.g. brief screening, functional
analysis, diagnosis)
121

Content validity
3. Assessment method
4. Item generation
– Deduction
– Experience
– Theory
– Literature
– Instrument
– Content expert
– Population
122

Content validity
5. Item alignment
– Use table of construct to map against items
– Generate multiple items/facet
– Adjust the number of items relatively to the
importance of facet
6. Item examination
– Suitability of items for a facet
– Consistency, accuracy, specificity, and clarity of
wording and definitions
– Remove redundant items
123

Content validity
7. Quantitative parameter
– Response formats and scales
– Time-stamping parameters
8. Instrumentation
– Create instructions to match with domain and
function of assessment instrument
– Clarify and strive for specificity and appropriate
grammatical structure
124

Content validity
9. Stimuli creation (e.g. social scenarios, audio
and video presentations)
10.Pre-test (with expert for steps 1–3 and 5–9)
11.Pilot test (with sample)
12.Item screening (using content validation
process by Lindell (2001), Lindell, Brandt, &
Whitney (1999), and Lindell & Brandt(1999))
• Ref: Haynes, Richard, & Kubany (1995)
125

Levels of measurement
126
Level Information required Example
Nominal Definitions of categories Sex
Graded
membership
Definitions of categories plus degrees of
membership or distance from prototype
Socio-economic status
Ordinal Definitions of categories plus ordering Rating scale
Interval Unit of measure plus zero point Degree Celsius
Log-interval Exponent to define intervals Richter magnitude scale
Extensive ratio Unit of measure (additive rule applies) Length, mass, time
Cyclic ratio Unit of measure plus length of cycle Angle
Derived ratio Unit of measures (formula of combination) Density, velocity
Counts Definition of objects counted Number of employees
Absolute Type Probability, proportion
Ref: Chrisman (1998, p. 236)

TO EXPLAIN
Latent variable theory
127

Measurement analysis
128
Rasch model
Test a scale
Item response theory
Explore a scale
Confirmatory factor analysis
Test a latent variable
Exploratory factor analysis
Explore latent variables
Generalizability theory
Incorporate multiple errors
Incorporate an error
Principal component analysis
Construct an index

Latent variable models
129
Factor
analysis
Item
response
theory
Mixture
model
Latent
class
analysis
Manifest variables are
Continuous
Continuous
Categorical
Categorical
Latent variables are

Types of indicators
F
X1 X2 X3 X4
e1 e2 e3 e4
F
X1 X2 X3 X4
e1 e2 e3 e4
F
X1 X2 X3 X4
Ref: Hayduk et al. (2007)
130

Type of measurement models
ξ
x1 x2 x3 x4
δ1 δ2 δ3 δ4
1 2 3 4
1  2  3  4
x1  x2  x3  x4
1  2  3  4
1  2  3  4
x1  x2  x3  x4
1  2  3  4

1  2  3  4
x1  x2  x3  x4
1  2  3  4
1  2  3  4
x1  x2  x3  x4
1  2  3  4
131
Ref: Graham (2006)

Type of measurement models
132
ξ = 1
x1 x2 x3 x4
0 0 0 0
1 2 3 4
η4
η3
η1 η2
1 2 3 4
ζ ζ ζ ζ
Ref: Jöreskog (1978)

Measurement parameters
• Mean is precision and difficulty.
• Factor loading, or slope, is scale and
discrimination.
• Error variance is error and unique variance.
Theoretically, it should be random error.
However, if there is systematic error, 2+ errors
will be correlated.
133

Exploratory factor analysis (EFA)
• Purpose: To create a
psychometric
measurement by
discovering an
underlying pattern
and conceptualising a
latent variable
134
X1 X2
F1
X3 X4
F2
e3 e4
e1 e2
Item
Type: interval

EFA use
• To partial the measurement error out of the
observe scores
• To explore underlying patterns in the data
• To determine the number of latent variables
• To reduce the number of variables
• To assess the reliability of each item
• To eliminate multi-dimensional items (i.e.
cross-loaded variables)
135

Dimensionality
Multi-dimensional
construct
Uni-dimensional
construct
Uni-dimensional
factor
Multi-dimensional
factor
Multi-dimensional
item
Uni-dimensional
item
136

EFA processes
137
Extract Rotate Interpret

Extraction
• When you assume data to be the population:
– Principal axis factoring assumes that factors are
hypothetical and that they can be estimated from
variables.
– Image factoring assumes that factors are real and
that they can be estimated from variables.
138

Extraction
• When you assume data to be a sample randomly selected
from the population:
– ULS attempts to minimise the sum of squared differences
between estimated and observed correlation matrices,
excluding the diagonal.
– GLS attempts to minimise the sum of squared differences
between estimated and observed correlation matrices while
accounting uniqueness of variables (i.e. the more the
uniqueness of variables is, the less weight the variables have).
– ML attempts to estimate parameters which are likely to produce
the observed correlation matrix. The estimated correlation
matrix is accounted for the uniqueness of variables.
– Kaiser’s alpha factoring assumes that variables are randomly
sampled from a universe of variables. It attempts to maximise
reliability of factors.
139

Rotation
• Orthogonal rotation assumes factors to be
uncorrelated from one another:
– Quartimax maximises the sum of variances of
loadings in rows of the factor matrix. It attempts to
minimise the number of factors needed to explain
each variable. This method tends to make a number of
variables highly loaded on a single factor.
– Varimax maximises the sum of variances of loadings in
columns of the factor matrix. It attempts to minimise
the number of variables highly loaded on each factor.
– Equamax combines Quartimax and Varimax
approaches, but it is reported to behave erratically.
140

Rotation
• Oblique rotation assumes factors to be
correlated with one another:
– Direct oblimin allows you to adjust a degree of
correlation between factors. Delta value of 0
allows factors to be moderately correlated. On the
other hand, delta value of +0.8 allows factors to
be more correlated while delta value of -0.8 allows
factors to be less correlated.
– Promax is quicker than direct oblimin. It is useful
with a large sample.
141

Extraction & rotation
F1
F2
F1
F2
142

5-min exercise: Principal axis factoring
1. Click Analyze  Dimension Reduction  Factor
2. Select Q2–Q65 into Variables
3. Click Descriptives; tick Coefficients, Significance levels,
Determinant, KMO and Bartlett’s test of sphericity,
Reproduce, and Anti-image; click Continue
4. Click Extraction, choose Principal axis factoring for Method,
tick Scree plot, click Continue
5. Click Rotation, choose Direct Oblimin, click Continue
6. Click Options, tick Sort by size and Suppress small
coefficients, type .33, click Continue
7. Click OK
143

Reading EFA Results
• Correlation matrix is a good start to look for bad items: those that
do not correlate with others (r<0.3) and those that correlate highly
with other (r>0.9). These items are subject for elimination. In
addition, a determinant value higher than 10-5 signifies that there is
no multicollinearity issue.
• Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy tells
you whether data is factorable:
– >0.9 is marvellous
– >0.8 is meritorious
– >0.7 is middling
– >0.6 is mediocre
– >0.5 is miserable
– <0.5 is unacceptable
144

Reading EFA Results
• Bartlett’s test of sphericity determines whether the
observed correlation matrix is different from the identity
matrix, meaning that you cannot analyse data with EFA if
the data is the identity matrix since there is no correlation
between variables. The null hypothesis is:
– H0: The observed correlation matrix and the identity matrix have
the same value.
• Anti-image correlation matrix contains the KMO measure
of each variable (i.e. a diagonal element) and negatives of
partial correlation among variables (i.e. off-diagonal
elements). The KMO measure less than 0.5 means that a
particular variable is subject for elimination. In addition, a
value of negative partial correlation should be small.
145

Reading EFA Results
• Total Variance Explained table shows the proportion of
variance explained by factors:
– Based on Kaiser’s criterion, factors having eigenvalues
greater than 1 should be retained. The logic behind this
argument is that a factor should explain at least one
variable. Generally, this criterion leads to an overfactoring
issue. However, this justification is accurate when:
• The number of variables is less than 30 and extracted
communalities are all greater than 0.7.
• The sample size is more than 200 and extracted communalities are
0.6 or higher.
• Communalities table shows the percentage of variance
of each variable explained by factors. It is item
reliability (R2).
146

Reading EFA Results
• Reproduced Correlations table displays the predicted
correlation matrix. Ideally, it should be the same as the
observed correlation matrix. It also shows residual
which is the difference between the predicted
correlation matrix and the observed correlation matrix.
The percentage of non-redundant residuals with an
absolute value higher than 0.5 should be less than
50%.
• Scree Plot depicts eigenvalues gained from an
additional factor. The cut-off point should be at where
the slope changes dramatically. Use of this graph is
controversial as being subjective.
147

Reading Results
• Factor Matrix shows the correlation between
items and factors (i.e. a factor loading) before
rotation is taken place.
• Pattern Matrix shows the correlation between
items and factors after rotation is taken place.
• Structure Matrix shows the correlation between
items and factors, accounted for relationships
between factors. Actually, it is the product of the
pattern matrix and the factor correlation matrix.
• Factor Correlation Matrix shows the correlation
between factors.
148

How many factors?
• You should use a combination of:
– Your measurement
– Your conceptual model
– Theories
– Literature
– Eigenvalues (Kaiser’s criterion)
– Pattern matrix
– Communalities (item reliability)
– Scree plot
– Parallel analysis
149

Parallel analysis
• To determine the maximum number of factors
to be extracted by assessing eigenvalues of
the data against those of the simulation.
Factors to be extracted must have eigenvalues
higher than those of the simulated ones.
• You may use the SPSS script provided by
O’Connor (2000) or the online engine
provided by Patil, Singh, Mishra, and Donovan
(2008).
150
File: rawpar

Confirmatory factor analysis (CFA)
• Purpose: To test a
psychometric
measurement by
hypothesising an
underlying pattern
based on a known
construct.
151
X1 X2
F1
X3 X4
F2
e3 e4
e1 e2
Item
Type: interval

CFA
• To test a specific measurement model (i.e. parallel, 𝜏-
equivalent, essentially 𝜏-equivalent, congeneric, and
variable-length models)
• To test a higher-order factor model
• To test construct validity (i.e. convergent validity,
discriminant validity, and factorial validity)
• To assess to what extent the measurement fits the data
• To perform multiple-group or invariance analysis
• To prepare the measurement model for structural
equation modeling (SEM)
152

Measurement validation
• Convergent validity
– To test dimensionality of a factor and items
– To assess construct reliability
– Method: one-factor model
• Discriminant validity
– To test that 2 factors represent different things
– Method: nested two-factor model (i.e. one model assumes two
factors to be the same thing and the other does not), average
variance extracted (𝜌𝑣𝑐 AKA average variance extract (AVE))
• Factorial validity
– To test that all factors in the measurement fits the data
– Method: multi-factor model
153

Convergent validity
154
File: Convergent validity

Discriminant validity
155
File: Discriminant validity

Correlation–AVE comparison
1. Calculate AVE using the formula 𝜌𝑣𝑐 =
𝑖=1
𝑝
𝜆𝑖
2
𝑖=1
𝑝
𝜆𝑖
2 + 𝑖=1
𝑝
𝑣𝑎𝑟(𝜀𝑖)
– Ref: Fornell & Larcker (1981)
2. Produce a correlation matrix for every pair of factors.
3. Calculate a squared correlation matrix (𝛾2
)
4. Evaluate discriminant validity with the following
criteria:
– 𝜌𝑣𝑐 ≥ .5
– 𝜌𝑣𝑐 > 𝛾2
• Example: Molla, Cooper, & Pittayachawan (2011)
156

Factorial validity
157
File: Factorial validity

Construct validity
158
Convergent validity
Fit?
Discriminant validity
Fit?
Fix/free parameter
Split factor
Drop item
Factorial validity
Fit?
Fix/free parameter
Drop item
Diff? Combine factors
Fix/free parameter
Drop item
Ref: Molla, Cooper, & Pittayachawan (2011)

Correlated errors
• Freeing a correlational parameter between
error terms may be a post hoc practice to
improve model fit. However, it should be
supported by a theoretical explanation.
• Gerbing (1984) explains that one possibility to
have a correlated errors is due to multi-
dimensionality.
159

TO DESCRIBE
160

161
Ref: Alreck & Settle (2004, p. 58)

Reliability
• The extent that your measurement can
produce consistent results (i.e. precision)
True
Score
Error
Observed
Score
162

163
Error
Bias
Observation
Errors
Interviewer
Respondent
Instrument
Mode
Non-observation
Errors
Coverage
Non-response
Sampling
Variance
Observation
Errors
Interviewer
Respondent
Instrument
Mode
Non-observation
Errors
Coverage
Non-response
Sampling
Groves (2004, p. 10)

Error
• Consists of 2 components:
– Bias is a constant error caused by research design.
– Variance is a variable error caused by obtaining data
from different respondents, using different
interviewers, and asking different questions.
• Both bias and variance consist of 2 components:
– Observation error is deviation of observed scores
from true scores.
– Non-observation error is caused by failure to include
other samples.
164

Error in observation
• Observation error consists of 4 components:
– Interviewer error is caused by ways of
administration made by interviewers.
– Respondent error is caused by different
individuals give responses with a different amount
of error.
– Instrument error is caused by design of
instruments.
– Mode error is caused by using different modes of
enquiry.
165

Error in non-observation
• Non-observation error consists of 3
components:
– Coverage error is caused by failure to include
samples into a sampling frame.
– Non-response error is caused by respondents
cannot be located or refuse to respond.
– Sampling error is caused by statistics producing
results based on a subset of the population which
may exhibits responses differently from other
subsets.
166

Error in non-response
• Non-response may cause by:
– Respondents lack motivation or time
– Fear of being registered
– Travelling
– Unlisted, wrong, or changed contact details
– Answering machine
– Telephone number display
– Illness or impairment
– Language problems
– Business staff, owner, or structure changes
– Too difficult or boring
– Business policy
– Low priority
– Survey is too costly, or lack of time or staff
– Sensitive or bad questions
• Ref: Biemer & Lyberg (2003, p. 93)
167

Error in statistics
• There are 2 components:
– Systematic error represents something that is not
captured in a model.
– Random error, AKA measurement error, represents
something that uniquely causes observed scores to
deviate from true scores. This type of error is
supported by latent variable models. It is a
combination of error generated by interviewers,
respondents, and instruments. When multitrait–
multimethod (MTMM) is used, mode error can also be
incorporated.
168

Instrument error
• Unstated criteria
– Wrong: How important is it for stores to carry a large
variety of different brands of this product?
– Right: How important is it to you that the store you
shop at carries a large variety of different brands?
• Inapplicable questions
– Wrong: How long does it take you to find a parking
place after you arrive at the plant?
– Right: If you drive to work, how long does it take you
to find a parking place after you arrive at the plant?
169

Instrument error
• Example containment
– Wrong: What small appliances, such as countertop
appliances, have you purchased in the past month?
– Right: Aside from major appliances, what other
smaller appliances have you bought in the past
month?
• Over-demanding recall
– Wrong: How many times did you go out on a date
with your spouse before you were married?
– Right: How many months were you dating your
spouse before you were married?
170

Instrument error
• Over-generalisations
– Wrong: When you buy” fast food”, what percentage of the
time do you order each of the following type of food?
– Right: Of the last 10 times you bought “fast food”, how
many times did you eat each type of food?
• Over-specificity
– Wrong: When you visited the museum, how many times
did you read the plaques that explain what the exhibit
contained?
– Right: When you visited the museum, how often did you
read the plaques that explain what the exhibit contained?
Would you say always, often, sometimes, rarely, or never?
171

Instrument error
• Over-emphasis
– Wrong: Would you favour increasing taxes to cope
with the current fiscal crisis?
– Right: Would you favour increasing taxes to cope
with the current fiscal problem?
• Ambiguity of wording
– Wrong: About what time do you ordinarily eat
dinner?
– Right: About what time do you ordinarily dine in
the evening?
172

Instrument error
• Double-barrelled questions
– Wrong: Do you regularly take vitamins to avoid getting
sick?
– Right: Do you regularly take vitamins? Why or why not?
• Leading questions
– Wrong: Don’t you see some danger in the new policy?
– Right: Do you see any danger in the new policy?
• Loaded questions
– Wrong: Do you advocate a lower speed limit to save
human lives?
– Right: Does traffic safely require a lower speed limit?
173

Respondent error
• Social desirability
– Response based on what is perceived as being socially
acceptable or respectable.
• Acquiescence
– Response based on respondent’s perception of what
would be desirable to the sponsor.
• Yea- and nay-saying
– Response influenced by the global tendency toward
positive or negative answers.
• Prestige
– Response intended to enhance the image of the
respondent in the eyes of others.
174

Respondent error
• Threat
– Response influenced by anxiety or fear instilled by
the nature of the question.
• Hostility
– Response arising from feelings of anger or
resentment engendered by the response task.
• Auspices
– Response dictated by the image or opinion of the
sponsor rather than the actual question.
175

Respondent error
• Mental set
– Cognitions or perceptions based on previous items
influence response to later ones.
• Order
– The sequence in which a series is listed affects the
responses to the items.
• Extremity
– Clarity of extremes and ambiguity of mid-range
options encourage extreme responses.
176

Error reduction
Type Method
Coverage error Identify which cases are missing from the sampling frame
Use multiple sampling frames
Remove duplicates and erroneous inclusions from a frame
Non-response error Theories of survey participation (Cialdini, 1990; Groves & Couper 1998):
• reciprocation (e.g. incentive),
• consistency (e.g. data vs personal and social benefits),
• social validation (e.g. participation of similar respondents),
• authority (e.g. reputation),
• scarcity (e.g. rare opportunities), and
• liking (e.g. interviewers are similar to respondents).
Tailored design method (TDM) by Dillman, Smyth, & Christian (2009)
Guarantee privacy and confidentiality
Reduction of respondents’ burden
Sampling error Use probability sampling
Weight cases for non-probability sampling
Interviewer, respondent, and
instrument error
Use latent variable models (e.g. SEM, IRT, or LCM)
Train interviewers
Thoroughly develop instruments
Use scales that capture bias
Use indirect questions
Mode error Use multitrait–multimethod (MTMM)
177
Ref: Biemer & Lyberg (2003)

Internal consistency
• Cronbach’s  is internal consistency based on inter-item correlation.
• Split-half reliability is to randomly separate measurement into two parts in order to calculate
correlation between them. It assumes variance in each part is equal. This option produces
Spearman–Brown split-half reliability coefficient and Guttman split-half reliability coefficient.
• Guttman’s lower bounds calculates six reliability coefficients:
– 1: an intermediate coefficient used to calculate other 
– 2: coefficient which is more complex than Cronbach’s 
– 3: an equivalent version of Cronbach’s 
– 4: Guttman split-half reliability
– 5: recommended when a single item is highly correlated with others, which lack high
correlation among themselves
– 6: recommended when inter-item correlation is low in relation to square multiple
correlation
• Parallel model assumes equal variance among items and among their error
• Strict parallel model assumes equal variance among items and among their error and items
have equal mean
178

Internal consistency
• Internal consistency assumes factor loadings of all
items are equal (i.e. essentially 𝜏-equivalent
model). When this assumption is met, internal
consistency is reliability (Novick & Lewis, 1967).
However, when this assumption is violated,
internal consistency under-estimates reliability.
• Internal consistency assumes uni-dimensionality.
As a result, its value will increase even though
random items are thrown into the analysis.
179

5-min exercise: Internal consistency
1.Calculate Cronbach’s  (Analyze  Scale 
Reliability Analysis) based on the results that
you have from CFA. Then try to add any item
in a subsequent analysis and observe how its
value changes.
180

Construct reliability
• When a model is not essentially 𝜏-equivalent
model (i.e. congeneric model), coefficient 𝐻 is
recommended Hancock & Mueller (2001).
• Coefficient H has a good theoretical property
that its value is not less than the most reliable
item in the construct.
• The formula is 𝐻 =
1
1+
1
𝑖=1
𝑝 𝜆𝑖
2
1−𝜆𝑖
2
181

5-min exercise: Construct reliability
1.Calculate coefficient H using the spread sheet
and factor loadings from CFA.
2.Observe the differences in values between
Cronbach’s  and coefficient H.
182

Use of reliability index
• To report construct reliability
• To be used as a priori parameter values (i.e. factor
loading and error variance), especially for
creating a composite variable (i.e. parcelling)
• Munck (1979) demonstrates that you may use a
single-indicator factor and still makes a model
identified by calculating a factor loading (𝜆) and
error variance (𝜃) with the following formulae:
• 𝜆 = 𝜎𝑥 𝑟
• 𝜃 = 𝜎𝑥
2
1 − 𝑟
183

Up to this point
• Before conducting SEM, you must ensure that:
– All factors hold construct validity, reliability, and
uni-dimensionality.
– All items hold uni-dimensionality.
– Preferably, each factor has local independence
(i.e. absence of correlation between errors).
184

30-min exercise: SEM
1. Create a structural model to test the
following hypotheses:
– Security positively affects trust.
– Web design positively affects trust.
– Web design positively affects security.
2. You may also try to combine SEM with
multiple-group analysis or invariance
analysis.
185

SEM method
Conceptualisation Instrumentation Identification Mensuration
Preparation
Specification
Estimation
Evaluation
Rectification Alternation Selection Explanation
187

SEM method
1. Conceptualisation: This is done during literature review or
following a subsequent study. Theories are used to explain
a phenomenon.
2. Instrumentation: Concepts are linked into manifest
variables to create an instrument and a procedure for data
collection.
3. Identification: A model must be guaranteed that it will be
identified. If it is not, then an additional variable must be
included.
4. Mensuration: Data are collected while minimising
measurement error.
5. Preparation: Data are collated, cleaned, and structured in
a format supported by software.
188

SEM methodology
6. Specification: A model is specified properly in
software. The difficulty level depends on which
software and analysis are used.
7. Estimation: An appropriate estimator is chosen and
used. Different estimators have different statistical
assumptions and require different sample sizes.
Estimators sometimes are limited by types of analysis.
8. Evaluation: A model is evaluated at both model and
parameter levels that it fits data. It is often done in the
following procedures: EFA, CFA, and SEM.
189

SEM methodology
9. Rectification: In practice, a model often does not fits
data (e.g. 𝑝 < .05) and needs to be respecified. MI is
normally used as a tool to identify a cause of misfit.
After a model is respecified, it needs to be evaluated.
10.Alternation: In some disciplines, fitting a model is not
a goal. In fact, no one can prove that the fitted model
is the true one. Since SEM’s condition is over-
identified, countless models can fit data. It is
encouraged that a researcher also recognises and
identifies an alternative hypothesis to explain a
phenomenon. This can be achieved in 3 ways: nested
model, equivalent model, and competitive model.
190

SEM methodology
11.Selection: After original and alternative
models have been fitted, one model must be
selected to represent a phenomenon.
12.Explanation: A fitted, and selected, model
must be explained. This also includes
correlation between error terms.
191

References
• Alreck, P. L., & Settle, R. B. (2003). The survey research handbook (3rd ed.). New York, NY, USA: McGraw–Hill/Irwin.
• Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal modeling. Annual Review of Psychology,
31(1), 419–456.
• Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ, USA: John Wiley & Sons, Inc.
• Bollen, K. A. (1989). Structural equations with latent variables. New York, NY, USA: Wiley.
• Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge, UK:
Cambridge University Press.
• Bunge, M. (2009). Causality and modern science (4th ed.). New Brunswick, NJ, USA: Transaction Publishers.
• Chrisman, N. R. (1995). Beyond Stevens: A revised approach to measurement for geographic information. In 13th
International Symposium on Computer-Assisted Cartography (pp. 271–280). Presented at the 13th International
Symposium on Computer-Assisted Cartography, Charlotte, NC, USA.
• Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and
measurement error. Journal of Marketing Research, 18, 39–50.
• Gerbing, D. W., & Anderson, J. C. (1984). On the meaning of within-factor correlated measurement errors. The
Journal of Consumer Research, 11(1), 572–580.
• Goodhue, D., Lewis, W., & Thompson, R. (2006). PLS, small sample size, and statistical power in MIS research.
Presented at the Hawaii International Conference on System Sciences, IEEE.
• Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and
how to use them. Educational and Psychological Measurement, 66(6), 930–944.
• Groves, R. M. (2004). Survey errors and survey costs. Hoboken, NJ, USA: John Wiley & Sons, Inc.
• Hancock, G.R. and R.O. Mueller (2001). Rethinking Construct Reliability Within Latent Variable Systems. In Cudeck,
R., S. Du Toit, and D. Söbom (eds.). Structural Equation Modeling: Present and Future, Lincolnwood: Scientific
Software International, Inc., pp. 195–216.
192

References
• Hayduk, L. A., Robinson, H. P., Cummings, G. G., Boadu, K., Verbeek, E. L., & Perks, T. A. (2007). The weird world,
and equally weird measurement models: Reactive indicators and the validity revolution. Structural Equation
Modeling, 14(2), 280–310.
• Haynes, S. N., Richard, D. C., & Kubany, E. S. (1995). Content validity in psychological assessment: a functional
approach to concepts and methods. Psychological Assessment, 7(3), 238–247.
• Jaccard, J., & Jacoby, J. (2009). Theory construction and model-building skills: A practical guide for social scientists.
New York, NY, USA: The Guilford Press.
• Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43(4), 443–477.
• Lance, C. E. (1988). Residual centering, exploratory and confirmatory moderator analysis, and decomposition of
effects in path models containing interactions. Applied Psychological Measurement, 12(2), 163–175.
• Lindell, M. K. (2001). Assessing and testing interrater agreement on a single target using multi-item rating scales.
Applied Psychological Measurement, 25(1), 89–99.
• Lindell, M. K., & Brandt, C. J. (1999). Assessing interrater agreement on the job relevance of a test: A comparison
of CVI, T, rWG(J), and r*WG(J) indexes. Journal of Applied Psychology, 84(4), 640.
• Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A revised index of interrater agreement for multi-item ratings
of a single target. Applied Psychological Measurement, 23(2), 127–135.
• MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for
covariance structure modeling. Psychological Methods, 1(2), 130–149.
• Molla, A., Cooper, V., & Pittayachawan, S. (2011). The green IT readiness (g-readiness) of organizations: An
exploratory analysis of a construct and instrument. Communications of the Association for Information Systems,
29(1), 67–96.
• Mulaik, S. A. (1998). Parsimony and model evaluation. The Journal of Experimental Education, 66(3), 266–273.
• Munck, I. M. E. (1979). Model building in comparative education: Applications of the LISREL method to cross-
national survey data. International Association for the Evaluation of Educational Achievement Monograph Series
No. 10. Stockholm: Almqvist & Wiksell.
193

References
• Novick, M. R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika,
32(1), 1–13.
• O’connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis
and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402. Springer.
• Patil, V. H., Singh, S. N., Mishra, S., & Todd Donavan, D. (2008). Efficient theory development and factor retention
criteria: Abandon the “eigenvalue greater than one” criterion. Journal of Business Research, 61(2), 162–170.
• Pearl, J. (1995). Causal diagrams for empirical research. Biometrika, 82(4), 669–688.
• Pearl, J. (1998). Graphs, causality, and structural equation models. Sociological Methods & Research, 27(2), 226–
284.
• Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. TRENDS in Cognitive Sciences, 6(10), 421–425.
• Russell, B. (1903). Principles of mathematics. Cambridge, UK: Cambridge University Press.
• Schmidt, F. L., & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance
testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no
significance tests? (pp. 37–64). Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc.
• Wright, S. (1923). The theory of path coefficients: A reply to Niles's criticism. Genetics, 8(3), 239–255.
• Yu, C.-Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous
outcomes. PhD, University of California, Los Angeles, Los Angeles.
194

Structural_equation_modeling_SEM_worksho (2).pptx

Recommended

Recommended

More Related Content

Similar to Structural_equation_modeling_SEM_worksho (2).pptx

Similar to Structural_equation_modeling_SEM_worksho (2).pptx (20)

Recently uploaded

Recently uploaded (20)

Structural_equation_modeling_SEM_worksho (2).pptx