Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Complex sampling in latent variable... by Daniel Oberski 999 views
- ESRA2015 course: Latent Class Analy... by Daniel Oberski 3378 views
- Identifying “mischievous” responder... by Center for Innova... 133 views
- A measure to evaluate latent variab... by Daniel Oberski 724 views
- How good are administrative registe... by Daniel Oberski 576 views
- Multidirectional survey measurement... by Daniel Oberski 462 views

974 views

Published on

Published in:
Science

No Downloads

Total views

974

On SlideShare

0

From Embeds

0

Number of Embeds

13

Shares

0

Downloads

25

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in latent class models Daniel Oberski Department of methodology and statistics (Based on joint work with Jeroen Vermunt and Geert Van Kollenburg) Detecting local dependence in latent class models Daniel Oberski
- 2. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; Detecting local dependence in latent class models Daniel Oberski
- 3. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; • If local dependence, severe bias can occur. Detecting local dependence in latent class models Daniel Oberski
- 4. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; • If local dependence, severe bias can occur. Detecting local dependence in latent class models Daniel Oberski
- 5. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; Detecting local dependence in latent class models Daniel Oberski
- 6. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: Detecting local dependence in latent class models Daniel Oberski
- 7. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), Detecting local dependence in latent class models Daniel Oberski
- 8. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), Detecting local dependence in latent class models Daniel Oberski
- 9. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), • Expected parameter change (EPC). Detecting local dependence in latent class models Daniel Oberski
- 10. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), • Expected parameter change (EPC). Detecting local dependence in latent class models Daniel Oberski
- 11. Example LCA Local dependence BVR and MI EPC Conclusions References ..1 Example LCA ..2 Local dependence ..3 BVR and MI ..4 EPC ..5 Conclusions Detecting local dependence in latent class models Daniel Oberski
- 12. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
- 13. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
- 14. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Here is an xray of a person who possibly has caries: Detecting local dependence in latent class models Daniel Oberski
- 15. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
- 16. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) Detecting local dependence in latent class models Daniel Oberski
- 17. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) Detecting local dependence in latent class models Daniel Oberski
- 18. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) y4 : Yes (1) Detecting local dependence in latent class models Daniel Oberski
- 19. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) y4 : Yes (1) Detecting local dependence in latent class models Daniel Oberski
- 20. Example LCA Local dependence BVR and MI EPC Conclusions References Actual dentists' ratings collected by Espeland & Handelman (1989) > dentists Var1 Var2 Var3 Var4 Var5 Observed 1 0 0 0 0 0 1880 2 0 0 0 0 1 789 3 0 0 0 1 0 43 4 0 0 0 1 1 75 5 0 0 1 0 0 23 6 0 0 1 0 1 63 7 0 0 1 1 0 8 8 0 0 1 1 1 22 9 0 1 0 0 0 188 10 0 1 0 0 1 191 11 0 1 0 1 0 17 ... etc. Detecting local dependence in latent class models Daniel Oberski
- 21. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists Detecting local dependence in latent class models Daniel Oberski
- 22. Example LCA Local dependence BVR and MI EPC Conclusions References Local independence latent class model Patternwise likelihood: Pr(Y = y) = ∑ t Pr(Y = y|ξ = t)Pr(ξ = t), (1) where the conditional probability of the response patterns is Pr(Y = y|ξ = t) = exp(Xθ) 1T exp(Xθ) (2) With X a design matrix containing: • Observed variables main effects (intercepts); • Latent class × observed variables interaction. Detecting local dependence in latent class models Daniel Oberski
- 23. Example LCA Local dependence BVR and MI EPC Conclusions References Estimation of local independence latent class model Log-likelihood ℓ(θ) = nT log Pr(Y = y), with vector n the observed frequency of each response pattern. Total sample size is N. Maximum likelihood estimates ˆθ = arg max θ∈Rq ℓ(θ) by expectation-maximization, quasi-Newton, or a combination. This gives expected frequencies ˆµ := N · Pr(Y = y|θ = ˆθ) Detecting local dependence in latent class models Daniel Oberski
- 24. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists' ratings Possible goals: Goal: How LCA is useful: * See how well dentists rate xrays ``sensitivity'' & ``specificity'' (conditional probabilities) from intercepts and slopes * Classify each photograph Pattern posteriors give probabilistic classification * Estimate ``true'' caries prevalence Latent class proportions Detecting local dependence in latent class models Daniel Oberski
- 25. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists' ratings Possible goals: Goal: How LCA is useful: * See how well dentists rate xrays ``sensitivity'' & ``specificity'' (conditional probabilities) from intercepts and slopes * Classify each photograph Pattern posteriors give probabilistic classification * Estimate ``true'' caries prevalence Latent class proportions Local dependence biases all of these results. Detecting local dependence in latent class models Daniel Oberski
- 26. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Detecting local dependence in latent class models Daniel Oberski
- 27. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Model fit: L2 X2 df BIC AIC AIC3 CAIC 129.85 132.00 20 -35.37 89.85 69.85 -35.37 Detecting local dependence in latent class models Daniel Oberski
- 28. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Model fit: L2 X2 df BIC AIC AIC3 CAIC 129.85 132.00 20 -35.37 89.85 69.85 -35.37 • The model does not fit. What could be a reason? • What should be done? Detecting local dependence in latent class models Daniel Oberski
- 29. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependence latent class models Detecting local dependence in latent class models Daniel Oberski
- 30. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local dependence model for dentists Detecting local dependence in latent class models Daniel Oberski
- 31. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependence latent class model Conditional probability of the response patterns still Pr(Y = y|ξ = t) = exp(Xθ) 1T exp(Xθ) But now X is a design matrix containing: • Observed variables main effects (intercepts); • Latent class × observed variables interaction; • (Some) observed variables × observed variables 2-way interactions; Detecting local dependence in latent class models Daniel Oberski
- 32. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? Detecting local dependence in latent class models Daniel Oberski
- 33. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? Detecting local dependence in latent class models Daniel Oberski
- 34. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? •• Goal: Examine fit of each pair of variables (dentist ratings) to the local independence assumption without fitting all possible alternative models. Detecting local dependence in latent class models Daniel Oberski
- 35. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? •• Goal: Examine fit of each pair of variables (dentist ratings) to the local independence assumption without fitting all possible alternative models. Detecting local dependence in latent class models Daniel Oberski
- 36. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependencies with the BVR and MI Detecting local dependence in latent class models Daniel Oberski
- 37. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 Detecting local dependence in latent class models Daniel Oberski
- 38. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 BVR1,3 = r11 2 ∑ k,l ˆµ−1 kl = (32.6)2 ∑ k,l ˆµ−1 kl ≈ 1063(0.0154) ≈ 16.3 Detecting local dependence in latent class models Daniel Oberski
- 39. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 BVR1,3 = r11 2 ∑ k,l ˆµ−1 kl = (32.6)2 ∑ k,l ˆµ−1 kl ≈ 1063(0.0154) ≈ 16.3 MI1,3 = r11 2 Var(r11)−1 = (32.6)2 /(31.3) ≈ 1063(0.0320) ≈ 34.0 Detecting local dependence in latent class models Daniel Oberski
- 40. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependencies between five dentists' x-ray ratings for caries. Dentist dependence MI BVR 1 ↔ 2 3.1 1.4 1 ↔ 3 34.0 ** 16.3 1 ↔ 4 13.1 ** 7.7 1 ↔ 5 2.7 0.8 2 ↔ 3 6.8 * 1.7 2 ↔ 4 1.8 0.6 2 ↔ 5 16.4 ** 4.7 3 ↔ 4 2.7 1.0 3 ↔ 5 5.1 * 0.8 4 ↔ 5 3.5 1.0 Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple testing, * without correction Detecting local dependence in latent class models Daniel Oberski
- 41. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson "chi-square" residual in cross-table of a pair of observed variables: ∑ (observed - expected)2/expected • Surprise! Not a chi square statistic! Detecting local dependence in latent class models Daniel Oberski
- 42. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson "chi-square" residual in cross-table of a pair of observed variables: ∑ (observed - expected)2/expected • Surprise! Not a chi square statistic! Detecting local dependence in latent class models Daniel Oberski
- 43. Example LCA Local dependence BVR and MI EPC Conclusions References q q q q q q q qq q qq qqqqqqqqqqqqqqqqqq q q q q qq q q q q q q 0 20 40 60 80 100 0 20 40 60 80 100 MI equals chi−square improvement... χ2 MI q q q q q q q qq q qq qqqqqqqqqqqqqqqqqq q q q q qq q q qq q q 0 20 40 60 80 100 0 20 40 60 80 100 ... BVR does not. χ2 BVR Detecting local dependence in latent class models Daniel Oberski
- 44. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) Detecting local dependence in latent class models Daniel Oberski
- 45. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) • Under the null hypothesis, chi-square distributed (1 df) Detecting local dependence in latent class models Daniel Oberski
- 46. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) • Under the null hypothesis, chi-square distributed (1 df) Detecting local dependence in latent class models Daniel Oberski
- 47. Example LCA Local dependence BVR and MI EPC Conclusions References BVR not even close to chi-square distributed, MI is. Monte Carlo simulation under the null hypothesis Condition α for nominal 5% Empirical distribution BVR MI BVR λ n Naive Boot MI Mean Var Mean Var 0.5 200 0.000 0.050 0.051 0.97 1.7 0.33 0.2 0.5 500 0.000 0.020 0.050 1.06 2.3 0.36 0.2 0.5 1000 0.000 0.060 0.065 0.96 1.9 0.33 0.2 0.5 5000 0.000 0.085 0.055 0.97 2.0 0.34 0.2 0.8 200 0.000 0.065 0.040 1.04 1.7 0.25 0.1 0.8 500 0.000 0.070 0.060 1.05 2.0 0.25 0.1 0.8 1000 0.000 0.060 0.090 1.22 2.6 0.30 0.2 0.8 5000 0.000 0.035 0.060 1.16 3.1 0.28 0.2 Should be: 0.050 0.050 0.050 1.00 2.0 1.00 2.0 Detecting local dependence in latent class models Daniel Oberski
- 48. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: * See paper. Detecting local dependence in latent class models Daniel Oberski
- 49. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; * See paper. Detecting local dependence in latent class models Daniel Oberski
- 50. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. * See paper. Detecting local dependence in latent class models Daniel Oberski
- 51. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: * See paper. Detecting local dependence in latent class models Daniel Oberski
- 52. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; * See paper. Detecting local dependence in latent class models Daniel Oberski
- 53. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. * See paper. Detecting local dependence in latent class models Daniel Oberski
- 54. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. * See paper. Detecting local dependence in latent class models Daniel Oberski
- 55. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. •• Can bootstrap BVR values, leading to performance similar to, though slightly below, MI* * See paper. Detecting local dependence in latent class models Daniel Oberski
- 56. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. •• Can bootstrap BVR values, leading to performance similar to, though slightly below, MI* * See paper. Detecting local dependence in latent class models Daniel Oberski
- 57. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size of local dependencies with the EPC Detecting local dependence in latent class models Daniel Oberski
- 58. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; Detecting local dependence in latent class models Daniel Oberski
- 59. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: Detecting local dependence in latent class models Daniel Oberski
- 60. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; Detecting local dependence in latent class models Daniel Oberski
- 61. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; Detecting local dependence in latent class models Daniel Oberski
- 62. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; Detecting local dependence in latent class models Daniel Oberski
- 63. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); Detecting local dependence in latent class models Daniel Oberski
- 64. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; Detecting local dependence in latent class models Daniel Oberski
- 65. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; • Regression coefficients of covariates if present. Detecting local dependence in latent class models Daniel Oberski
- 66. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; • Regression coefficients of covariates if present. Detecting local dependence in latent class models Daniel Oberski
- 67. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; Detecting local dependence in latent class models Daniel Oberski
- 68. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; • Assesses substantive rather than statistical significance. Detecting local dependence in latent class models Daniel Oberski
- 69. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; • Assesses substantive rather than statistical significance. Detecting local dependence in latent class models Daniel Oberski
- 70. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependencies between five dentists' x-ray ratings for caries. Dentist dependence EPC MI BVR 1 ↔ 2 -0.081 3.1 1.4 1 ↔ 3 -0.261 34.0 ** 16.3 1 ↔ 4 -0.146 13.1 ** 7.7 1 ↔ 5 -0.117 2.7 0.8 2 ↔ 3 -0.140 6.8 * 1.7 2 ↔ 4 -0.058 1.8 0.6 2 ↔ 5 -0.157 16.4 ** 4.7 3 ↔ 4 0.074 2.7 1.0 3 ↔ 5 -0.191 5.1 * 0.8 4 ↔ 5 -0.104 3.5 1.0 Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple testing, * without correction Detecting local dependence in latent class models Daniel Oberski
- 71. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. Detecting local dependence in latent class models Daniel Oberski
- 72. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. Detecting local dependence in latent class models Daniel Oberski
- 73. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) Detecting local dependence in latent class models Daniel Oberski
- 74. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) Detecting local dependence in latent class models Daniel Oberski
- 75. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) Detecting local dependence in latent class models Daniel Oberski
- 76. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) • Best two-class model in lit. (Qu et al. 1996): BIC −83.4 Detecting local dependence in latent class models Daniel Oberski
- 77. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) • Best two-class model in lit. (Qu et al. 1996): BIC −83.4 Detecting local dependence in latent class models Daniel Oberski
- 78. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions Detecting local dependence in latent class models Daniel Oberski
- 79. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; Detecting local dependence in latent class models Daniel Oberski
- 80. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. Detecting local dependence in latent class models Daniel Oberski
- 81. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. Detecting local dependence in latent class models Daniel Oberski
- 82. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: Detecting local dependence in latent class models Daniel Oberski
- 83. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; Detecting local dependence in latent class models Daniel Oberski
- 84. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. Detecting local dependence in latent class models Daniel Oberski
- 85. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. Detecting local dependence in latent class models Daniel Oberski
- 86. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. Detecting local dependence in latent class models Daniel Oberski
- 87. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. •• Dentist example: obtained more parsimonious and easy-to-interpret two-class model than current literature. Detecting local dependence in latent class models Daniel Oberski
- 88. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. •• Dentist example: obtained more parsimonious and easy-to-interpret two-class model than current literature. Detecting local dependence in latent class models Daniel Oberski
- 89. Example LCA Local dependence BVR and MI EPC Conclusions References • Class-specific local dependence; • Multivariate MI, EPC, (polytomous items); • Effect of freeing local dependencies on params of interest; • Problem of parameter dependence; • Problem of multiple comparisons; • (Post-hoc) power of MI (Satorra); • MI for other params than locdeps (e.g. item bias). Detecting local dependence in latent class models Daniel Oberski
- 90. Example LCA Local dependence BVR and MI EPC Conclusions References Thank you for your attention! Daniel Oberski doberski@uvt.nl Working papers on this topic: Oberski, D., Van Kollenburg, G., and Vermunt, J. (submitted). A Monte Carlo evaluation of three methods to detect local dependence in binary data latent class models. Oberski, D. and Vermunt, J. (submitted). The Expected Parameter Change (EPC) for local dependence assessment in binary data latent class models. Oberski, D. (submitted). Change in SEM parameters of interest as a criterion for partial measurement invariance: The EPC-interest. Detecting local dependence in latent class models Daniel Oberski
- 91. Appendix Appendix Detecting local dependence in latent class models Daniel Oberski
- 92. Appendix 0.5 0.8 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 −0.4−0.2−0.050.050.20.4 200 500 1000 5000 200 500 1000 5000 Log(sample size) Power p−value type BVR BVR_bootstrap MI Detecting local dependence in latent class models Daniel Oberski
- 93. Appendix Convergence to noncentral chi-square in worst condition qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q qq q q Sample size: 200 Q−Q plot, noncentral χ1 2 (4.7) Theoretical quantiles MI 0 10 20 30 40 0 5 10 15 20 25 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q q q Sample size: 500 Q−Q plot, noncentral χ1 2 (11.8) Theoretical quantiles MI 0 10 20 30 40 50 60 70 0 10 20 30 40 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q Sample size: 1000 Q−Q plot, noncentral χ1 2 (23.7) Theoretical quantiles MI 0 20 40 60 80 10 20 30 40 50 60 q qqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqq qqqqq qq q Sample size: 5000 Q−Q plot, noncentral χ1 2 (118) Theoretical quantiles MI 50 100 150 200 80 120 160 Detecting local dependence in latent class models Daniel Oberski
- 94. Appendix Local dependencies between indicators of Hispanic ethnicity in the U.S. Census Local dependence EPCL TL EPCgs TGS Ancestry-re ↔ Language-in 0.92 5.0 1.45 7.9 0. Ancestry-re ↔ Origin-in -0.76 2.5 -1.23 4.1 -0. Ancestry-re ↔ Origin-re 2.94 45.6 1.32 20.5 1. Language-in ↔ Origin-in 4.14 97.1 1.59 37.2 3. Language-in ↔ Origin-re -1.08 7.9 -1.76 12.8 1. Origin-in ↔ Origin-re 1.10 6.1 2.20 12.2 0. Detecting local dependence in latent class models Daniel Oberski

No public clipboards found for this slide

Be the first to comment