Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Detecting local dependence in latent class models

974 views

Published on

Latent class (mixture) models are often used in wide range of fields. These models assume that the observed variables are independent given the latent classes: local independence. What if this assumption does not hold?

Published in: Science
  • Be the first to comment

Detecting local dependence in latent class models

  1. 1. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in latent class models Daniel Oberski Department of methodology and statistics (Based on joint work with Jeroen Vermunt and Geert Van Kollenburg) Detecting local dependence in latent class models Daniel Oberski
  2. 2. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; Detecting local dependence in latent class models Daniel Oberski
  3. 3. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; • If local dependence, severe bias can occur. Detecting local dependence in latent class models Daniel Oberski
  4. 4. Example LCA Local dependence BVR and MI EPC Conclusions References Latent class analysis • Latent class analysis (LCA) used for: model-based classification, clustering, latent structure analysis, estimating false positives and false negatives, (specificity and sensitivity) of error-prone variables; • assumes local independence; • If local dependence, severe bias can occur. Detecting local dependence in latent class models Daniel Oberski
  5. 5. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; Detecting local dependence in latent class models Daniel Oberski
  6. 6. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: Detecting local dependence in latent class models Daniel Oberski
  7. 7. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), Detecting local dependence in latent class models Daniel Oberski
  8. 8. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), Detecting local dependence in latent class models Daniel Oberski
  9. 9. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), • Expected parameter change (EPC). Detecting local dependence in latent class models Daniel Oberski
  10. 10. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependence in LCA • Local dependence in LCA leads to model misfit; • Usual reaction: increase number of classes; This talk: • Do not increase number of classes; • Model the local dependencies directly; • Use specifically tailored measures to detect which local dependencies should be modeled: • Bivariate residuals (BVR), • Score test or ``modification index'' (MI), • Expected parameter change (EPC). Detecting local dependence in latent class models Daniel Oberski
  11. 11. Example LCA Local dependence BVR and MI EPC Conclusions References ..1 Example LCA ..2 Local dependence ..3 BVR and MI ..4 EPC ..5 Conclusions Detecting local dependence in latent class models Daniel Oberski
  12. 12. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
  13. 13. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
  14. 14. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Here is an xray of a person who possibly has caries: Detecting local dependence in latent class models Daniel Oberski
  15. 15. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays Detecting local dependence in latent class models Daniel Oberski
  16. 16. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) Detecting local dependence in latent class models Daniel Oberski
  17. 17. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) Detecting local dependence in latent class models Daniel Oberski
  18. 18. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) y4 : Yes (1) Detecting local dependence in latent class models Daniel Oberski
  19. 19. Example LCA Local dependence BVR and MI EPC Conclusions References Example LCA: dentists rating caries in xrays y1 : Yes (1) y2 : No (0) y3 : No (0) y4 : Yes (1) Detecting local dependence in latent class models Daniel Oberski
  20. 20. Example LCA Local dependence BVR and MI EPC Conclusions References Actual dentists' ratings collected by Espeland & Handelman (1989) > dentists Var1 Var2 Var3 Var4 Var5 Observed 1 0 0 0 0 0 1880 2 0 0 0 0 1 789 3 0 0 0 1 0 43 4 0 0 0 1 1 75 5 0 0 1 0 0 23 6 0 0 1 0 1 63 7 0 0 1 1 0 8 8 0 0 1 1 1 22 9 0 1 0 0 0 188 10 0 1 0 0 1 191 11 0 1 0 1 0 17 ... etc. Detecting local dependence in latent class models Daniel Oberski
  21. 21. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists Detecting local dependence in latent class models Daniel Oberski
  22. 22. Example LCA Local dependence BVR and MI EPC Conclusions References Local independence latent class model Patternwise likelihood: Pr(Y = y) = ∑ t Pr(Y = y|ξ = t)Pr(ξ = t), (1) where the conditional probability of the response patterns is Pr(Y = y|ξ = t) = exp(Xθ) 1T exp(Xθ) (2) With X a design matrix containing: • Observed variables main effects (intercepts); • Latent class × observed variables interaction. Detecting local dependence in latent class models Daniel Oberski
  23. 23. Example LCA Local dependence BVR and MI EPC Conclusions References Estimation of local independence latent class model Log-likelihood ℓ(θ) = nT log Pr(Y = y), with vector n the observed frequency of each response pattern. Total sample size is N. Maximum likelihood estimates ˆθ = arg max θ∈Rq ℓ(θ) by expectation-maximization, quasi-Newton, or a combination. This gives expected frequencies ˆµ := N · Pr(Y = y|θ = ˆθ) Detecting local dependence in latent class models Daniel Oberski
  24. 24. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists' ratings Possible goals: Goal: How LCA is useful: * See how well dentists rate xrays ``sensitivity'' & ``specificity'' (conditional probabilities) from intercepts and slopes * Classify each photograph Pattern posteriors give probabilistic classification * Estimate ``true'' caries prevalence Latent class proportions Detecting local dependence in latent class models Daniel Oberski
  25. 25. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local independence model for dentists' ratings Possible goals: Goal: How LCA is useful: * See how well dentists rate xrays ``sensitivity'' & ``specificity'' (conditional probabilities) from intercepts and slopes * Classify each photograph Pattern posteriors give probabilistic classification * Estimate ``true'' caries prevalence Latent class proportions Local dependence biases all of these results. Detecting local dependence in latent class models Daniel Oberski
  26. 26. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Detecting local dependence in latent class models Daniel Oberski
  27. 27. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Model fit: L2 X2 df BIC AIC AIC3 CAIC 129.85 132.00 20 -35.37 89.85 69.85 -35.37 Detecting local dependence in latent class models Daniel Oberski
  28. 28. Example LCA Local dependence BVR and MI EPC Conclusions References Questions for you • What does the latent class variable represent? • And if the number of classes is increased? Model fit: L2 X2 df BIC AIC AIC3 CAIC 129.85 132.00 20 -35.37 89.85 69.85 -35.37 • The model does not fit. What could be a reason? • What should be done? Detecting local dependence in latent class models Daniel Oberski
  29. 29. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependence latent class models Detecting local dependence in latent class models Daniel Oberski
  30. 30. Example LCA Local dependence BVR and MI EPC Conclusions References 2-class local dependence model for dentists Detecting local dependence in latent class models Daniel Oberski
  31. 31. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependence latent class model Conditional probability of the response patterns still Pr(Y = y|ξ = t) = exp(Xθ) 1T exp(Xθ) But now X is a design matrix containing: • Observed variables main effects (intercepts); • Latent class × observed variables interaction; • (Some) observed variables × observed variables 2-way interactions; Detecting local dependence in latent class models Daniel Oberski
  32. 32. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? Detecting local dependence in latent class models Daniel Oberski
  33. 33. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? Detecting local dependence in latent class models Daniel Oberski
  34. 34. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? •• Goal: Examine fit of each pair of variables (dentist ratings) to the local independence assumption without fitting all possible alternative models. Detecting local dependence in latent class models Daniel Oberski
  35. 35. Example LCA Local dependence BVR and MI EPC Conclusions References • Can we find a locally dependent two-class model that fits? • Which local dependence parameters are needed? •• Goal: Examine fit of each pair of variables (dentist ratings) to the local independence assumption without fitting all possible alternative models. Detecting local dependence in latent class models Daniel Oberski
  36. 36. Example LCA Local dependence BVR and MI EPC Conclusions References Detecting local dependencies with the BVR and MI Detecting local dependence in latent class models Daniel Oberski
  37. 37. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 Detecting local dependence in latent class models Daniel Oberski
  38. 38. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 BVR1,3 = r11 2 ∑ k,l ˆµ−1 kl = (32.6)2 ∑ k,l ˆµ−1 kl ≈ 1063(0.0154) ≈ 16.3 Detecting local dependence in latent class models Daniel Oberski
  39. 39. Example LCA Local dependence BVR and MI EPC Conclusions References Fit of two-way cross-table between dentists 1 and 3 Observed No Yes No 3250 280 Yes 123 216 Expected No Yes No 3217 313 Yes 156 183 Bivariate residuals No Yes No 32.6 -32.6 Yes -32.6 32.6 BVR1,3 = r11 2 ∑ k,l ˆµ−1 kl = (32.6)2 ∑ k,l ˆµ−1 kl ≈ 1063(0.0154) ≈ 16.3 MI1,3 = r11 2 Var(r11)−1 = (32.6)2 /(31.3) ≈ 1063(0.0320) ≈ 34.0 Detecting local dependence in latent class models Daniel Oberski
  40. 40. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependencies between five dentists' x-ray ratings for caries. Dentist dependence MI BVR 1 ↔ 2 3.1 1.4 1 ↔ 3 34.0 ** 16.3 1 ↔ 4 13.1 ** 7.7 1 ↔ 5 2.7 0.8 2 ↔ 3 6.8 * 1.7 2 ↔ 4 1.8 0.6 2 ↔ 5 16.4 ** 4.7 3 ↔ 4 2.7 1.0 3 ↔ 5 5.1 * 0.8 4 ↔ 5 3.5 1.0 Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple testing, * without correction Detecting local dependence in latent class models Daniel Oberski
  41. 41. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson "chi-square" residual in cross-table of a pair of observed variables: ∑ (observed - expected)2/expected • Surprise! Not a chi square statistic! Detecting local dependence in latent class models Daniel Oberski
  42. 42. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson "chi-square" residual in cross-table of a pair of observed variables: ∑ (observed - expected)2/expected • Surprise! Not a chi square statistic! Detecting local dependence in latent class models Daniel Oberski
  43. 43. Example LCA Local dependence BVR and MI EPC Conclusions References q q q q q q q qq q qq qqqqqqqqqqqqqqqqqq q q q q qq q q q q q q 0 20 40 60 80 100 0 20 40 60 80 100 MI equals chi−square improvement... χ2 MI q q q q q q q qq q qq qqqqqqqqqqqqqqqqqq q q q q qq q q qq q q 0 20 40 60 80 100 0 20 40 60 80 100 ... BVR does not. χ2 BVR Detecting local dependence in latent class models Daniel Oberski
  44. 44. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) Detecting local dependence in latent class models Daniel Oberski
  45. 45. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) • Under the null hypothesis, chi-square distributed (1 df) Detecting local dependence in latent class models Daniel Oberski
  46. 46. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI ``Bivariate residual'' (BVR): • Pearson residual in cross-table of a pair of observed variables: (observed - expected)2/expected; • Not a chi square statistic. ``Modification index'' (MI): • ``Score test'' (``Lagrange multiplier'' test) for introducing a local dependency between a pair of variables; • Turns out to use the same residual as the BVR, but with the correct variance! (thanks Jeroen) • Under the null hypothesis, chi-square distributed (1 df) Detecting local dependence in latent class models Daniel Oberski
  47. 47. Example LCA Local dependence BVR and MI EPC Conclusions References BVR not even close to chi-square distributed, MI is. Monte Carlo simulation under the null hypothesis Condition α for nominal 5% Empirical distribution BVR MI BVR λ n Naive Boot MI Mean Var Mean Var 0.5 200 0.000 0.050 0.051 0.97 1.7 0.33 0.2 0.5 500 0.000 0.020 0.050 1.06 2.3 0.36 0.2 0.5 1000 0.000 0.060 0.065 0.96 1.9 0.33 0.2 0.5 5000 0.000 0.085 0.055 0.97 2.0 0.34 0.2 0.8 200 0.000 0.065 0.040 1.04 1.7 0.25 0.1 0.8 500 0.000 0.070 0.060 1.05 2.0 0.25 0.1 0.8 1000 0.000 0.060 0.090 1.22 2.6 0.30 0.2 0.8 5000 0.000 0.035 0.060 1.16 3.1 0.28 0.2 Should be: 0.050 0.050 0.050 1.00 2.0 1.00 2.0 Detecting local dependence in latent class models Daniel Oberski
  48. 48. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: * See paper. Detecting local dependence in latent class models Daniel Oberski
  49. 49. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; * See paper. Detecting local dependence in latent class models Daniel Oberski
  50. 50. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. * See paper. Detecting local dependence in latent class models Daniel Oberski
  51. 51. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: * See paper. Detecting local dependence in latent class models Daniel Oberski
  52. 52. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; * See paper. Detecting local dependence in latent class models Daniel Oberski
  53. 53. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. * See paper. Detecting local dependence in latent class models Daniel Oberski
  54. 54. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. * See paper. Detecting local dependence in latent class models Daniel Oberski
  55. 55. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. •• Can bootstrap BVR values, leading to performance similar to, though slightly below, MI* * See paper. Detecting local dependence in latent class models Daniel Oberski
  56. 56. Example LCA Local dependence BVR and MI EPC Conclusions References BVR and MI: main findings • Many (most?) published applied studies using BVR pretend it is a chi-square statistic*; • BUT, if pretend BVR is chi-square: • BVR does not provide nominal α levels; • BVR power to detect local dependencies miserably low*. • Referring MI to chi-square: • MI provides approximately nominal α levels; • Power of MI is adequate except for very small dependencies (±0.05)*. •• Can bootstrap BVR values, leading to performance similar to, though slightly below, MI* * See paper. Detecting local dependence in latent class models Daniel Oberski
  57. 57. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size of local dependencies with the EPC Detecting local dependence in latent class models Daniel Oberski
  58. 58. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; Detecting local dependence in latent class models Daniel Oberski
  59. 59. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: Detecting local dependence in latent class models Daniel Oberski
  60. 60. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; Detecting local dependence in latent class models Daniel Oberski
  61. 61. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; Detecting local dependence in latent class models Daniel Oberski
  62. 62. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; Detecting local dependence in latent class models Daniel Oberski
  63. 63. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); Detecting local dependence in latent class models Daniel Oberski
  64. 64. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; Detecting local dependence in latent class models Daniel Oberski
  65. 65. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; • Regression coefficients of covariates if present. Detecting local dependence in latent class models Daniel Oberski
  66. 66. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Modification index'' (MI): • Statistical ``score'' test of the hypothesis that the local dependence between two variables, say ψ, equals zero. • ψ corresponds to the observed variable × observed variable 2-way interaction in conditional prob. model; • Only statistical significance → MI depends on: • Sample size; • Size of the loadings of the two variables; • Size of the loadings of the other variables; • Latent variable intercept(s); • Observed variable intercepts; • Regression coefficients of covariates if present. Detecting local dependence in latent class models Daniel Oberski
  67. 67. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; Detecting local dependence in latent class models Daniel Oberski
  68. 68. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; • Assesses substantive rather than statistical significance. Detecting local dependence in latent class models Daniel Oberski
  69. 69. Example LCA Local dependence BVR and MI EPC Conclusions References Assessing substantive size besides statistical siginificance ``Expected parameter change'' (EPC): • Estimate of ψ, the strength of the local dependence between two variables; • No need to fit alternative model; • Assesses substantive rather than statistical significance. Detecting local dependence in latent class models Daniel Oberski
  70. 70. Example LCA Local dependence BVR and MI EPC Conclusions References Local dependencies between five dentists' x-ray ratings for caries. Dentist dependence EPC MI BVR 1 ↔ 2 -0.081 3.1 1.4 1 ↔ 3 -0.261 34.0 ** 16.3 1 ↔ 4 -0.146 13.1 ** 7.7 1 ↔ 5 -0.117 2.7 0.8 2 ↔ 3 -0.140 6.8 * 1.7 2 ↔ 4 -0.058 1.8 0.6 2 ↔ 5 -0.157 16.4 ** 4.7 3 ↔ 4 0.074 2.7 1.0 3 ↔ 5 -0.191 5.1 * 0.8 4 ↔ 5 -0.104 3.5 1.0 Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple testing, * without correction Detecting local dependence in latent class models Daniel Oberski
  71. 71. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. Detecting local dependence in latent class models Daniel Oberski
  72. 72. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. Detecting local dependence in latent class models Daniel Oberski
  73. 73. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) Detecting local dependence in latent class models Daniel Oberski
  74. 74. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) Detecting local dependence in latent class models Daniel Oberski
  75. 75. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) Detecting local dependence in latent class models Daniel Oberski
  76. 76. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) • Best two-class model in lit. (Qu et al. 1996): BIC −83.4 Detecting local dependence in latent class models Daniel Oberski
  77. 77. Example LCA Local dependence BVR and MI EPC Conclusions References Based on EPC and MI, • keep two-classes, but... • ...free five out of ten possible bivariate local dependencies. •• 15 degrees of freedom (was 20) • L2 = 28.4 (p = 0.07) (was 129.9) • BIC is −95.5 (was −35.37) • Best two-class model in lit. (Qu et al. 1996): BIC −83.4 Detecting local dependence in latent class models Daniel Oberski
  78. 78. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions Detecting local dependence in latent class models Daniel Oberski
  79. 79. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; Detecting local dependence in latent class models Daniel Oberski
  80. 80. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. Detecting local dependence in latent class models Daniel Oberski
  81. 81. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. Detecting local dependence in latent class models Daniel Oberski
  82. 82. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: Detecting local dependence in latent class models Daniel Oberski
  83. 83. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; Detecting local dependence in latent class models Daniel Oberski
  84. 84. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. Detecting local dependence in latent class models Daniel Oberski
  85. 85. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. Detecting local dependence in latent class models Daniel Oberski
  86. 86. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. Detecting local dependence in latent class models Daniel Oberski
  87. 87. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. •• Dentist example: obtained more parsimonious and easy-to-interpret two-class model than current literature. Detecting local dependence in latent class models Daniel Oberski
  88. 88. Example LCA Local dependence BVR and MI EPC Conclusions References Conclusions • Local dependence is a serious problem in LCA; • Sometimes modeling dependence directly is preferable to increasing no. classes; • Need to monitor local dependence. •• Introduced MI and BVR, tests for local dependence: • MI works well as chi-square statistic; • BVR does not, but can bootstrap p-values. • Introduced EPC: local dependence substantive size. •• Dentist example: obtained more parsimonious and easy-to-interpret two-class model than current literature. Detecting local dependence in latent class models Daniel Oberski
  89. 89. Example LCA Local dependence BVR and MI EPC Conclusions References • Class-specific local dependence; • Multivariate MI, EPC, (polytomous items); • Effect of freeing local dependencies on params of interest; • Problem of parameter dependence; • Problem of multiple comparisons; • (Post-hoc) power of MI (Satorra); • MI for other params than locdeps (e.g. item bias). Detecting local dependence in latent class models Daniel Oberski
  90. 90. Example LCA Local dependence BVR and MI EPC Conclusions References Thank you for your attention! Daniel Oberski doberski@uvt.nl Working papers on this topic: Oberski, D., Van Kollenburg, G., and Vermunt, J. (submitted). A Monte Carlo evaluation of three methods to detect local dependence in binary data latent class models. Oberski, D. and Vermunt, J. (submitted). The Expected Parameter Change (EPC) for local dependence assessment in binary data latent class models. Oberski, D. (submitted). Change in SEM parameters of interest as a criterion for partial measurement invariance: The EPC-interest. Detecting local dependence in latent class models Daniel Oberski
  91. 91. Appendix Appendix Detecting local dependence in latent class models Daniel Oberski
  92. 92. Appendix 0.5 0.8 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 −0.4−0.2−0.050.050.20.4 200 500 1000 5000 200 500 1000 5000 Log(sample size) Power p−value type BVR BVR_bootstrap MI Detecting local dependence in latent class models Daniel Oberski
  93. 93. Appendix Convergence to noncentral chi-square in worst condition qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q qq q q Sample size: 200 Q−Q plot, noncentral χ1 2 (4.7) Theoretical quantiles MI 0 10 20 30 40 0 5 10 15 20 25 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q q q Sample size: 500 Q−Q plot, noncentral χ1 2 (11.8) Theoretical quantiles MI 0 10 20 30 40 50 60 70 0 10 20 30 40 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q Sample size: 1000 Q−Q plot, noncentral χ1 2 (23.7) Theoretical quantiles MI 0 20 40 60 80 10 20 30 40 50 60 q qqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqq qqqqq qq q Sample size: 5000 Q−Q plot, noncentral χ1 2 (118) Theoretical quantiles MI 50 100 150 200 80 120 160 Detecting local dependence in latent class models Daniel Oberski
  94. 94. Appendix Local dependencies between indicators of Hispanic ethnicity in the U.S. Census Local dependence EPCL TL EPCgs TGS Ancestry-re ↔ Language-in 0.92 5.0 1.45 7.9 0. Ancestry-re ↔ Origin-in -0.76 2.5 -1.23 4.1 -0. Ancestry-re ↔ Origin-re 2.94 45.6 1.32 20.5 1. Language-in ↔ Origin-in 4.14 97.1 1.59 37.2 3. Language-in ↔ Origin-re -1.08 7.9 -1.76 12.8 1. Origin-in ↔ Origin-re 1.10 6.1 2.20 12.2 0. Detecting local dependence in latent class models Daniel Oberski

×