Fitting and understanding Multilevel Models-Andrew Gelman
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Fitting and understanding Multilevel Models-Andrew Gelman

on

  • 554 views

Fitting and understanding Multilevel Models-Andrew Gelman

Fitting and understanding Multilevel Models-Andrew Gelman

Statistics

Views

Total Views
554
Views on SlideShare
554
Embed Views
0

Actions

Likes
0
Downloads
16
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Fitting and understanding Multilevel Models-Andrew Gelman Presentation Transcript

  • 1. EffectivenessUbiquityWay of lifeFitting and understanding multilevel (hierarchical)modelsAndrew GelmanDepartment of Statistics and Department of Political ScienceColumbia University8 December 2004Andrew Gelman Fitting and understanding multilevel models
  • 2. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 3. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 4. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 5. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 6. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 7. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 8. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 9. EffectivenessUbiquityWay of lifeMaking more use of existing informationThe problem: not enough data to estimate effects withconfidenceThe solution: make your studies broader and deeperBroader: extend to other countries, other years, otheroutcomes, . . .Deeper: inferences for individual states, demographicsubgroups, components of outcomes, . . .The solution: multilevel modelingRegression with coefficients grouped into batchesNo such thing as “too many predictors”Andrew Gelman Fitting and understanding multilevel models
  • 10. EffectivenessUbiquityWay of lifeFitting and understanding multilevel modelsThe effectiveness of multilevel modelsMultilevel models in unexpected placesMultilevel models as a way of lifecollaborators:Iain Pardoe, Dept of Decision Sciences, University of OregonDavid Park, Dept of Political Science, Washington UniversityJoe Bafumi, Dept of Political Science, Columbia UniversityBoris Shor, Dept of Political Science, Columbia UniversityNoah Kaplan, Dept of Political Science, University of HoustonShouhao Zhao, Dept of Statistics, Columbia UniversityZaiying Huang, Circulation, New York TimesAndrew Gelman Fitting and understanding multilevel models
  • 11. EffectivenessUbiquityWay of lifeFitting and understanding multilevel modelsThe effectiveness of multilevel modelsMultilevel models in unexpected placesMultilevel models as a way of lifecollaborators:Iain Pardoe, Dept of Decision Sciences, University of OregonDavid Park, Dept of Political Science, Washington UniversityJoe Bafumi, Dept of Political Science, Columbia UniversityBoris Shor, Dept of Political Science, Columbia UniversityNoah Kaplan, Dept of Political Science, University of HoustonShouhao Zhao, Dept of Statistics, Columbia UniversityZaiying Huang, Circulation, New York TimesAndrew Gelman Fitting and understanding multilevel models
  • 12. EffectivenessUbiquityWay of lifeFitting and understanding multilevel modelsThe effectiveness of multilevel modelsMultilevel models in unexpected placesMultilevel models as a way of lifecollaborators:Iain Pardoe, Dept of Decision Sciences, University of OregonDavid Park, Dept of Political Science, Washington UniversityJoe Bafumi, Dept of Political Science, Columbia UniversityBoris Shor, Dept of Political Science, Columbia UniversityNoah Kaplan, Dept of Political Science, University of HoustonShouhao Zhao, Dept of Statistics, Columbia UniversityZaiying Huang, Circulation, New York TimesAndrew Gelman Fitting and understanding multilevel models
  • 13. EffectivenessUbiquityWay of lifeFitting and understanding multilevel modelsThe effectiveness of multilevel modelsMultilevel models in unexpected placesMultilevel models as a way of lifecollaborators:Iain Pardoe, Dept of Decision Sciences, University of OregonDavid Park, Dept of Political Science, Washington UniversityJoe Bafumi, Dept of Political Science, Columbia UniversityBoris Shor, Dept of Political Science, Columbia UniversityNoah Kaplan, Dept of Political Science, University of HoustonShouhao Zhao, Dept of Statistics, Columbia UniversityZaiying Huang, Circulation, New York TimesAndrew Gelman Fitting and understanding multilevel models
  • 14. EffectivenessUbiquityWay of lifeFitting and understanding multilevel modelsThe effectiveness of multilevel modelsMultilevel models in unexpected placesMultilevel models as a way of lifecollaborators:Iain Pardoe, Dept of Decision Sciences, University of OregonDavid Park, Dept of Political Science, Washington UniversityJoe Bafumi, Dept of Political Science, Columbia UniversityBoris Shor, Dept of Political Science, Columbia UniversityNoah Kaplan, Dept of Political Science, University of HoustonShouhao Zhao, Dept of Statistics, Columbia UniversityZaiying Huang, Circulation, New York TimesAndrew Gelman Fitting and understanding multilevel models
  • 15. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 16. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 17. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 18. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 19. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 20. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 21. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 22. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 23. EffectivenessUbiquityWay of lifeOutline of talkThe effectiveness of multilevel modelsState-level opinions from national polls(crossed multilevel modeling and poststratification)Multilevel models in unexpected placesEstimating incumbency advantage and its variationBefore-after studiesMultilevel models as a way of lifeBuilding and fitting modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 24. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationNational opinion trends1940 1950 1960 1970 1980 1990 200050607080YearPercentagesupportforthedeathpenaltyAndrew Gelman Fitting and understanding multilevel models
  • 25. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 26. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 27. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 28. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 29. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 30. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationState-level opinion trendsGoal: estimating time series within each stateOne poll at a time: small-area estimationIt works! Validated for pre-election pollsCombining surveys: model for parallel time seriesMultilevel modeling + poststratificationPoststratification cells: sex × ethnicity × age × education ×stateAndrew Gelman Fitting and understanding multilevel models
  • 31. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 32. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 33. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 34. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 35. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 36. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationMultilevel modeling of opinionsLogistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsBayesian inference, summarize by posterior simulations of β:Simulation θ1 · · · θ751 ** · · · **............1000 ** · · · **Andrew Gelman Fitting and understanding multilevel models
  • 37. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 38. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 39. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 40. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 41. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 42. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 43. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationInterlude: why “multilevel” = “hierarchical”Logistic regression: Pr(yi = 1) = logit−1((Xβ)i )X includes demographic and geographic predictorsGroup-level model for the 16 age × education predictorsGroup-level model for the 50 state predictorsCrossed (nonnested) structure of age, education, stateSeveral overlapping “hierarchies”Andrew Gelman Fitting and understanding multilevel models
  • 44. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 45. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 46. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 47. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 48. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 49. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationPoststratification to estimate state opinionsImplied inference for θj = logit−1(Xβ) in each of 3264 cells j(e.g., black female, age 18–29, college graduate,Massachusetts)PoststratificationWithin each state s, average over 64 cells:j∈s Nj θj j∈s NjNj = population in cell j (from Census)1000 simulation draws propagate to uncertainty for each θjAndrew Gelman Fitting and understanding multilevel models
  • 50. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 51. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 52. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 53. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 54. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 55. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 56. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 57. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 58. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 59. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationCBS/New York Times pre-election polls from 1988Validation study: fit model on poll data and compare toelection resultsCompeting estimates:No pooling: separate estimate within each stateComplete pooling: no state predictorsHierarchical model and poststratifyMean absolute state errors:No pooling: 10.4%Complete pooling: 5.4%Hierarchical model with poststratification: 4.5%Andrew Gelman Fitting and understanding multilevel models
  • 60. EffectivenessUbiquityWay of lifeState-level opinions from national pollsPoststratificationValidationValidation study: comparison of state errors1988 election outcome vs. poll estimate0.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0no pooling of state effectsEstimated Bush supportActualelectionoutcome0.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0complete pooling (no state effects)Estimated Bush supportActualelectionoutcome0.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0multilevel modelEstimated Bush supportActualelectionoutcomeAndrew Gelman Fitting and understanding multilevel models
  • 61. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 62. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 63. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 64. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 65. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 66. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel models alwaysAnything worth doing is worth doing repeatedlyA “method” is any procedure applied more than onceCity planningOutward expansion: fitting a model to other countries, otheryears, other outcomes, . . .Infilling: inferences for individual states, demographicsubgroups, components of data, . . .“Frequentist” statistical theory of repeated inferencesAndrew Gelman Fitting and understanding multilevel models
  • 67. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 68. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 69. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 70. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 71. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 72. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesIncumbency advantage in U.S. House electionsRegression approach (Gelman and King, 1990):For any year, compare districts with and without incs runningControl for vote in previous electionControl for incumbent partyvit = β0 + β1vi,t−1 + β2Pit + ψIit + itOther estimates (sophomore surge, etc.) have selection biasAndrew Gelman Fitting and understanding multilevel models
  • 73. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesEstimated incumbency advantage from lagged regressionsYearEstincadvfromlaggedregression1900 1920 1940 1960 1980 20000.00.050.100.15Andrew Gelman Fitting and understanding multilevel models
  • 74. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCan we do better?Regression estimate: vit = β0 + β1vi,t−1 + β2Pit + ψIit + it“Political science” problem: ψ is assumed to be same in alldistricts“Statistics” problem: the model doesn’t fit the dataWe’ll show pictures of the model not fittingWe’ll set up a model allowing inc advantage to varyAndrew Gelman Fitting and understanding multilevel models
  • 75. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCan we do better?Regression estimate: vit = β0 + β1vi,t−1 + β2Pit + ψIit + it“Political science” problem: ψ is assumed to be same in alldistricts“Statistics” problem: the model doesn’t fit the dataWe’ll show pictures of the model not fittingWe’ll set up a model allowing inc advantage to varyAndrew Gelman Fitting and understanding multilevel models
  • 76. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCan we do better?Regression estimate: vit = β0 + β1vi,t−1 + β2Pit + ψIit + it“Political science” problem: ψ is assumed to be same in alldistricts“Statistics” problem: the model doesn’t fit the dataWe’ll show pictures of the model not fittingWe’ll set up a model allowing inc advantage to varyAndrew Gelman Fitting and understanding multilevel models
  • 77. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCan we do better?Regression estimate: vit = β0 + β1vi,t−1 + β2Pit + ψIit + it“Political science” problem: ψ is assumed to be same in alldistricts“Statistics” problem: the model doesn’t fit the dataWe’ll show pictures of the model not fittingWe’ll set up a model allowing inc advantage to varyAndrew Gelman Fitting and understanding multilevel models
  • 78. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCan we do better?Regression estimate: vit = β0 + β1vi,t−1 + β2Pit + ψIit + it“Political science” problem: ψ is assumed to be same in alldistricts“Statistics” problem: the model doesn’t fit the dataWe’ll show pictures of the model not fittingWe’ll set up a model allowing inc advantage to varyAndrew Gelman Fitting and understanding multilevel models
  • 79. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesModel misfitUnder the model, parallel lines are fitted to the circles (open seats)and dots (incs running for reelection)Democratic vote in 1986Democraticvotein19880.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0ooooooooo oooooooooooooYearCoefficientsforlaggedvote1900 1920 1940 1960 1980 20000.20.40.60.81.01.2incumbents runningopen seatsAndrew Gelman Fitting and understanding multilevel models
  • 80. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 81. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 82. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 83. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 84. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 85. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 86. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 87. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesMultilevel modelfor t = 1, 2: vit = 0.5 + δt + αi + φitIit + itδ2 − δ1 is the national vote swingαi is the “normal vote” for district i: mean 0, sd σα.φit is the inc advantage in district i at time t: mean ψ, sd σφit’s are independent errors: mean 0 and sd σ .Candidate-level incumbency effects:If the same incumbent is running in years 1 and 2, thenφi2 ≡ φi1Otherwise, φi1 and φi2 are independentAndrew Gelman Fitting and understanding multilevel models
  • 88. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 89. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 90. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 91. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 92. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 93. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesFitting the multilevel modelBayesian inferenceLinear parameters: national vote swings, district effects,incumbency effects3 variance parameters: district effects, incumbency effects,residual errorsNeed to model a selection effect: information provided by theincumbent party at time 1Solve analytically for Pr(inclusion), include factor in thelikelihoodGibbs-Metropolis sampling, program in SplusAndrew Gelman Fitting and understanding multilevel models
  • 94. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesEstimated incumbency advantage and its variationYearAverageIncumbencyAdvantage1900 1920 1940 1960 1980 20000.00.040.080.12YearSDofDistrictEffects1900 1920 1940 1960 1980 20000.00.050.100.15YearSDofIncumbencyAdvantage1900 1920 1940 1960 1980 20000.00.020.040.06YearResidualSDofElectionResults1900 1920 1940 1960 1980 20000.00.020.040.06Andrew Gelman Fitting and understanding multilevel models
  • 95. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCompare oldand new estimatesYearEstincadvfromlaggedregression1900 1920 1940 1960 1980 20000.00.050.100.15YearAverageIncumbencyAdvantage 1900 1920 1940 1960 1980 20000.00.040.080.12ntage0.06Andrew Gelman Fitting and understanding multilevel models
  • 96. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesNo-interaction modelBefore-after data with treatment and control groupsDefault model: constant treatment effectsFisher’s classical null hyp: effect is zero for all casesRegression model: yi = Ti θ + Xi β + icontroltreatment"before" measurement, x"after"measurement,yAndrew Gelman Fitting and understanding multilevel models
  • 97. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesNo-interaction modelBefore-after data with treatment and control groupsDefault model: constant treatment effectsFisher’s classical null hyp: effect is zero for all casesRegression model: yi = Ti θ + Xi β + icontroltreatment"before" measurement, x"after"measurement,yAndrew Gelman Fitting and understanding multilevel models
  • 98. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesNo-interaction modelBefore-after data with treatment and control groupsDefault model: constant treatment effectsFisher’s classical null hyp: effect is zero for all casesRegression model: yi = Ti θ + Xi β + icontroltreatment"before" measurement, x"after"measurement,yAndrew Gelman Fitting and understanding multilevel models
  • 99. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesNo-interaction modelBefore-after data with treatment and control groupsDefault model: constant treatment effectsFisher’s classical null hyp: effect is zero for all casesRegression model: yi = Ti θ + Xi β + icontroltreatment"before" measurement, x"after"measurement,yAndrew Gelman Fitting and understanding multilevel models
  • 100. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 101. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 102. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 103. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 104. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 105. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesActual data show interactionsTreatment interacts with “before” measurementBefore-after correlation is higher for controls than for treatedunitsExamplesAn observational study of legislative redistrictingAn experiment with pre-test, post-test dataCongressional elections with incumbents and open seatsAndrew Gelman Fitting and understanding multilevel models
  • 106. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesObservational study of legislative redistrictingbefore-after dataEstimated partisan bias in previous electionEstimatedpartisanbias(adjustedforstate)-0.05 0.0 0.05-0.050.00.05no redistrictingbipartisan redistrictDem. redistrictRep. redistrict.. .. ............................ .. ............ .... ........ ............... ............... ......... .............. oooooooxxxxxxxxxx••• ••••••••••••(favors Democrats)(favors Republicans)Andrew Gelman Fitting and understanding multilevel models
  • 107. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesExperiment: correlation between pre-test and post-testdata for controls and for treated unitsgradecorrelation1 2 3 40.80.91.0controlstreatedAndrew Gelman Fitting and understanding multilevel models
  • 108. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesCorrelation between two successive Congressional electionsfor incumbents running (controls) and open seats (treated)1900 1920 1940 1960 1980 20000.00.20.40.60.8yearcorrelationincumbentsopen seatsAndrew Gelman Fitting and understanding multilevel models
  • 109. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 110. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 111. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 112. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 113. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 114. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 115. EffectivenessUbiquityWay of lifeGeneral frameworkEstimating incumbency advantage and its variationInteractions in before-after studiesInteractions as variance componentsUnit-level “error term” ηiFor control units, ηi persists from time 1 to time 2For treatment units, ηi changes:Subtractive treatment error (ηi only at time 1)Additive treatment error (ηi only at time 2)Replacement treatment errorUnder all these models, the before-after correlation is higherfor controls than treated unitsAndrew Gelman Fitting and understanding multilevel models
  • 116. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsSome new toolsBuilding and fitting multilevel modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 117. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsSome new toolsBuilding and fitting multilevel modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 118. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsSome new toolsBuilding and fitting multilevel modelsDisplaying and summarizing inferencesAndrew Gelman Fitting and understanding multilevel models
  • 119. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBuilding and fitting multilevel modelsA reparameterization can change a model(even if it leaves the likelihood unchanged)Redundant additive parameterizationRedundant multiplicative parameterizationWeakly-informative prior distribution for group-level varianceparametersAndrew Gelman Fitting and understanding multilevel models
  • 120. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBuilding and fitting multilevel modelsA reparameterization can change a model(even if it leaves the likelihood unchanged)Redundant additive parameterizationRedundant multiplicative parameterizationWeakly-informative prior distribution for group-level varianceparametersAndrew Gelman Fitting and understanding multilevel models
  • 121. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBuilding and fitting multilevel modelsA reparameterization can change a model(even if it leaves the likelihood unchanged)Redundant additive parameterizationRedundant multiplicative parameterizationWeakly-informative prior distribution for group-level varianceparametersAndrew Gelman Fitting and understanding multilevel models
  • 122. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBuilding and fitting multilevel modelsA reparameterization can change a model(even if it leaves the likelihood unchanged)Redundant additive parameterizationRedundant multiplicative parameterizationWeakly-informative prior distribution for group-level varianceparametersAndrew Gelman Fitting and understanding multilevel models
  • 123. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBuilding and fitting multilevel modelsA reparameterization can change a model(even if it leaves the likelihood unchanged)Redundant additive parameterizationRedundant multiplicative parameterizationWeakly-informative prior distribution for group-level varianceparametersAndrew Gelman Fitting and understanding multilevel models
  • 124. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 125. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 126. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 127. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 128. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 129. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant parameterizationData model: Pr(yi = 1) = logit−1β0 + βageage(i) + βstatestate(i)Usual model for the coefficients:βagej ∼ N(0, σ2age), for j = 1, . . . , 4βstatej ∼ N(0, σ2state), for j = 1, . . . , 50Additively redundant model:βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Why add the redundant µage, µstate?Iterative algorithm moves more smoothlyAndrew Gelman Fitting and understanding multilevel models
  • 130. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant additive parameterizationModelPr(yi = 1) = logit−1β0+ βageage(i) + βstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered parameters:˜βagej = βagej − ¯βage, for j = 1, . . . , 4˜βstatej = βstatej − ¯βstate, for j = 1, . . . , 50Redefine the constant term:˜β0= β0+ ¯βage+ ¯βageAndrew Gelman Fitting and understanding multilevel models
  • 131. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant additive parameterizationModelPr(yi = 1) = logit−1β0+ βageage(i) + βstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered parameters:˜βagej = βagej − ¯βage, for j = 1, . . . , 4˜βstatej = βstatej − ¯βstate, for j = 1, . . . , 50Redefine the constant term:˜β0= β0+ ¯βage+ ¯βageAndrew Gelman Fitting and understanding multilevel models
  • 132. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant additive parameterizationModelPr(yi = 1) = logit−1β0+ βageage(i) + βstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered parameters:˜βagej = βagej − ¯βage, for j = 1, . . . , 4˜βstatej = βstatej − ¯βstate, for j = 1, . . . , 50Redefine the constant term:˜β0= β0+ ¯βage+ ¯βageAndrew Gelman Fitting and understanding multilevel models
  • 133. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant multiplicative parameterizationNew modelPr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered and scaled parameters:˜βagej = ξage(βagej − ¯βage), for j = 1, . . . , 4˜βstatej = ξstateβstatej − ¯βstate, for j = 1, . . . , 50Faster convergenceMore general model, connections to factor analysisAndrew Gelman Fitting and understanding multilevel models
  • 134. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant multiplicative parameterizationNew modelPr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered and scaled parameters:˜βagej = ξage(βagej − ¯βage), for j = 1, . . . , 4˜βstatej = ξstateβstatej − ¯βstate, for j = 1, . . . , 50Faster convergenceMore general model, connections to factor analysisAndrew Gelman Fitting and understanding multilevel models
  • 135. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant multiplicative parameterizationNew modelPr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered and scaled parameters:˜βagej = ξage(βagej − ¯βage), for j = 1, . . . , 4˜βstatej = ξstateβstatej − ¯βstate, for j = 1, . . . , 50Faster convergenceMore general model, connections to factor analysisAndrew Gelman Fitting and understanding multilevel models
  • 136. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRedundant multiplicative parameterizationNew modelPr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Identify using centered and scaled parameters:˜βagej = ξage(βagej − ¯βage), for j = 1, . . . , 4˜βstatej = ξstateβstatej − ¯βstate, for j = 1, . . . , 50Faster convergenceMore general model, connections to factor analysisAndrew Gelman Fitting and understanding multilevel models
  • 137. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsWeakly informative prior distribution for the multilevelvariance parameterRedundant multiplicative parameterization:Pr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Separate prior distributions on the ξ and σ parameters:Normal on ξInverse-gamma on σ2Generalizes and fixes problems with the standard choices ofprior distributionsAndrew Gelman Fitting and understanding multilevel models
  • 138. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsWeakly informative prior distribution for the multilevelvariance parameterRedundant multiplicative parameterization:Pr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Separate prior distributions on the ξ and σ parameters:Normal on ξInverse-gamma on σ2Generalizes and fixes problems with the standard choices ofprior distributionsAndrew Gelman Fitting and understanding multilevel models
  • 139. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsWeakly informative prior distribution for the multilevelvariance parameterRedundant multiplicative parameterization:Pr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Separate prior distributions on the ξ and σ parameters:Normal on ξInverse-gamma on σ2Generalizes and fixes problems with the standard choices ofprior distributionsAndrew Gelman Fitting and understanding multilevel models
  • 140. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsWeakly informative prior distribution for the multilevelvariance parameterRedundant multiplicative parameterization:Pr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Separate prior distributions on the ξ and σ parameters:Normal on ξInverse-gamma on σ2Generalizes and fixes problems with the standard choices ofprior distributionsAndrew Gelman Fitting and understanding multilevel models
  • 141. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsWeakly informative prior distribution for the multilevelvariance parameterRedundant multiplicative parameterization:Pr(yi = 1) = logit−1β0+ ξageβageage(i) + ξstateβstatestate(i)βagej ∼ N(µage, σ2age), for j = 1, . . . , 4βstatej ∼ N(µstate, σ2state), for j = 1, . . . , 50Separate prior distributions on the ξ and σ parameters:Normal on ξInverse-gamma on σ2Generalizes and fixes problems with the standard choices ofprior distributionsAndrew Gelman Fitting and understanding multilevel models
  • 142. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsDisplaying and summarizing inferencesDisplaying parameters in groups rather than as a long listAverage predictive effectsR2 and partial pooling factorsAnalysis of varianceAndrew Gelman Fitting and understanding multilevel models
  • 143. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsDisplaying and summarizing inferencesDisplaying parameters in groups rather than as a long listAverage predictive effectsR2 and partial pooling factorsAnalysis of varianceAndrew Gelman Fitting and understanding multilevel models
  • 144. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsDisplaying and summarizing inferencesDisplaying parameters in groups rather than as a long listAverage predictive effectsR2 and partial pooling factorsAnalysis of varianceAndrew Gelman Fitting and understanding multilevel models
  • 145. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsDisplaying and summarizing inferencesDisplaying parameters in groups rather than as a long listAverage predictive effectsR2 and partial pooling factorsAnalysis of varianceAndrew Gelman Fitting and understanding multilevel models
  • 146. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsDisplaying and summarizing inferencesDisplaying parameters in groups rather than as a long listAverage predictive effectsR2 and partial pooling factorsAnalysis of varianceAndrew Gelman Fitting and understanding multilevel models
  • 147. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRaw display of inferencemean sd 2.5% 25% 50% 75% 97.5% Rhat n.effB.0 0.402 0.147 0.044 0.326 0.413 0.499 0.652 1.024 110b.female -0.094 0.102 -0.283 -0.162 -0.095 -0.034 0.107 1.001 1000b.black -1.701 0.305 -2.323 -1.910 -1.691 -1.486 -1.152 1.014 500b.female.black -0.143 0.393 -0.834 -0.383 -0.155 0.104 0.620 1.007 1000B.age[1] 0.084 0.088 -0.053 0.012 0.075 0.140 0.277 1.062 45B.age[2] -0.072 0.087 -0.260 -0.121 -0.054 -0.006 0.052 1.017 190B.age[3] 0.044 0.077 -0.105 -0.007 0.038 0.095 0.203 1.029 130B.age[4] -0.057 0.096 -0.265 -0.115 -0.052 0.001 0.133 1.076 32B.edu[1] -0.148 0.131 -0.436 -0.241 -0.137 -0.044 0.053 1.074 31B.edu[2] -0.022 0.082 -0.182 -0.069 -0.021 0.025 0.152 1.028 160B.edu[3] 0.148 0.112 -0.032 0.065 0.142 0.228 0.370 1.049 45B.edu[4] 0.023 0.090 -0.170 -0.030 0.015 0.074 0.224 1.061 37B.age.edu[1,1] -0.044 0.133 -0.363 -0.104 -0.019 0.025 0.170 1.018 1000B.age.edu[1,2] 0.059 0.123 -0.153 -0.011 0.032 0.118 0.353 1.016 580B.age.edu[1,3] 0.049 0.124 -0.146 -0.023 0.022 0.104 0.349 1.015 280B.age.edu[1,4] 0.001 0.116 -0.237 -0.061 0.000 0.052 0.280 1.010 1000B.age.edu[2,1] 0.066 0.152 -0.208 -0.008 0.032 0.124 0.449 1.022 140B.age.edu[2,2] -0.081 0.127 -0.407 -0.137 -0.055 0.001 0.094 1.057 120B.age.edu[2,3] -0.004 0.102 -0.226 -0.048 0.000 0.041 0.215 1.008 940B.age.edu[2,4] -0.042 0.108 -0.282 -0.100 -0.026 0.014 0.157 1.017 170B.age.edu[3,1] 0.034 0.135 -0.215 -0.030 0.009 0.091 0.361 1.021 230B.age.edu[3,2] 0.007 0.102 -0.213 -0.039 0.003 0.052 0.220 1.019 610B.age.edu[3,3] 0.033 0.130 -0.215 -0.029 0.009 0.076 0.410 1.080 61B.age.edu[3,4] -0.009 0.109 -0.236 -0.064 -0.005 0.043 0.214 1.024 150B.age.edu[4,1] -0.141 0.190 -0.672 -0.224 -0.086 -0.003 0.100 1.036 270B.age.edu[4,2] -0.014 0.119 -0.280 -0.059 -0.008 0.033 0.239 1.017 240B.age.edu[4,3] 0.046 0.132 -0.192 -0.024 0.019 0.108 0.332 1.010 210B.age.edu[4,4] 0.042 0.142 -0.193 -0.022 0.016 0.095 0.377 1.015 160B.state[1] 0.201 0.211 -0.131 0.047 0.172 0.326 0.646 1.003 960B.state[2] 0.466 0.252 0.008 0.310 0.440 0.603 1.047 1.001 1000Andrew Gelman Fitting and understanding multilevel models
  • 148. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsRaw graphical display80% interval for each chain R−hat−4 −2 0 2 1 1.5 2+1 1.5 2+1 1.5 2+B.0 qqqb.female qqqb.black qqqb.female.black qqqB.age[1] qqq[2] qqq[3] qqq[4] qqqB.edu[1] qqq[2] qqq[3] qqq[4] qqqB.age.edu[1,1] qqq[1,2] qqq[1,3] qqq[1,4] qqq[2,1] qqq[2,2] qqq[2,3] qqq[2,4] qqq[3,1] qqq[3,2] qqqB.state[1] qqq[2] qqq[3] qqq[4] qqq[5] qqq[6] qqq[7] qqq[8] qqq[9] qqq[10] qqqB.region[1] qqq[2] qqq[3] qqq[4] qqq[5] qqq**medians and 80% intervalsB.000.20.40.6qqqb.female−0.3−0.2−0.100.1qqqb.black−2.5−2−1.5−1qqqb.female.black−1−0.500.5qqqB.age−0.4−0.200.20.4qqq111111111qqq222222222qqq333333333qqq444444444B.edu−0.500.5qqq111111111qqq222222222qqq333333333qqq444444444B.age.edu−1−0.500.5qqq111111111111111111qqq222222222qqq333333333qqq444444444qqq222222222111111111qqq222222222qqq333333333qqq444444444qqq333333333111111111qqq222222222qqq333333333qqq444444444qqq444444444111111111qqq222222222qqq333333333qqq444444444B.state−4−202qqq111111111qqq222222222qqq333333333qqq444444444qqq555555555qqq666666666qqq777777777qqq888888888qqq999999999qqq101010101010101010qqq qqq121212121212121212qqqqqq141414141414141414qqq qqq161616161616161616qqq qqq181818181818181818qqq qqq202020202020202020qqq qqq222222222222222222qqq qqq242424242424242424qqq qqq262626262626262626qqq qqq282828282828282828qqq qqq303030303030303030qqq qqq323232323232323232qqq qqq343434343434343434qqq qqq363636363636363636qqq qqq383838383838383838qqq qqq404040404040404040*B.region−1−0.500.51qqq111111111qqq222222222qqq333333333qqq444444444qqq555555555Sigma.age00.20.40.6qqqSigma.edu00.51qqqSigma.age.edu00.10.20.3qqqBugs model at "C:/books/multilevel/election88/model4.bug", 3 chains, each with 2001 iterationsAndrew Gelman Fitting and understanding multilevel models
  • 149. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBetter graphical display 1: demographicsfemaleblackfemale x black18−2930−4445−6465+no h.s.high schoolsome collegecollege grad18−29 x no h.s.18−29 x high school18−29 x some college18−29 x college grad30−44 x no h.s.30−44 x high school30−44 x some college30−44 x college grad45−64 x no h.s.45−64 x high school45−64 x some college45−64 x college grad65+ x no h.s.65+ x high school65+ x some college65+ x college grad−2.5−2.5−2−2−1.5−1.5−1−1−0.5−0.5000.50.511Andrew Gelman Fitting and understanding multilevel models
  • 150. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBetter graphical display 2: within states−2.0 −1.0 0.00.00.40.8Alaskalinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Arizonalinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Arkansaslinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Californialinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Coloradolinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Connecticutlinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8Delawarelinear predictorPr(supportBush)−2.0 −1.0 0.00.00.40.8District of Columbialinear predictorPr(supportBush)Andrew Gelman Fitting and understanding multilevel models
  • 151. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsBetter graphical display 3: between statesNortheastR vote in prev electionsregressionintercept0.5 0.6 0.7−0.50.00.5CTDEMEMDMANHNJNYPARIVTWVMidwestR vote in prev electionsregressionintercept0.5 0.6 0.7−0.50.00.5ILINIAKSMIMNMONENDOHSDWISouthR vote in prev electionsregressionintercept0.5 0.6 0.7−0.50.00.5ALAR FLGAKY LAMSNCOKSCTNTXVAWestR vote in prev electionsregressionintercept0.5 0.6 0.7−0.50.00.5AKAZCA COHIIDMTNVNMORUTWAWYAndrew Gelman Fitting and understanding multilevel models
  • 152. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?uvyAndrew Gelman Fitting and understanding multilevel models
  • 153. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 154. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 155. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 156. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 157. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 158. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAverage predictive effectsWhat is E(y | x1 = high) − E(y | x1 = low), with all other x’sheld constant?In general, difference can depend on xAverage over distribution of x in the dataYou can’t just use a central value of xCompute APE for each input variable xMultilevel factors are categorical input variablesAndrew Gelman Fitting and understanding multilevel models
  • 159. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsAPE: why you can’t just use a central value of x−6 −4 −2 0 2 4 6 80.00.20.40.60.81.0vE(y|u,v)u = 1u = 0| || || ||| | |||| | ||| ||| | || || |||| ||| || ||| | | |avg pred effect = 0.03pred effect at E(v) = 0.24−6 −4 −2 0 2 4 6 80.00.20.40.60.81.0vE(y|u,v)u = 1u = 0||| | | || ||| ||| ||| | | |||| || | |||| ||| |||||| ||avg pred effect = 0.11pred effect at E(v) = 0.03Andrew Gelman Fitting and understanding multilevel models
  • 160. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 161. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 162. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 163. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 164. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 165. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 166. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsUnderstanding sources of variationGeneralization of R2 (explained variance), defined at eachlevel of the modelPartial pooling factor, defined at each levelAnalysis of varianceSummarize the scale of each batch of predictorsGo beyond classical null-hypothesis-testing frameworkOpen question: how to construct models with deepinteraction structures?Andrew Gelman Fitting and understanding multilevel models
  • 167. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsConclusionsMultilevel modeling is not just for grouped dataNew ideas needed to fit, understand, display, and summarizeeach level of the modelGeneral framework for modeling treatment effects that varyIt’s not just about “data fitting” or “getting the rightstandard errors”Andrew Gelman Fitting and understanding multilevel models
  • 168. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsConclusionsMultilevel modeling is not just for grouped dataNew ideas needed to fit, understand, display, and summarizeeach level of the modelGeneral framework for modeling treatment effects that varyIt’s not just about “data fitting” or “getting the rightstandard errors”Andrew Gelman Fitting and understanding multilevel models
  • 169. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsConclusionsMultilevel modeling is not just for grouped dataNew ideas needed to fit, understand, display, and summarizeeach level of the modelGeneral framework for modeling treatment effects that varyIt’s not just about “data fitting” or “getting the rightstandard errors”Andrew Gelman Fitting and understanding multilevel models
  • 170. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsConclusionsMultilevel modeling is not just for grouped dataNew ideas needed to fit, understand, display, and summarizeeach level of the modelGeneral framework for modeling treatment effects that varyIt’s not just about “data fitting” or “getting the rightstandard errors”Andrew Gelman Fitting and understanding multilevel models
  • 171. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsConclusionsMultilevel modeling is not just for grouped dataNew ideas needed to fit, understand, display, and summarizeeach level of the modelGeneral framework for modeling treatment effects that varyIt’s not just about “data fitting” or “getting the rightstandard errors”Andrew Gelman Fitting and understanding multilevel models
  • 172. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 173. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 174. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 175. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 176. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 177. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 178. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models
  • 179. EffectivenessUbiquityWay of lifeBuilding and fitting modelsDisplaying and summarizing inferencesConclusionsMultilevel modelingWhy?Make use of lots of information out there that’s alreadycollectedUse MLM to adjust for time effects, state effects,survey-organization effectsDon’t freak out because you have “too many predictors”How?Make your studies broader and deeperMake plots and compute summaries to understand theestimated modelAndrew Gelman Fitting and understanding multilevel models