Successfully reported this slideshow.

Mixed Effects Models - Introduction

0

Share

Loading in …3
×
1 of 57
1 of 57

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Mixed Effects Models - Introduction

  1. 1. Using Mixed Effects Models in Psychology Scott Fraundorf sfraundo@pitt.edu Zoom office hours: Wed 11-12, or by appointment
  2. 2. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  3. 3. Course Goals ! We will: - Understand form of mixed effects models - Apply mixed effects models to common designs in psychology and related fields (e.g., factorial experiments, educational interventions, longitudinal studies) - Fit mixed effects models in R using lme4 - Diagnose and address common issues in using mixed effects models ! We won’t: - Cover algorithms used by software to compute mixed effects models
  4. 4. Course Requirements ! We’ll be fitting models in R ! Free & runs on basically any computer ! Next week, will cover basics of using R
  5. 5. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  6. 6. Why Mixed Effects Models?
  7. 7. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  8. 8. • Inferential statistics you may be familiar with: • ANOVA • Regression • Correlation • All of these methods involve random sampling out of a larger population • To which we hope to generalize Problem 1: Multiple Random Effects Subject 1 Subject 2 Subject 3
  9. 9. • Inferential statistics you may be familiar with: • ANOVA • Regression • Correlation • Standard assumption: All observations are independent • Subject 1’s score doesn’t tell us anything about Subject 2’s Problem 1: Multiple Random Effects Subject 1 Subject 2 Subject 3
  10. 10. • Independence assumption is fair if we randomly sample 1 person at a time • e.g., you recruit 40 undergrads from the Psychology Subject Pool • But maybe this isn’t all we should be doing… (Henrich et al., 2010, Nature) Problem 1: Multiple Random Effects
  11. 11. • Important assumption! • Does testing benefit learning of biology? Problem 1: Multiple Random Effects Section A n=50 Practice quizzes Section C n=50 Practice quizzes Section B n=50 Restudy Section D n=50 Restudy
  12. 12. • Impressive if 100 separate people who did practice tests learned better than 100 people who reread the textbook • Not so impressive when we realize these aren’t fully independent • Need to account for differences in instructor, time of day • Problem 1: Multiple Random Effects
  13. 13. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  14. 14. • But many sensible, informative research designs involve more complex sampling procedures • Example: Sampling multiple children from the same family • Kids from the same family will be more similar • a/k/a clustering Child 2 FAMILY 1 FAMILY 2 Child 3 Child 1 Problem 1A: Nested Random Effects
  15. 15. • But many sensible, informative research designs involve more complex sampling procedures • Or: Two people in a relationship • Or any other small group (work teams, experimentally formed groups, etc.) COUPLE 1 Problem 1A: Nested Random Effects
  16. 16. • But many sensible, informative research designs involve more complex sampling procedures • Or: A longitudinal study • Each person is sampled multiple times • Some characteristics don’t change over time—or don’t change much Problem 1A: Nested Random Effects
  17. 17. • But many sensible, informative research designs involve more complex sampling procedures • Or: Kids in classrooms in schools • Kids from the same school will be more similar • Kids in same classroom will be even more similar! Problem 1A: Nested Random Effects Student 2 CLASS- ROOM 1 Student 3 Student 1 CLASS- ROOM 2 SCHOOL 1
  18. 18. • One way to describe what’s going on here is that there are several levels of sampling, each nested inside each other • Each level is what we’ll call a random effect (a thing we sampled) Problem 1A: Nested Random Effects SAMPLED SCHOOLS SAMPLED CLASSROOMS in those schools SAMPLED STUDENTS in those classrooms Student 2 CLASS- ROOM 1 Student 3 Student 1 CLASS- ROOM 2 SCHOOL 1
  19. 19. • Two challenges: • Statistically, we need to take account for this non- independence (similarity) • Even a small amount of non-independence can lead to spurious findings (Quené & van den Bergh, 2008) • We might want to characterize differences at each level! • Are classroom differences or school differences bigger? Problem 1A: Nested Random Effects Student 2 CLASS- ROOM 1 Student 3 Student 1 CLASS- ROOM 2 SCHOOL 1
  20. 20. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  21. 21. Problem 1B: Crossed Random Effects • A closely related problem shows up in many experimental studies • Experimental / research materials are often sampled out of population of possible items • Words or sentences • Educational materials • Hypothetical scenarios • Survey items • Faces
  22. 22. Problem 1B: Crossed Random Effects • We might ask: • Do differences in stimuli used account for group / condition differences? • Study word processing with a lexical decision task EAGLE PLERN
  23. 23. Problem 1B: Crossed Random Effects • We might ask: • Do differences in stimuli used account for group / condition differences? • e.g., Maybe easier vocab words used in one condition Unprimed Primed EAGLE PENGUIN EAGLE MONKEY
  24. 24. Problem 1B: Crossed Random Effects • We might ask: • Do differences in stimuli used account for group / condition differences? • Do our results generalize to the population of all relevant items? • All vocab words • All fictional resumes • All questionnaire items that measure extraversion • All faces
  25. 25. Problem 1B: Crossed Random Effects • Again, we are sampling two things—subjects and items • Arrangement is slightly different because each subject gets each item • Crossed random effects • Still, problem is that we have multiple random effects (things being sampled) Subject 1 Subject 2 EAGLE MONKEY
  26. 26. • Robustness across stimuli has been a major concern in psycholinguistics for a long time Problem 1B: Crossed Random Effects
  27. 27. • Robustness across stimuli has been a major concern in psycholinguistics for a long time • If you are doing research related to language processing, you’ll be expected to address this • (But stats classes don’t always teach you how) • Now growing interest in other fields, too Problem 1B: Crossed Random Effects
  28. 28. • Robustness across stimuli has been a major concern in psycholinguistics for a long time • If you are doing research related to language processing, you’ll be expected to address this • (But stats classes don’t always teach you how) • Now growing interest in other fields, too • Be a statistical pioneer! • Test generalization across items • Characterize variability • Increase power (more likely to detect a significant effect) Problem 1B: Crossed Random Effects
  29. 29. F1(1,3) = 18.31, p < .05 F2(1,4) = 22.45, p < .01 Problem 1B: Crossed Random Effects ! OLD ANOVA solution: Do 2 analyses ! Subjects analysis: Compare each subject (averaging over all of the items) ! Does the effect generalize across subjects? ! Items analysis: Compare each item (averaging over all of the subjects) ! Does the effect generalize across items? SUBJECT ANALYSIS ITEM ANALYSIS
  30. 30. Problem 1B: Crossed Random Effects ! OLD ANOVA solution: Do 2 analyses ! Subjects analysis ! Items analysis ! Problem: We now have 2 different sets of results. Might conflict! ! Possible to combine them with min F’, but not widely used F1(1,3) = 18.31, p < .05 F2(1,4) = 22.45, p < .01 SUBJECT ANALYSIS ITEM ANALYSIS
  31. 31. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  32. 32. Problem 2: Categorical Data ! Basic linear model assumes our response (dependent) measure is continuous ! But, we often want to look at categorical data RT: 833 ms Does student graduate high school or not? Item recalled or not? What predicts DSM diagnosis of ASD? Income: $30,000
  33. 33. Problem One: Categorical Data ! Traditional solution: Analyze percentages ! Maybe with some transformation (e.g., arcsine, logit) ! Violates assumptions of linear model - Among other issues: Linear model assumes normal DV, which has infinite tails - But percentages are clearly bounded - Model could predict impossible values like 110% Problem 2: Categorical Data But 0% ≤ percentages ≤ 100% ∞ - ∞ 100% 0%
  34. 34. Problem One: Categorical Data ! Traditional solution: Analyze percentages ! Violates assumptions of ANOVA - Among other issues: ANOVA assumes normal distribution, which has infinite tails - But proportions are clearly - Model could predict impossible values like 110% Problem 2: Categorical Data ∞ - ∞ But 0% ≤ percentages ≤ 100% 100% 0%
  35. 35. Problem One: Categorical Data ! Traditional solution: Analyze percentages ! Maybe with some transformation (e.g., arcsine, logit) ! Violates assumptions of linear model ! Can lead to: ! Spurious effects (false positives / Type I error) ! Missing real effects (false negatives / Type II error) Problem 2: Categorical Data
  36. 36. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  37. 37. ! Many interesting independent variables vary continuously ! e.g., Word frequency Problem 3: Continuous Predictors pitohui eagle penguin
  38. 38. ! Many interesting independent variables vary continuously ! Or: Second language proficiency or reading skill ! ANOVAs require division into categories ! e.g., median split Problem 3: Continuous Predictors
  39. 39. ! Many interesting independent variables vary continuously ! Or: Second language proficiency or reading skill ! ANOVAs require division into categories ! e.g., median split ! Or: extreme groups design Problem 3: Continuous Predictors
  40. 40. ! Many interesting independent variables vary continuously ! Or: Second language proficiency or reading skill ! ANOVAs require division into categories ! Problem: Can only ask “is there a difference?”, not form of relationship ! Loss of statistical power (Cohen, 1983) Problem 3: Continuous Predictors
  41. 41. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R Power of subjects analysis! Power of items analysis! Captain Mixed Effects to the rescue!
  42. 42. ! Biggest contribution of mixed-effects models is to incorporate multiple random effects into the same analysis Mixed Effects Models to the Rescue! How does the effect of parental stress on screen time generalize across children and families? How does the effect of aphasia on sentence processing generalize across subjects and sentences?
  43. 43. Mixed Models to the Rescue! ! Extension of linear model (regression) = School + + GPA Motiv- ational intervention Subject Outcome Problem 1A solved! 3.07
  44. 44. ! For many experimental designs, this means a change in what gets analyzed ! ANOVA: Unit of analysis is cell mean ! Mixed effects models: Unit of analysis is individual trial! Mixed Effects Models to the Rescue!
  45. 45. Mixed Models to the Rescue! ! Extension of linear model (regression) ! Look at individual trials/observations (not means) = Item + + RT Prime? Subject Lexical decision RT Problem 1B solved!
  46. 46. Mixed Models to the Rescue! ! In a linear model (regression), easy to include independent variables that are continuous = Item + + RT Lexical Freq. Subject Lexical decision RT Problem 3 solved! 2.32
  47. 47. Mixed Models to the Rescue! ! Link functions allow us to relate model to DV that isn’t normally distributed = Item + + Lexical Freq. Subject Odds of correct response on this trial (yes or no?) Accuracy Problem 2 solved!
  48. 48. Mixed Models to the Rescue! ! Link functions allow us to relate model to DV that isn’t normally distributed = Odds of graduating college Yes or No? Problem 2 solved! School + + Motiv- ational intervention Subject
  49. 49. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  50. 50. A Terminological Note… ! “Mixed effects models” is a more general term ! Any model that includes sampled subjects, classrooms, or items (a “random effect”) plus predictor variables of interest (“fixed effects”) ! “Mixing” these two types of effects ! One specific type that we’ll be discussing are “hierarchical linear models”
  51. 51. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  52. 52. R ! How do we run mixed effects models? ! Multiple software packages could be used to fit the same conceptual model ! Most popular solution: R with lme4
  53. 53. R Pros R Cons ! Free & open-source! ! Runs on any computer ! Lots of add-ons— can do just about any type of model ! Widespread use ! Makes analyses clear & more reproducible ! Documentation / help files not the best ! Other online resources ! Requires some programming, not just menus
  54. 54. Two Ways to Use R ! Regular R < www.r-project.org > ! RStudio < www.rstudio.com > ! Different interface ! Some additional windows/tools to help you keep track of what you’re doing ! Same commands, same results ! Also available for just about any platform ! Recommended (but not required) ! Requires you to download regular R first ! Homework for next class: Download them!
  55. 55. Mixed Effects Models Intro ! Course goals & requirements ! Flex @ Pitt considerations ! Motivation for mixed effects models - Multiple random effects - Nested random effects - Crossed random effects - Categorical data - Continuous predictors ! Big picture view of mixed effects models ! Terminology ! R
  56. 56. Wrap-Up ! Mixed effects models solve three common problems with linear model ! Multiple random effects (subjects, items, classrooms, schools) ! Categorical outcomes ! Continuous predictors ! For next week: Download R! ! Next class, we’ll get started using R ! Canvas survey about your research background & Flex@Pitt implementation

×