SlideShare a Scribd company logo
Course Business
! Midterm assignment due on Canvas next
Monday at 1:30 PM
! Unsure if an article is suitable? Can run it by me
! Add-on packages to install for today:
! performance (may have gotten this last week)
! emmeans
! Lab materials on Canvas
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Unbalanced Factors
• Sometimes, we may have differing
numbers of observations per level
• Possible reasons:
• Some categories naturally more common
• e.g., college majors
• Categories may be equally common in the
population, but we have sampling error
• e.g., ended up 60% female participants, 40% male
• Study was designed so that some conditions are
more common
• e.g., more “control” subjects than “intervention” subjects
• We wanted equal numbers of observations, but lost
some because of errors or exclusion criteria
• e.g., data loss due to computer problems
• Dropping subjects below a minimum level of performance
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Weighted Coding
• “For the average student, does course size
predict probability of graduation?”
• Random sample of 200 Pitt undergrads
• 5 are student athletes and 195 are not
• How can we make the intercept reflect the
“average student”?
• We could try to apply effects coding to the
StudentAthlete variable by centering around the
mean and getting (0.5, -0.5), but…
Weighted Coding
• An intercept at 0 would no longer correspond to the
overall mean
• As a scale, this would be totally
unbalanced
• To fix balance, we need to assign
a heavier weight to Athlete
ATHLETE
(5)
NOT ATHLETE
(195)
.5 -.5
0
-.475
Zero is here
But “not athlete” is actually far more common
Weighted Coding
• Change codes so the mean is 0
• c(.975, -.025)
• contr.helmert.weighted() function in my
psycholing package will calculate this
ATHLETE
(5)
NOT ATHLETE
(195)
.975 -.025
0
Weighted Coding
• Weighted coding: Change the codes so that
the mean is 0 again
• Used when the imbalance reflects something real
• Like Type II sums of squares
• “For the average student, does course size
predict graduation rates?”
• Average student is not a student athlete, and our
answer to the question about an “average student”
should reflect this!
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Unweighted Coding
• Last week we looked at aphasia.csv:
• Response times (RT) in a sentence verification task
• Effect of SubjectType (aphasia vs. control)
• Effect of SentenceType (active vs. passive)
• And their interaction
Unweighted Coding
• Oops! Our experiment loaded up the wrong image
for one of our Passive sentences (“Groceries”)
• It may have been sabotaged
• UsableItem column is No for this item
• First, can we remove this from our data?
• Some possibilities:
• aphasia %>% filter(UsableItem == 'Yes') -> aphasia
• aphasia %>% filter(UsableItem != 'No') -> aphasia2
• etc.
Unweighted Coding
• Oops! Our experiment loaded up the wrong image
for one of our Passive sentences (“Groceries”)
• Now, there’s an imbalance, but it’s an accident and
not meaningful
• In fact, we’d like to get rid of it!
Unweighted Coding
• Oops! Our experiment loaded up the wrong image
for one of our Passive sentences (“Groceries”)
• Now, there’s an imbalance, but it’s an accident and
not meaningful
• In fact, we’d like to get rid of it!
• Retain the (-0.5, 0.5) codes
• Weights the two conditions equally—because the
imbalance isn’t meaningful
• Like Type III sums of squares
• Probably what you want for factorial experiments
Unbalanced Factors: Summary
• Weighted coding: Change the codes so that the
mean is 0
• Use when the imbalance reflects something real
• Can be done with contr.helmert.weighted()
• Unweighted coding: Keep the codes as -0.5 and
0.5
• Use when the imbalance is an accident that we
want to eliminate
• With balanced factors, these are identical
Mean across each individual:
Mean of the two levels:
Mean of the
active sentences
Mean of the
passive sentences
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Post-hoc Comparisons
• Maximal model for the aphasia data was:
• model.Maximal <- lmer(RT ~
1 + SentenceType * SubjectType +
(1 + SentenceType|Subject) +
(1 + SubjectType|Item),
data = aphasia)
• This didn’t converge:
• boundary (singular) fit: see ?isSingular
Probably
overparameterized
Post-hoc Comparisons
• Maximal model for the aphasia data was:
• model.Maximal <- lmer(RT ~
1 + SentenceType * SubjectType +
(1 + SentenceType|Subject) +
(1 + SubjectType|Item),
data = aphasia)
• So, let’s simplify:
• model2<- lmer(RT ~
1 + SentenceType * SubjectType +
(1 + SentenceType|Subject) +
(1|Item),
data = aphasia)
• Doesn’t seem to harm model fit—p > .20
• anova(model.Maximal, model2)
Post-hoc Comparisons
Intercept: RT for healthy
controls, active sentences
Significant RT difference for
passive sentences (among
healthy controls)
Not a significant RT
difference for aphasics
(among active
sentences)
Significant special effect of
aphasia + passive sentence
• With treatment coding, we get
estimates of simple-effects:
Post-hoc Comparisons
• The estimates from a model are enough to fully
describe differences among conditions
• With simple effects:
ACTIVE,
CONTROL
RT ≈ 1716 ms
PASSIVE,
CONTROL
RT ≈ 2269 ms
SubjectType
SentenceType
Active Passive
Aphasia
Control
Passive
simple
effect
+553 ms
Post-hoc Comparisons
• The estimates from a model are enough to fully
describe differences among conditions
• With simple effects:
ACTIVE,
APHASIA
RT ≈ 1801 ms
ACTIVE,
CONTROL
RT ≈ 1716 ms
PASSIVE,
CONTROL
RT ≈ 2269 ms
SubjectType
SentenceType
Active Passive
Aphasia
Control
Passive
simple
effect
+553 ms
Aphasia
simple
effect
+85
ms
Post-hoc Comparisons
• The estimates from a model are enough to fully
describe differences among conditions
• With simple effects:
ACTIVE,
APHASIA
RT ≈ 1801 ms
PASSIVE,
APHASIA
RT ≈ 2542 ms
ACTIVE,
CONTROL
RT ≈ 1716 ms
PASSIVE,
CONTROL
RT ≈ 2269 ms
SubjectType
SentenceType
Active Passive
Aphasia
Control
Passive
simple
effect
+553 ms
Aphasia
simple
effect
+85
ms
+553 ms
+85
ms
Interaction
effect
+188
m
s
Post-hoc Comparisons
• But, sometimes we want to compare individual
combinations (e.g., people w/ aphasia seeing
active vs passive sentences)
• i.e., individual cells
ACTIVE,
APHASIA
RT ≈ 1801 ms
PASSIVE,
APHASIA
RT ≈ 2542 ms
ACTIVE,
CONTROL
RT ≈ 1716 ms
PASSIVE,
CONTROL
RT ≈ 2269 ms
SubjectType
SentenceType
Active Passive
Aphasia
Control
?
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Post-hoc Comparisons: Tukey Test
• But, sometimes we want to compare individual
combinations (e.g., people w/ aphasia seeing
active vs passive sentences)
• i.e., individual cells
• emmeans(Model.Maximal,pairwise~SentenceType*SubjectType)
• Requires emmeans package to be loaded
• library(emmeans)
• Uses Tukey
test to correct
for multiple
comparisons
so overall
α still = .05
• Which two cells don’t significantly differ?
Name of the model not the
original dataframe
Comparisons of each pair of cells
The independent variables
(for now, all of them)
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Estimated Marginal Means
• emmeans also returns estimated means and std.
errors for each cell of the design
• EMMs represent what the means of the different
groups would look like if they didn’t differ in other
(fixed or random) variables
Estimated Marginal Means
• As an example, let’s add SentenceLength as a
covariate
• Task was to read the sentence & judge whether it matches a
picture, so length of sentence would plausibly affect time
needed to do this
• Not perfectly balanced across conditions
Estimated Marginal Means
• New model & estimated marginal means:
• model.Length <- lmer(RT ~ 1 + SentenceType *
SubjectType + SentenceLength + (SentenceType|Subject)
+ (1|Item), data=aphasia)
• emmeans(model.Length,
pairwise~SentenceType*SubjectType + SentenceLength)
Our raw data (where the
average sentence length
differs slightly across
conditions)
What we’d
expect if the
sentences were
all of equal
length
Estimated Marginal Means
• EMMs are a hypothetical
• What the means of the different
groups would look like if they didn’t
differ in this covariate
• Like unweighted coding in the
case of missingness
• Based on our statistical model, so if our
model is wrong (e.g., we picked the wrong
covariate), the adjusted means will be too
• Unlike the raw sample means
• Need to use some caution in interpreting them
• Be clear what you are reporting
Estimated Marginal Means
• Also possible to test whether each of these
estimated cell means significantly differs from 0
• ls_means(model.Length)
• Silly in case of RTs, but could be relevant for some other
DVs (e.g., preference)
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab
Comparing Marginal Means
• emmeans can also test marginal means:
• emmeans(model2,pairwise~SubjectType)
• Effect of one variable averaging over the other
• e.g., aphasic participants (averaging over all sentence
types) vs. controls (averaging over all sentence types)
• These are what main effects are testing
Now, include just one variable (for which
we want marginal means)
Week 8.1: Post-Hoc Comparison
! Unbalanced Factors
! Weighted Coding
! Unweighted Coding
! Post-Hoc Comparisons
! Tukey Test
! Estimated Marginal Means
! Comparing Marginal Means
! Lab

More Related Content

What's hot

Mixed Effects Models - Random Slopes
Mixed Effects Models - Random SlopesMixed Effects Models - Random Slopes
Mixed Effects Models - Random Slopes
Scott Fraundorf
 
Mixed Effects Models - Empirical Logit
Mixed Effects Models - Empirical LogitMixed Effects Models - Empirical Logit
Mixed Effects Models - Empirical Logit
Scott Fraundorf
 
Mixed Effects Models - Effect Size
Mixed Effects Models - Effect SizeMixed Effects Models - Effect Size
Mixed Effects Models - Effect Size
Scott Fraundorf
 
Mixed Effects Models - Crossed Random Effects
Mixed Effects Models - Crossed Random EffectsMixed Effects Models - Crossed Random Effects
Mixed Effects Models - Crossed Random Effects
Scott Fraundorf
 
Mixed Effects Models - Growth Curve Analysis
Mixed Effects Models - Growth Curve AnalysisMixed Effects Models - Growth Curve Analysis
Mixed Effects Models - Growth Curve Analysis
Scott Fraundorf
 
Mixed Effects Models - Signal Detection Theory
Mixed Effects Models - Signal Detection TheoryMixed Effects Models - Signal Detection Theory
Mixed Effects Models - Signal Detection Theory
Scott Fraundorf
 
Mixed Effects Models - Autocorrelation
Mixed Effects Models - AutocorrelationMixed Effects Models - Autocorrelation
Mixed Effects Models - Autocorrelation
Scott Fraundorf
 
状態空間モデル等による多変量時系列データ解析
状態空間モデル等による多変量時系列データ解析状態空間モデル等による多変量時系列データ解析
状態空間モデル等による多変量時系列データ解析
businessanalytics
 
第4回DARM勉強会 (構造方程式モデリング)
第4回DARM勉強会 (構造方程式モデリング)第4回DARM勉強会 (構造方程式モデリング)
第4回DARM勉強会 (構造方程式モデリング)
Yoshitake Takebayashi
 
Vectorization In NLP.pptx
Vectorization In NLP.pptxVectorization In NLP.pptx
Vectorization In NLP.pptx
Chode Amarnath
 
心理学者のためのJASP入門(操作編)[説明文をよんでください]
心理学者のためのJASP入門(操作編)[説明文をよんでください]心理学者のためのJASP入門(操作編)[説明文をよんでください]
心理学者のためのJASP入門(操作編)[説明文をよんでください]
daiki hojo
 
指数分布とポアソン分布のいけない関係
指数分布とポアソン分布のいけない関係指数分布とポアソン分布のいけない関係
指数分布とポアソン分布のいけない関係
Nagi Teramo
 
Rで実験計画法 前編
Rで実験計画法 前編Rで実験計画法 前編
Rで実験計画法 前編
itoyan110
 
13.01.20. 第1回DARM勉強会資料#4
13.01.20. 第1回DARM勉強会資料#413.01.20. 第1回DARM勉強会資料#4
13.01.20. 第1回DARM勉強会資料#4
Yoshitake Takebayashi
 
統計的検定と例数設計の基礎
統計的検定と例数設計の基礎統計的検定と例数設計の基礎
統計的検定と例数設計の基礎
Senshu University
 
Forward-Forward Algorithm
Forward-Forward AlgorithmForward-Forward Algorithm
Forward-Forward Algorithm
Dong Heon Cho
 
第4回DARM勉強会 (多母集団同時分析)
第4回DARM勉強会 (多母集団同時分析)第4回DARM勉強会 (多母集団同時分析)
第4回DARM勉強会 (多母集団同時分析)
Masaru Tokuoka
 
西山計量経済学第8章 制限従属変数モデル
西山計量経済学第8章 制限従属変数モデル西山計量経済学第8章 制限従属変数モデル
西山計量経済学第8章 制限従属変数モデル
Katsuya Ito
 
ベイズファクターとモデル選択
ベイズファクターとモデル選択ベイズファクターとモデル選択
ベイズファクターとモデル選択
kazutantan
 
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)Keiku322
 

What's hot (20)

Mixed Effects Models - Random Slopes
Mixed Effects Models - Random SlopesMixed Effects Models - Random Slopes
Mixed Effects Models - Random Slopes
 
Mixed Effects Models - Empirical Logit
Mixed Effects Models - Empirical LogitMixed Effects Models - Empirical Logit
Mixed Effects Models - Empirical Logit
 
Mixed Effects Models - Effect Size
Mixed Effects Models - Effect SizeMixed Effects Models - Effect Size
Mixed Effects Models - Effect Size
 
Mixed Effects Models - Crossed Random Effects
Mixed Effects Models - Crossed Random EffectsMixed Effects Models - Crossed Random Effects
Mixed Effects Models - Crossed Random Effects
 
Mixed Effects Models - Growth Curve Analysis
Mixed Effects Models - Growth Curve AnalysisMixed Effects Models - Growth Curve Analysis
Mixed Effects Models - Growth Curve Analysis
 
Mixed Effects Models - Signal Detection Theory
Mixed Effects Models - Signal Detection TheoryMixed Effects Models - Signal Detection Theory
Mixed Effects Models - Signal Detection Theory
 
Mixed Effects Models - Autocorrelation
Mixed Effects Models - AutocorrelationMixed Effects Models - Autocorrelation
Mixed Effects Models - Autocorrelation
 
状態空間モデル等による多変量時系列データ解析
状態空間モデル等による多変量時系列データ解析状態空間モデル等による多変量時系列データ解析
状態空間モデル等による多変量時系列データ解析
 
第4回DARM勉強会 (構造方程式モデリング)
第4回DARM勉強会 (構造方程式モデリング)第4回DARM勉強会 (構造方程式モデリング)
第4回DARM勉強会 (構造方程式モデリング)
 
Vectorization In NLP.pptx
Vectorization In NLP.pptxVectorization In NLP.pptx
Vectorization In NLP.pptx
 
心理学者のためのJASP入門(操作編)[説明文をよんでください]
心理学者のためのJASP入門(操作編)[説明文をよんでください]心理学者のためのJASP入門(操作編)[説明文をよんでください]
心理学者のためのJASP入門(操作編)[説明文をよんでください]
 
指数分布とポアソン分布のいけない関係
指数分布とポアソン分布のいけない関係指数分布とポアソン分布のいけない関係
指数分布とポアソン分布のいけない関係
 
Rで実験計画法 前編
Rで実験計画法 前編Rで実験計画法 前編
Rで実験計画法 前編
 
13.01.20. 第1回DARM勉強会資料#4
13.01.20. 第1回DARM勉強会資料#413.01.20. 第1回DARM勉強会資料#4
13.01.20. 第1回DARM勉強会資料#4
 
統計的検定と例数設計の基礎
統計的検定と例数設計の基礎統計的検定と例数設計の基礎
統計的検定と例数設計の基礎
 
Forward-Forward Algorithm
Forward-Forward AlgorithmForward-Forward Algorithm
Forward-Forward Algorithm
 
第4回DARM勉強会 (多母集団同時分析)
第4回DARM勉強会 (多母集団同時分析)第4回DARM勉強会 (多母集団同時分析)
第4回DARM勉強会 (多母集団同時分析)
 
西山計量経済学第8章 制限従属変数モデル
西山計量経済学第8章 制限従属変数モデル西山計量経済学第8章 制限従属変数モデル
西山計量経済学第8章 制限従属変数モデル
 
ベイズファクターとモデル選択
ベイズファクターとモデル選択ベイズファクターとモデル選択
ベイズファクターとモデル選択
 
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)
Rによる特徴抽出 第48回R勉強会@東京(#TokyoR)
 

Similar to Mixed Effects Models - Post-Hoc Comparisons

Topic 5 (multiple regression)
Topic 5 (multiple regression)Topic 5 (multiple regression)
Topic 5 (multiple regression)
Ryan Herzog
 
NLP_KASHK:Evaluating Language Model
NLP_KASHK:Evaluating Language ModelNLP_KASHK:Evaluating Language Model
NLP_KASHK:Evaluating Language Model
Hemantha Kulathilake
 
LecccccccccccccProgrammingLecture-09.pdf
LecccccccccccccProgrammingLecture-09.pdfLecccccccccccccProgrammingLecture-09.pdf
LecccccccccccccProgrammingLecture-09.pdf
AmirMohamedNabilSale
 
Topic 5 (multiple regression)
Topic 5 (multiple regression)Topic 5 (multiple regression)
Topic 5 (multiple regression)
Ryan Herzog
 
Topic 2 - More on Hypothesis Testing
Topic 2 - More on Hypothesis TestingTopic 2 - More on Hypothesis Testing
Topic 2 - More on Hypothesis Testing
Ryan Herzog
 
Mixed Effects Models - Fixed Effect Interactions
Mixed Effects Models - Fixed Effect InteractionsMixed Effects Models - Fixed Effect Interactions
Mixed Effects Models - Fixed Effect Interactions
Scott Fraundorf
 
probability.pptx
probability.pptxprobability.pptx
probability.pptx
bisan3
 
Chapter37
Chapter37Chapter37
Chapter37
Ying Liu
 
Inferential Statistics
Inferential StatisticsInferential Statistics
Inferential Statistics
Neny Isharyanti
 
regression.pptx
regression.pptxregression.pptx
regression.pptx
aneeshs28
 
Mixed Effects Models - Missing Data
Mixed Effects Models - Missing DataMixed Effects Models - Missing Data
Mixed Effects Models - Missing Data
Scott Fraundorf
 
powerpoint 1-19.pdf
powerpoint 1-19.pdfpowerpoint 1-19.pdf
powerpoint 1-19.pdf
JuanPicasso7
 
5954987.ppt
5954987.ppt5954987.ppt
5954987.ppt
MukhtiarKhan5
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
Shimi Bandiel
 
Thermal Expansion Lab ProjectThe data listed in Table 1 be
Thermal Expansion Lab ProjectThe data listed in Table 1 beThermal Expansion Lab ProjectThe data listed in Table 1 be
Thermal Expansion Lab ProjectThe data listed in Table 1 be
GrazynaBroyles24
 
Formulating a Hypothesis
Formulating a HypothesisFormulating a Hypothesis
Formulating a Hypothesis
bjkim0228
 
Dimd_m_004 DL.pdf
Dimd_m_004 DL.pdfDimd_m_004 DL.pdf
Dimd_m_004 DL.pdf
juan631
 
TEST #1Perform the following two-tailed hypothesis test, using a.docx
TEST #1Perform the following two-tailed hypothesis test, using a.docxTEST #1Perform the following two-tailed hypothesis test, using a.docx
TEST #1Perform the following two-tailed hypothesis test, using a.docx
mattinsonjanel
 
Spss
SpssSpss
Lab report walk through
Lab report walk throughLab report walk through
Lab report walk through
serenaasya
 

Similar to Mixed Effects Models - Post-Hoc Comparisons (20)

Topic 5 (multiple regression)
Topic 5 (multiple regression)Topic 5 (multiple regression)
Topic 5 (multiple regression)
 
NLP_KASHK:Evaluating Language Model
NLP_KASHK:Evaluating Language ModelNLP_KASHK:Evaluating Language Model
NLP_KASHK:Evaluating Language Model
 
LecccccccccccccProgrammingLecture-09.pdf
LecccccccccccccProgrammingLecture-09.pdfLecccccccccccccProgrammingLecture-09.pdf
LecccccccccccccProgrammingLecture-09.pdf
 
Topic 5 (multiple regression)
Topic 5 (multiple regression)Topic 5 (multiple regression)
Topic 5 (multiple regression)
 
Topic 2 - More on Hypothesis Testing
Topic 2 - More on Hypothesis TestingTopic 2 - More on Hypothesis Testing
Topic 2 - More on Hypothesis Testing
 
Mixed Effects Models - Fixed Effect Interactions
Mixed Effects Models - Fixed Effect InteractionsMixed Effects Models - Fixed Effect Interactions
Mixed Effects Models - Fixed Effect Interactions
 
probability.pptx
probability.pptxprobability.pptx
probability.pptx
 
Chapter37
Chapter37Chapter37
Chapter37
 
Inferential Statistics
Inferential StatisticsInferential Statistics
Inferential Statistics
 
regression.pptx
regression.pptxregression.pptx
regression.pptx
 
Mixed Effects Models - Missing Data
Mixed Effects Models - Missing DataMixed Effects Models - Missing Data
Mixed Effects Models - Missing Data
 
powerpoint 1-19.pdf
powerpoint 1-19.pdfpowerpoint 1-19.pdf
powerpoint 1-19.pdf
 
5954987.ppt
5954987.ppt5954987.ppt
5954987.ppt
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Thermal Expansion Lab ProjectThe data listed in Table 1 be
Thermal Expansion Lab ProjectThe data listed in Table 1 beThermal Expansion Lab ProjectThe data listed in Table 1 be
Thermal Expansion Lab ProjectThe data listed in Table 1 be
 
Formulating a Hypothesis
Formulating a HypothesisFormulating a Hypothesis
Formulating a Hypothesis
 
Dimd_m_004 DL.pdf
Dimd_m_004 DL.pdfDimd_m_004 DL.pdf
Dimd_m_004 DL.pdf
 
TEST #1Perform the following two-tailed hypothesis test, using a.docx
TEST #1Perform the following two-tailed hypothesis test, using a.docxTEST #1Perform the following two-tailed hypothesis test, using a.docx
TEST #1Perform the following two-tailed hypothesis test, using a.docx
 
Spss
SpssSpss
Spss
 
Lab report walk through
Lab report walk throughLab report walk through
Lab report walk through
 

Recently uploaded

The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
History of Stoke Newington
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Nguyen Thanh Tu Collection
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
Colégio Santa Teresinha
 
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
Leena Ghag-Sakpal
 
Advanced Java[Extra Concepts, Not Difficult].docx
Advanced Java[Extra Concepts, Not Difficult].docxAdvanced Java[Extra Concepts, Not Difficult].docx
Advanced Java[Extra Concepts, Not Difficult].docx
adhitya5119
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
iammrhaywood
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
Celine George
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
HajraNaeem15
 
Solutons Maths Escape Room Spatial .pptx
Solutons Maths Escape Room Spatial .pptxSolutons Maths Escape Room Spatial .pptx
Solutons Maths Escape Room Spatial .pptx
spdendr
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
Priyankaranawat4
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
MJDuyan
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
siemaillard
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
ssuser13ffe4
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
ZK on Polkadot zero knowledge proofs - sub0.pptx
ZK on Polkadot zero knowledge proofs - sub0.pptxZK on Polkadot zero knowledge proofs - sub0.pptx
ZK on Polkadot zero knowledge proofs - sub0.pptx
dot55audits
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
Nicholas Montgomery
 
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching AptitudeUGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
S. Raj Kumar
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
Celine George
 

Recently uploaded (20)

The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
 
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
Bed Making ( Introduction, Purpose, Types, Articles, Scientific principles, N...
 
Advanced Java[Extra Concepts, Not Difficult].docx
Advanced Java[Extra Concepts, Not Difficult].docxAdvanced Java[Extra Concepts, Not Difficult].docx
Advanced Java[Extra Concepts, Not Difficult].docx
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
 
Solutons Maths Escape Room Spatial .pptx
Solutons Maths Escape Room Spatial .pptxSolutons Maths Escape Room Spatial .pptx
Solutons Maths Escape Room Spatial .pptx
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
 
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumPhilippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) Curriculum
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
ZK on Polkadot zero knowledge proofs - sub0.pptx
ZK on Polkadot zero knowledge proofs - sub0.pptxZK on Polkadot zero knowledge proofs - sub0.pptx
ZK on Polkadot zero knowledge proofs - sub0.pptx
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
 
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching AptitudeUGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
 

Mixed Effects Models - Post-Hoc Comparisons

  • 1. Course Business ! Midterm assignment due on Canvas next Monday at 1:30 PM ! Unsure if an article is suitable? Can run it by me ! Add-on packages to install for today: ! performance (may have gotten this last week) ! emmeans ! Lab materials on Canvas
  • 2. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 3. Unbalanced Factors • Sometimes, we may have differing numbers of observations per level • Possible reasons: • Some categories naturally more common • e.g., college majors • Categories may be equally common in the population, but we have sampling error • e.g., ended up 60% female participants, 40% male • Study was designed so that some conditions are more common • e.g., more “control” subjects than “intervention” subjects • We wanted equal numbers of observations, but lost some because of errors or exclusion criteria • e.g., data loss due to computer problems • Dropping subjects below a minimum level of performance
  • 4. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 5. Weighted Coding • “For the average student, does course size predict probability of graduation?” • Random sample of 200 Pitt undergrads • 5 are student athletes and 195 are not • How can we make the intercept reflect the “average student”? • We could try to apply effects coding to the StudentAthlete variable by centering around the mean and getting (0.5, -0.5), but…
  • 6. Weighted Coding • An intercept at 0 would no longer correspond to the overall mean • As a scale, this would be totally unbalanced • To fix balance, we need to assign a heavier weight to Athlete ATHLETE (5) NOT ATHLETE (195) .5 -.5 0 -.475 Zero is here But “not athlete” is actually far more common
  • 7. Weighted Coding • Change codes so the mean is 0 • c(.975, -.025) • contr.helmert.weighted() function in my psycholing package will calculate this ATHLETE (5) NOT ATHLETE (195) .975 -.025 0
  • 8. Weighted Coding • Weighted coding: Change the codes so that the mean is 0 again • Used when the imbalance reflects something real • Like Type II sums of squares • “For the average student, does course size predict graduation rates?” • Average student is not a student athlete, and our answer to the question about an “average student” should reflect this!
  • 9. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 10. Unweighted Coding • Last week we looked at aphasia.csv: • Response times (RT) in a sentence verification task • Effect of SubjectType (aphasia vs. control) • Effect of SentenceType (active vs. passive) • And their interaction
  • 11. Unweighted Coding • Oops! Our experiment loaded up the wrong image for one of our Passive sentences (“Groceries”) • It may have been sabotaged • UsableItem column is No for this item • First, can we remove this from our data? • Some possibilities: • aphasia %>% filter(UsableItem == 'Yes') -> aphasia • aphasia %>% filter(UsableItem != 'No') -> aphasia2 • etc.
  • 12. Unweighted Coding • Oops! Our experiment loaded up the wrong image for one of our Passive sentences (“Groceries”) • Now, there’s an imbalance, but it’s an accident and not meaningful • In fact, we’d like to get rid of it!
  • 13. Unweighted Coding • Oops! Our experiment loaded up the wrong image for one of our Passive sentences (“Groceries”) • Now, there’s an imbalance, but it’s an accident and not meaningful • In fact, we’d like to get rid of it! • Retain the (-0.5, 0.5) codes • Weights the two conditions equally—because the imbalance isn’t meaningful • Like Type III sums of squares • Probably what you want for factorial experiments
  • 14. Unbalanced Factors: Summary • Weighted coding: Change the codes so that the mean is 0 • Use when the imbalance reflects something real • Can be done with contr.helmert.weighted() • Unweighted coding: Keep the codes as -0.5 and 0.5 • Use when the imbalance is an accident that we want to eliminate • With balanced factors, these are identical Mean across each individual: Mean of the two levels: Mean of the active sentences Mean of the passive sentences
  • 15. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 16. Post-hoc Comparisons • Maximal model for the aphasia data was: • model.Maximal <- lmer(RT ~ 1 + SentenceType * SubjectType + (1 + SentenceType|Subject) + (1 + SubjectType|Item), data = aphasia) • This didn’t converge: • boundary (singular) fit: see ?isSingular Probably overparameterized
  • 17. Post-hoc Comparisons • Maximal model for the aphasia data was: • model.Maximal <- lmer(RT ~ 1 + SentenceType * SubjectType + (1 + SentenceType|Subject) + (1 + SubjectType|Item), data = aphasia) • So, let’s simplify: • model2<- lmer(RT ~ 1 + SentenceType * SubjectType + (1 + SentenceType|Subject) + (1|Item), data = aphasia) • Doesn’t seem to harm model fit—p > .20 • anova(model.Maximal, model2)
  • 18. Post-hoc Comparisons Intercept: RT for healthy controls, active sentences Significant RT difference for passive sentences (among healthy controls) Not a significant RT difference for aphasics (among active sentences) Significant special effect of aphasia + passive sentence • With treatment coding, we get estimates of simple-effects:
  • 19. Post-hoc Comparisons • The estimates from a model are enough to fully describe differences among conditions • With simple effects: ACTIVE, CONTROL RT ≈ 1716 ms PASSIVE, CONTROL RT ≈ 2269 ms SubjectType SentenceType Active Passive Aphasia Control Passive simple effect +553 ms
  • 20. Post-hoc Comparisons • The estimates from a model are enough to fully describe differences among conditions • With simple effects: ACTIVE, APHASIA RT ≈ 1801 ms ACTIVE, CONTROL RT ≈ 1716 ms PASSIVE, CONTROL RT ≈ 2269 ms SubjectType SentenceType Active Passive Aphasia Control Passive simple effect +553 ms Aphasia simple effect +85 ms
  • 21. Post-hoc Comparisons • The estimates from a model are enough to fully describe differences among conditions • With simple effects: ACTIVE, APHASIA RT ≈ 1801 ms PASSIVE, APHASIA RT ≈ 2542 ms ACTIVE, CONTROL RT ≈ 1716 ms PASSIVE, CONTROL RT ≈ 2269 ms SubjectType SentenceType Active Passive Aphasia Control Passive simple effect +553 ms Aphasia simple effect +85 ms +553 ms +85 ms Interaction effect +188 m s
  • 22. Post-hoc Comparisons • But, sometimes we want to compare individual combinations (e.g., people w/ aphasia seeing active vs passive sentences) • i.e., individual cells ACTIVE, APHASIA RT ≈ 1801 ms PASSIVE, APHASIA RT ≈ 2542 ms ACTIVE, CONTROL RT ≈ 1716 ms PASSIVE, CONTROL RT ≈ 2269 ms SubjectType SentenceType Active Passive Aphasia Control ?
  • 23. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 24. Post-hoc Comparisons: Tukey Test • But, sometimes we want to compare individual combinations (e.g., people w/ aphasia seeing active vs passive sentences) • i.e., individual cells • emmeans(Model.Maximal,pairwise~SentenceType*SubjectType) • Requires emmeans package to be loaded • library(emmeans) • Uses Tukey test to correct for multiple comparisons so overall α still = .05 • Which two cells don’t significantly differ? Name of the model not the original dataframe Comparisons of each pair of cells The independent variables (for now, all of them)
  • 25. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 26. Estimated Marginal Means • emmeans also returns estimated means and std. errors for each cell of the design • EMMs represent what the means of the different groups would look like if they didn’t differ in other (fixed or random) variables
  • 27. Estimated Marginal Means • As an example, let’s add SentenceLength as a covariate • Task was to read the sentence & judge whether it matches a picture, so length of sentence would plausibly affect time needed to do this • Not perfectly balanced across conditions
  • 28. Estimated Marginal Means • New model & estimated marginal means: • model.Length <- lmer(RT ~ 1 + SentenceType * SubjectType + SentenceLength + (SentenceType|Subject) + (1|Item), data=aphasia) • emmeans(model.Length, pairwise~SentenceType*SubjectType + SentenceLength) Our raw data (where the average sentence length differs slightly across conditions) What we’d expect if the sentences were all of equal length
  • 29. Estimated Marginal Means • EMMs are a hypothetical • What the means of the different groups would look like if they didn’t differ in this covariate • Like unweighted coding in the case of missingness • Based on our statistical model, so if our model is wrong (e.g., we picked the wrong covariate), the adjusted means will be too • Unlike the raw sample means • Need to use some caution in interpreting them • Be clear what you are reporting
  • 30. Estimated Marginal Means • Also possible to test whether each of these estimated cell means significantly differs from 0 • ls_means(model.Length) • Silly in case of RTs, but could be relevant for some other DVs (e.g., preference)
  • 31. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab
  • 32. Comparing Marginal Means • emmeans can also test marginal means: • emmeans(model2,pairwise~SubjectType) • Effect of one variable averaging over the other • e.g., aphasic participants (averaging over all sentence types) vs. controls (averaging over all sentence types) • These are what main effects are testing Now, include just one variable (for which we want marginal means)
  • 33. Week 8.1: Post-Hoc Comparison ! Unbalanced Factors ! Weighted Coding ! Unweighted Coding ! Post-Hoc Comparisons ! Tukey Test ! Estimated Marginal Means ! Comparing Marginal Means ! Lab