Beyond the ANOVA:
Alternative techniques for statistical
analyses of neuropsychological data.
Philip Schatz, PhD
Saint Jos...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Philip:
• Background/Intro
• “va”, ANCOVA, and Multicol...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Alt: “How to become a stats geek without trying”
Why th...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=1?
“va”
QuickTime™ and a
TIFF (Un...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=20? (range=31; 6-36)
“va”
QuickTi...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=20? (range=6;1-6)
“va”
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=100?
The developer of Swampy Acre...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=100?
Swampy Acres Retirement Home...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What is “variance”? N=100? “normally” distributed?
“va”...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Inter-relationship between variables
AKA: Shared varian...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Multicollinearity
National Academy of Neuropsychology, San Antonio, TX, October 2006
Why use
a covariate?
Non-
Equivalent
Groups
Design
To A...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What are the “rules” or “assumptions for ANCOVA?
1. DV ...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What are the “rules” or “assumptions for ANCOVA?
1. DV ...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What are the “rules” or “assumptions for ANCOVA?
1. DV ...
National Academy of Neuropsychology, San Antonio, TX, October 2006
1. DV is linear and normally distributed - yes!
• IV: C...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What are the “rules” or “assumptions for ANCOVA?
2. The...
National Academy of Neuropsychology, San Antonio, TX, October 2006
2. There is a homogeneity of regression slopes
(slope o...
National Academy of Neuropsychology, San Antonio, TX, October 2006
2. There is a homogeneity of regression slopes - yes
* ...
National Academy of Neuropsychology, San Antonio, TX, October 2006
What are the “rules” or “assumptions for ANCOVA?
3. The...
National Academy of Neuropsychology, San Antonio, TX, October 2006
3. There is a theoretical and statistical correlation
b...
National Academy of Neuropsychology, San Antonio, TX, October 2006
3. There is a theoretical and statistical correlation
b...
National Academy of Neuropsychology, San Antonio, TX, October 2006
3. There is a theoretical and statistical correlation
b...
National Academy of Neuropsychology, San Antonio, TX, October 2006
3. There is a theoretical and statistical correlation
b...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Between Groups Differences (MANOVA)
IV: Concussion (Yes...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Can you predict Group membership with p<.001?
Beyond (M...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Can you predict Group membership with ANOVA?
Beyond (M)...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Can you predict Group membership with ANOVA?
Beyond (M)...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Discriminant Analysis
• Characterizes the relationship ...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Characterizes the relationship between a set of IVs
wit...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Flip the MANOVA upside-down:
Between Groups Differences...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Assumptions:
• IVs are either dichotomous or measuremen...
National Academy of Neuropsychology, San Antonio, TX, October 2006
1. We have a between group difference
Discriminant Anal...
National Academy of Neuropsychology, San Antonio, TX, October 2006
2. The DA returns 1 “function”
-it is significant (Chi-...
National Academy of Neuropsychology, San Antonio, TX, October 2006
3. Classification Table reveals “success” in
discrimina...
National Academy of Neuropsychology, San Antonio, TX, October 2006
4. Standardized canonical discriminant function
coeffic...
National Academy of Neuropsychology, San Antonio, TX, October 2006
5. Correlation coefficients - how much each variable
co...
National Academy of Neuropsychology, San Antonio, TX, October 2006
6. Calculating sensitivity
Correct “positive” hits = 81...
National Academy of Neuropsychology, San Antonio, TX, October 2006
7. Calculating specificity
Correct “negative” hits = 89...
National Academy of Neuropsychology, San Antonio, TX, October 2006
8. Calculating positive likelihood ratio
prob of an ind...
National Academy of Neuropsychology, San Antonio, TX, October 2006
9. Calculating negative likelihood ratio
prob of an ind...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Dependent samples analyses:
• Change from Time 1 to Tim...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Is change from Time 1 to Time 2 a reliable change?
• Re...
National Academy of Neuropsychology, San Antonio, TX, October 2006
RCI Articles:
Jacobson & Traux (1991). Journal of Consu...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Beyond ANOVA - Reliable Change?
(Table adapted from Ive...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Beyond ANOVA - Reliable Change?
Using Excel:
QuickTime™...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Beyond ANOVA - Reliable Change?
Baseline-to-Time 1:
58%...
National Academy of Neuropsychology, San Antonio, TX, October 2006
Upcoming SlideShare
Loading in …5
×

Beyond_ANOVA.ppt

810
-1

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
810
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Beyond_ANOVA.ppt

  1. 1. Beyond the ANOVA: Alternative techniques for statistical analyses of neuropsychological data. Philip Schatz, PhD Saint Joseph’s University Stephen T. Moelter, PhD University of Sciences in Philadelphia online reprint: http://schatz.sju.edu/nan/ pschatz@sju.edu National Academy of Neuropsychology, San Antonio, TX, October 2006
  2. 2. National Academy of Neuropsychology, San Antonio, TX, October 2006 Philip: • Background/Intro • “va”, ANCOVA, and Multicollinearity • Discriminant Analysis - Sensitivity & Specificity • Reliable Change Index (RCI) Stephen: • Cluster Analysis • Multidimensional Scaling • Correspondence Analysis
  3. 3. National Academy of Neuropsychology, San Antonio, TX, October 2006 Alt: “How to become a stats geek without trying” Why this talk at NAN? Resources: Texts: Stevens, Tabachnick & Fidel Web calculators: http://calculators.stat.ucla.edu/ Web sites: www2.chass.ncsu.edu/garson/pa765/statnote.htm Background
  4. 4. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=1? “va” QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  5. 5. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=20? (range=31; 6-36) “va” QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  6. 6. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=20? (range=6;1-6) “va”
  7. 7. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=100? The developer of Swampy Acres Retirement Home is attempting to sell lots in a "Southern Paradise" to northern buyers. The "marks" express concern that flooding might occur. The developer reassures them by explaining that the average elevation of the lots is 78.5 feet and that the water has never exceeded 25 feet. Would you buy a “used Retirement Home” from this data? “va”
  8. 8. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=100? Swampy Acres Retirement Home Would you buy a “used Retirement Home” from this data? “va” QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  9. 9. National Academy of Neuropsychology, San Antonio, TX, October 2006 What is “variance”? N=100? “normally” distributed? “va” QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  10. 10. National Academy of Neuropsychology, San Antonio, TX, October 2006 Inter-relationship between variables AKA: Shared variance Which variable contributes the MOST variance? Which variables then contribute the most UNIQUE variance that is not shared with the first? Multicollinearity
  11. 11. National Academy of Neuropsychology, San Antonio, TX, October 2006 Multicollinearity
  12. 12. National Academy of Neuropsychology, San Antonio, TX, October 2006 Why use a covariate? Non- Equivalent Groups Design To ANCOVA or not to ANCOVA…
  13. 13. National Academy of Neuropsychology, San Antonio, TX, October 2006 What are the “rules” or “assumptions for ANCOVA? 1. DV is linear and normally distributed 2. There is a homogeneity of regression slopes (slope of regression line is same in each group) 3. There is a theoretical and statistical correlation between the DV and proposed CV To ANCOVA or not to ANCOVA…
  14. 14. National Academy of Neuropsychology, San Antonio, TX, October 2006 What are the “rules” or “assumptions for ANCOVA? 1. DV is linear and normally distributed - easy! • Know your data • Frequency distributions, graphs To ANCOVA or not to ANCOVA…
  15. 15. National Academy of Neuropsychology, San Antonio, TX, October 2006 What are the “rules” or “assumptions for ANCOVA? 1. DV is linear and normally distributed • IV: Concussion Group None, 1+, 2+ Recent • DV: Attention Subscale (RBANS) • CV: GPA? Moser RS, Schatz P. & Jordan, B. (2005), Neurosurgery, 57, 300-306. To ANCOVA or not to ANCOVA…
  16. 16. National Academy of Neuropsychology, San Antonio, TX, October 2006 1. DV is linear and normally distributed - yes! • IV: Concussion Group None, 1+, 2+ Recent • DV: Attention Subscale (RBANS) • CV: GPA? Moser RS, Schatz P. & Jordan, B. (2005), Neurosurgery, 57, 300-306. To ANCOVA or not to ANCOVA…
  17. 17. National Academy of Neuropsychology, San Antonio, TX, October 2006 What are the “rules” or “assumptions for ANCOVA? 2. There is a homogeneity of regression slopes (slope of regression line is same in each group) To ANCOVA or not to ANCOVA…
  18. 18. National Academy of Neuropsychology, San Antonio, TX, October 2006 2. There is a homogeneity of regression slopes (slope of regression line is same in each group) • Simple, but little known method: SPSS syntax MANOVA DV by IV (X,X) WITH CV /PRINT=SIGNIF(BRIEF) /ANALYSIS = DV /METHOD = SEQUENTIAL /DESIGN CV IV CV by IV. To ANCOVA or not to ANCOVA…
  19. 19. National Academy of Neuropsychology, San Antonio, TX, October 2006 2. There is a homogeneity of regression slopes - yes * * * * * * A n a l y s i s o f V a r i a n c e * * * * * * Tests of Significance for NEWATTN using SEQUENTIAL Sums of Squares Source of Variation SS DF MS F Sig of F WITHIN+RESIDUAL 22194.41 225 98.64 GPA 1226.46 1 1226.46 12.43 .001 NEWGRP 744.49 3 248.16 2.52 .059 GPA BY NEWGRP 81.18 3 27.06 .27 .844GPA BY NEWGRP 81.18 3 27.06 .27 .844 (Model) 2052.13 7 293.16 2.97 .005 (Total) 24246.53 232 104.51 R-Squared = .085 Adjusted R-Squared = .056 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - To ANCOVA or not to ANCOVA…
  20. 20. National Academy of Neuropsychology, San Antonio, TX, October 2006 What are the “rules” or “assumptions for ANCOVA? 3. There is a theoretical and statistical correlation between the DV and proposed CV - not so easy! • Statistical - Pearson’s r - simple • Theoretical - based on literature, knowledge - not so simple To ANCOVA or not to ANCOVA…
  21. 21. National Academy of Neuropsychology, San Antonio, TX, October 2006 3. There is a theoretical and statistical correlation between the DV and proposed CV - not so easy! • Statistical - Pearson’s r - simple To ANCOVA or not to ANCOVA…
  22. 22. National Academy of Neuropsychology, San Antonio, TX, October 2006 3. There is a theoretical and statistical correlation between the DV and proposed CV - not so easy! • Theoretical - ugh! “No means of achieving the superficially appealing goal of ‘correcting” or ‘controlling for’ real group differences on a potential covariate” Miller & Chapman (2001). Misunderstanding Analysis of Covariance, J Abn Psych, 110, 40-48 To ANCOVA or not to ANCOVA…
  23. 23. National Academy of Neuropsychology, San Antonio, TX, October 2006 3. There is a theoretical and statistical correlation between the DV and proposed CV - not so easy! • Makes no sense to ask how students with low GPA in school would score on attention test if they had high GPA… • Not all students have a high GPA… • GPA and concussion are intimately related To ANCOVA or not to ANCOVA…
  24. 24. National Academy of Neuropsychology, San Antonio, TX, October 2006 3. There is a theoretical and statistical correlation between the DV and proposed CV - not so easy! • Removal of variance in attention associated with GPA may remove considerable variance in attention associate with concussion To ANCOVA or not to ANCOVA…
  25. 25. National Academy of Neuropsychology, San Antonio, TX, October 2006 Between Groups Differences (MANOVA) IV: Concussion (Yes/No) DV: ImPACT Subscales, Symptom MANOVA: [F(25,514)=4.4; p<.001] Schatz P, Pardini JE, Lovell MR, Collins MW, & Podell K. (2006), Arch Clin Neuropsy, 21, 91-99. Beyond (M)ANOVA…
  26. 26. National Academy of Neuropsychology, San Antonio, TX, October 2006 Can you predict Group membership with p<.001? Beyond (M)ANOVA…
  27. 27. National Academy of Neuropsychology, San Antonio, TX, October 2006 Can you predict Group membership with ANOVA? Beyond (M)ANOVA…
  28. 28. National Academy of Neuropsychology, San Antonio, TX, October 2006 Can you predict Group membership with ANOVA? Beyond (M)ANOVA…
  29. 29. National Academy of Neuropsychology, San Antonio, TX, October 2006 Discriminant Analysis • Characterizes the relationship between a set of IVs with a categorical DV with relatively few categories • Creates a linear combination of the IVs that best characterizes the differences among the groups • Predictive discriminant analysis focus on creating a rule to predict group membership • Descriptive DA studies the relationship between the DV and the IVs. Beyond (M)ANOVA…
  30. 30. National Academy of Neuropsychology, San Antonio, TX, October 2006 Characterizes the relationship between a set of IVs with a categorical DV with relatively few categories • Creates a linear combination of the IVs that best characterizes the differences among the groups • Predictive DA - focus on creating a rule to predict group membership • Descriptive DA - studies the relationship between the DV and the IVs. Discriminant Analysis
  31. 31. National Academy of Neuropsychology, San Antonio, TX, October 2006 Flip the MANOVA upside-down: Between Groups Differences (MANOVA) • IV: Concussion (Yes/No) • DV: ImPACT Subscales, Symptom Predict Group Membership (DA) • DV = “Predictant” = Concussion (yes/no) • IVs = Predictors = ImPACT Subscales, Symptom Discriminant Analysis
  32. 32. National Academy of Neuropsychology, San Antonio, TX, October 2006 Assumptions: • IVs are either dichotomous or measurement • “Normality” • Homogeneity of variances Steps: • Screen data: (outliers, poor predictors, etc) • Run DA • Check for the prediction/classification rate • Check for the importance of individual variables Discriminant Analysis
  33. 33. National Academy of Neuropsychology, San Antonio, TX, October 2006 1. We have a between group difference Discriminant Analysis QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  34. 34. National Academy of Neuropsychology, San Antonio, TX, October 2006 2. The DA returns 1 “function” -it is significant (Chi-Square) -predicts a specific amount of variance (Eigen, Cannonical Correlation) Discriminant Analysis QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  35. 35. National Academy of Neuropsychology, San Antonio, TX, October 2006 3. Classification Table reveals “success” in discriminating between groups Discriminant Analysis
  36. 36. National Academy of Neuropsychology, San Antonio, TX, October 2006 4. Standardized canonical discriminant function coefficients - like beta weights in a regression - rank importance of each variable Discriminant Analysis
  37. 37. National Academy of Neuropsychology, San Antonio, TX, October 2006 5. Correlation coefficients - how much each variable correlates with the function Discriminant Analysis
  38. 38. National Academy of Neuropsychology, San Antonio, TX, October 2006 6. Calculating sensitivity Correct “positive” hits = 81.9% (e.g., the probability that a test result will be positive when a concussion is present) (See: http://www.poems.msu.edu/InfoMastery/Diagnosis/likelihood_ratios.htm) Discriminant Analysis Concussed Control Concussed 81.9% 18.1% Not Conc 10.6% 89.4%
  39. 39. National Academy of Neuropsychology, San Antonio, TX, October 2006 7. Calculating specificity Correct “negative” hits = 89.4% (e.g., the probability that a test result will be negative when a concussion is not present) Discriminant Analysis Concussed Control Concussed 81.9% 18.1% Not Conc 10.6% 89.4%
  40. 40. National Academy of Neuropsychology, San Antonio, TX, October 2006 8. Calculating positive likelihood ratio prob of an indiv with the condition having a pos test = 81.9 = 7.73:1 prob of an indiv w/out the condition having a pos test 10.6 (ratio between the probability of a positive test result given the presence of a concussion and the probability of a positive test result given the absence of a concussion) Discriminant Analysis Predicted-> Concussed Control Concussed 81.9% 18.1% Not Conc 10.6% 89.4%
  41. 41. National Academy of Neuropsychology, San Antonio, TX, October 2006 9. Calculating negative likelihood ratio prob of an indiv with the condition having a neg test = 18.1 = .20:1 prob of an indiv w/out the condition having a neg test 84.9 (e.g., ratio between the probability of a negative test result given the presence of a concussion and the probability of a negative test result given the absence of a concussion) Discriminant Analysis Predicted-> Concussed Control Concussed 81.9% 18.1% Not Conc 10.6% 89.4%
  42. 42. National Academy of Neuropsychology, San Antonio, TX, October 2006 Dependent samples analyses: • Change from Time 1 to Time 2 • IV=Baseline versus Post-concussion • DV=ImPACT • Wilke’s Lambda revealed a multivariate within-subjects effect (time) on ImPACT performance [F(2,76)=1670; p=.000; ES=.98], with significant declines noted from baseline-to-3 days (eta squared=.56). Beyond ANOVA
  43. 43. National Academy of Neuropsychology, San Antonio, TX, October 2006 Is change from Time 1 to Time 2 a reliable change? • Reliable Change Index (RCI) is computed for each case: • Time 2 - Time 1 / Sediff • Report total percentage of cases with “reliable change” • Report RCI intervals for that measure Beyond ANOVA - Reliable Change?
  44. 44. National Academy of Neuropsychology, San Antonio, TX, October 2006 RCI Articles: Jacobson & Traux (1991). Journal of Consulting and Clinical Psychology, 59, 12-19. Chelune, et al. (1993). Neuropsychology, 7, 41-52. Collie, et al. (2003). British Journal of Sports Medicine, 37, 370-372 Beyond ANOVA - Reliable Change?
  45. 45. National Academy of Neuropsychology, San Antonio, TX, October 2006 Beyond ANOVA - Reliable Change? (Table adapted from Iverson, et al., 2001, Lupus)
  46. 46. National Academy of Neuropsychology, San Antonio, TX, October 2006 Beyond ANOVA - Reliable Change? Using Excel: QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
  47. 47. National Academy of Neuropsychology, San Antonio, TX, October 2006 Beyond ANOVA - Reliable Change? Baseline-to-Time 1: 58% showed “reliable change’ in at least one composite score 27% showed “reliable change’ in two or more composite score Required “Change Intervals”: Reaction Time: 0.17 Verbal Memory: 19% Visual memory: 29% Processing Spd: 19.2 Symptom Score: 15 Covassin, Schatz, Swanik, (in review), Neurosurgery
  48. 48. National Academy of Neuropsychology, San Antonio, TX, October 2006

×