Statistical applications in GraphPad Prism

1. 4/6/2010
1
Statistical Applications in PrismStatistical Applications in Prism
Presented by: Jeff Skinner, M.S.
Biostatistics Specialist
Bioinformatics and Computational Biosciences Branch
National Institute of Allergy and Infectious Diseases
Office of Cyber Infrastructure and Computational Biology
GraphPad PRISM® at NIAID
• Free site license download• Free site license download
• Free training, consultation and
support from BSIP ***
• Statistical results and help
menus designed specifically for
biological researchers *** Please contact:
ScienceApps@niaid.nih.gov

2. 4/6/2010
2
NIAID Site License
http://graphpad.com/paasl/index.cfm?sitecode=NIHNIAID
http://bioinformatics.niaid.nih.gov/
Statistical Analyses in PRISM®
• XY data tables • Grouped data tables
– Linear regression
– Nonlinear regression
– Correlations
– Analysis of covariance
(ANCOVA)
• Column data tables
– Two‐way ANOVA
– Repeated measures ANOVA
• Contingency data tables
– Chi‐square tests
– Relative Risks, odds ratios,
sensitivity and specificity
Column data tables
– One‐sample, two‐sample
and paired t‐tests
– One‐way analysis of variance
(ANOVA)
– Nonparametric tests
• Survival data tables
– Kaplan‐Meier tests
– Log‐rank tests

3. 4/6/2010
3
Research Problems
• Dr. Smith wants to compare pre‐ and post‐treatment polyp counts
among men and women in a colon cancer studyamong men and women in a colon cancer study
– Paired t‐tests and two‐sample t‐tests
• Dr. Lee wants to evaluate the relationship between gestation age
of a parasite and the efficacy of a new anti‐parasite drug
– Linear regression methods
• Dr. Herrera wants compare purified protein yield among three pH
values and two temperatures
– One‐way and two‐way ANOVA methods
T‐tests and Nonparametrics
• Dr. Smith wants to test a new drug that
reduces the number of colon polyps in
patients to help prevent colon cancer
– Do patients have fewer colon polyps after
taking the new drug? (i.e. paired t‐test)
– Is the drug more effective for either women or
men? (i.e. two‐sample t‐test)
• What is a t‐test and how does it work?
– Why use a one‐sided test?
– Why use a paired t‐test?
– Why use a nonparametric test?

4. 4/6/2010
4
Statistical Testing
• Formulate null and alternative hypotheses
– Null and alternative hypotheses are mutually exclusive and
exhaustive statements about the population
– Typically assume the null hypothesis is true, until we find
evidence to refute the null in favor of the alternative
– E.g. H0: µ = 0 versus HA: µ ≠ 0
C l l t th i t t t t ti ti d fi d it• Calculate the appropriate test statistic and find its
probability under the null hypothesis
• Make a statistical decision and biological conclusion
T‐tests and Sampling Distributions


errorstandard
valuenullstatistic*
t
• A statistical test is typically the ratio of a difference over an error








n
dev.std.sample
0meansample
errorstandard
– Is the difference between the statistic and null value large relative to the error?
• The Central Limit Theorem implies the means from numerous samples of size n
will have a normal distribution centered on the population mean
– t* follows a student’s t‐distribution to account for the unknown std. dev.
• One‐sided and two‐sided tests are determined by our hypotheses
– One‐sided tests ignore some alternatives for more power

5. 4/6/2010
5
Paired vs. Independent Data
• Often experiments collect two types of
t f h t t bj t
Subject Pre- Post- Diff.
measurements from each test subject
– E.g. pre‐ and post‐treatment measurements
• Two observations collected from the same
subject should be more similar than any
two measurements from different subjects
(repeated measures)
PRISM t “ i i ffi i ”
Fred 87 43 44
Barney 23 14 9
Wilma 45 44 1
Betty 54 52 2
– PRISM reports “pairing efficiency”
• Examine differences between paired
measurements to increase the power to
detect differences between groups
Pebbles 45 21 24
Bambam 45 29 16
… … … …
Finding Things in PRISM
Tools are accessed using
menu buttons
Use the “Info” folder to
store laboratory notes
Hold cursor over any folder
item for a thumbnail image
Cut and paste data into a
data table from MS Excel
Graphs and results from
statistical tests are stored
in separate folders
Use “Layouts” folder to tile
and group figures or overlay
one figure on another
Insert floating notes to
leave a helpful reminder to
yourself or other users

6. 4/6/2010
6
Paired T‐tests
• Did the treatment reduce the number of
polyps in the male patients?p yp p
• Click Analyze > t‐tests (and
non‐parametrics), then select paired t‐test
• Results yield a t* test statistic, p‐value,
pairing efficiency, the average paired
difference and its confidence interval
– If the mean paired difference is significantly
different from zero the treatment worksdifferent from zero, the treatment works
• Choose the Before / After figure or create
a column of paired differences to produce
a box plot of the differences
Two‐sample T‐tests
• Do men and women experience equal
red ctions in pol ps post treatment?
Two-sample t-test
reductions in polyps post treatment?
• Click Analyze > t‐tests (and non‐
parametrics) and select unpaired t‐test
• Results yield a t* test statistic, p‐value,
individual means and CI’s, difference
b h d
-40
-20
0
between the two means and its CI
– If the difference between male and female
samples is significantly different from zero, the
treatment is more effective for one sex

7. 4/6/2010
7
Nonparametric Statistics
• T‐tests are a parametric test, because they
d ll d b dassume data are normally distributed
• Nonparametric tests evaluate medians and ranks
with no assumption of normal distributions
– Wilcoxon Signed Rank used in place of one‐sample t‐g p p
test or paired t‐test
– Mann‐Whitney (Wilcoxon Rank Sum) used in place of
two‐sample t‐test
Simple Linear Regression
• Dr. Lee thinks the relationship
between age and drug efficacy is
Linear regression
250
between age and drug efficacy is
described by a straight line
• Linear regression finds the best
fitting line through the XY plot
using “least squares”
Regression equation:
0 2 4 6 8 10
0
50
100
150
200
Gestation Age
Efficacy(#parasiteskilled)
• Is the slope significantly different
from zero?
– Zero slope = no relationship
between X and Y
Regression equation:
Efficacy = 28.48 +17.96*(Gestation Age)
Regression equation is used to
predict efficacy at a specific value
of gestation age
Y = 0 + 1X + 

8. 4/6/2010
8
What Is Least Squares?
• The best fitting line minimizes simple linear regression
the errors between XY data
points and the fitted line
• Errors are both positive and
negative, so we square the errors
to simplify minimization
-10 -8 -6 -4 -2
-100
0
100
200
300
400
500
error 
2
• Squared errors are minimized by
calculus to find the best fit
estimates of slope and y‐int


 j
efficacyi
observed value
 
 0  1agei
predicted value
  
error  






i
  0
ˆ1 
Xi  X Yi Y 
i

Xi  X 
2
i

, ˆ0  Y  ˆ1X
Entering Data and Creating Graphs
• Choose the XY table and graph
– Select your graph type
– Select options for simple XY pairs,
replicates or error measurements
• Cut and paste data from Excel®
• Find data tables, graphs and
statistical results in the PRISM®
Navigator bar on the left menuNavigator bar on the left menu
• Graphs automatically created, but
easily edited using point and click
menus in PRISM®

9. 4/6/2010
9
Perform the Regression Analysis
• Click Analyze > Linear Regressiony g
• Choose options for interpolation,
graphing, regression through the origin
(RTO) and replicate values
• Select graphing options to modify the g p g p y
automatically created figure
• Check the option box for a residual plot
to check model assumptions
Linear Regression Results
• Find coefficients and
confidence intervals for
slopes and both Y‐ and
X‐intercepts
• R2 and sy.x model
goodness of fit stats
– R2 is the coefficient of
determination
– sy.x = std. dev. of the
residuals
• Hypothesis test results
for the slope

10. 4/6/2010
10
More About R2
• R2 is the percent of the variation in Y that is p
explained by the changes in X
– R2 = SSR / SST = var(model) / var(total)
• When two models meet their assumptions,
the model with the higher R2 fits best
• R2 is meaningless if one or both models fail to
meet their assumptions
Graphs and Diagnostics
• Regression procedure automatically adds
fitted regression line to figures
Linear regression
300g g
• Add confidence or prediction bands with
90%, 95% or 99% confidence
– Confidence interval describes the certainty of
the estimated regression line
– Prediction interval describes the certainty of a
single predicted observation
0 2 4 6 8 10
0
100
200
Gestation Age
Efficacy(#parasiteskilled)
Linear regression:Residuals
• Check model assumptions with residual
plot and tests
0 5 10 15
-15
-10
-5
0
5
10
15
Gestation Age
Residuals

11. 4/6/2010
11
Model Assumptions
• Residuals, or random errors, should be
Nonlinear regression:Residuals
100
Old Drug
N D
es dua s, o a do e o s, s ou d be
independent and identically normally
distributed
• Plot of residuals vs X variable should
show constant variance
– Good: rectangle or oval shape
5000 10000 15000 20000
-40
-20
0
20
40
Eliza Units
Residuals
-12 -10 -8 -6 -4 -2
-100
-50
0
50
New Drug
Dose
g p
– Bad: strong “cone” shape
• Histogram of residuals should be
normal, or bell‐shaped
-60
-60
-50
-40
-30
-20
-10
0
10
20
30
0
10
20
30
40
50
Bin Center
Percent
Some Limitations of PRISM®
• PRISM® cannot perform most multiple regressionsp p g
– PRISM® does not accept multiple X predictors or covariates
– Analysis of covariance (ANCOVA) or multiple regression with dummy
variables can be performed using multiple Y‐variables
– Polynomial regression is available among the non‐linear regression
model procedures
• PRISM® cannot perform logistic regression models orPRISM cannot perform logistic regression models or
proportional hazards regression models
– Logistic regression has a categorical response
– Proportional hazard regression models typically evaluate factors
affecting time until death or failure

12. 4/6/2010
12
ANCOVA Example
• Dr. Lee wants to compare the new drug to an older
drug
D b th d h th l ti hi ith– Do both drugs share the same relationship with
gestation age? I.e. are slopes equal?
– Is the new drug always better than the older drug?
I.e. are means equal?
• Two statistical approaches to these questions
– ANCOVA is a comparison of means in the presence of
a continuous nuisance factor
– Regression with dummy variables produces a single
d f lprediction equation for several groups
• PRISM’s approach is a nice compromise
– Separate regressions for each group
– Global tests for common slope and intercepts
Perform the ANCOVA
• Click Analyze > Linear Regression and
select both responses
• Be sure to check the box labeled “test
whether slopes and intercepts are
significantly different”
• Other options identical to simple linear
regression, shown earlier

13. 4/6/2010
13
Results and Diagnostics
• ANCOVA produces simple linear
regression results for all response
Dummy variables (ANCOVA)
250
d)
regression results for all response
variables and a separate page for tests of
equal slopes and means
– If slopes are unequal, analysis stops
– If slopes are equal, common slope is reported
and means are tested
• Check model diagnostics, just like in
0 5 10 15
0
50
100
150
200
Old Drug
New Drug
Gestation Age
Efficacy(#parasiteskilled
Check model diagnostics, just like in
simple linear regression
• Must find coefficients for dummy
variable and interaction by hand
Nonlinear Regression Methods
• PRISM® offers a number of non‐linear
log-dose vs response
500
No inhibitor
regression models, many designed for
specific experiments
– Dose / Response models with EC50
– Binding and enzyme kinetics models
– Sine curves, exponentials, Gaussian models
and Lowess smoothing curves
– Polynomial regression in this menu
-10 -8 -6 -4 -2
-100
0
100
200
300
400
No inhibitor
Inhibitor
log[Agonist], M
response
y g
• A BSIP training seminar titled
“Nonlinear Regression in PRISM”
explains these methods in detail

14. 4/6/2010
14
Survival Analyses
• If Dr Lee could measure time until Survival Analyses:Survival proportionsIf Dr. Lee could measure time until
death for each parasite, a
population survival curve can be
estimated with survival tables
• Kaplan‐Meier survival curves and
estimates with 95% CIs
Survival Analyses:Survival proportions
0 500 1000 1500 2000
0
20
40
60
80
100 Standard
Experimental
Days
Percentsurvival
estimates with 95% CIs
• Log‐rank tests compare survival
curves among groups
Contingency Tables and Chi‐Square
• Dr. Lee has categorical measures from
a meta‐analysis studya meta‐analysis study
– High and low gestation ages
– Large and small % parasites killed
• Choose the contingency table data
format and enter a 2 x 2 table
• Chi‐square analysis tests for a
significant association between
columns and rows of the table

15. 4/6/2010
15
Pearson Chi‐Square Test
• Chi‐square tests always have the Obs Exp 
2
Chi square tests always have the
same hypotheses
– H0: no relationship between rows
and columns of table
– HA: there is a relationship
• Calculate expected values for
• Calculate the test statistic to
determine if observed data
have no relationship between
rows and columns
2

Obsij  Expij 
Expiji, j

p
each cell under the null, H0
rows and columns
– Small 2 statistics support the
null hypothesis, while a larger 2
statistic refutes the null
– P‐values found on theoretical chi‐
square distributiontotal
totalcolumnrow total
Exp
ji
ij


Calculate the Chi‐Square Test
• Click Analyze > Chi‐square test
• Select chi‐square test or Fisher’s exact
test from the menu
– Chi‐square requires large sample sizes
– Fisher’s exact has strong assumptions
• Choose options for relative risk, odds
ratios, sensitivity and specificity, etc.
– Relative risk and odds ratio help interpret the
strength of the association

16. 4/6/2010
16
Chi‐square Test Results
• P‐value from the chi‐square test
indicates if relationship between the
Chi-square analysis
300indicates if relationship between the
two variables is significant
• Relative risk and odds ratios indicate
strength of association
– RR = 1 or OR = 1 indicates there is no
relationship between the variables
• Sensitivity and specificity reflect the
Old Drug New Drug
0
100
200
300
Most Parasites Killed
Few Parasites Killed
Count
accuracy of a (medical) test
– Sensitivity = Pr( + | have disease )
– Specificity = Pr( ‐ | no disease )
Some Limitations of PRISM®
• PRISM® cannot perform McNemar’s test for paired
contingency table data
– E.g. McNemar’s test would be used when paired observations are
made for two categorical variables in a cross‐over design
• PRISM® cannot perform Mantel‐Haenszel tests for multiple 2
x 2 contingency tables
– E.g. MHC tests are used to determine if chi‐square test results differ
(Si ’ d )among groups (Simpson’s paradox)
• PRISM® cannot be used for log‐linear models
– E.g. Log‐linear models are used to analyze more complicated
experiments with categorical responses

17. 4/6/2010
17
One‐way and Two‐way ANOVA
• Dr. Herrera wants to maximize yield in a
fprotein purification experiment
– Do yields differ among three pH groups?
– Do temperature and pH both affect yield?
– What pH and temperature has highest yield?
• ANOVA model also fit using least squares with• ANOVA model also fit using least squares with
the same assumptions as regression
– Independent and identical normal errors
Variation Within and
Between Groups
• ANOVA is used to compare 3 orANOVA is used to compare 3 or
more “cell means”, but it really
divides the variance into two
different partitions
– Within group variation, sW
2
– Between group variation, sB
2
• If sB
2 is larger than sW
2, the sampling
distributions do not overlap and F =
2 / 2 i lsB
2 / sW
2 is large
• If sB
2 is smaller than sW
2, the
sampling distributions overlap and F
is small (not significant)

18. 4/6/2010
18
Post Hoc Multiple Comparisons Tests
• The ANOVA F‐test indicates there are differences
among groups, but not where the differences are
• Individuals tests or CI’s reveal specific differences
among groups, but often yield false positives
– Alpha5 tests = 1 ‐ (0.95)5 = 0.23 > 0.05p 5 tests ( )
• Post Hoc tests like Bonferroni, Tukey and Dunnett tests
correct for this multiple testing problem
One‐way ANOVA Results
• Dr. Herrera wants to compare purified
protein yield among three pH values
One-way ANOVA
150
protein yield among three pH values
• Enter data into a column data sheet
• Click Analyze > One‐way ANOVA (and non‐
parametric) , then select option for a post
hoc multiple comparisons test
• Results include F* and its p‐value, a Bartlett’s
test for equal variances, an ANOVA table and
B
asic
N
eutral
A
cidic
0
50
100
pH
ProteinYield
q
post hoc test results
• Use the post hoc tests and graph to
determine how the groups differ

19. 4/6/2010
19
Two‐way ANOVA Results
• Dr. Herrera also wants to test two
t t d i th i t
Two-way ANOVA
100
Room Temperature
temperatures used in the experiment
• Enter data into a grouped data sheet
• Click Analyze > Two‐way ANOVA, then
select regular two‐way ANOVA or one of the
repeated measures options
• Select options for post hoc tests
B
asic
N
eutral
A
cidic
0
20
40
60
80 Increased Temperature
pH
ProteinYield
• Output includes F* stats and p‐values for
each factor, post hoc test results and an
explanatory narrative page
Some Notes on Two‐way ANOVA
• You must have replication in your experiment to ou us a e ep ca o you e pe e o
evaluate an interaction effect
– I.e. at least two observations for each unique combination of
predictor variable values
• You DO NOT interpret main effects when there is a
statistically significant interaction
• You do not typically evaluate interactions, when one
predictor represents “blocks”
– E.g. no interaction in a RCB design, etc.

20. 4/6/2010
20
Nonparametric Statistics
• ANOVA is parametric, but some nonparametric
lt ti il bl i PRISMalternatives are available in PRISM
– Use Kruskal‐Wallis for one‐way ANOVA
– Friedman’s test for nonparametric repeated measures is found
in the columns data table
– Nonparametric two‐way ANOVA with interactions is a more
difficult problem p
• Nonparametric analyses compare sums of ranks or
medians instead of means and variances
Some Limitations of PRISM®
• PRISM® does not estimate random effects
– PRISM® only calculates Type I fixed effects, like SAS PROC GLM
and other software like Minitab, etc.
– Modern procedures like SAS PROC MIXED and R/Splus
calculate Type II and Type III random and mixed effects, with
covariance structures specified by the user (e.g. AR1 models,
Toeplitz structure, etc.)
• PRISM® cannot evaluate nested effects
– E.g. car manufacturer and model (i.e. Ford Mustang)