Comparing Correlated Proportions With McNemar’s Test
Suppose you are evaluating the effectiveness of an intervention which is
designed to make patients more likely to comply with their physicians’ prescriptions.
Prior to introducing the intervention, each of 200 patients is classified as compliant or
not. Half (100) are compliant, half (100) are not. After the intervention you reclassify
each patient as compliant (120) or not (80). It appears that the intervention has raised
compliance from 50% to 65%, but is that increase statistically significant?
McNemar’s test is the analysis traditionally used to answer a question like this.
To conduct this test you first create a 2 x 2 table where each of the subjects is classified
in one cell. In the table below, cell A includes the 45 patients who were compliant at
both times, cell B the 55 who were compliant before the intervention but not after the
intervention, cell C the 85 who were not compliant prior to the intervention but were after
the intervention, and cell D the 15 who were noncompliant at both times.
After the Intervention
Compliant (1) Not (0) Marginals
Prior to the Compliant (1) 45 A 55 B 100
Not (0) 85 C 15 D 100
Marginals 130 70 200
McNemar’s Chi-square, with a correction for continuity, is computed this way:
(| b − c | −1)2 (| 55 − 85 | −1)2
. For the data above, χ =
= 6.007 . The chi-square is
b+c 55 + 85
evaluated on one degree of freedom, yielding, for these data, a p value of .01425.
If you wish not to make the correction for continuity, omit the “−1.” For these data
that would yield a chi-square of 6.429 and a p value of .01123. I have not investigated
whether or not the correction for continuity provides a better approximation of the exact
(binomial) probability or not, but I suspect it does.
McNemar Done as an Exact Binomial Test
Simply use the binomial distribution to test the null hypothesis that p = q = .5
where the number of successes is either count B or count C from the table and N = B +
C. For our data, that is, obtain the probability of getting 55 or fewer failures in 85 + 55 =
140 trials of binomial experiment when p = q = .5. The syntax for doing this with SPSS
and the value computed is .01396.
McNemar Done With SAS
Input Prior After Count; Cards;
1 1 45
1 0 55
0 1 85
0 0 15
Proc Freq; Tables Prior*After / Agree; Weight Count; run;
Statistics for Table of Prior by After
Statistic (S) 6.4286
Asymptotic Pr > S 0.0112
Exact Pr >= S 0.0140
The “Statistic” reported by SAS is the chi-square without the correction for
continuity. The “Asymptotic Pr” is the p value for that uncorrected chi-square. The
“Exact Pr” is an exact p value based on the binomial distribution.
McNemar Done With SPSS
Analyze, Descriptive Statistics, Crosstabs. Prior into “Rows” and After into
“Columns.” Click “Statistics” and select “McNemar.” Continue, OK
Prior * After Crosstabulation
0 1 Total
Prior 0 15 85 100
1 55 45 100
Total 70 130 200
Exact Sig. (2-
McNemar Test .014a
N of Valid Cases 200
a. Binomial distribution used.
Notice that SPSS does not give you a chi-square approximate p value, but rather
an exact binomial p value.
McNemar Done With Vasser Stats
Go to http://faculty.vassar.edu/lowry/propcorr.html
Enter the counts into the table and click “Calculate.”
Very nice, even an odds ratio with a confidence interval. I am impressed.
More Than Two Blocks
The design above could be described as one-way, randomized blocks, two levels
of the categorical variable (prior to intervention, after intervention). What if there were
more than two levels of the treatment – for example, prior to intervention, immediately
after intervention, six months after intervention. An appropriate analysis here might be
the Cochran test. See http://en.wikipedia.org/wiki/Cochran_test.
See also McNemar Tests of Marginal Homogeneity at
Return to Wuensch’s Stats Lessons
Karl L. Wuensch, July, 2008