Sign Test testing the creativity of an Ad campaign Classic situation where we need to use a nonparametric test. This is because the samples are small (17 observations), and the variables are not normally distributed (check Skewness and Kurtosis).
Sign Test testing the creativity of an Ad campaign (continued)
Put both samples together. Rank values in ascending order. For repeated numbers (ties) across samples, use the average of their ranks so that identical numbers get identical ranks.
Find the average rank for each sample.
Calculate Difference in average rank.
Find the Standard Error for the average difference in the ranks: (n 1 + n 2 )[SQRT(n 1 + n 2 + 1)/(12n 1 n 2 )].
Divide the Difference in avg. rank (step 3) by the Standard Error (step 4) to find the test statistic (a Z value).
Calculate P Value using NORMSDIST.
Note Steps 2 through 6 are similar to the unpaired t Test except it uses Ranks instead of Values .
Example: Testing income of mortg. applicants Testing if applicants for fixed-rate mortgages have higher income than applicants for variable-rate mortgages. The fixed-rate applicants have one high income value ($240,000). Kurtosis and Skewness of both samples confirm they are not normally distributed. The unpaired t test would not work well.
Sorting and Ranking Ranked in ascending order. The figures in yellow are identical ($36,500). They originally ranked 12 th , 13 th , and 14 th . So, they all received the tied ranking of 13 th .
P Value (probability difference is due to chance) Based on ranks (not values), there is an 18.3% probability the two samples come from same population. There is a 81.7% probability that the Variable Rate mortgage applicants have a higher income because they have a higher average rank (17.79 vs 13.50).
Breaking off the ties may not have much impact. Having redone the last example without breaking the ties, depending on how the yellow figures got ranked you get P values of 17.0% or 19.8% not much different than the 18.3%.
Important caveat . You need at least 10 observations for each of the two unpaired samples you test for to obtain a valid Z variable to calculate a P value.
Mann-Whitney U is calculated differently than as shown that reflects calculations by Andrew Siegel that gets the same result faster. See Appendix on next slide.
Appendix: The actual Mann-Whitney U Calculation