The document discusses hypothesis testing and proportion tests. It provides an overview of hypothesis testing terminology and steps. It also gives examples of using one-proportion and two-proportion tests to analyze business data on regulatory compliance documentation and workload balance between regions. The null hypothesis is tested in each example to determine if there are statistically significant differences between the proportions.

1. Proportion Testing Chris Connors, Ph.D. Jay Armstrong, MSc., M.C.E. October 2, 2009 Statistics Symposium

2. Applied Statistics in Business, Healthcare, Pharmaceuticals, Education,and Industry

3. Outline What we are covering and what we are not covering today Virtual Scavenger Hunt Statistical Decisions and Risk Six Sigma DMAIC application The Business Approach Hypothesis Test Approach Understanding Distributions Sample Size Test of Independence Example 1: Regulatory Compliance Documentation Example 2: Workload Balance (Productivity) References and Web Sites Q&A

4. Hypothesis Tests: What we are covering? Continuous Data Attribute Data 1 sample t-test : Δ mean from known test mean 2 sample t-test: Δ mean between 2 independent sample means Paired t-test: Δ mean between 2 dependent sample means One Way ANOVA : At least 1 sample mean Δ between 3 or more samples Kruskal Wallis & Mood’s Median: At least 1 sample median Δ between 3 or more samples F-test, Levene’s test, & Bartlett’s test: At least 1 sample standard deviation Δ between 3 or more samples Correlation/Regression/DOE: 2 or more factors are correlated/ Predictor affects the sampled process 1 proportion test: A sample proportion Δ against a known value 2 proportion test: Proportions from the two samples are different Chi Square test: At least one sample proportion Δ from others:

5. Scavenger Hunt Find another person who can sign off on these statements. Each person can only sign once. 1. Has used Chi-Square or Proportion Test 2. Has more than $50 on them 3. Used Minitab to determine sample size 4. Worked on a project with a value proposition >$1 million 5. Knows Chris Connors' middle name 6. Has more than three children 7. Has met a movie star or celebrity (and was not arrested) 8. Knows the difference between a confidence interval and a confidence level 9. Knows what a quark or a fantod is 10. Has more than one academic degree, license or certification

6. Statistical Decision: Setting up your risk level Type I and II errors There are two kinds of errors that can be made in significance testing: a true null hypothesis can be incorrectly rejected and a false null hypothesis can fail to be rejected. The former error is called a Type I error and the latter error is called a Type II error. These two types of errors are defined in the table. The probability of a Type I error is designated by the Greek letter alpha (  ) and is called the Type I error rate; the probability of a Type II error (the Type II error rate) is designated by the Greek letter beta (ß). A Type II error is only an error in the sense that an opportunity to reject the null hypothesis correctly was lost. It is not an error in the sense that an incorrect conclusion was drawn since no conclusion is drawn when the null hypothesis is not rejected.

7. Six Sigma DMAIC method: Hypothesis Tests Six Sigma DMAIC method has 5 phases: Define Opportunity/Problem Measure Performance Analyze Process and Performance Improve Process and Performance Control Process and Performance I typically use this diagram to depict the continuous focus of measurement in the Six Sigma method by placing Measure in the center of the DMAIC method. Measure Define Control Improve Analyze

8. 6S Black Belt Level of Cognition for Hypothesis Testing *1= learned, 2= know, 3 = used, 4 = taught Topic Level of Cognition My Development Introduction to Statistical Comparisons 2* Normality and Transformation 2 Correlation Analysis 3 Regression Analysis 3 Introduction to Multiple Linear Regression 1 t-tests 3 ANOVA 3 1 and 2 proportion test 3 Chi-Square Analysis 3 Binary Logistic Regression 1

9. 6S BB Level of Cognition for Hypothesis Testing *1= learned, 2= know, 3 = used, 4 = taught Topic Level of Cognition My Development Introduction Experimental Design 2* Background on Experimental design 2 DOE Designs and terminology 2 Full Factorial design 2 Half factorial designs 2 Robust Designs 2 Checklists for designing and conducting DOE BBC exercise: DOE Simulation 2 Results of DOE Simulation 2

10. The Business Approach Proportion Tests When we want to make a statistical comparison of a discrete variable with a target, or between two discrete variables, Proportion Tests should be used. Statistical Problem Statistical Problem Business Problem Business Solution Decision Statistical Solution Potential Root Causes Identified Root Causes Verified

11. Selecting the Right Statistical Tool Discrete Discrete Continuous Proportion Tests Logistic Regression t test ANOVA DOE Correlation Regression X Y Continuous

12. Determine if a statistically significant difference of proportion exists between: - A sample and a target - Two independent samples - Two samples or less Tests of Proportion Use samples to make inferences about population proportions 1 Proportion Test 1 Sample Comparing Proportions 2 Proportion Test Chi-Square Test More Than 2 Samples 2 Samples

13. Proportion Test Approach State the null and alternative hypotheses Null H 0 P 1 = P 2 Number of tails = 2 P 1 - P 2  0 Number of tails = 1 P 1 - P 2  0 Number of tails = 1 Alternatives H a P 1 - P 2  0 P 1  P 2 Number of tails = 1, left or right P 1 - P 2  0 P 1 - P 2  0 2. Formulate an analysis plan : 1 Proportion to known value (z) or 2 Proportions test 3. Analyze sample data Independence Test: Fisher’s, Barnard’s, G-Test Pooled sample proportion to compute standard error P value for test statistic 4. Interpret results: for a statistical decision (hopefully a business decision, not not always) If P is low, H 0 must be no go

14. One Tail or Two Tails: Placing the Alpha Risk

15. Useful Discrete Distributions Binomial distribution for: The number X of successes (or failures!) in n trials when p is the chance of success (or failure!) or each trial. Examples: number X of faulty expense reports out of n =100 submitted in a particular month, when the faulty expense report rate typically runs at p =0.03 (i.e., 3%) number of voters out of a random sample of n =800 expressing approval of the President’s performance, when the approval rating in the entire population of voters is p =0.42 (i.e., 42%) X is discrete : it must be one of 0, 1, 2, … , n

16. Binomial - key facts Useful fact: has approximately a normal distribution when n is large (more than 25 or 30) and np and n( 1 -p) are not too small (say >5).

17. Binomial - Normal Approximation

18. Histogram: n=20

19. Histogram: n=100

20. Sample Size General Guidelines (if not followed, test may not run): Each Sample includes at least 10 failures and 10 successes (some texts say 5) The sample is from a population 10 x the sample Use Minitab sample size calculator Use TI 83 or TI 84 Graphing Calculator (see web)

21. Hypothesis testing - terms Null hypothesis (H 0 ) – e.g., µ 1 = µ 2 - this is the hypothesis to be tested and should be in the form of a true/false statement . This hypothesis states that there is NO DIFFERENCE between the data sets or samples or populations. Null hypotheses are never accepted – we either reject them or fail to reject them. The null hypothesis has PRIORITY and should not be rejected unless there is strong statistical evidence to do so. Alternate hypothesis (H 1 , H A ) – e.g., µ 1 ≠ µ 2 - the alternative to the null hypothesis – states that there IS A DIFFERENCE between the data sets or populations. Type 1 error – rejecting the null hypothesis when it is really true – e.g., “convicting the innocent” Type 2 error – failing to reject the null hypothesis when it really is false – e.g., “letting the guilty go free” Level (or size) of a test = Alpha ( α ) – is the probability of a type 1 error – default = 5% Beta ( β ) – is the probability of a type 2 error – default = 10% Power of a test or power – is the probability of correctly rejecting a false null hypothesis. Since β is the probability of a type I error, power is calculated by the formula (1 - β ). Power = (1 - β ) when the null hypothesis is false. The default value for power is 90% This means that you have an 90% chance of finding a difference when you really want to find it. Critical region (rejection region) – set of values of the test statistic that cause the null hypothesis to be rejected. If the test statistic falls into the rejection region, the null hypothesis is rejected.

22. Hypothesis testing steps State the null hypothesis H 0 and the alternate hypothesis H A (e.g., the mean incomes of college graduates does not equal that of other people) Choose the level of significance, alpha ( α default = 0.05) and the sample size (default n = 25) Choose the appropriate statistical techniques (t test, Chi-square, etc.,) and test statistic (e.g., mean) Collect the data and calculate the sample value of the test statistic Calculate the p value based on the test statistic and compare it with alpha ( α = 0.05) Make a statistical decision – if p is greater than or equal to alpha, fail to reject the null hypothesis. If the p value is less than alpha, reject the null hypothesis.

23. Hypothesis tests are either one tailed or two tail tests Fail to Reject H 0 Reject H 0 1% or 5% significance level Fail to Reject H 0 Reject H 0 One tail test - Answers only ONE question - is the test statistic less than or greater than the known distribution Fail to Reject H 0 Reject H 0 Reject H 0 Two tailed test – Only asks if the test statistic is different from the known distribution – H A usually has “not equal to” in the wording 2.5% significance level 2.5% significance level

24. Clinical Testing One-tailed example by hand The “Feel Good” Drug company has discovered a new drug which prevents acne. Since the market for skin care products is larger for woman than men, the company would like to be able to show a treatment advantage for women vs men. The company statistician chooses a simple random sample of 110 women and 207 men from a population of 100,000 healthy volunteers. After 6 months, 48% of women had no acne, vs 61% of men. Can the company claim a benefit for women vs men at the 0.01 level of significance? What are the hypotheses? Calculate the pooled sample proportion and the Standard Error and consult the z-score statistic What do the results tell us?

25. Clinical Testing One-tailed example by hand 1) What are the hypotheses? Ho - P1 = P2 Ha – P1 < > P2 The null hypothesis will be rejected if the proportion of women developing acne (p1) is substantially smaller than the proportion of men developing acne (p2) Calculate the pooled sample proportion and the Standard Error and consult the z-score statistic: P = (p1 * n1 + p2 * n2)/(n1 + n2) = [(0.48 *110) + (0.61 * 207)]/(110 + 207) = 52.8 + 126.3 / 317 = 0.564 SE = sqrt { p * (1 - p) * [(1/n1) + (1/n2)]} = [ 0.564 * 0.436 * (1/110 + 1/207) = sqrt 0.245 * (0.009 + 0.005) = 0.058 Z = (p1 - p2)/SE = (0.48 - 0.61) / 0.058 = -2.24 Since this is a one tailed test, the P value is the probability that the z-score is less than -2.24. The Normal distribution calculator for P (z < -2.24) = 0.013 P value = 0.013. Since 0.013 is greater than the chosen significance level (0.01), WE FAIL TO REJECT THE NULL HYPOTHESIS – THERE IS NO STATISTICAL DIFFERENCE BETWEEN THE POPULATIONS

26. Test of Independence Fisher’s Exact Test is most commonly used for 2 x 2 tables to determine if there is a nonrandom relationship between two categorical variables. Fisher’s calculates conditional probability for the observed row and column matrix . Fisher’s exact test in Minitab: Rows: adverse Columns: drug new old All n 90 80 170 y 210 120 330 All 300 200 500 Cell Contents: Count Fisher's exact test: P-Value = 0.0265193

27. Regulatory Compliance Documentation Sample Size: Minitab

28. The Business Approach 1-Proportion Test Statistical Problem Statistical Problem Business Problem Business Solution Decision Statistical Solution Potential Root Causes Identified Root Causes Verified

29. A Black Belt is studying the company’s ability to get regulatory compliance documentation to the record center with in 5 days from project completion. What is the binomial characteristic? A random sample of 130 project documentation records showed that 74 of them met the 5 day deadline. The business was heard saying “at least we’re over the half way mark!” Test the hypothesis at 95% confidence that more than 50% of engagements met the deadline. What is the Null Hypothesis? Regulatory Compliance Documentation Example

30. H o : The proportion of compliance documentation filed at the record center on time is 50% (interim target value). H a : The proportion of external work papers filed at the record center on time is greater than 50%. Note: Typically the alternative is stated as “there is a difference.” Why does this example state “greater than?” Regulatory Compliance Documentation Example - Hypothesis

31. Tool Bar Menu > Stat > Basic Statistics > 1 Proportion Analysis Compliance Documentation Example – Minitab Commands target

32. Compliance Documentation Example – Minitab Results What’s our interpretation? Test and CI for One Proportion Test of p = 0.5 vs p > 0.5 95% Lower Exact Sample X N Sample p Bound P-Value 1 74 130 0.569231 0.493309 0.068

33. Regulatory Compliance Documentation Sample Size Power and Sample Size Test for Two Proportions Testing proportion 1 = proportion 2 (versus <) Calculating power for proportion 2 = 0.7 Alpha = 0.05 Sample Target Proportion 1 Size Power Actual Power 0.6 388 0.9 0.900148 0.6 281 0.8 0.800923 The sample size is for each group. Is the sample size a concern?

34. The Business Approach 2-Proportion Test Statistical Problem Statistical Problem Business Problem Business Solution Decision Statistical Solution Potential Root Causes Identified Root Causes Verified

35. Analysis of Proportions for Workload Balance Jack Lairdieson, MBB, Vanguard Interpret as an Interval Plot for Multiple Proportions Total Region 5 Region 6 Region 3 Region 1 Region 3 Region 2

36. The Workload Balance (WLB) metrics were being discussed at a regional meeting. The Region 1 representative scoffed at the Region 2 representative that the Region 2’s “In-range” WLB performance metrics were at the “bottom of the barrel”. The Region 2 representative quickly responded, “Really, Region 1 is no better than Region 2.” Once back to the office the concerned Region 1 representative gave the following Workload Balance data to a Black Belt. WLB Stats In-Range Staff Region 1 663 1411 Region 2 141 353 Should Region 1 be concerned about his conclusion? What is the null hypothesis? Workload Balance Example

37. H o : The proportion of Region 1 “In-Range” staff is equal to the proportion of Region 2 “In-Range” staff. H a : The proportion of Region 1 “In-Range” staff is not equal to the proportion of Region 2 “In-Range” staff. or H a : The proportion of Region 1 “In-Range” staff is greater than the proportion of Region 2 “In-Range” staff. Workload Balance Example - Hypothesis

38. Tool Bar Menu > Stat > Basic Statistics > 2 Proportion Analysis through MINITAB™ Workload Balance Example – Minitab Commands

39. Workload Balance Example – Minitab Results Session Window Output What’s our interpretation? What Hypothesis did we choose to test? Is the sample size a concern? Test and CI for Two Proportions Sample X N Sample p 1 663 1411 0.469880 2 141 353 0.399433 Difference = p (1) - p (2) Estimate for difference: 0.0704461 95% lower bound for difference: 0.0223190 Test for difference = 0 (vs > 0): Z = 2.41 P-Value = 0.008

40. Sample Size: Minitab Testing proportion 1 = proportion 2 (versus >) Calculating power for proportion 2 = 0.399 Alpha = 0.05 Sample Target Proportion 1 Size Power Actual Power 0.469 857 0.9 0.900072 0.469 619 0.8 0.800094 The sample size is for each group.

41. References Fisher RA (1925). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh Barnard GA (1945). A new test for 2 x 2 tables. Nature 156:177 Chan I (1998) Exact tests of equivalence and efficacy with non-zero lower bound for comparative studies. Statistics in Medicine 17, 1403-1413 Mehta CR and Senchaudhuri P (2003). Conditional versus unconditional tests for comparing two binomials. Cytel Software. Web Sites: http://www.minitab.com/support/documentation/answers/ SampleSize2p.pdf www.statsoft.com/textbook/stathome http://sofia.fhda.edu/gallery/statistics/lessons/lesson10-2

42. Six Sigma Links Six Sigma Motorola, Inc. - Motorola University Six Sigma - What is Six Sigma? i Six Sigma - Six Sigma Quality Resources for Achieving Six Sigma Results General Electric : Our Company : What is Six Sigma? Quality American Society for Quality - ASQ TQM Virtual CoursePack SPC Press - Home Statistics http://www. statsoft .com/textbook/ stathome .html Penn State Statistical Education Resource Kit--Overview of Statistics Data Statistics Video Course The Sofia Open Content Initiative - Elementary Statistics Resource: Learning Math: Data Analysis, Statistics, and Probability Lean Six Sigma Kaizen and Lean Manufacturing Consulting: Gemba Research - | Kaizen Products Conquering Complexity, Fast Innovation, Lean Six Sigma Quality. George Group Consulting Six Sigma Training Book LEAN.org - Lean Enterprise Institute| Lean Production| Lean Manufacturing| LEI| Lean Services| Lean Enterprise Training Course| Lean Consumption| Lean Resources| Lean Experts| Lean Healthcare| Lean in Healthcare| Training on Lean Manufacturing| Lean Business Excel Statistics Add on http://www.qimacros.com/