Power and Sample Size Test for One Proportion Testing proportion = 0.1 (versus not = 0.1) Alpha = 0.05 Alternative Sample Target Proportion Size Power Actual Power 0.12 2523 0.9 0.900079 0.15 438 0.9 0.900409 0.20 122 0.9 0.901723 0.25 59 0.9 0.903729
Transcript
1.
Sample Size Determination Deliverable 10A
2.
Analyze Module Roadmap Define 1D – Define VOC, VOB, and CTQ’s 2D – Define Project Boundaries 3D – Quantify Project Value 4D – Develop Project Mgmt. Plan Measure 5M – Document Process 6M – Prioritize List of X’s 7M – Create Data Collection Plan 8M – Validate Measurement System 9M – Establish Baseline Process Cap. Analyze 10A – Determine Critical X’s Improve 12I – Prioritized List of Solutions 13I – Pilot Best Solution Control 14C – Create Control System 15C – Finalize Project Documentation Green 11G – Identify Root Cause Relationships Queue 1 Queue 2
Beta risk can be selected by the Black Belt, but we don‘t control it the same way we do risk. The best we can do is adjust sample size so that is no greater than a specified value.
When a beta error ( ) occurs, we have missed detecting a difference (good or bad).
Power is defined as (1 - ). It represents the probablitily that we can detect an important effect in the process
Typical values of power in experiments are between 0.80 to 0.90
Delta ( ) is the minimum change that needs to be detected during analysis
Example: if the average cycle time to perform a laboratory test was 120 minutes, you as the supervisor may not be concerned if the average time shifted to 121 minutes, but you would want to know if it increased to 130 minutes. In this case, 10 minutes is the smallest increment of concern ( = 10 minutes).
It is the acceptable window of uncertainty around the estimate
As delta decreases (more precision), the sample size increases
As delta increases (less precision), the sample size decreases
You are going to perform a statistical test to determine if there is a difference in the average suspended solids level for two processing lines at a wastewater treatment plant. A suspended solids difference of 10 units or less is unimportant to you for the purpose of this test, but you would like to detect a difference > 10. The historical process standard deviation is 5.
12.
Wastewater Sample Size Example Stat > power and sample size > 2-Sample t
13.
Minitab Output Power and Sample Size 2-Sample t Test Testing mean 1 = mean 2 (versus not =) Calculating power for mean 1 = mean 2 + difference Alpha = 0.05 Assumed standard deviation = 5 Sample Target Difference Size Power Actual Power 10 7 0.9 0.929070 The sample size is for each group.
“ Wow! Seven samples are not that many. I was prepared to gather 25 samples. How small of a difference can I detect if I collect 10,15, 20 or the entire 25 samples”?
Power and Sample Size 2-Sample t Test Testing mean 1 = mean 2 (versus not =) Calculating power for mean 1 = mean 2 + difference Alpha = 0.05 Assumed standard deviation = 5 Sample Size Power Difference 10 0.9 7.66846 15 0.9 6.13222 20 0.9 5.25996 25 0.9 4.67878 The sample size is for each group. Notice how sample size increases dramatically as the difference to detect becomes smaller and smaller.
Our old friend Pat is starting to wonder about the validity of a great number of past decisions. In this case, Pat now realizes that the past practice of guessing at the number of invoices to inspect (as was done in previous modules) wasn’t the most reliable. How many data points will Pat need to inspect to rule in/out that the process does not have a 10% defect rate if the samples inspected had a 12%, 15%, 20%, or 25% defect rate?
Now that we know how many data points to include in the statistical test, we need to identify which samples should be placed in the test.
Assume you have several hundred data points collected over time, but the sample size calculation showed you need only 35 for the statistical test. How do we pick the appropriate 35?
The 35 “best” or “worst” will certainly skew our conclusions
35 from the center of the data will not show the appropriate variability
20.
Randomly Selected Data This procedure works equally well with text or numerical values (a wonderful way to select the sequence for Black Belts to present their projects in class).
Be the first to comment