Introduction to Statistical Applications for Process Validation

12,323 views
12,020 views

Published on

This presentation from IVT's 2nd Annual Validation Week Canada covers the 2011 FDA Process validation and the subsequent statistical processes. Statistics in process validation is introduced as well as the integration with six sigma and solutions to common mistakes.

Published in: Health & Medicine

Introduction to Statistical Applications for Process Validation

  1. 1. Introduction to Statistical Applications for Process ValidationEugenie KhlebnikovaSr. Validation Specialist, CQEMcNeil Consumer Healthcare 1
  2. 2. AGENDA Regulatory Expectations for Statistical Analysis Statistical Tools Six Sigma and Process Validation Common Mistakes to Avoid 2
  3. 3. REGULATORY EXPECTATIONS 3
  4. 4. PV GUIDELINES• statistical tools to be used in • Emphasis on process design the analysis of data elements, and maintaining• the number of process runs process control based on carried out and observations knowledge gained throughout made should be sufficient to commercialization allow the normal extent of • Emphasize to have good variation and trends to be knowledge to detect and to established to provide control variability through use sufficient data for evaluation. of statistical analysis 4
  5. 5. PROCESS VALIDATION LIFE CYCLE Variation analysis, capability, Stage 2: Process stability analysis QualificationStage 1: ProcessDesignStatistics toanalyze andoptimizeresults (DOE,variationanalysis, etc) Process Capability Stage 3: Process Monitoring Control Charts and Improvement 5
  6. 6. PROCESS UNDERSTANDING Testing the finalproduct and passing specifications doesnot give knowledge of the process Variation at each production stage Knowledge of stability and capability 6
  7. 7. PROCESS UNDERSTANDING – KNOW VARIATION“Understanding variation is the key to success inquality and business” W. Edwards Deming (Father ofModern Process Control)The customers “feel” variation and lack ofconsistency in a product much more so than the“average” (Jack Welch) 7
  8. 8. FDA PV GUIDANCE RECOMMENDATIONSINTEGRATED TEAM APPROACH industrial pharmacy Recommended that a statistician quality orprocess assurance person with adequate training inengineeringand statistical process control techniquemanufacturing analytical develop the data collection plan and chemistry statistical methods and procedures used in measuring and evaluating microbiology process stability and process capability. statistics 8
  9. 9. DESCRIPTIVE VS INFERENTIAL STATISTICS This distinction is based onwhat you’re trying to do with The Division Between your data Descriptive and Inferential Statistics 9
  10. 10. DESCRIPTIVE STATISTICS• Summarizing or displaying the facts Mean = Sum of all observations/ # of observations Range = Max - Min Standard Deviation Variance = std dev2 Relative Standard Deviation or CV = std dev*100/mean 10
  11. 11. RELATIVE STANDARD DEVIATIONExample 1: Example 2:Group Size Avg St Dev RSD Group Size Avg St Dev RSD 1 10 80 0.8 1.0 1 10 80 1.0 1.4 2 10 90 0.9 1.0 2 10 90 1.0 1.1 3 10 100 1.0 1.0 3 10 100 1.0 1.0 4 10 110 1.1 1.0 4 10 110 1.0 0.9 5 10 120 1.2 1.0 5 10 120 1.0 0.8Standard deviation is proportional to the %RSD is changing because the average isaverage and the %RSD is unchanged changing, not the standard deviation 11
  12. 12. EXAMPLE: BLEND UNIFORMITY Tote Batch 1 Batch 2 Batch 3Location Specification: 1 101 100 102 90-110% RSD ≤ 5.0% 2 98 99 104 3 99 101 99 4 100 103 97 5 103 97 101 6 102 102 100 7 101 100 102 8 100 101 98 9 102 102 103 10 104 99 102 12
  13. 13. EXAMPLE: BLEND UNIFORMITY Tote Batch 1 Batch 2 Batch 3 Minitab Output:Location Descriptive Statistics: Batch 1, Batch 2, Batch 3 1 101 100 102 Variable Mean StDev CoefVar Minimum Maximum 2 98 99 104 Batch 1 101.00 1.83 1.81 98.00 104.00 Batch 2 100.40 1.78 1.77 97.00 103.00 3 99 101 99 Batch 3 100.80 2.25 2.23 97.00 104.00 4 100 103 97 5 103 97 101 6 102 102 100 7 101 100 102 8 100 101 98 9 102 102 103 10 104 99 102 13
  14. 14. EXAMPLE: BLEND UNIFORMITY 14
  15. 15. INFERENTIAL STATISTICS 15
  16. 16. INFERENTIAL STATISTICS• A decision about the batch is based on a relative small sample taken since it is not realistic to test the entire batch.• To confirm that the data is representative of the batch, inference statistics (confidence and tolerance intervals) can be used to predict the true mean. 16
  17. 17. CONFIDENCE INTERVAL• A confidence interval is an interval within which it is believed the true mean lies CI = ± where is sample mean, s is sample standard deviation, N is the sample size, and t value is a constant obtained from t-distribution tables based on the level of confidence. Note the value of t should correspond to N-1. 17
  18. 18. TOLERANCE INTERVAL• A tolerance interval is an interval within which it is believed the individual values lie, TI = ± k*s where is sample mean, s is sample standard deviation, N is the sample size, and k value is a constant obtained from factors for two-sided tolerance limits for normal distributions table believed the true mean lies. 18
  19. 19. EXAMPLEA batch of tablets was tested forcontent uniformity. The meanvalue of 10 tablets tested was99.1% and a standard deviationwas 2.6%. 19
  20. 20. EXAMPLE: Confidence Interval• t from a table• N-1=10-1=9• t=3.25• probability of 99% covering 99% of data 20
  21. 21. EXAMPLE: Confidence Interval• CI = ± = 99.1 ± =96.4 to 101.8• Then we can say that we are 99% certain that the true batch mean will be between 96.4% and 101.8 %. 21
  22. 22. EXAMPLE: TOLERANCE INTERVALN=10,k =5.594probabilityof 99%covering99% of data 22
  23. 23. EXAMPLE: TOLERANCE INTERVAL• N=10, mean=99.1, s =2.6, k =5.594 TI = ± k*s• Probability of 99% covering 99% of data: TI =99.1 ± (5.594*2.6) TI = 84.6% to 113.6% 23
  24. 24. EXAMPLE: Confidence and Tolerance Interval• If a sample has the mean value of 10 tablets at 99.1% and a standard deviation at 2.6%.• Then we can say that we are 99% certain that 99% of the tablet content uniformity lies between 80.6 and 117.6% and we are 99% certain that the true batch mean will be between 96.4 and 101.8 %. 24
  25. 25. SAMPLING 25
  26. 26. SAMPLING• The cGMPs mention samples, sampling plans, or sampling methods repeatedly.• Firms are expected: – To use a sampling plan that utilizes basic elements of statistical analysis – Provide a scientific rationale for sampling that would vary the amount of samples taken according to the lot size – Define a confidence limit to ensure an accurate and representative sampling of the product 26
  27. 27. WARNING LETTER EXAMPLE211.165 - Testing and release for distribution:(d) Acceptance criteria for the sampling and testing conducted by thequality control unit shall be adequate to assure that batches of drugproducts meet each appropriate specification and appropriatestatistical quality control criteria as a condition for their approval andrelease. The statistical quality control criteria shall include appropriateacceptance levels and/or appropriate rejection levels.“For example, your firms finished product sampling plan product A isnot representative of the batch produced. A total of 13 units aresampled per lot, with 3 tested for bacterial endotoxin and 10 tested forbioburden. This sampling of 13 units is irrespective of lot size, whichmay vary from X to Z units (vials) per lot” 27
  28. 28. CHOOSING SAMPLES Sampling Method: •Simple Random •Convenience •Systematic •Cluster •Stratified 28
  29. 29. SAMPLING METHODSSIMPLE RANDOM SYSTEMATIC 0 min 30 min 1 hrCONVENIENCE CLUSTER STRATIFIED top middle bottom 29
  30. 30. SAMPLING RISK DISPOSITION IMPACT IF LOT IMPACT IF LOT BAD GOOD Lot is accepted Correct Decision Incorrect Decision (Type II or Consumer’s risk) Lot is rejected Incorrect Decision Correct Decision (Type I or Producer’s risk)Expressed as Acceptable Quality Level (AQL): maximum average percentdefective that is acceptable for the product being evaluated. 30
  31. 31. ACCEPTANCE SAMPLINGAcceptance Sampling is a form of inspection applied to lots orbatches of items before or after a process to judgeconformance to predetermined standards.Sampling Plans specify the lot size, sample size, number ofsamples and acceptance/rejection criteria. Lot Random sample 31
  32. 32. OPERATING CHARACTERISTIC CURVE• The operating-characteristic (OC) curve measures the performance of an acceptance-sampling plan.• The OC curve plots the probability of accepting the lot versus the lot fraction defective.• The OC curve shows the probability that a lot submitted with a certain fraction defective will be either accepted or rejected. 32
  33. 33. OC CURVES Ideal OC Curve Reject all lots with more than 2.5% defective and accept all lots with less than 2.5% defective The only way to assure is 100% inspection 100 90acceptance (%) 80Probability of 70 60 50 40 30 20 10 1 1.5 2 2.5 3 3.5 Percent defective (%) 33
  34. 34. OCCs for Single Sampling Plans An Operating Characteristic Curve (OCC) is a probability curve for a sampling plan that shows the probabilities of accepting lots with various lot quality levels (% defectives). 1 0.9 Under this sampling plan, if the lot has 3% defectiveProbability of accepting lot the probability of accepting the lot is 90% 0.8 the probability of rejecting the lot is 10% 0.7 0.6 0.5 If the lot has 20% defective 0.4 it has a small probability (5%) of being accepted 0.3 the probability of rejecting the lot is 95% 0.2 0.1 0 0 .05 .10 .15 .20 Lot quality (% defective) 34
  35. 35. SAMPLING PLANSSampling plans involve:  Single sampling  Double sampling  Multiple samplingProvisions for each type of sampling plan include 1. Normal inspection 2. Tightened inspection 3. Reduced inspection 35
  36. 36. SWITCHING RULES “and” conditions: 2 out of 5 Production Steady Start consecutive 10 consecutive lots lots rejected accepted Approved by responsibility authority TightenedReduced Normal 5 consecutive “or” conditions: lots accepted 10 consecutive Lot rejected lots remain on Irregular production tightened A lot meets neither inspection the accept nor the reject criteria Other conditions Discontinue warrant return to inspection normal inspection 36
  37. 37. SAMPLING BY ATTRIBUTES: ANSI Z1.4 2008• The acceptable quality level (AQL) is a primary focal point of the standard• The AQL is generally specified in the contract or by the authority responsible for sampling.• Different AQLs may be designated for different types of defects (critical, major, minor).• Tables for the standard provided are used to determine the appropriate sampling scheme. 37
  38. 38. ANSI Z1.4 2008PROCEDURE:1. Choose the AQL2. Choose the inspection level3. Determine the lot size4. Find the appropriate sample size code letter from Table I-Sample Size Code Letters5. Determine the appropriate type of sampling plan to use (single, double, multiple)6. Check the appropriate table to find the acceptance criteria. 38
  39. 39. SAMPLE SIZE DETERMINATION Table I - Sample Size Letter Codes Special Inspection Levels General Inspection LevelsLot or Batch Size S-1 S-2 S-3 S-4 I II III 2 to 8 A A A A A A B 9 to 15 A A A A A B C 16 to 25 A A B B B C D 26 to 50 A B B C C D E 51 to 90 B B C C C E F 91 to 150 B B C D D F G 151 to 280 B C D E E G H 281 to 500 B C D E F H J 501 to 1200 C C E F G J K 1201 to 3200 C D E G H K L 3201 to 10000 C D F G J L M 10001 to 35000 C D F H K M N 35001 to 150000 D E G J L N P150001 to 500000 D E G J M P Q500001 to over D E H K N Q R 39
  40. 40. SAMPLE SIZE DETERMINATION 40
  41. 41. SINGLE SAMPLING PLAN - EXAMPLEDefect: any color except of redN = lot size = 25 applesFrom Sample Size Code Letters: Lot or batch size General Inspection Level 16-25 BFrom Normal Single Level Inspection Sampling Sample Size AQL 0.010 Size Code Letter B 3 0/1 Scenario 1: Scenario 2: 0 defects 2 defectsn = sample size =3 Accept Reject 41C=acceptance number = 0 Accept/1 Reject
  42. 42. SINGLE SAMPLING PLAN - EXAMPLEN = lot size = 120,000From Sample Size Code Letters: Lot or batch size General Inspection Level 35,001-150,000 NNormal InspectionFrom Normal Single Level Inspection Sampling Size Sample Critical Major Minor Code Letter Size AQL 0.010 AQL 0.65 AQL 4.0 N 500 ACC 0 / REJ 1 ACC 7/ REJ 8 ACC 21 / REJ 22 42
  43. 43. STATISTICAL PROCESS CONTROL• The principle of SPC analysis is to understand the process and detect the process change.• Statistical Process Control (SPC) charts are used to detect process variation. 43
  44. 44. STATISTICAL PROCESS CONTROL• The Current Good Manufacturing Practices for Process Validation published by the FDA in January 2011 states "homogeneity within a batch and consistency between batches are goals of process validation activities." Control charts explicitly compare the variation within subgroups to the variation between subgroups, making them very suitable tools for understanding processes over time (stability). 44
  45. 45. VARIABLE CONTROL CHARTS n=1 2<n<9 n is ‘small’ n is ‘large’ median 3<n<5 n > 10X & Rm X&R X&R X&SUsed for measured data 45
  46. 46. CONTROL CHART SELECTION: ATTRIBUTE DATA Defect or Defective Data Nonconformity DataConstant Variable Constant VariableSample Size Sample Size n > 50 n > 50C chart u chart p or np chart p chartUsed for count (attribute) data 46
  47. 47. Stable and Unstable ProcessesA stable (or “incontrol”) process is UCLone in which thekey processresponses show nosigns of special LCLcauses.An unstable (or UCL“out of control”)process has bothcommon andspecial causes LCLpresent. 47
  48. 48. CONTROL CHARTTablet Weight 305 UCL 303.7 302 300 mean 298.0 296.3 LCL 285 280 1 hr 30 2hr 30 0 min 30 min 1 hr min 2 hr min 48
  49. 49. PROCESS CAPABILITY• Is the process capable of consistently delivering quality products?• Is the process design confirmed as being capable of reproducible commercial manufacturing?• Process capability is expressed as a ratio of specifications/process variability 49
  50. 50. PROCESS CAPABILITY INDECES Lower Cust. Tolerance Upper Spec. Spec. 0 .4 Limit Limit 0 .3 0 .2 0 .1 0 .0 -5.33 -4.0 -2.67 -1.33 0 1.33 2.67 4.0 5.33 Lower Upper Spec. Cust. Tolerance Spec. 0 .4 Limit Limit 0 .3Cpk < 1 - not capable 0 .2Cpk = 1 - marginally capable 0 .1 0 .0Cpk > 1 - capable -5.33 -4.0 -2.67 -1.33 0 1.33 2.67 4.0 5.33 50
  51. 51. PROCESS CAPABILITY Accurate and precise Accurate but not precise Precise but not accurate Desired DesiredDesired Current Current Situation Situation LSL T USL LSL T USL LSL T USL 51
  52. 52. PROCESS CAPABILITY INDECES• Short-term (Cp and Cpk) and/or long term (Pp and Ppk) are commonly used to evaluate process performance.• Cpk attempts to answer the question "does my current production sample meet specification?"• Ppk attempts to answer the question "does my process in the long run meet specification?" 52
  53. 53. EXAMPLE: PROCESS CAPABILITY Process Capability Sixpack of Hardness Xbar Chart Capabilit y Hist ogram LSL USL UC L=20.239 20.0 SpecificationsSample Mean _ LSL 16 _ X=19.599 USL 23 19.5 19.0 LC L=18.959 1 2 3 4 5 6 7 8 9 10 16 17 18 19 20 21 22 23 S Chart Normal Prob Plot 1.2 AD: 0.304, P: 0.564 UC L=1.126 Sample StDev 0.8 _ S=0.656 0.4 LC L=0.186 1 2 3 4 5 6 7 8 9 10 18.0 19.5 21.0 22.5 Last 10 Subgroups Capabilit y Plot 21.0 Within Within Overall StDev 0.674453 StDev 0.673974Values Cp 1.73 Pp 1.73 19.5 Overall Cpk 1.68 Ppk 1.68 Cpm * 18.0 Specs 2 4 6 8 10 Sample 53
  54. 54. PROCESS CAPABILITY• At a minimum, 50 individual values or 25 subgroups for sub-grouped data are required to calculate process capability; and 100 individual values provide a stronger basis for the assessment.• Use SPC charts to check if the process is stable• Check the distribution (normal vs not normal)• Use the Cpk value which represents the process under consideration 54
  55. 55. PROCESS CAPABILITY EXAMPLE• A client had to meet Cpk requirement of ≥ 1.20.• When data was assumed to be normally distributed, the Cpk =0.8• When the non-normal behavior was accounted for, the Cpk = 1.22 55
  56. 56. SIX SIGMA AND PROCESS VALIDATON• Six Sigma and Process Validation• Use the process knowledge to make improvements 56
  57. 57. SIX SIGMA AND PROCESS VALIDATONSix Sigma – process improvement methodologyDMAICDefine  Objective  To improve compression processMeasure  Measure hardness during PVAnalyze  Statistical analysis, calculate Cp/CpkImprove  Decrease variationControl  Control variation 57
  58. 58. Cpk and SigmaSigma 1,Cpk =0.33 Sigma 3, Sigma 5, Cpk = 1 Cpk = 1.67 Sigma 2, Sigma 4, Cpk = Cpk = 0.67 1.33
  59. 59. COMMON MISTAKES• Incorrect use of statistical tools: – ANSI Attribute Sampling for measurement data (pH) – Incorrect sampling size – Distribution is not checked – Process in not stable – Incorrect uses of Cpk (equivalency between equipment, large specification limits, etc) 59
  60. 60. WARNING LETTER: EQUIPMENT COMPARABILITY AND CAPABILITY• The firm referenced the Cpk values for processes using a double-sided tablet press and the single-sided tablet press to demonstrate statistical equivalence.• FDA evaluation : – The Cpk value alone was not appropriate metric to demonstrate statistical equivalence. Cpk analysis requires a normal underlying distribution and a demonstrated state of statistical process control. – Statistical equivalence between the two processes could have been shown by using either parametric or non-parametric (based on distribution analysis) approaches and comparing means and variances. – Firm did not use the proper analysis to support their conclusion that no significant differences existed between the two compression processes. 60
  61. 61. STATISTICAL EVALUATION• Is required by statute• Is an expectation of the regulatory inspector during inspection of the firm as it relates to process validation of products• Use statistical tools that are meaningful and useful to understand the baseline performance of the process• Is invaluable as a troubleshooting tool post validation 61
  62. 62. QUESTIONS 62

×