Developing Bioanalytical MethodsBalancing the Statistical Tightrope
“Lee: can I use this number?”Process DevelopmentGSK, 1997                                2
“it’s about 40” “about 40?” “probably...”                  3
Enlightenment?
5
Blooms Taxonomy                              the 4 stages of competence                                Incompetent     Com...
A Statistical GodMe
Using Statistics
Why? Six Reasons1. Potency assays are   key   in making medicines2. Bioassays   are very   variable3. Statistics will help...
The Regulator & Assay Control Regulators have been asking for this for years! QbD1. Pharmaceutical cGMPs for the 21st   Ce...
StatisticsThe complete solution?
Or this?           Your assay?                         12
Or this?           or your assay?                            13
Statistics - an Amazing Transition                                     14
Bioassays will always be variableYou can improve it- by understanding it- Focusing effort in right places- This brings con...
Why assay variation matters?     product variation +                                                 A few unsatisfactory ...
Our Control Strategy                  What does the scientist need to achieve?Define            i.e. selectivity, accuracy...
Generating Bioassay       Data                      18
The Rules1. Speak with your statistician before   generating data2.See Rule 1                                         19
Lot’s data ≠ Value                     20
21
Statistics are a tool                        22
QC                   Which Tools?                     UCL                                   Stage 4                     Te...
What’s Appropriate Knowledge?• Learning takes time• Will you use it often enough?• It’s not an academic pursuit• Activitie...
Scope  &Design
Define & ScopeHow is the assay performing?   Prec/TOL2-sided =   6 x 16.76                                                ...
Parameters (e.g. 15)pDNANaClpHTube LengthTimeSeeding DensityRatio of TransfectionTemperatureAgitation and levelVector – ty...
Q. How Many parameters?Q. Which parameters?Q. What ranges?A. Existing knowledgeA. Common senseA. Practical limits
Define & ScopeDrill down - map out assay - build understanding & scope                                         Assay Flow ...
Define & Scope   Drill down & map out assay to build understanding & scopeAttention is focusedtoward key stepsand the para...
Scope & Screen            Scope ranges with simple experimentsScoping Experiments                           Explore mildes...
Revealing the Big Hitters                            32
Temptation
Building UnderstandingOFATProvides estimatesof effects at setconditions of the    NaClother factors andno interaction     ...
Building UnderstandingFactorial Design                2400           2600                                 1300            ...
Optimisation                   Optimise the parameters that                   survived the initial screeningwork towards a...
Simulations The tools allow you to simulate scenarios using the data you’ve built upVisual simulation of expected performa...
Is the Model Correct?                        38
Validate & Verify The evaluation of robustness should be considered during the development phase and should show the relia...
Assay Control: control the parameters inside boundaries                                                          40
Working within the controlboundaries will keep theassay under control                             Even if you go outside  ...
Summary - Data Driven Development    Scope               Screen               Optimize                 Verify             ...
43
PrecisionIt may be considered at three levels:1.   Repeatability2.   Intermediate precision3.   ReproducibilityICH Q2A, 1994
Repeatability1 analyst in 1 laboratory on 1 day injecting 6 times Summary Statistics            Number of             Stan...
Intermediate Precision•   1 analyst in 1 laboratory on•   1 day•   injecting 6 samples•   each tested 6 times          As ...
Intermediate PrecisionSo we compare the mean values for each sample(over replicate results per sample)             Varianc...
and the others…..?Precision within a laboratory but withdifferent analysts, on different days, withdifferent equipment…ref...
Intermediate PrecisionData collect using several analysts using several instrumentsover several days:                     ...
Intermediate PrecisionPotentially misleading: large analyst-to-analyst variationpresent:                                  ...
Intermediate Precision      better examined looking at multiple      sources of variation within an assaywant to understan...
Intermediate Precision                         52
Intermediate PrecisionCan also perform Unbalanced designsOne operator performs multiple injections on singlepreparation;Tw...
Reproducibility               multiple laboratories; typically run as an inter-               laboratory cross-over study,...
ReproducibilityCan use analysis of variance (ANOVA) to look fordifferences or biases between labs      Alternatively look ...
Risk ManagementThe level of effort, formality and documentation....should be commensurate with the level of risk          ...
Before & AfterHow is the assay performing?   P/TOL2-sided =    6 x 16.76                                                  ...
Before & AfterBetter               P/TOL2-sided =     6 x 6.99                                          100               ...
Risk ManagementMethod Understanding, Control and Capability (MUCC)               Understand impact of variation           ...
Risk Management                Understanding?                                               Understanding?Capable?        ...
Risk Management                                         P/TOL2-sided =                  6 x 6.99                          ...
Summary1.Build a good basic understanding of  stats but don’t need to become guru2.Involve a statistician, at least at the...
“Lee: can I use this number?”                                63
“Yes – it’s 42 … ”    0.05 with 95% Confidencefor the statisticians in the audience                                      ...
AcknowledgmentsDr. Paul Nelson – Prism TC LtdPictures from “The Cartoon Guide to Statistics”Larry Gonick & Woollcott Smith...
Ibc biological assay development & validation 2011 gra presentation
Upcoming SlideShare
Loading in …5
×

Ibc biological assay development & validation 2011 gra presentation

1,268 views
1,175 views

Published on

Presentation at IBC's Biological Assay Development and Validation conference 12 May 2011

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,268
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ibc biological assay development & validation 2011 gra presentation

  1. 1. Developing Bioanalytical MethodsBalancing the Statistical Tightrope
  2. 2. “Lee: can I use this number?”Process DevelopmentGSK, 1997 2
  3. 3. “it’s about 40” “about 40?” “probably...” 3
  4. 4. Enlightenment?
  5. 5. 5
  6. 6. Blooms Taxonomy the 4 stages of competence Incompetent Competent ConsciousConsciousness Unconscious 6
  7. 7. A Statistical GodMe
  8. 8. Using Statistics
  9. 9. Why? Six Reasons1. Potency assays are key in making medicines2. Bioassays are very variable3. Statistics will help you understand your data4. Understanding your data will reveal if control exists5. Your level of control allows you to judge RISK6. Regulators globally require it 9
  10. 10. The Regulator & Assay Control Regulators have been asking for this for years! QbD1. Pharmaceutical cGMPs for the 21st Century2. PAT3. ICH Q2: Validation of Analytical Procedures4. ICH Q8: Pharmaceutical Development5. ICH Q9: Quality Risk Management6. ICH Q10: Quality Pharmaceutical Systems 10
  11. 11. StatisticsThe complete solution?
  12. 12. Or this? Your assay? 12
  13. 13. Or this? or your assay? 13
  14. 14. Statistics - an Amazing Transition 14
  15. 15. Bioassays will always be variableYou can improve it- by understanding it- Focusing effort in right places- This brings control- You can manage expectations- This is understood by regulators 15
  16. 16. Why assay variation matters? product variation + A few unsatisfactory assay variation + batches may even inaccuracy pass specification due to a combination of assay method and process variability Many satisfactory OOS batches likely to fail (potentially costing £Ms)because of combination of assay method & process inaccuracy & variation 16
  17. 17. Our Control Strategy What does the scientist need to achieve?Define i.e. selectivity, accuracy, precision linearity Identify & prioritise analytical CNX parametersMeasure Control Noise eXperimental parameters parameters parametersAnalyse Fix & control e.g., MSA, e.g., DoE Input into Precision Regression Method MethodImprove Ruggedness Robustness Method Control Strategy & reduce Risk prior toControl Validation → Routine Use & Continuous Improvement 17
  18. 18. Generating Bioassay Data 18
  19. 19. The Rules1. Speak with your statistician before generating data2.See Rule 1 19
  20. 20. Lot’s data ≠ Value 20
  21. 21. 21
  22. 22. Statistics are a tool 22
  23. 23. QC Which Tools? UCL Stage 4 Technology QC Tools CELLULA, Shewhart chart, YES Transfer LCL CUSUM NO TIME Stage 1: Qualification Tool Stage 3: Fishbone, Minitab Validation Tools Nested, CELLULAPrecision Stage 2:AccuracyLinearity etc. Development Tools DX8, JMP, Minitab Design
  24. 24. What’s Appropriate Knowledge?• Learning takes time• Will you use it often enough?• It’s not an academic pursuit• Activities must add value  do what’s necessary 24
  25. 25. Scope &Design
  26. 26. Define & ScopeHow is the assay performing? Prec/TOL2-sided = 6 x 16.76 100 = 1.01 26
  27. 27. Parameters (e.g. 15)pDNANaClpHTube LengthTimeSeeding DensityRatio of TransfectionTemperatureAgitation and levelVector – type, concAddition Order
  28. 28. Q. How Many parameters?Q. Which parameters?Q. What ranges?A. Existing knowledgeA. Common senseA. Practical limits
  29. 29. Define & ScopeDrill down - map out assay - build understanding & scope Assay Flow 29
  30. 30. Define & Scope Drill down & map out assay to build understanding & scopeAttention is focusedtoward key stepsand the parametersinvolved in thesesteps Cause & Effect Diagram (Fishbone) helps think your assay through Identify & prioritise analytical CNX parameters 30
  31. 31. Scope & Screen Scope ranges with simple experimentsScoping Experiments Explore mildest to most forcing conditions 31
  32. 32. Revealing the Big Hitters 32
  33. 33. Temptation
  34. 34. Building UnderstandingOFATProvides estimatesof effects at setconditions of the NaClother factors andno interaction pDNAeffects. pH 34
  35. 35. Building UnderstandingFactorial Design 2400 2600 1300 900 1800Estimates effects atdifferent conditions toestimate interactions 350 600 250 300 500Design of ExperimentsDOE 35
  36. 36. Optimisation Optimise the parameters that survived the initial screeningwork towards aRobust Optimum 36
  37. 37. Simulations The tools allow you to simulate scenarios using the data you’ve built upVisual simulation of expected performance relative to specification 37
  38. 38. Is the Model Correct? 38
  39. 39. Validate & Verify The evaluation of robustness should be considered during the development phase and should show the reliability of an analysis with respect to deliberate variations in method parameters ICH Q2B, 1994Method stretch…what if? Ideal Settings Control Space Design Space 39
  40. 40. Assay Control: control the parameters inside boundaries 40
  41. 41. Working within the controlboundaries will keep theassay under control Even if you go outside the control boundaries, the assay will have enough flexibility to deal with it without an OOS 41
  42. 42. Summary - Data Driven Development Scope Screen Optimize Verify QC/TT Transfer to QC to validate on batches & bring into routine useExplore mildest Identify few potential Estimate & utilizeto most forcing key parameters interactions to move Rattle the cage toconditions Focus on vital few & towards optimum deliver a design narrow ranges conditions space
  43. 43. 43
  44. 44. PrecisionIt may be considered at three levels:1. Repeatability2. Intermediate precision3. ReproducibilityICH Q2A, 1994
  45. 45. Repeatability1 analyst in 1 laboratory on 1 day injecting 6 times Summary Statistics Number of Standard Coefficient Lower 95% CI Upper 95% Values Mean Deviation of Variation for Mean CI for Mean t30 PS 6 223.27 6.43 2.88% 216.52 230.02 45
  46. 46. Intermediate Precision• 1 analyst in 1 laboratory on• 1 day• injecting 6 samples• each tested 6 times As well as sample variation, this study still provides information on repeatability 46
  47. 47. Intermediate PrecisionSo we compare the mean values for each sample(over replicate results per sample) Variance Components Factor df Variance % Total Sample 5 27.8535 21% Repeat 30 102.6361 79% 35 130.4896 100% Standard Mean Deviation RSD 47 216.24 11.4232 5.28%
  48. 48. and the others…..?Precision within a laboratory but withdifferent analysts, on different days, withdifferent equipment…reflects the realconditions within one laboratory ICH Q2A 1995 48
  49. 49. Intermediate PrecisionData collect using several analysts using several instrumentsover several days: Y 56000 55500 55000 54500 Peak Area 54000 53500 53000 52500 52000 0 5 10 15 20 25 Sample 49
  50. 50. Intermediate PrecisionPotentially misleading: large analyst-to-analyst variationpresent: Y 56000 55500 55000 54500 Peak Area 54000 53500 53000 52500 52000 0 5 10 15 20 25 Sample Analyst 1 Analyst 2 Analyst 3 50
  51. 51. Intermediate Precision better examined looking at multiple sources of variation within an assaywant to understandmajor sources ofvariation such assample, prep,analyst etc. 51
  52. 52. Intermediate Precision 52
  53. 53. Intermediate PrecisionCan also perform Unbalanced designsOne operator performs multiple injections on singlepreparation;Two operators perform single injections on multiplepreparations 53
  54. 54. Reproducibility multiple laboratories; typically run as an inter- laboratory cross-over study, with each participating lab sending samples to every other lab and analysing all samples (including own) …. sent to and analysed by other lab A B CSamples from A    laboratory: B    C    54
  55. 55. ReproducibilityCan use analysis of variance (ANOVA) to look fordifferences or biases between labs Alternatively look for “analytical equivalence”
  56. 56. Risk ManagementThe level of effort, formality and documentation....should be commensurate with the level of risk ICH Q9Evaluation of the risk to quality should be based onscientific knowledge & ultimately link to theprotection of the patientIs the bioassay fit for purpose and under control? 56
  57. 57. Before & AfterHow is the assay performing? P/TOL2-sided = 6 x 16.76 100 = 1.01 57
  58. 58. Before & AfterBetter P/TOL2-sided = 6 x 6.99 100 = 0.42 58
  59. 59. Risk ManagementMethod Understanding, Control and Capability (MUCC) Understand impact of variation upon risk… Risk Understanding?Capable? Management Loop Statistical Capability Process Control& Precision (SPC) Charts Control? 59
  60. 60. Risk Management Understanding? Understanding?Capable? P/TOL2-sided = 6 x 16.76 Capable? 100 Control? = 1.01 Capability& Precision 60
  61. 61. Risk Management P/TOL2-sided = 6 x 6.99 100 I-MR Chart of t30 PS Summary Report = 0.42 Is the process mean stable? I Chart Evaluate the % of out-of-control points. Investigate out-of-control points. 0% > 5% 225 UCL=220.77Yes No 210 0.0% t30 PS _ X=199.87 195 Comments 180 LCL=178.96 The process mean is stable. No data points are out of control 1 6 11 16 21 26 31 36 41 46 on the I chart. 61 Observation
  62. 62. Summary1.Build a good basic understanding of stats but don’t need to become guru2.Involve a statistician, at least at the beginning3.Build understanding of your bioassay (QbD) – it’s a must4.Get to grips with Bioassay Variability 62
  63. 63. “Lee: can I use this number?” 63
  64. 64. “Yes – it’s 42 … ”  0.05 with 95% Confidencefor the statisticians in the audience 64
  65. 65. AcknowledgmentsDr. Paul Nelson – Prism TC LtdPictures from “The Cartoon Guide to Statistics”Larry Gonick & Woollcott Smith 65

×