Validation Exceptions – Managinglife’s little (and big) unexpected events ( g) p Brian K. Nunnally, Ph.D.
What is an exception? pAn error or a failure which occurs duringVerification or ValidationWilliam Shakespeare: "It is neither good norbad, but thinking makes it so."
Types of Exceptions yp pThree levels of exceptions: Critical Non-critical N iti l Simple
Simple Exceptions p pSimple exceptions are documentation orprotocol errors These errors are found prior to execution or are Th f d i t ti “obvious” errors or typosThese e ceptions ha e no impact on the exceptions havevalidation (e.g. no risk)
Simple Exceptions examples p p pWrong test selected for productIncorrect specification for productLoss of power to the equipment beingtested.Other examples from the participants?
Non-critical Exceptions pNon-criticalNon critical exceptions are errors in theprotocol or execution of the validation whichhave no impact on the validation These errors are found during or after execution Assessment is needed t d t A ti d d to determine no i i impactt Need to document the risk (or lack thereof)
Non-critical Exceptions examples lOperator not trained to perform the operationAn interruption in the computer system ortheth equipment’s performance i t’ fOther examples from the participants?
Critical Exceptions pCritical exceptions are errors which have animpact on the validation These errors are found during or after execution Th f dd i ft ti Assessment is used to determine impact Need to document th risk N dt d t the i k
Critical Exceptions examples p pAcceptance criterion failureAn interruption in the computer system ortheth equipment’s performance i t’ fOther examples from the participants?
Basic Exception Process Flow p SME involvement throughout is critical You cannot document enough, even if all of it does not make it into the final event write-up CAPA may not be needed for simple events, but consider if there are many of them or the same ones repeatedly
General Process ConsiderationsCommence exception resolution process as soonas event is observedInvolve SMEs/QA throughout process asappropriate Get agreement on next steps and/or results g p Prior to execution, or after the completion Including all elements of the processImmediately notify the relevant SME/QA if there isany potential Product/Material impactConduct a risk assessment to determine impactand actions
What is the risk and impact of the exception?Event severity x Testing Stage = ImpactEvent Severity: What kind of event is it? How was the event caused?Testing Stage: Where are we in the validation/verification process? Installation? Performance Qualification?
Event Types ypTypographical errors These errors are found prior to execution or are “obvious” errors or typosExternal Issues Failures caused by factors external to the process or system under testExecution Errors Failures to follow instructions within the Test Document or referenced procedures.Document Generation Errors Incorrect detail in procedures, acceptance criteria or referenced documentationAcceptance Criteria Failures Pre-defined acceptance criteria have not been achieved.
Testing Stage g gNew system installation and the exceptiondoes not impact the validation intentNew system i t ll ti and th exceptionN t installation d the tidoes impact the validation intentNew or Existing system qualificationProcess ValidationRe-qualification
Should testing stop? g pDepends on the impact scoreDepends on consultation with SMEs and QAAre you willing to live with a failure?Will it open more q p questions than it solves?Note: It does not depend on the timeline!
Exception example #1 p pAn acceptance criterion was failed failed. However, the acceptance criterion was poorly set ( (limited knowledge was available to set the acceptance g p criterion).What level of exception is this?What do you do?
Exception example #2 p pThe acceptance criterion was passed but the passed,result was right at the limit. Assay variability is high for the test.Is this an exception? Why or why not?What do you do?
Exception example #3 p pDuring validation of the cleaning process the process,Operator executing the protocol failed to samplethe correct locations.Is this an exception? Why or why not?What do you do?
The key to preventing exceptions tiUnderstanding and reducing variation variation.Systems thinking (managing the flow through theentire process as an entity) entity).
Types of Variation (Including Sampling, Sampling Analytical and Production)Variation in Quality Characteristic (specific activity activity,potency, pH, temperature…etc.Process variation Variation in the way our variation.processes behave and are managed or controlled.Variation in volume flowing through a process.
Why focus on variability? y yReduced variation in Quality Characteristicsreduces rejects, rework and deviations. Itprovides high First Pass Yields.Reduced variation in Volume (smoother flow)increases output capacity and reduces capitalrequirements.requirementsReduced variation in Volume (smoother flow)decreases cycle time and Work-In-Progress. y g
Which one will be more likely to fail an acceptance criterion? Day shift Following night shift 2600 2400 22001-Point Average 2000 1800 1600 1400 1200 1000 50 70 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
The lesson from TribusBring a team together to focus on ensuring the aimis clear to all participantsFocus on systems to reduce and remove possibilityfor errorsImprove systems as a way to prevent human error.Be very careful when assigning blame to humanerror or to a specific p p person
Three keys to exceptions y pDocumentDocumentDocument!
Keep it simple! p pDocument the standard questions: Who What Wh t When Why Where How
Other requirements qRoot Cause Determination If root cause is unknown, a study should be co duc ed conductedImpact Determination How severe is the exception? What is the impact to intent?Corrective Action Determination How will this be prevented in routine use?Re Testing Re executionRe-Testing / Re-execution Is it necessary? Why or why not?
Exception example #4 p pDuring the annual re qualification / re-verification of re-qualification re verificationthe performance of an autoclave which has beenused for manufacturing of p g product, the ,thermocouples did not achieve the requiredtemperature.Is this an exception? Why or why not?What do you do?
Exception example #5 p pDuring the verification of the installation of Vesselwhich has not been used to manufacture product,ppower was lost during testing of the agitator. g g gIs this an exception? Why or why not?What do you do?
Exception example #6 p pDuring the re validation / re verification of Cleaning re-validation re-verificationvalidation of a process which has NOT been usedto manufacture product, some samples were p , paccidentally not taken.Is this an exception? Why or why not? p y yWhat do you do?
Exception example #7 p pDuring the execution of an alarm check(operational verification) for a system not used tomanufacture product, the test document had the p ,incorrect sequence of events.Is this an exception? Why or why not? p y yWhat do you do?