Spreadsheets Validation (IVT San Diego 2012)


Published on

Published in: Health & Medicine
  • Nice presentation, very helpful
    Are you sure you want to  Yes  No
    Your message goes here
  • Very nice work. Clear and concise. Would it be possible to have a copy ? Thanks, Emmet
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Spreadsheets Validation (IVT San Diego 2012)

  2. 2. ABOUT THE AUTHOR• 19+ years of experience in the pharmaceutical, medical devices, biotechnology, and consumer electronic industries• MS Biotechnology & Biomedical Engineering• BS Mechanical Engineering• I have led validation / qualification efforts in many scenarios: • High-speed manufacturing and packaging equipment • Laboratory : information systems and equipment / instruments • Enterprise resource planning applications (i.e. SAP) • IT network infrastructure • Software applications • Product improvements, material changes, vendor changes• Contact information: • Raul Soto rasove_21@yahoo.com 2
  3. 3. Table of Contents1. Regulatory Perspective2. Building the Spreadsheet3. Validating the Spreadsheet
  5. 5. FDA WARNING LETTERS• http://www.fda.gov/ICECI/EnforcementActi ons/WarningLetters/default.htm 5
  6. 6. FDA WARNING LETTERSMedical Devices:http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2011/ucm256498.htm 6
  7. 7. FDA WARNING LETTERSMedical Devices:http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2002/ucm145296.htm
  8. 8. FDA WARNING LETTERSPharmaceuticals:http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2001/ucm178334.htm
  9. 9. FDA WARNING LETTERSMedical Devices:http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2003/ucm147927.htm
  10. 10. FDA WARNING LETTERSMedical Devices:http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2001/ucm178612.htm
  11. 11. FDA WARNING LETTERSBiologics :http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2006/ucm075766.htm
  12. 12. MEDICAL DEVICES• 21 CFR 820.70(i): Automated processes• When computers or automated data processing systems are used as part of production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol.• All software changes shall be validated before approval and issuance.• These validation activities and results shall be documented. 12
  13. 13. MEDICAL DEVICES• Let’s take this apart:• Applies to computer / data processing systems used for production or quality• The manufacturer is responsible for validation• Validation must be for the intended use(s)• Must have an established protocol• Changes must be validated before implemented• Document all validation activities and results 13
  14. 14. PHARMACEUTICALS• 21 CFR 211.165(e): Testing and Release for Distribution • (e) The accuracy, sensitivity, specificity, and reproducibility of test methods employed by the firm shall be established and documented. Such validation and documentation may be accomplished in accordance with 211.194(a)(2). 14
  15. 15. BIOLOGICS• 21 CFR 606.140: Laboratory Controls• Laboratory control procedures shall include:• (a) The establishment of scientifically sound and appropriate specifications, standards and test procedures to assure that blood and blood components are safe, pure, potent and effective.• (b) Adequate provisions for monitoring the reliability, accuracy, precision and performance of laboratory test procedures and instruments.• (c) Adequate identification and handling of all test samples so that they are accurately related to the specific unit of product being tested, or to its donor, or to the specific recipient, where applicable. 15
  16. 16. ISO 13485 DESIGN AND MANUFACTURE OF MEDICAL DEVICES• Clause• The organization shall establish documented procedures for the validation of the application of computer software (and changes to such software and/or its application) for production and service provision that affect the ability of the product to conform to specified requirements.• Such software applications shall be validated prior to initial use. 16
  17. 17. FDA’S INTERNAL SPREADSHEETS• http://www.fda.gov/ScienceResearch/FieldScience /LaboratoryManual/ucm174286.htm 17
  18. 18. 18
  19. 19. FDA’S INTERNAL SOP’S FOR SPREADSHEET VALIDATION• Single-user spreadsheets: • www.spreadsheetvalidation.com/pdf/LIB_Part_II_Single- User.pdf• Multiple-user spreadsheets: • www.spreadsheetvalidation.com/pdf/LIB_Design_Multi- User2A.pdf 19
  20. 20. 20
  22. 22. SCOPE• Microsoft Excel file that may contain • mathematical calculations • Statistical analysis • Graphing• Outside our scope: • Macros or custom code • Database lookups 22
  23. 23. IMPORTANT !!!• Locking / unlocking cells• Protect spreadsheets /worksheets• Cell formatting: • Data types • Look and feel : Font, Colors• Validation criteria definitions for input cells• Define formulas• Use of logical statements in formulas• Use of Page Set-up 23
  24. 24. CHANGE CONTROL• Spreadsheets used for GxP processes should be subjected to formal version control and validation.• The organization’s change management system should be used to document whenever we: • Create a new spreadsheet for regulated / validated uses • Make changes to a validated spreadsheet • Retire/decommission a validated spreadsheet • Move a validated spreadsheet to a different version of Excel (i.e. from Excel 2007 to Excel 2010) from the one used in development and validation • Move a validated spreadsheet to a different operating system (i.e. from Windows to Linux or MacOS) 24
  25. 25. VERSION CONTROL• The following information (at minimum) should be included on specific cells of the main worksheet: • Spreadsheet Name • Version number • Date of last modification • Every printout should contain this information 25
  26. 26. CHANGE HISTORY• Each spreadsheet should have a Change History worksheet• Every time the spreadsheet is updated, an entry should be made in this Change History: • Version number • Date of change approval • Change control number • Name of person(s) who made the change • Spreadsheet Validation Protocol document number • Location of spreadsheet file in the network • Itemized description of ALL changes made (with DETAILS) 26
  27. 27. CREATING THE SPREADSHEET• Verify formulas against sources• Use correct units of measurement• Lock all non-input cells• Disable column/row add/remove• Unlock input cells• Distinguish input, calculation (formula) vs output cells • Cell background color • Font : bold, italics, size, color• Use conditional formatting on output cells • Different cell background color and/or font color for results that pass vs results that fail 27
  28. 28. CREATING THE SPREADSHEET• Add a field where the user will enter his/her name every time the spreadsheet is used.• Use Page Setup to print the following information: • Spreadsheet filename • Correct date and time • Page numbers and total pages• Make the spreadsheet read-only 28
  29. 29. DATA VALIDATION CRITERIA• Use Excel’s Data Validation tools to enforce limits in input fields.• Examples: • Integer input between 1 and 20 • Alphanumeric field, from 1 to 16 characters • Date or time • Decimal number greater, lower or than a specific value; or between specific values• Excel’s Data Validation also allows you to: • Display custom messages on input cells • Display custom error messages when input is outside parameters 29
  31. 31. DATA VALIDATION• Types of inputs: • Any value • Whole number • Decimal • List • restrict input to items from a specified list • List should be included in a protected column in the spreadsheet • Date • Time • Text length • Custom 31
  32. 32. DATA VALIDATION• Custom: • Example: ensure the entry in a column is a number 1.Select the column 2.select Data, Validation 3.Select the Settings tab 4.Select Custom from the Allow list 5.In the Formula field, enter =ISNUMBER(c1) 6.Click OK • The ISNUMBER function returns TRUE if the cell contains a number 32
  33. 33. UNITS CONVERSION• You can use the Excel CONVERT function to convert values from one unit to another in your formulas• Click HELP in Excel for a list of all units included. 33
  34. 34. SECURITY LAYERS• Lock all cells except user input cells. • HOME tab/ CELLS group/ Format / Protection• Password-protect the worksheet • Prepare / Encrypt document• Install the validated spreadsheet in a protected folder on a networked shared drive • Access to the folder should only be given to authorized personnel 34
  36. 36. VALIDATING YOUR SPREADSHEET• Validation : Provide documented evidence that the spreadsheet does what it is intended to do, correctly, and consistently• Validation Protocol• General Information• Requirements• IQ : Installation and configuration• OQ : Test Scripts• Validation Report 36
  37. 37. VALIDATION PROTOCOL• General Information • Spreadsheet name • Description and Purpose : • What the spreadsheet will be used for • Processes and areas where it will be used • References • Analytical methods, SOPs, etc. 37
  38. 38. VALIDATION PROTOCOL• Requirements • MS Excel version required • Excel Add-Ins required, if any • OS version in which it is developed • For example: MS Windows 7 Home Premium SP1, Mac OS X 10.6 • Input cell and formula cell requirements • Formatting • Data validation (field type, size, etc.) • Descriptions for every formula 38
  39. 39. VALIDATION PROTOCOL• Requirements • Charts and Graphs • Title and Description of each • Required labels for each axis • Security requirements • Password protection • Output requirements • Color printer, PDF-creating software • Specific folder in a specific shared drive • Automatic eMail to specific people 39
  40. 40. VALIDATION PROTOCOL• IQ : Installation and Configuration • State all prerequisites, ensure they are met prior to execution • List all spreadsheet files that have to be installed • List location and path where spreadsheet files will be installed • Development environment – does not have to be validated • Test environment – may need to be validated • Production environment – must be validated 40
  41. 41. VALIDATION PROTOCOL• OQ • State all prerequisites, ensure they are met prior to execution • Verify that all requirements documented in the Requirements section are met • Ensure all outputs have the correct formatting • Provide printouts of all formulas, input values used, output results. • Positive v Negative testing • Security testing • Performance testing • Simulate output issues 41
  42. 42. VALIDATION PROTOCOL• OQ • Positive testing : test and verify every single formula using values that should produce correct results • Ensure the spreadsheet produces correct results for various input data sets • i.e. two data sets • Data set 1: known to result in a PASS decision • Data set 2: known to result in a FAIL decision • Verify spreadsheet results against results from manual calculations 42
  43. 43. VALIDATION PROTOCOL• OQ • Negative testing : • Use incorrect inputs to ensure the spreadsheet is robust, able to handle them without crashing or producing false or incorrect outputs, and that it is not possible to bypass security 43
  44. 44. VALIDATION PROTOCOL• OQ - Negative testing examples: • Required data entry : ensure input fields cannot be left blank • Field type test: ensure data entered is of the correct type (i.e. dates, alphanumeric, etc.) • Enter alpha or special (@%^#) characters on a numeric field • Enter a decimal number in an integer field • Field size test: ensure user can only enter the specified number of characters in a field 44
  45. 45. VALIDATION PROTOCOL• OQ - Negative testing examples: • Date bounds test: test upper and lower bounds for dates. • Numeric bounds test: for example, if the number in a field should be between 10 and 50, the system should not accept numbers lower than 10, and higher than 50. • Numeric limits test: make sure the spreadsheet doesn’t crash or produce incorrect output when entering a number outside of Excel’s limits: • http://office.microsoft.com/en-us/excel-help/excel-specifications-and- limits-HP005199291.aspx 45
  46. 46. VALIDATION PROTOCOL• OQ : • Output issues : disconnect printer, close the PDF-rendering application, ensure the spreadsheet behaves as expected • Security testing: • Ensure it is not possible to unlock locked cells / worksheets • Ensure it is not possible to alter formulas, output results, output graphs /charts/ tables • Performance testing: • Execute spreadsheet while various other applications are running • Open several instances of the spreadsheet at the same time 47
  47. 47. VALIDATION PROTOCOL• OQ : Formula Auditing • Excel’s Formula Auditing tools allows us to demonstrate that the inputs come from the expected cells 48
  48. 48. VALIDATION PROTOCOL• Executing the protocol • Run all test cases • Compare the actual vs expected results • Document pass / fail of test steps • Take screenshots as documented evidence • Turn on Track Changes • Save the Change Log 49
  49. 49. VALIDATION REPORT• Include all executed test scripts• Include all screenshots, printed outputs, manual calculations• Include the Change Log• Post-approvals 50
  50. 50. HOW MUCH IS TOO MUCH?• Depends on the level of risk 51
  51. 51. QUESTIONS 52
  52. 52. Thank You ! 53