This document discusses quality metrics and their importance in measuring and improving pharmaceutical quality. It defines quality metrics as standards used to assess efficiency, performance, and quality. Quality metrics help ensure what is measured gets recorded, monitored, controlled, and improved. The document outlines key quality metrics like batch acceptance rate, product complaint rate, and invalid OOS rate. It also discusses FDA's guidance around voluntary submission of quality metrics data and the benefits this can provide manufacturers. Overall quality metrics are presented as an important tool for continual improvement and assurance of patient safety.

1. Dr. A. Amsavel

2. An Overview
 Objective
 Requirement
 Guidance
 Quality metrics
 Measurements
 Conclusions

3. Introduction
 GMP and Quality systems are dynamic .
 Regulations and requirements for Pharmaceutical
industries become stringent YOY
 Pharmaceutical Quality for the 21st Century -ICH Q9, Q10
& Q11 in addition to Q7 and other guideline
 Latest expectation from Regulatory Agencies is QM

4. Metric- Meaning
 Standards of measurement by which efficiency,
performance, progress, or Quality of a Plan,
Process, or Product can be assessed

5. Measures to Improvements
What does not get Measured, Can not be Recorded
What does not get Recorded, Can not be Monitored
What does not get Monitored, Can not be Controlled
What does not get Controlled, Can not be Improved

6. Why Quality Metrics and Who
 Manufacturers have extensive knowledge about critical
product and process parameters and quality attributes
 Organization to measure performance both inside and in
its competitive environment
 Uses data and analysis in decision making
 Manufacturers strive for continual improvement
Use QM for Improvements & Growth

7. FDA Expectation
 “A maximally efficient, agile, flexible pharmaceutical
manufacturing sector that reliably produces high
quality drugs without extensive regulatory oversight.”
-Janet Woodcock

8. Quality Metrics & FDA Guidance
 Establish Quality Metrics
 Objective/goal, measurement and method
 Measure and PDCA
 Register to FDA as volunteer to Submit QM
 Submit the Quality metrics to FDA for review

9. Quality Metrics-FDA
 Supports continual improvement of process performance and
product quality.
 Supports continual improvement of PQS
 Important element of oversight and controls over the
manufacture of drugs to ensure quality (section 501 FD&C Act)
 Encourage use of modern, more efficient manufacturing
technologies.
 Improvement by industry leads to more robust manufacturing
processes
 Regulatory flexibility for post-approval changes

10. Quality Metrics Submission -FDA
 Submission of QM to FDA is not a requirement, It is voluntary
reporting program.
 FDA does not intend to take enforcement action based on
errors in a quality metrics data submission, provided the
submission is made in good faith
 FDA does not publicly disclose information
 Providing industry an incentive too
 Participation in the program demonstrates:
 A willingness to proactively engage with the Agency
 A commitment to increasing transparency between industry &
FDA

11. Benefits of Participation
 Work with establishments towards early resolution of
potential quality problems
 Improved Inspection effectiveness
 FDA is considering use of calculated metrics as an
element of the post-approval manufacturing change
reporting program
 Reduction in inspection frequency
 Inclusion on the Quality Metrics Reporters List

12. Quality Metrics
 Quality can be measured on different levels and for many
processes
 It must be useful and it can enable a company to reach a
high quality performance
 Quality Metrics are not just a measure of Quality
Assurance activities but a measure all activities that
ensure quality in the product.
 This should be effective.

13. Quality Metrics: Key Measures
# Process Measurement
1 Robustness of Commercial
Manufacturing Process
Batch acceptance/
rejection rate
2 Robustness of Laboratory
Operation
Invalidate OOS rate
3 Voice of the Patient/Customer Product complaint rate
4 Periodic & timely quality
review
Deviation from timely
PQR

14. Batch Acceptance Rate
 Batch Acceptance Rate =
The number of accepted lots /
No. of lots produced ( no exclusion) in a given period
 Possibly intermediates in in a given period.
 Calculate lot acceptance rate by the month,
 Based on each month trend
 Set Goal for lot acceptance
 Show improvement each month

15. Product Complaint Rate
 Product Complaint Rate =
The number of product quality complaints received for
the product
= ----------------------------------------------------------------
The total number of lots of the product released
in the same timeframe
 Calculate complaint rate by the month / Quarter
 Trend
 Set Goal to reduce the complaint
 Eliminate the repeated complaint.

16. Invalid OOS Rate
Measure for both Batch approval & Stability OOS
Invalidated Out-of-Specification (OOS) Rate (IOOSR)=
Number of OOS test results for lot release invalidated due to an
aberration of the measurement process/ The total number of lot
release OOS test results in the current reporting timeframe of tests
performed in the same timeframe.
= No. of invalidated OOS test results for lot release
The total No. of batch release
= No of OOS test results in long-term stability tested invalidated
Total No . of long-term stability OOS tested
 As per guideline – No of tests not even batches Eg. Assay, related
substances , LOD, ash

17. APR Completion Rate
Measure for APR or PQR
 If the associated APRs or PQRs were completed within 30 days
of annual due date for the product.
 The number of APRs or PQRs required for the product.
 Calculate lot acceptance rate by the month, each month.
Trend Goal – improvement each month
 Establish a goal of reduction of the trend
 = No. of invalidated OOS test results for lot release
The total No. of batch release
= No of OOS test results in long-term stability tested invalidated
Total No . of long-term stability OOS tested
 As per guideline – No of tests not even batches Eg. Assay, related
substances , LOD, ash

18. Goals for FDA’s application of QM
 Develop objective measures
 Conduct continual monitoring, assessment, and reporting
on the state of quality across the inventory of drug
products and facilities regulated by FDA
 Monitor the quality of products and sites by utilizing
information from exploratory and explanatory studies to
detect critical signals (e.g. trends, seasonal effects, cyclic
patterns).
 Senior management commitment to quality
 CAPA effectiveness (retraining, preventive actions)

19. Other Metrics
Facilities & Utilities
 Percentage failure rate for PW system
 TOC/Conductivity being out of range
 Alert & Action exceedances
 HVAC excursions
 Temperature/RH/pressure differentials
 Out of calibration rates for equipment
 Percentage unplanned maintenance

20. Other Metrics
Production
 Right first time
 Documentation errors
 Efficiency
 Quality Failure
 Rework rate
 Reprocessing rate
 EM excursion rates
 Deviations

21. Other Metrics
QC
 QC Laboratory System
 OOS rate - Invalidated OOS
 Raw material/ Finished Product/ Stability
 Testing completed on time
 Instrument failure
Supplier quality requirements ▫ Distribution system
measurements

22. Other Metrics-QA
QA
 Right-First-Time
 Process performance and capability -CP & Cpk
 Deviations without assigned root cause
 Repeated OOS / Deviations
 Quality complaint – other than quality
 Supplier quality requirements
 Distribution system measurements

23. Purpose
to ensure patient safety and Quality
 Ultimately Pharmaceutical Quality is
to Assure every dose is Safe and
Effective, free of Defects and
Adulteration/comtamination.

24. Reference
Submission of Quality Metrics Data Guidance for Industry:
U.S. Department of Health and Human Services
Food and Drug Administration
November 2016