This document discusses post-market surveillance of medical devices in Lesotho, focusing on point-of-care testing for early infant HIV diagnosis. It describes analyzing invalid and error rates for two assays from January to May 2017, with the Alere assay peaking at 9.3% in February. Corrective actions like cleaner technique reduced further errors. Two lots of Cepheid HIV test cartridges showed high error rates, which the company is investigating while providing replacement cartridges. The issues were reported to the WHO Prequalification Programme to aid investigation and information sharing.

1. Optimizing Early Infant HIV Diagnosis
through the Introduction of Point of Care
Testing
Post-Market Surveillance
Case Study
Lesotho, October 2017

2. Post Market surveillance
Post-marketing surveillance (PMS) (also post market
surveillance) is the practice of monitoring the safety of
a pharmaceutical drug or medical device after it has
been released on the market and is an important part of
the science of pharmacovigilance.

3. Why ?
• Devices offer opportunities for improved diagnosis and management of
disease, they also can carry substantial risks.
• Gvt & Regulatory bodies need to balance the goals of expanding
therapeutic options with safeguarding public health
• In recognition of the importance of medical devices, the WHO established a
Medical Device Unit
– focus research and policy on prioritizing access to medical devices in low-
resource settings
– Dissemination of innovations, and
– Training of biomedical personnel to support the use of devices worldwide

4. Two Types of PMS

5. Reactive Post-market Surveillance
• Complaint reporting, including vigilance of mild,
moderate and severe adverse events
• Evaluation of data from external quality assessment
schemes (proficiency testing)
• End-user quality control programmes.

6. Proactive Post-market Surveillance Activities
• Lot verification testing (pre-distribution and post-distribution
to end-users).
• Lot verification testing aims to identify any catastrophic
product failure and to determine variation from lot to the
next.
• Lots may either be consecutively or randomly sampled for
testing on a panel of well-characterised biological
specimens.

7. • evaluated strategies for PMS of medical devices in the United States, European
Union, Japan, and China.
• Several common elements, including primary reliance on passive adverse event
collection for marketed devices, but vary widely in their allocation of stakeholder
responsibilities and mechanisms for evaluating the performance and safety of
approved devices.
• Greater system transparency, scheduled re-examination of approved devices, and
balancing central and local control

8. Key features of four device post-market
surveillance systems
https://doi.org/10.1371/journal.pmed.1001519.t001
Daniel B et al, 2013, Plos Medicine

9. Example of IQC failures analysis reporting in Lesotho
Sources
• IQC failure Incident log at testing site
• WhatsApp group of Qaulity Assurance Officers and
users
• IQC failure, Invalid and No Result Analysis Report
• Maintenance log

10. IQC Failure, Invalid And No Result Analysis Report
Instrument (All)
Date (Multiple Items)
Years 2017
Count of Test Result Error code
Row Labels 1006 2005 2037 2096 2097 2126 5002 5011 5016 12002 23204 23205 24100 (blank) 5007 5015 18004 23207 no error code Grand Total
Likotsi Filter Clinic 1 1 2 1 7 3 15
Mafeteng Hospital 1 5 2 5 1 20 1 1 8 44
Maluti SDA Hospital 1 1 1 8 3 2 16
Maputsoe SDA 1 2 6 1 2 1 13
Seboche Hospital 2 2 2 1 7
Grand Total 2 1 1 1 7 2 8 4 37 1 2 6 1 4 4 3 2 1 8 95
Site InstrumentDate Test Result Description Detail Error CodeCorrective Action Sample ID
User Module
Addressed? (Y/N)
Comments (if any)
Mafeteng Hospital
GeneXpert 8/1/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 3.9 was above the maximum of 3.0
5016 Repeated with leftover sample in B4 and passed
002459 Nthabeleng Sesheme
B3 Y Repeat sample HIV-1 NOT DETECTED
Mafeteng Hospital
GeneXpert 8/4/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 4.1 was above the maximum of 3.0
5016 Repeated with leftover sample in B1 and passed
001419 Nthabeleng Sesheme
B4 Y Repeat sample HIV-1 NOT DETECTED
Mafeteng Hospital
GeneXpert 8/8/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 6.9 was above the maximum of 3.0
5016 Repeated with leftover sample in B2 and passed
000433 Nthabeleng Sesheme
B2 Y Repeat sample HIV-1 NOT DETECTED
Mafeteng Hospital
GeneXpert 8/8/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 12.1 was above the maximum of 3.0
5016 Repeated with leftover sample in B2 and passed
001211 Nthabeleng Sesheme
B1 Y Repeat sample HIV-1 NOT DETECTED
Seboche Hospital GeneXpert 8/8/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [HIV-1]. The RMS error of 4.2 was above the maximum of 3.0
5016 Caregiver contacted002786 Ithabeleng Lesaoana
A3 N Repeat sample clotted, caregiver informed, samp
Maluti SDA Hospital
GeneXpert 8/1/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 3.4 was above the maximum of 3.0
5016 Repeated with leftover sample in module A3 and passed
004803 Malillo Sepiriti
A2 Y Repeat sample HIV 1 NOT DETECTED
Maluti SDA Hospital
GeneXpert 8/3/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 3.3 was above the maximum of 3.0
5016 Repeated with leftover sample in module A4 and passed
002220 Malillo Sepiriti
A1 Y Repeat sample HIV 1 NOT DETECTED
Maluti SDA Hospital
GeneXpert 8/7/2017 ERROR Post-run analysis error
Error 5011: Signal loss detected in the amplification curve for analyte [HIV-1]. 64.2 decrease in signal with 20.7 % degrees at cycle 25
5011 Repeated with leftover sample in module A1 and passed
001030 Malillo Sepiriti
A4 Y Repeat sample HIV 1 NOT DETECTED
Likotsi Filter ClinicGeneXpert 8/8/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 12.1 was above the maximum of 3.0
5016 Repeated with leftover sample in A4 and passed
004811 Thabang Likhojana
A1 Y Repeat sample HIV-1 NOT DETECTED
Likotsi Filter ClinicGeneXpert 8/8/2017 ERROR Post-run analysis error
Error 5011: Signal loss detected in the amplification curve for analyte [HIV-1]. 55.2 decrease in signal with 21.1 % degrees at cycle 30
5011 Repeated with leftover sample in A1 and passed
004815 Manapo Matsoso
A1 Y Repeat sample HIV-1 NOT DETECTED
Likotsi Filter ClinicGeneXpert 8/21/2017 INVALID Post-run analysis error
Error 5016: Failed to verify valid background error for analyte [SPC]. The RMS error of 12.1 was above the maximum of 3.0
5016 Repeated with leftover sample in A4 and passed
004828 Thabang Likhojana
A4 Y Repeat sample HIV-1 NOT DETECTED
ERROR, INVALID AND NO RESULT ANALYSIS REPORT

11. Example of IQC failure analysis reporting in Lesotho
0%
2%
4%
6%
8%
10%
Jan Feb March April May
Error
rate
cepheid Xpert Alere q
In February, the Alere rate peaked at 9.3%. Recurrent IQC failures suggested possible causes related to
dirty instrument optics and incorrect cartridge loading. Corrective actions, taken in early March included
use of powder-free gloves and retraining of end-users on key pre-analytical steps of testing; no further
errors on the Alere assay were detected from mid-March on

12. Process followed to report the Cepheid cartridges
problem to WHO-PQ
• EGPAF reached out to Cepheid regarding lot Lot 30401 and Lot
30502
• Cameroon had similar rates for the same Lot numbers (56 %)
site
started
testing
number of tests
since 27 March
to 14 July 2017
total #
of
errors
overall
IQC error
rate
# of IQC
error 5016
only
Proportion of
Code 5016
overall error
rate
excluding
code 5016
Mafeteng 22-Dec-16 456 34 7% 20 59% 3%
Likotsi 2-Jan-17 123 10 8% 7 70% 2%
Maluti SDA 8-Jun-17 164 16 10% 8 50% 5%
Seboche 6-Jun-17 60 5 8% 2 40% 5%
Totals Xpert 803 65 8% 37 57% 3%

13. Sharing with the Consortium
• Irregularities noticed for Xpert HIV-1 Qual cartridge lots
30401 and 30502
• From kits lot 1000055129 and 1000057681 respectively)
• Expiry dates of 2017-06-25 and 2017-12-10
• Cepheid voluntarily credited us cartridges to account for the
number of 5016 codes obtained while they are performing
further investigation.

14. Officially notified WHO-PQ
• Diagnostic Advisor notified WHO on 10 Aug 2017
• A post-market surveillance effort
• For proper documentation, investigation and dissemination.
• Recalled the lots in question from sites mid Aug
• preliminary investigations- “a change was made to the product to ameliorate the
possibility for false “HIV-1 detected” results, had an unintended consequence of
increasing errors than before.”
• 1 month post recall – error 5016 significantly reduced

15. Example of IQC failure analysis reporting in
Lesotho
• IQC failure reporting log was introduced for
clinic end-users (health workers) to
investigate all failures
• IQC failures observed in the first five months
(January-May 2017) of routine clinical use of
the two POC assays in three maternal and
child health clinics were analyzed monthly
for event counts and error/invalid types
0%
2%
4%
6%
8%
10%
12%
Error rates from Jan - May 2017
cepheid Xpert Alere q
Fig 1- Monthly IQC failure rates from Jan – Aug 2017

16. This document was made possible thanks to Unitaid’s
support. Unitaid accelerates access to innovation so that
critical health products can reach the people who most
need them.
For more information, please contact: The EGPAF
Innovation and New Technology Team at
(innovation@pedaids.org).
www.pedaids.org | www.unitaid.org