This document discusses challenges in literature screening for pharmacovigilance and potential solutions. The key challenges are the high volume of scientific literature, poor signal-to-noise ratio, compliance risks for audits and inspections, duplicate articles, and increased workload from regulatory requirements. Technology solutions like automation, prioritization, text mining and machine learning can help address these challenges by improving workflow efficiency and compliance. Outsourcing literature screening services can also help reduce costs and resources needed while maintaining oversight and accountability.

1. Marnix Wieffer, Senior marketing manager drug safety
m.wieffer@elsvier.com
https://www.linkedin.com/in/marnix-wieffer/
Improving Effectiveness
And Compliance Of
Literature screening
Automation, Prioritization And Outsourcing

2. What we would like to leave you with….
With workflow solutions, automation and partnerships we can
significantly improve efficiency and reduce regulatory issues
Biomedical literature is an important source
of post-market adverse event reporting
Significant challenges around costs,
efficiencies, oversight and regulations
Pharma companies are continuously
juggling the benefits of in and outsourcing

3. Why screen literature for Pharmacovigilance?
1. Screening literature for
adverse events is a regulatory
requirement
25-9-2019
2. Literature is an important
source for adverse event
identification​
➢ Part of inspection ➢ Case reporting,
aggregated reporting
signal detection and
validation
➢ Often adverse drug
reactions are not
reported spontaneously
and may only be
reported in the literature

4. Literature as important source of reports on
adverse drug reactions
Tamsulosin is used for benign prostatic hyperplasia, kidney stones or
acute urinary retention
Intraoperative floppy iris syndrome (IFIS) in ≈2% of their cataract surgery
population
Appeared to be associated with Tamsulosin
After retrospective data analysis (511 patients) a prospective study (741
patients)
The clinicians Chang & Campbell published their observations in 2005
At the time of publication, none of these cases had been reported to
Regulatory Authorities!
IFIS (patients) Retrospective
Study
Prospective
Study
No Tamsulosin 0 / 495* 1 / 725
With Tamsulosin 10 / 16 15 / 16
* No IFIS in 11 patients on other systemic α1-blockers for treatment of BPH Intraoperative floppy iris syndrome associated with tamsulosin
Chang D.F., Campbell J.R. Journal of Cataract and Refractive
Surgery 2005 31:4 (664-673)
Tamulosin and Floppy Iris Syndrom

5. Data retrieved from www.Embase.com
Number of articles in Embase per year
25-9-2019
Challenge #1 High Volume of Scientific Literature
400m
1,417,665 records in 2018...

6. A good search?
• Broad and comprehensive to make sure all
relevant literature is retrieved
Outcome: results in a high recall with low
precision.
Why is it a bottleneck?:
• Experienced resource are not spending time on
value added functions.
• Reviewing high volume of non-relevant
citations and abstracts.
25-9-2019
Challenge #2 Poor signal to noise
Precision Recall

7. Why is it a bottleneck?
• Inspection requests can be lengthy and
time consuming
• Need to ensure that all decisions are
documented for auditability and traceability
• Important to have the information readily
available to be provided to the inspectors
25-9-2019
'My previous PV experience
was that literature was on the
agenda for every audit and
inspection that occurred'
Avoid
• Inspection findings
• Additional work with authorities
• Increase costs for remediation work
• Reputation with regulators
• Damaged brand image
Challenge #3 Compliance Risks for Audits and
Inspections

8. 1. Multiple databases used for searching
with same journals (e.g. Medline and
Embase).
2. Different search strategies for ICSR,
aggregate, signals but the same article
is retrieved in each search results.
3. Updates to metadata (e.g. updated
indexing, when “in press” articles are
published) these records will be
updated and received in your results
4. EMA-MLM Service
25-9-2019
Pharma
Biosis
previews
Derwent
drug file
Sources of
duplicates
Challenge #4 Duplicate Articles Increasing
Workload

9. 25-9-2019
Challenge #5: Impact of Increased Workload due
to EMA’s MLM
2. Missed Duplicates identified from
EMA-MLM results and from ICSR
searching (re-review)
MLM
Case
report
biomedical
database
biomedical
database
1. Poor Integration of EMA-
MLM service into Pharma’s
PV literature workflow
EMAPharma

10. 25-9-2019
Technology
Supporting Compliance and Automation

11. Addressing Challenges with Technology
Opportunity:
MAHs have a requirement for a PV Literature monitoring and review solution which
meets the demands
✓ Optimized Search Strategy in Bio-medical database
✓ Automated Import of Search results from Bio-medical databases
✓ De-duplication of retrieved search results
✓ Automated ordering of unique journals and translations
✓ Efficiency in Workflow for ICSRs to avoid re-ordering articles and keep focus on review
✓ Auditability and traceability of all decisions and actions that were completed to provide
inspection ready information
✓ Identifying valid ICSRs from an average of 95% Invalid ICSRs
25-9-2019

12. 25-9-2019
Technology to Assess AE from Literature
▪ Automated ingestion of literature search results
▪ Customizable workflows
▪ Deduplication
• Oversight
• Inspection ready
• Operate GxP compliant
• Audit Trail

13. Approach Purpose
Rule based
approach
To identify relevant information from
literature and separate invalid ICSRs
Text mining
& NLP
To identify and extract entity relationship
and feed the ICSR prioritization algorithm.
Continuously build Gold set to finetune
Machine Learning model.
Added flexibility to expand MAH criteria.
AI and
Machine
Learning
Support with case identification and
extraction of information to meet decision
criteria of MAH’s requirements
25-9-2019
Our Journey to an Intelligent Solution
0
50
100
150
200
250
Rules
Based
Automation
Text Mining
and NLP
Machine
Learning
via entity
extraction
Impact with scales
Efficiencygainvsmanual

14. 25-9-2019
Prioritization
Using the power of Automation

15. Identification of relevant records is very inefficient
25-9-2019
Incoming
records
Incoming records
Manual ICSR
Screening
!
Individual Case Safety
Reports (ICSRs)
95% of analyzed record is not
relevant for ICSR reporting

16. Screening can be focused using ICSR prioritization
25-9-2019
Incoming
records
ICSR
Prioritization
ICSR XML export
!
Incoming records
!
ICSR
reviewIncoming records

17. 25-9-2019
Progress with ICSR Prioritization- Ready for Market
Sample
Size
Precision Recall
4500+ 99.9% 65%
140,100 99.9% 52.5%
With the initial assumption that 95% of the citations are not relevant to potentially be screened as ICSRs.
Approximately 5-10% would be potential “hits” for ICSRs identification and close to 5% would be confirmed
ICSRs.
Reiterating the algorithm on sample sets, including 1 year’s data to find invalids / non ICSRs
Test Results of the Algorithm

18. High efficiency Gain with ICSR Prioritization
Records/
month
Avg Review
time in min
Review Time
in hours
FTE / month
13000
45 9750 40
45 *1500
10 *11500
3041 12
0
10
20
30
40
50
60
23 24 25 26 27 28 29
Predicted* ISCR Review Time
Time Predicted
* Assumption that in the predicted weeks, there is no case reported.
** Increments are symbolic based on assumption
Proposed method for review
• ICSR Prioritization gives the visual decision support
• Enable reviewer review abstracts for non-cases
• Algorithm reduces error from full manual review
• Encourage review with full text when reporting cases
• Decision support provides time-savings
Assumption
- 95% of invalids can be reviewed in ~10min
- 5% of valid ICSRs can be reviewed in ~45 min
- Average of 1500 cases /month
- Average of 700 citations/ day or 13000 citations/ month

19. Rule based approach algorithm can be
used for ICSR Prioritization
To prioritize the likeliness of ICSRs into
high, medium, and low potential
backlogs.
Benefits:
• Enable PV reviewers to focus on the
most relevant citation/abstracts first in
order to meet compliance.
• Possibly eliminating errors in the review
process.
25-9-2019
ICSR Prioritization Rule-based approach
Embase
ICSR + PV
Management Solution
Quosa PV
Homegrown PV
Literature management
ICSR Prioritization
Service

20. Next step: Automated case extraction
Drug
Adverse event
Reporter
Patient

21. 1. AdverseReaction: signs and symptoms,worseningmedicalconditions,
changes in laboratoryparameters,and changes in other measures of
critical bodyfunction,such as vital signs and ECG results.
2. Severity: Measurementofthe severity of a specific AdverseReaction.This
can be qualitativeterms (e.g., "major", "critical", "serious","life-
threatening") or quantitativegrades (e.g.,"grade 1", "Grade 3-4", "3 times
upper limit of normal (ULN)", "240 mg/dL").
3. DrugClass:The class of drug that the specific drug belongs to.
4. Negation: Trigger word for event negation.
5. Animal: Animalspecies in which an AdverseReaction was observed.
6. Factor: May, risk, potential,references to the placebo arm ofa clinical
trial, or specific sub-populations(e.g.,pregnancy, fetus)
1. Negated: A Negation or Factor that negates an AdverseReactionforthe
drug.
2. Hypothetical: An Animal,DrugClass,or Factor that speculates about,or
qualifies thedefinitivenessofthe drug's relationshipwith
an AdverseReaction.
3. Effect: A Severity of an AdverseReactionfor the drug.
Entities
Relations
Enti
ty
Enti
ty
Relati
on
TAC 2018 gold set & Elsevier gold set
Research ahead: Entity data extraction using ML

22. Drug safety information extraction using ML
Extraction of information at the level
Drug | Dosage | Patient| Adverse Event |Adverse Drug Reaction
Going Forward, feed output of Entity data extraction into building gold set and training the Machine Learning model

23. 25-9-2019
Outsourcing
Towards a stronger partnership

24. Outsourcing of literature screening is common
• Reduce required in-house resources
needed
• Stay Flexible
• Respond fully to regulatory obligations
• Avoid competitive hiring of experienced
safety personnel
ProCon
• Costs can be high!
• MAH remains accountable
• Lack of oversight

25. Screening Review Case Processing Submission
ProcessesPeople
Highly skilled and experienced science graduates trained in handling literature and E2B R3 compliant safety databases
Highly skilled medical professionals for medical review and signal detection
Software
ICSR
prioritization
E2B-R3
Outsourced PV screening using Elsevier technology
backbone

26. Cost effective and compliant full
literature screening services with
oversight

27. 25-9-2019
Elsevier’s Current and Future Solutions
• Industry standard
biomedical literature database
• Suggested by regulators
• Optimized for PV search
strategies
• Inspection ready info
• Oversight for internal and
external resources
• Increased workflow efficiency
• Signal validation
• Safety data extracted
from regulatory
approval documents
• Includes the FDA
FAERS data

28. What we would like to leave you with….
With workflow solutions, automation and partnerships we can
significantly improve efficiency and reduce regulatory issues
Biomedical literature is an important source
of post-market adverse event reporting
Significant challenges around costs,
efficiencies, oversight and regulations
Pharma companies are continuously
juggling the benefits of in and outsourcing

29. Thank You
Marnix Wieffer, Senior marketing manager drug safety
m.wieffer@elsvier.com
https://www.linkedin.com/in/marnix-wieffer/
Matthias Bandilla, Information specialist
m.bandilla@elsevier.com
https://www.linkedin.com/in/matthias-bandilla-0943285a/
Jean-Dominique Pierret, Customer Consultant Pharmacovigilance & Safety
j.pierret@elsevier.com
https://www.linkedin.com/in/jdpierret/