This document discusses redesigning post-market drug safety surveillance. It proposes developing an integrated commercial off-the-shelf post-market surveillance platform to analyze real-world data on prescribing, patient outcomes, and adverse events in a standardized way. This could provide valuable insight into drug safety and efficacy using real-world evidence from clinical practice settings. Currently, post-market surveillance relies on voluntary reporting which results in underreporting and limited data for analyzing trends. The proposed platform aims to address these limitations through consistent analysis of real-world data sources.

1. REDESIGNING
POST-MARKET DRUG
SAFETY SURVEILLANCE
ARETE-ZOE, LLC
December 30, 2017

2. INFORMATION ENVIRONMENT
11/4/2019https://www.aretezoe.com/
2
OBSERVATION
S
Unfolding
circumstances
Outside
information
Unfolding
interaction with
environment
Implicit guidance
and control
ORIENTATION
Cultural
traditions
Genetic
heritage
New
information
Previous
Experiences
Analysis &
Synthesis
Implicit guidance
and control
Unfolding
interaction with
environment
DECISION
Hypothesis
ACTION
Test
Col. John Boyd’s OODA Loop

3. SYSTEM RESPONSIVENESS
11/4/2019https://www.aretezoe.com/
3
The direct impact of an improved information environment
Leveson, N: Applying System Engineering to Pharmaceutical Safety

4. STAKEHOLDERS
11/4/2019https://www.aretezoe.com/
4
Manufacturers&Sponsors
•ROI products
•Compliance
•Continue sales
•New indications
•New target
populations
CROs
•ROI service
•Conduct trials /
studies
•Produce data
for sponsors
Regulators
•Public health
•Safety, efficacy
& security
•Accurate
information
•Facilitate
innovation
Healthcareproviders
•Quality of care
•Access
•Cost
•Community
benefit
•Comparative
efficacy
Insurers,Payers,CMS
•Cost-
effectiveness
•Safety, Efficacy
•Pharmaco-
economics
•System
responsiveness
•Avoidable injury
Patients
Value & Co-pays
Investors
Minimize financial risk

5. PROBLEM STATEMENT
Existing pharmacovigilance post-market surveillance system is based
on reporting of adverse drug reactions to national databases in real
time and post-marketing safety studies.
Quantitative evaluation of incidence, prevalence trends and patterns
of use from reporting data is problematic due to under-reporting and
missing data on exposure. This is especially true if drugs are used off-
label, in populations they were not intended for, or in combination
with other medications.
Whilst the case study below emphasizes post-market surveillance of
marketed drugs, the same principles are applicable to other areas
within clinical research and drug development, outcomes research,
monitoring and evaluation of the quality of provided care and
pharmaco-economic applications.
11/4/2019https://www.aretezoe.com/
5

6. The design and development of an integrated COTS PMS
platform can provide valuable insight into the safety and
efficacy of marketed drugs by analyzing a stream of real-world
data from prescribing information, patient outcomes, and
adverse events in a consistent and systematic manner using
standardized pre-approved Master Study Protocols.
11/4/2019https://www.aretezoe.com/
6
INTELLIGENT DESIGN
INTEGRATED DATA PLATFORM

7. WHAT
Real-world data
Product data
Patient characteristics
Key indicators of disease
Selected biomarkers
Interventions
Treatment outcomes
Complications
ADRs/ADEs
Real-world evidence
Patterns of use
Benefits and risks
New indications
Special populations
Effectiveness of REMS
Output
Post-authorization safety
studies (PASS)
Post-authorization efficacy
studies (PAES)
Pharmaco-economic studies
Comparative studies
11/4/2019https://www.aretezoe.com/
7

8. WHY
• Consistent medical evidence
• Real-world treatment setting, reflects actual use
• Comparable across therapeutic/indication group
• Complement Phase I – IV studies
• RWE to support
• Regulatory submissions (PASS/PAES)
• New indications
• New populations
• Risk-benefit profile
• Effectiveness of REMS
• Pharmacoeconomic studies and HTAs
11/4/2019https://www.aretezoe.com/
8
Improved quality of evidence and increased speed of generating
and evaluating such evidence would increase system
responsiveness to avoidable patient injury.

9. 11/4/2019https://www.aretezoe.com/
9
SAFETY BY DESIGN
Post-approval drug lifecycleWHEN
Healthcare setting levelWHERE
• WHY? PHI are sacred. We share study results. We do not pool
PHI. PHI do not leave the provider/insurer ecosystem.
Per protocol analysisHOW
• DESIGN: Large simple trials, pragmatic, observational
• MASTER PROTOCOLS: Consistency, standardization
• STANDARD FORMAT OF STUDY RESULTS

10. 11/4/2019https://www.aretezoe.com/
10.
Respectstakeholders’informationneeds
Planning Direction
Collection
Processing
Analysis
Production
Distribution
Evaluation
Per Protocol
Analysis
RWE Production
RWD in
Hospital
Setting
Customer
feedback
Communicating
findings
Insurers
HCPs
MAHs
Research
sponsors
Regulators
Academia
Investors
Stakeholders’ needs
analysis
Standardized output format
Easy to understand
Layered information
RWE Product pipeline
Master Protocol
Approval
Critical (CIR) & Priority
(PIR) Information
Requirements

11. 11/4/2019https://www.aretezoe.com/
11
Planning Direction
Collection
Processing
Analysis
Production
Distribution
Evaluation
Per Protocol
Analysis
RWE Production
RCD/RWD
collection
infrastructure
Customer feedback
Redefined CIR/PIR
Report format adjustment
Regulatory landscape change
Information systems upgrade
Communicating
findings
Insurers
HCPs
MAHs
Research
sponsors
Regulators
Academia
Investors
Stakeholders’ needs analysis, determining research strategy
Standardized output format
Layered informationRWE Product pipeline
Master Protocol
Approval
Critical (CIR) & Priority (PIR)
Information Requirements

12. 11/4/2019https://www.aretezoe.com/
12
SYSTEM RESPONSIVENESS
The direct impact of an improved information environment
Leveson, N: Applying System Engineering to Pharmaceutical Safety

13. WHO BENEFITS
11/4/2019https://www.aretezoe.com/
13
Manufacturers&Sponsors
•ROI products
•Compliance
•Continue sales
•New indications
•New target
populations
CROs
•ROI service
•Conduct trials &
studies
•Produce data
for sponsors
Regulators
•Public health
•Safety, efficacy
& security
•Accurate
information
•Facilitate
innovation
Healthcareproviders
•Quality of care
•Access
•Cost
•Community
benefit
•Comparative
efficacy
Insurers,Payers,CMS
•Cost-
effectiveness
•Safety, Efficacy
•Pharmaco-
economics
•System
responsiveness
•Avoidable injury
Patients
Value & Co-pays
Investors
Minimize financial risk

14. SITUATION
11/4/2019https://www.aretezoe.com/
14
PV reporting as part of Post-Market Surveillance
Signal Detection Methods in PV
Post-Market Safety Events as Public Health Challenge
PMS Events: Adverse Drug Events vs. Adverse Drug Reactions
Safety Withdrawals
Level of Evidence
Outcome Measures; Study Endpoints; Novel Biomarkers
Communicating Findings
Data security

15. PHARMACOVIGILANCE REPORTING AS
PART OF POST-MARKET SURVEILLANCE
11/4/2019https://www.aretezoe.com/
15.
USA Europe Global
Reporting Form MedWatch CIOMS Mostly CIOMS
PV Database FAERS EudraVigilance
National DBs
WHO: VigiBase
National DBs
Reporting Industry mandatory
HCPs voluntary
Mandatory for HCPs
and industry
Mostly mandatory
Compliance varies
Access Public dashboard FOIA
requests
Rights differ by
stakeholders
Searchable online
Limitations:
• Under-reporting (1-10%), reporting bias, duplication, varying quality of reports
• The system does not provide information on incidence and prevalence, the
numerator is uncertain and denominator is projected from drug utilization data.

16. SIGNAL DETECTION METHODS IN PV
11/4/2019https://www.aretezoe.com/
16.
ICSRs: Clinical trials, PMS, PASS, Literature
Company databases, monitoring systems
Signaldetection
No action
Triage of Outputs & Interpretation
Impact Assessment
ICSR review
Aggregate reports, PSURs
NI studies, class analysis
Data mining algorithms
Disproportionality
reporting ratios
Signal evaluation
Signalprioritization
• ICSR review
• Aggregate analysis
• Periodic reports
• NI studies
• Class analysis

17. MAIN LIMITATIONS
Underreporting
Reporting bias
Varying quality of reports
Varying quality of industry
reporting systems
No incidence and prevalence
Reliance on disproportionality
analysis
11/4/2019https://www.aretezoe.com/
17.
SIGNAL DETECTION METHODS IN PV

18. POST-MARKET SAFETY EVENTS
AS A PUBLIC HEALTH CHALLENGE
11/4/2019https://www.aretezoe.com/
18.
Measure: Rate of visits to U.S.
hospital EDs for injury from
Hospital Quality Management Systems (ADEs)
Numerator Denominator
Oral anticoagulants Number of visits to U.S.
hospital EDs for injury
from the drug
Number of patients receiving the
concerned drug dispensed in U.S.
retail outpatient settings
Insulin
Opioid analgesics
• 82% of American adults take at least one medication
• 29% take five or more
• ADEs cause more than 1 million emergency department visits a year
• ADE case 280,000 hospitalizations a year
• $3.5 billion a year is spent on excess medical costs of ADEs
• 40% of ambulatory (non-hospital) ADEs are preventable
Highestnumberofpatientinjures
attributabletothreedrugclasses
CDC, 2017
HRET, 2017 and DHHS, 2014

19. POST-MARKET SAFETY EVENTS
ADVERSE DRUG EVENTS VS. ADVERSE DRUG REACTIONS
11/4/2019https://www.aretezoe.com/
19.
Adverse Drug Events (ADE): All harms that occur during medical care that are directly
caused by the drug including medication errors, ADRs, allergic reactions and overdoses
Medication error: “Inappropriate use of a drug that may or may not result in harm”
Adverse Drug Reaction (ADR): “Harm directly caused by a drug at normal doses”
Highestnumberofpatientinjures
attributabletothreedrugclasses
Disconnected PV reporting systems
(ADRs) and hospital QMS (ADEs)
ADR/ADEs are processed depending
on the relevant compliance track,
utilizing separate systems, burdening
physicians with avoidable
bureaucracy.
Adverse
Drug
Reactions
Medication Errors Adverse Drug Events

20. SAFETY WITHDRAWALS
11/4/2019https://www.aretezoe.com/
20.
Level of evidence
Number (%) of withdrawals
All marketed drugs
(n = 462)
Marketed drugs launched since
1950 (n = 286)
Level 1: Systematic reviews 6 (1.3%) 6 (2.1%)
Level 2: Randomized studies 27 (5.8%) 25 (8.7%)
Level 3: Non-randomized studies 43 (9.3%) 30 (10.5%)
Level 4: Case reports 330 (71.4%) 189 (66.1%)
Level 5: Mechanism-based reasoning 56 (12.1%) 36 (12.6%)
From 2001 to 2010, the FDA approved 183 novel pharmaceuticals and 39
biologics. One-third of the newly approved therapeutics was affected by
at least one post-market safety event: 3 withdrawals, 61 boxed warnings,
and 59 safety communications. The most affected groups were
psychiatric medications and biologics.
• Quality of evidence the current PV reporting systems can provide
• Time lapse between identification of a safety concern and action
• Toxicity caused by abnormal metabolism is not detectable
Key limitations

21. LEVEL OF EVIDENCE
POST-AUTHORIZATION STUDIES
11/4/2019https://www.aretezoe.com/
21.
# Studies Have Results No Results
Observational Phase 4 studies 874 80 794
Active, not recruiting 12 0 12
Completed 626 69 557
Enrolling by invitation 3 0 3
Not yet recruiting 4 0 4
Recruiting 24 0 24
Terminated 61 11 50
Suspended 3 0 3
Unknown status 107 0 107
Withdrawn 34 0 34
Interventional Phase 4 studies 23,327 3,948 19,379
Active, not recruiting 765 18 747
Completed 13,273 3,420 9,853
Enrolling by invitation 173 0 173
Not yet recruiting 929 0 929
Recruiting 3,045 0 3,045
Terminated 1,527 506 1,021
Suspended 70 1 69
Unknown status 2,950 4 2,946
Withdrawn 600 0 600

22. LEVEL OF EVIDENCE
11/4/2019https://www.aretezoe.com/
22.
1A Systematic Review of RCTs
1B Individual RCTs
1C All or none studies
2A Systematic Review of cohort studies
2B Individual cohort study, low-quality RCT
2C Outcomes research, ecological studies
3A individual case-control study
3B Individual case-control study
4 Case series, poor quality case-control studies
5 Expert opinion without explicit critical appraisal
PV reporting systems
Observational
research

23. LEVEL OF EVIDENCE
11/4/2019https://www.aretezoe.com/
23.
PV systems produce LEVEL 4 evidence
Post-authorization phase of the product lifecycle
currently lacks adequate insight into the drug’s
true clinical utility and cost-effectiveness.
Information environment that lacks transparency
is characteristic by significant ambiguity that
creates misplaced incentives for clinicians,
healthcare providers, insurers as well as investors
in the drug development and commercialization
enterprise.

24. OUTCOME MEASURES
STUDY ENDPOINTS
The reliance on surrogate endpoints increases
the need for accurate and timely post-market
surveillance.
A validated surrogate endpoint: supported
by a clear mechanistic rationale; strong clinical
evidence that the surrogate predicts a clinical
benefit
A reasonably likely surrogate endpoint:
supported by clear mechanistic and/or
epidemiologic rationale; clinical data
insufficient
A candidate surrogate endpoint: under
evaluation for its ability to predict clinical
benefit
11/4/2019https://www.aretezoe.com/
24.
The diversity of outcome
measures, biomarkers and
study endpoints makes
assessment of the total body
of evidence complex and
challenging.
.

25. NOVEL BIOMARKERS
Pharmacogenomic biomarkers are one of the important areas of interest for many
healthcare stakeholders. Toxicity relating to variant metabolism is currently not
detectable within the standard PV reporting systems.
11/4/2019https://www.aretezoe.com/
25.
Withdrawn

26. COMMUNICATING
FINDINGS
The disclosure of clinical trial findings and publication bias that
favors positive findings is a critical unmet need in the clinical
research and HTA information ecosystem.
Selective disclosure of results of clinical trials and sketchy PMS
provide an incomplete and misleading picture of the studied
products’ benefit:risk profile and clinical utility.
CONSORT is the most commonly used standard for the
presentation of results of randomized controlled trials.
Standards were also developed for the reporting of different
types of studies including non-interventional.
11/4/2019https://www.aretezoe.com/
26.

27. • Respect stakeholders’ information needs
• Concise and timely
• Format that is easily understood
• Presented in layered manner
• Trustworthiness of presented evidence
• Methodologically sound
AHRQ, 2017
11/4/2019https://www.aretezoe.com/
27
COMMUNICATING
FINDINGS
.

28. DATA BREACHES
IN HEALTHCARE
In August 2017, the FBI charged
Chinese national Yu Pingan with the
creation of malware that has
impacted multiple U.S. organizations
including the OPM and Anthem.
11/4/2019https://www.aretezoe.com/
28
The ANTHEM Breach (March 2015): 78,8 million patients
affected in a Hacking/IT incident that involved the
insurer’s Network server. The OPM breach exposed
information on 25.7 million Americans.
.

29. 11/4/2019https://www.aretezoe.com/
29
Incidents by state
Period from 2009 to 2016
Affected individuals by
state
Period from 2009 to 2016

30. 11/4/2019https://www.aretezoe.com/
30
Individuals affected by covered
entity type
Health plan
Healthcare provider
Business associate
(blank)
Healthcare clearing house
Business associate
Health plan
Healthcare clearing house
Healthcare provider
(blank)
Incidents by covered entity
type
Period from 2009 to 2016 Period from 2009 to 2016

31. 11/4/2019https://www.aretezoe.com/
31
INDIVIDUALS AFFECTED
BY BREACH TYPE AND LOCATION
Network
Server
Other
Desktop
Laptop
Period from 2009 to 2016
TheftHacking
IT Incident Loss

32. DATA BREACHES
IN HEALTHCARE
The black market value of EHRs
reach $1000 (25 cents for credit
card data).
Two thirds of healthcare data
breaches go undiscovered for
months or even years.
11/4/2019https://www.aretezoe.com/
32
Networked electronic systems make
confidential data more easily and
rapidly accessible to a wider circle of
recipients than compartmented
systems, with greater potential for
breaches of confidentiality.
Patients’ trust in data management
systems and confidence that the data
will not be shared inappropriately is
essential to their willingness to
participate in research.
Centralization of healthcare information
systems increases substantially the
value of such records to any threat
actors.
.

33. DATA BREACHES
IN HEALTHCARE
11/4/2019https://www.aretezoe.com/
33
Challenges
• Attractiveness of health data for
bad actors
• State-sponsored APTs
• Loss of trust in the ability of
providers to keep the data safe
• Reliance on designs that require
pooling of sensitive data and
systems that require/allow PHI
sharing for research and analysis.
Security by design
.

34. 11/4/2019https://www.aretezoe.com/
34
ARETE-ZOE, as a consultancy, provides solutions to complex
problems in the high stakes and high consequence environment
of Global Pharmaceuticals, including clinical research, healthcare
informatics, and public health. We blend established, Pharma
sector methodologies, innovation, and adaptations/transfers
from other sectors to identify and resolve consequential
practices that pose risk and often result in avoidable patient
casualty. However, we are specifically, not a patient advocacy
group but believe in optimizing organizational effectiveness
and that smart business is agile, competitive and profitable,
while intrinsically safe, secure, and resilient.
We work within a global context because transnational interests
influence national circumstances and choices at point of
prescription.
ARETE-ZOE, provides full spectrum organizational and
operational risk management consultancy. Our published
materials provide a glimpse of some aspects of our services to
demonstrate both knowledge and ongoing participation within
the Pharmaceutical Industry. Our analysis and consultancy
includes all channels of misuse, diversion, counterfeiting and
illicit exploitation of pharmaceuticals, medical devices, and
precursor chemicals. Our advisement is to manufactures,
jurisdictional entities, insurers, legislators, litigators, patients,
and health care providers.
This scope also frequently segues into the nexus of crime and
terrorism as significant influencers that undermine sector
integrity differentiated from other criminal activity. Obviously,
vulnerability assessment, information collection management
and intelligence production supporting decision-making for risk
reduction and interventions are routinely within the scope of
our services as well as design and implementation of
operational control measures.
.
ARETE-ZOE
Solution Development and
Implementation