Real world evidence (RWE) has potential to advance payers’ and HTA bodies’ evaluations of healthcare technologies by providing more relevant evidence. However, there are concerns about the quality and value of RWE. This presentation provides specific and actionable recommendations that highlight the role that each stakeholder group can play in overcoming the challenges and realising the potential for RWE. Author(s) and affiliation(s): Grace Hampson, Office of Health Economics Adrian Towse, Office of Health Economics Bill Dreitlein, Institute for Clinical and Economic Review Chris Henshall Steven Pearson, Institute for Clinical and Economic Review. Conference/meeting: Health Technology Assessment International (HTAi) 2018 Location: Vancouver 2018 Date: 03/06/2018

1. Optimizing The Use of RWE in
HTA: the ICER Membership
Summit 2017
Grace Hampson, Chris Henshall, Adrian Towse,
Bill Dreitlein, Steven Pearson.
HTAi
3rd June 2018

2. Real World Evidence for Coverage Decisions
Contents
• What is RWE?
• How is it used in the US?
• Challenges associated with use of RWE
• Opportunities to enhance generation and use of
RWE
• Development of a framework to support active
collaboration between stakeholders

3. Real World Evidence for Coverage Decisions
What is RWE?
• “Real-world data (RWD) are data relating to patient health
status and/or the delivery of health care routinely collected from
a variety of sources. Examples of RWD include data derived
from EHRs, claims and billing data, data from product and
disease registries, patient-generated data including in home-use
settings, and data gathered from other sources that can inform
on health status, such as mobile devices.”
• “Real world evidence (RWE) is the clinical evidence
regarding the usage, and potential benefits or risks, of a
medical product derived from analysis of RWD” (FDA, 2017).
• It is the setting in which evidence is collected (i.e. in health care
settings not a research environment), not the presence of a planned
intervention or randomization that is important (Sherman et al. 2016)
• We include pragmatic trials (PCTs) in this definition of RWE

4. Real World Evidence for Coverage Decisions
How is RWE being used? US (1)
Six areas RWE is being used in the US currently:
1. Drug development: to design the drug development pathway
2. Regulatory approval decisions: 21st Century Cures Act (2016) directs
the FDA to consider the use of RWE for approval of new indications. Use to
date has been limited.
3. Post-approval monitoring of safety signals:
• Sentinel Initiative: collects evidence on events using EHRs and claims data. Is it
having an impact? Two label changes to date at cost of $20m/year (Findlay, 2017).
• Adverse Event Reporting System (FAERS): collects information on adverse event
and medication errors from healthcare professionals and product manufacturers. 0.68
label changers per drug per year (1997-2016) (Mostaghim et al., 2017).
Figure 2: Experience Utilizing RWD and RWE for Regulatory Purposes
Source: Berger et al., 2017(a)

5. Real World Evidence for Coverage Decisions
How is RWE being used? US (2)
Six areas RWE is being used in the US currently (cont.):
4. HTA assessments and payer coverage decisions -
initial decisions: Epi and claims data used by payers to
determine eligible population size and estimate cost offsets
5. HTA assessments and payer coverage decisions –
reassessments: Provides opportunity to reconsider
coverage, formulary placement, and price/payment terms:
• Can be used to: identify/rule out safety signals; measure
adherence; establish effectiveness and value for money
within the health plan’s specific population; establish
effectiveness within sub-populations.
6. Outcomes-based contracting: payment linked to the
demonstrated real-world outcomes of patients. Few
examples in the US to date.

6. Real World Evidence for Coverage Decisions
Challenges in the use of RWE
1. Bias/
confounding
• Concerns around internal validity, selection bias and
reporting bias.
• Techniques exist to adjust for/mitigate these
concerns.
2. Incomplete
data
• Systematic omissions and misclassifications
• Gaps (for example test results).
3. Data
mining
• Re-examining existing datasets to generate new
information
• Concern that organisations can reanalyse datasets
until one that delivers preferential outcomes is
identified.
• The challenges are important, and have been well
documented in the literature
• We recap briefly here (for more information and references
see the background paper)

7. Real World Evidence for Coverage Decisions
Challenges (cont.)
4. Access
to data
• Sharing of data across organizations is not common.
• Inhibited by legal frameworks playing ‘catch up’.
5. Lack of
accepted
standards
• Various sets of principles exist for design, conduct,
analysis, reporting of RWE.
• None have been universally accepted/adopted.
6. Lack of
expertise
• Analysts must understand RWD well in order to be able
to interpret it properly and adjust for biases
appropriately
7. Obsolete
evidence
hierarchies
• Traditional evidence hierarchies that promote RCTs
were developed for a world without RWD.
• In reality the most appropriate study design depends
on the research question.

8. Real World Evidence for Coverage Decisions
Opportunities for the use of RWE
IMPROVING CURRENT USES:
• Evaluation of benefits and harms in a real world setting
• Evaluate outcomes not measured during standard
development process
• Evaluation of prevalence of the condition for budget impact
and cost-effectiveness analyses
• Development and use of innovative study designs (PCTs)
OPPORTUNITIES FOR THE FUTURE:
• Real time evidence based medicine
• Real time monitoring
• Accelerated access to innovative therapies

9. Real World Evidence for Coverage Decisions
Development of a framework
• Payers and manufacturers need a clear understanding of
the steps that can be taken to increase the validity of
observational evidence and ensure optimal application
• Meeting the most rigorous standards proposed for
observational evidence takes substantial time and
resources, creating a barrier
• There is therefore a need at the outset to select the
evidence standards that will need to be adopted for each
step in the process, in the light of the type of evidentiary
assertion that the evidence is intended to support and
the surrounding context
• Only then can the right balance be struck between rigor
and feasibility to produce RWE that will be persuasive in
coverage and formulary policy.

10. Real World Evidence for Coverage Decisions
Development of a framework

11. Real World Evidence for Coverage Decisions
Additional resources (ohe.org)

12. Real World Evidence for Coverage Decisions
Berger et al. 2017. A framework for regulatory use of real world evidence. White Paper.
The Duke-Margolis Centre for Health Policy.
FDA, 2017. Use of Real-World Evidence to Support Regulatory Decisions-Making for
Medical Devices: Guidance for Industry and Food and Drug Administration Staff. Real World
Evidence for Coverage Decisions. Document issued on August 31, 2017.
Findlay, S., 2017. Why an FDA Drug Safety Monitoring System Is Failing. Available at:
https://medshadow.org/your-meds/drug-safety-monitoring/ [Accessed November 2017]
Sherman, et al., 2016. Real-World Evidence - What Is It and What Can It Tell Us? The New
England Journal of Medicine, 2016, 375:2293-2297.
Mostagim et al., 2017. Safety related label changes for new drugs after approval in the US
through expedited regulatory pathways: retrospective cohort study. BMJ 2017, 358 :j3837.
References

13. Real World Evidence for Coverage Decisions
To enquire about additional information and analyses, please contact
Grace Hampson.
To keep up with the latest news and research, subscribe to our blog, OHE News
Follow us on Twitter @OHENews, LinkedIn and SlideShare
Office of Health Economics (OHE)
Southside, 7th Floor
105 Victoria Street
London SW1E 6QT
United Kingdom
+44 20 7747 8850
www.ohe.org
OHE’s publications may be downloaded free of charge from our website.
Thank you for listening