This document summarizes evidence-informed health care for rare diseases. It discusses: 1) Progress in rare disease research including increasing genetic tests available and approved orphan drugs. Clinical interventions and systems of care are important. 2) Why studying systems of care is important to reduce the "effectiveness gap" between clinical trials and real-world outcomes. Translational research from efficacy to effectiveness and population impact is needed. 3) Challenges in studying rare disease care including limited knowledge, defining meaningful outcomes, and study feasibility with small patient numbers. International collaboration is key.

1. Evidence-informed Health Care for
Rare Diseases
Beth Potter, Associate Professor
Julian Little, Professor & Director
School of Epidemiology, Public Health & Preventive Medicine
University of Ottawa
CORD Rare Disease Day Conference
March 9, 2016

2. From: International Rare Disease Research Consortium (IRDiRC), www.irdirc.org.
Rare disease research progress
# Rare diseases for which a genetic test is
available (data from Orphanet)
Cumulative # of medicinal products with
orphan designation & marketing approval
in Europe and US (data from EMA/FDA)

3. • Clinical interventions:
o Screening tests
o Orphan drugs and supplements
o Surgery (e.g. organ replacement)
o Physical therapy
o Diet and lifestyle modifications
• System of care:
o Identification/diagnosis (e.g. newborn screening,
diagnostic care)
o Organization & coordination of care – multidisciplinary
team care, regionalized treatment centres
o Programs to provide access to therapies
Interven2ons for rare diseases

4. Why study the system of care?
• The effectiveness
gap: outcomes in
highly controlled
trials (efficacy)
often different than
outcomes
experienced by
patients in real
world settings
(effectiveness)
• To achieve
effectiveness, new
treatments must be
implemented in high
quality health care
systems
From: Khoury et al, Am J Public Health 2012;102:34-37.

5. How do we study the system of care?
From: Khoury et al, Am J Public Health 2012;102:34-37.
“T2” translational
research: studies
to evaluate
efficacy and
effectiveness of
interventions
(phase III & IV
clinical trials;
observational
studies;
systematic
reviews)

6. From: Khoury et al, Am J Public Health 2012;102:34-37.
“T3” translational
research: studies
to inform the
successful
integration of
interventions into
existing systems
of care
How do we study the system of care?

7. From: Khoury et al, Am J Public Health 2012;102:34-37.
“T4” translational
research: studies
that evaluate
longer term
outcomes of
interventions and
their impact on
population health
How do we study the system of care?

8. Challenges in studying rare disease care
Hierarchy of evidence
1. RCT
2. Controlled observa2onal studies
3. Uncontrolled studies
4. Expert opinion
“It is unfortunate that scien3sts and clinicians dealing with the
very rare diseases o7en seem to be locked into the bo;om run
of this hierarchy” (Wilcken 2001)

9. • Limited knowledge of epidemiology and natural
history: are treatments leading to changes in
disease progression?
• Defining “effectiveness”: what are the right
outcomes? what is a meaningful change?
• Designing evaluative studies that are scientifically
sound and feasible in a rare disease setting (i.e.,
with few patients)
• Working together across geographic
boundaries and stakeholder communities
(patients/families, providers, policy-makers,
researchers) is key to addressing these
Challenges in studying rare disease care

10. Rare disease research networks

11. THE CHANGING FACE OF EPIDEMIOLOGY
Editors’ note: This series addresses topics that affect epidemiologists across a range of
specialties. Commentaries are first invited as talks at symposia organized by the Editors.
This paper was originally presented at the 2006 Congress of Epidemiology in Seattle.
The Emergence of Networks in Human Genome
Epidemiology
Challenges and Opportunities
Daniela Seminara,* Muin J. Khoury,† Thomas R. O’Brien,‡ Teri Manolio,§ Marta L. Gwinn,†
Julian Little,ʈ Julian P. T. Higgins,#¶ Jonine L. Bernstein,** Paolo Boffetta,†† Melissa Bondy,‡‡
Molly S. Bray,§§ Paul E. Brenchley,ʈʈ Patricia A. Buffler,¶¶ Juan Pablo Casas,##
Anand P. Chokkalingam,*** John Danesh,††† George Davey Smith,‡‡‡ Siobhan Dolan,§§§
Ross Duncan,ʈʈʈ Nelleke A. Gruis,¶¶¶ Mia Hashibe,†† David Hunter,¶¶¶
Marjo-Riitta Jarvelin,###**** Beatrice Malmer,†††† Demetrius M. Maraganore,‡‡‡‡
Julia A. Newton-Bishop,§§§§ Elio Riboli,†† Georgia Salanti,¶ Emanuela Taioli,ʈʈʈʈ Nic Timpson,†††
Andre´ G. Uitterlinden,¶¶¶¶ Paolo Vineis,####### Nick Wareham,***** Deborah M. Winn,*
Ron Zimmern,# and John P. A. Ioannidis†††††‡‡‡‡‡ for the Human Genome Epidemiology Network
and the Network of Investigator Networks
Large-scale “big science” is advocated as an approach to complex research problems in
many scientific areas.1
Epidemiologists have long recognized the value of large
collaborative studies to address important questions that are beyond the scope of a study
conducted at a single institution.2
We define networks (or, interchangeably, consortia) as
groups of scientists from multiple institutions who cooperate in research efforts involving,
but not limited to, the conduct, analysis, and synthesis of information from multiple

12. Challenges Faced by Inves2gator Networks
• Resources for establishing the ini2al infrastructure,
suppor2ng consor2a implementa2on, and adding new
partners
• Coordina2on: minimize administra2on to maximize scien2fic
progress and avoid conflicts
• Selec2on of target projects
• Variable data and biospecimen quality from par2cipa2ng
teams
• Handling of informa2on from nonpar2cipa2ng teams and of
nega2ve results
• Collec2on, management, and analysis of complex and
heterogeneous data sets
Seminara et al., 2007

13. Challenges Faced by Inves2gator Networks
• Anticipating future needs
• Communication and coordination
• Scientific credits and career development
• Access to the scientific community at large and
transparency
• Peer review
• Informed consent
Seminara et al., 2007

14. Networks
• BoXom up
• Top down

15. • The Canadian Inherited Metabolic Diseases Research
Network (CIMDRN): ~50 investigators
(multidisciplinary), 14 pediatric metabolic centres
• Funding: CIHR Emerging Team Grant (2012-17)
• Goal: generate evidence to improve care & outcomes
for pediatric inherited metabolic diseases (IMD)
• Interest in outcomes across the “triple aim”: clinical
outcomes, patient and family experiences, health
system impacts
• “Practice-based research”: collect observational data
about existing treatments & outcomes to make
inferences about patterns of care that work best
CIMDRN

16. • Consent-based enrollment of children born
2006-2015 and receiving care for one of ~30 IMD at
a participating centre (>420 families enrolled so far)
• Research (several completed/ongoing studies):
o Clinical interventions and outcomes (data abstracted
from charts with families’ permission)
o Patient and family-reported experiences (qualitative
interviews, survey)
o Health system impacts (health services use data)
• Successes: engaged investigators, enthusiastic
participation of families, high quality data
• Challenges: administration of multi-centre research,
sustainability
CIMDRN

17. Rare Diseases Clinical Research Network
(RDCRN -US Na&onal Center for Advancing Transla&onal Sciences )
• The RDCRN facilitates clinical research in rare diseases
through support for:
• Collabora2ve ac2vi2es, including mul2site longitudinal
studies of individuals with rare diseases, and/or clinical trials
• Training of clinical inves2gators in rare diseases research
• Pilot and demonstra2on projects
• Uniform data collec2on protocols
• Access to informa2on about rare diseases for basic and
clinical researchers, academic and prac2cing physicians,
pa2ents, and the public

18. RCDRN
• 22 clinical research consor2a
• Each consor2um
o focuses on at least three related rare diseases
o par2cipates in mul2site studies
o incorporates pa2ent advocacy groups as research partners
• Data Management and Coordina2ng Center intended to enable
uniform collec2on and analysis, and facilitate informa2on sharing
across the network
• Overall about 2,600 researchers at hundreds of clinical sites
interna2onally involved
• Since 2003, nearly 29,000 par2cipants enrolled in network clinical
studies
• Funding and scien2fic oversight provided by NCATS, 10 other NIH
en22es, and by pa2ent advocacy organiza2ons

19. Pa2ent-Centered Outcomes
Research Ins2tute
• PCORI’s authorizing law calls for a focus on rare diseases
(condi2ons that affect fewer than 200,000 people in US)
• special pool of funding
• Funded projects to compare treatments for urea cycle disorders,
non-cys2c fibrosis bronchiectasis, & syringomyelia
• PCORI requires that all studies compare two difference approaches
for preven2on, diagnosis, treatment or care delivery
• interested in funding comparisons of real-world clinical op2ons,
proposed studies will not qualify if there are no efficacy data or the
proposed interven2ons are not considered a clinical op2on for
individuals with a given rare disease

20. European networks of
reference for rare diseases
• Main added value of European Reference Networks (ERN) and
associated Centres of Reference is to facilitate improvements in
access to diagnosis and delivery of high-quality, accessible and cost-
effec2ve healthcare for pa2ents who have a medical condi2on
requiring a par2cular concentra2on of exper2se or resources,
par2cularly in medical domains where exper2se is rare
• ERN focal points for medical training and research, informa2on
dissemina2on and evalua2on
• Centres intended to serve as research and knowledge foci, upda2ng
and contribu2ng to the latest scien2fic findings, trea2ng pa2ents
from other Member States and ensuring the availability of
subsequent treatment facili2es where necessary

21. European networks of
reference for rare diseases
• Among Member States that have established such centres, no
common defini2on!
• process for iden2fying, selec2ng and designa2ng centres of
reference varies markedly between UK, France, Italy & Denmark
• Two main purposes:
1. provide a ra2ng scheme that enables consumers to iden2fy the appropriate health-care
resource
2. to enable health-care managers iden2fy where best to allocate financial resources
• European Union CommiXee of Experts on Rare Diseases (EUCERD)
has made 45 Recommenda2ons on Quality Criteria for Centres of
Exper2se
o Mission and Scope; Designa2on Criteria; Process of Designa2on and Evalua2on; European
Dimension
• ~12 projects funded

22. Pa2ent/family engagement
Figure 2, Shippee et al, Health Expectations 2013;18:1151-1166
• Engaging with
patients/families
recognized as key to
research that aims to
inform care
• Patient-oriented
research occurs on a
continuum: patients/
families as
participants,
consultants,
collaborators,
partners, or leaders
of research

23. Pa2ent involvement in rare disease research
• Providing patient-centred care
requires that outcomes of
importance to patients are
prioritized in research
• Patients with rare diseases and
their families are frequently
experts about health care
needs
• Rare disease organizations are
leading the way (e.g., PCORnet
patient-powered research
networks)

24. • Our group has undertaken two qualitative studies to
understand the experiences and needs of families of
children with rare inherited metabolic diseases (IMD)
related to the system of care:
1. Study recommended by patients/family members
on our network advisory board: perspectives of
patient advocacy organizations on families’
experiences and needs
2. Study of parents/caregivers of children with IMD
Examples of pa2ent perspec2ves on care
Funding from CIHR & the Rare Disease
Foundation
• Khangura et al., JIMD 2016;139-47
• Siddiq, MSc thesis

25. • Families are often experts about care and act as
educators/advocates; they report mixed
experiences with providers:
o “I think parents can be fantastic resources, and I think
doctors… are very receptive to hearing from
patients…”(study 1, participant 12)
o “…not all physicians are willing to accept that usually
the mom and dad know more about this particular
[disease]…” (study 1, participant 16)
o “When we go to the ER in crisis…we know what to say,
we know what's going to work… we've come up against
a couple of doctors in the ER that don't appreciate that,
they want to be the ones to when they walk into the
room, everyone looks to them for the answers…” (study
2, participant 21)
Selected findings

26. • In this patient population, parents/caregivers are
generally happy with the care their child receives
from the specialist metabolic clinic team:
o “...our experience is amazing. We have a fantastic
doctor and a fantastic dietitian. The backup dietitian
and the backup doctor are equally as
knowledgeable.” (study 2, participant 15)
o "Oh, when she was diagnosed it was great because
[metabolic physician] and [dietitian] would come in
together and we got a chance to see both. We have an
option to see them separately. So yah, there is
definitely a good coordination between them." (study
2, participant 1)
Selected findings

27. • But families frequently experience challenges
with care and its coordination outside the
specialist clinic:
o “…I think everybody who has a child with a complex
medical condition like this would benefit from having
more coordination. You know, being able to go into a
clinic and see several specialists in one day… And that
really doesn’t happen right now….the burden of getting
to all these appointments is huge.” (study 1, participant
3)
o “… I just have a lot of stress around mostly around the
health care that we received there [emergency
department] because I can't say that it's consistent at
all…I feel like almost like I have to prove that they are
sick…" (study 2, participant 2)
Selected findings

28. • Challenges with care often manifest as single
negative encounters, which accumulate and are
an important source of stress:
o “…like in a matter of three days that [hospital] room
had become my home, you know? And that was where
I’d stay, and I never left my son’s bedside. And then
when they’d tell me we were moving, like, my heart
would just sink and I’d almost be in tears over it
because I couldn’t handle any more change… and that
was very, very difficult as a family… (study 1,
participant 6)
o "…we were told not to use medication from home ...but
the pharmacy, when she was in Emerg that afternoon,
they were going to send up her meds from pharmacy,
which they didn't…" (study 2, participant 05)
Selected findings

29. • There is a need for patient- and family-oriented
health services research to complement the exciting
advances in the diagnosis and development of new
therapies for rare diseases
• This research will ensure that these treatments are
implemented in high quality systems that can
maximize their effectiveness
• This research should incorporate lessons that the rare
disease community has learned about working
together across boundaries, both geographic and
disciplinary
Conclusions

31. RCDRN Consor2a
1. Amyotrophic Lateral Sclerosis and Related Disorders
2. Autonomic Rare Diseases
3. Brain Vascular Malforma2on
4. BriXle Bone Disorders
5. Chronic Gran Versus Host Disease
6. Developmental Synaptopathies Associated with TSC, PTEN and SHANK3 Muta2ons
7. Dystonia
8. Eosinophilic Gastrointes2nal Disease
9. Frontotemporal Lobar Degenera2on
10. Gene2c Disorders of Mucociliary Clearance
11. Inherited Neuropathies
12. Lysosomal Disease
13. Mitochondrial Disease
14. Nephrolithiasis and Kidney Failure: The Rare Kidney Stone Consor2um
15. Nephro2c Syndrome
16. Porphyrias
17. Primary Immune Deficiency
18. Rare Lung Diseases (Molecular Pathway-Driven Diagnos2cs and Therapeu2cs)
19. ReX Syndrome, MECP2 Duplica2ons and ReX-Related Disorders
20. Sterol and Isoprenoid
21. Urea Cycle Disorders
22. Vasculi2s

32. ERN for rare diseases –
funded projects
1. Cys2c fibrosis
2. Dysmorphology
3. Alpha-1 an2trypsin deficiency
4. Porphyria
5. Rare Bleeding Disorders
6. paediatric Hodgkin’s lymphoma
7. Rare Paediatric Neurological Diseases (NEUROPED)
8. Langerhans cell his2ocytosis and associated syndrome
9. Severe Genodermatoses
10. Rare Anaemias
11. Refractory epilepsy and epilepsy surgery (E-PILEPSY)
12. Expert Paediatric Oncology Reference Network for Diagnos2cs and
Treatment (ExPO-r-NeT)

33. & REPORTING
ed
field,
nces,
n, York
ersity of
on
009
:c1066
Pragmatic trials are important for
informing routine clinical practice, but
current designs have shortcomings.
Clare Relton and colleagues outline
the new “cohort multiple randomised
controlled trial” design, which could help
address the problems associated with
existing approaches
Rethinking pragmatic randomised controlled trials:
introducing the “cohort multiple randomised
controlled trial” design
Clare Relton,1 3
David Torgerson,2
Alicia O’Cathain,1
Jon Nicholl1
Randomised controlled trials are generally held to be the
“gold standard” for establishing how well an intervention
works. Trials that aim to determine the efficacy of a treat-
ment by using a double blind, placebo controlled design
(that is, explanatory trials) are, however, sometimes criti-
cised. For example, although the design of explanatory tri-
als results in strong internal validity—we can depend upon
the results of a given trial—such trials may have limited
external validity: we can’t be confident that we can apply
the results to routine clinical practice. Pragmatic trials,1 2
which aim to inform healthcare decision making in prac-
tice, have been offered as a solution in that they retain the
rigour of randomisation (thus eliminate selection bias) but
retain the characteristics of normal clinical practice.
The implementation and interpretation of both prag-
matic and explanatory randomised controlled trials
are associated with significant problems. This article
describes a trial design that helps address these prob-
lems—the “cohort multiple randomised controlled trial”
approach.
Problems with randomised controlled trials
Existing clinical trial designs can have shortcomings in
four areas: recruitment; ethics; patient preferences; and
treatment comparisons.
Recruitment
The majority of randomised controlled trials have difficulty
recruiting sufficient numbers of patients. For example, one
investigation found that less than a third of 114 multicen-
tre, publicly funded UK trials recruited their original target
number of patients within the time originally specified.3
Failure to recruit to target may have implications for the
power and generalisability of trial results.
Moreover, many clinical trials exclude hard to reach
groups and ethnic minorities,4
resulting in disparities
between the “with need” (reference) population and the
trial population.5 6
Measures of real world effectiveness are
vital for analyses of benefit, harm, and cost effectiveness.
If the reference population is not adequately represented
in a trial and effectiveness is variable, then such analyses
cannot accurately inform real world decisions.
Ethics
The most common reason given by patients (and clini-
cians) for not participating in clinical trials is “concerns
with information and consent.”7
In routine real world
health care, patients are rarely told of treatments that
their clinicians cannot with certainty provide,8
nor are
patients told their treatment will be decided by chance.
On the other hand, in clinical trials providing this type of
“full” information before randomisation is regarded as an
ethical requirement.
Patient preferences
Standard “open” (unblinded) pragmatic trials often com-
pare an intervention with treatment as usual. Where the
S
andomised controlled trial” (cmRCT) design tackles some of the
Large observational cohort (N)
Regular outcome measurement
Eligible patients identified (NA)
Random selection of some
eligible patients (nA) and
outcomes compared with those
receiving usual care (NA–nA)
Eligible patients identified (NB)
Random selection of some
eligible patients (nB) and
outcomes compared with those
receiving usual care (NB–nB)

34. Using the cohort multiple
randomised controlled trial design
METHODS & REPORTING
Although this method shares several features with the
cmRCT design (features I, II, IV, and VII), it does not have
the capacity for multiple randomised controlled trials
(feature III) or use random selection of some instead of
random allocation of all (features V and VI).
We have obtained ethical approval for and have con-
ducted a pilot study of the cmRCT design.21
In this pilot,
a large observational cohort of 856 women aged 45-64
was recruited and their outcomes measured. A total of
72 women reported frequent or severe menopausal hot
flushes, or both. Of these 72 women, 48 were eligible for
the trial treatment (NA) and 24 were randomly selected to
be offered the treatment (nA). The outcomes of the ran-
domly selected patients were then compared with the
outcomes of those eligible patients not randomly selected
(NA − nA) using both intention to treat analysis and CACE
analysis.20
Patients were not told about the treatments that
they were not randomly selected to be offered.
The clinical outcomes of this pilot will be reported sepa-
rately. However, a post hoc evaluation of the design found
that the design was acceptable to patients, clinicians, and
the NHS Research ethics committee. The concept of multi-
ple trials within a single cohort of patients (feature III) has
not yet been tested.
The cmRCT design is currently being used to address
questions in the management of obesity (http://clahrc-sy.
nihr.ac.uk/theme-obesity.html). The 20 year study is pro-
jected to recruit a cohort of 20000 adults aged 16 years
Using the cohort multiple randomised controlled trial design
Most suited to:
Settings
Opentrialswith“treatmentasusual”asthecomparator
Studiesthataimtoinformhealthcaredecisionsinroutine
practice(pragmatictrials)
Researchquestionsthataddresseasilymeasuredand
collectedoutcomes
Populations
Stablepopulations
Easilyidentifiedpopulations
Clinicalconditions
Clinicalconditionsforwhichmanytrialswillbeconducted;
forexample,obesity,diabetes,chronicpain
Chronicconditions
Conditionsforwhichprevioustrialshavestruggledwith
recruitment
Treatments
Treatmentshighlydesiredbypatients
Expensivetreatments
Leastsuitedto:
Settings
Closedtrialdesignswithmaskingoraplaceboarm
Studiesthataimtofurtherknowledgeastohowandwhya
treatmentworks(efficacytrials)
Researchquestionsthataddresshardtomeasureandhard
tocollectoutcomes
Population
Populationswithhighattrition
for patients who comply with the protocol (albeit usually
with loss of power), unlike per protocol or on treatment
analysis.
Secondly, we could try to avoid some potential non-com-
pliance by presenting cohort patients with a list of possible
interventions at enrolment and asking which they would
consider agreeing to use if offered. This process identi-
Chronicconditions
Conditionsforwhichprevioustrialshavestruggledwith
recruitment
Treatments
Treatmentshighlydesiredbypatients
Expensivetreatments
Leastsuitedto:
Settings
Closedtrialdesignswithmaskingoraplaceboarm
Studiesthataimtofurtherknowledgeastohowandwhya
treatmentworks(efficacytrials)
Researchquestionsthataddresshardtomeasureandhard
tocollectoutcomes
Population
Populationswithhighattrition
Unstablepatientpopulations
Difficulttoidentifypopulations
Clinicalconditions
Acuteorshorttermconditions
Treatments
Treatmentsnothighlydesiredbypatients
Relton C et al. BMJ 2010;340:c1066