Methods description and application to estimate the effect of ivacaftor on health outcomes in cystic fibrosis. Authors: Ruth Keogh, Simon Newsome, Rhian Daniel, Diana Bilton, Siobhan Carr.

1. Using negative controls to estimate
causal effects of treatment in an
entirely treated cohort
Ruth Keogh
Department of Medical Statistics
London School of Hygiene & Tropical Medicine

2. Simon Newsome
London School of Hygiene &Tropical Medicine,UK
Novartis PharmaAG, Switzerland
Rhian Daniel
Cardiff University, UK
Diana Bilton, Siobhan Carr
Imperial College , UK
Royal Brompton and Harefield NHS FoundationTrust, UK

3. Motivation
The gold-standard study design is a randomized controlled trial
Cystic Fibrosis
• An inherited, chronic, progressive condition
• Affects >10,000 people in the UK
• New ‘precision medicines’ have been developed
which target the underlying defect
• These are called CFTR modulators – they work for
people with specific CF-causing genetic mutations
Ivacaftor (Kalydeco)
• Licenced in UK 2012
• Around 5% of the UK CF population are eligible

4. Studying the impact of ivacaftor
People who may benefit
from ivacaftor
Randomization
Ivacaftor
No Ivacaftor
Outcomes at 4-48 weeks
Primary outcome:
• Absolute change in
lung function (FEV1%)
Secondary outcomes:
• Use of IV antibiotics
• Pulmonary
exacerbations
• Quality of life
• …

5. Studying the impact of ivacaftor
People who may benefit
from ivacaftor
Randomization
• Randomized trials have short-term follow-up
• Are restricted to a subset of the eligible CF population
• It is of interest to use observational data to study longer
term impacts in the complete eligible CF population
Ivacaftor
No Ivacaftor
Outcomes at 4-48 weeks
Primary outcome:
• Absolute change in
lung function (FEV1%)
Secondary outcomes:
• Use of IV antibiotics
• Pulmonary
exacerbations
• Quality of life
• …

6. UK Cystic Fibrosis Registry
• A secure centralised database of
consenting with people with CF across
the UK
• Hosted and sponsored by the Cystic
Fibrosis Trust
• Data are collected at annual review
visits

7. Using registry data to study ivacaftor
People eligible for ivacaftor Almost all people are now receiving it
• How can we estimate the effect of ivacaftor?
• What is a suitable ‘control’ group?

8. Sawicki et al. Sustained benefit from ivacaftor demonstrated by combining clinical
trial and cystic fibrosis patient registry data. Am J RespirCrit Care Med.
2015;192:836–42.
Bessonova et al. Data from the US and UK cystic fibrosis registries support disease
modification by CFTR modulation with ivacaftor.Thorax. 2018;73:731–40.
Hubert et al. Retrospective observational study of French patients with cystic
fibrosis and a Gly551Asp-CFTR mutation after 1 and 2 years of treatment with
ivacaftor in a real-world setting. J Cyst Fibros. 2018;17:89–95.
Volkova et al. Disease progression in patients with cystic fibrosis treated with
ivacaftor: Data from national US and UK registries. J Cyst Fibros. 2020; 19: 68-79.
Using registry data to study ivacaftor

9. Using registry data to study ivacaftor
Pre-ivacaftor era
…-2011
Post-ivacaftor era
2012-…
Eligible people

10. Using registry data to study ivacaftor
Eligible people
Ineligible people
Pre-ivacaftor era
…-2011
Post-ivacaftor era
2012-…

11. Using registry data to study ivacaftor
Eligible people
Ineligible people
Pre-ivacaftor era
…-2011
Post-ivacaftor era
2012-…
Time period comparison
Genotype
comparison

12. Using registry data to study ivacaftor
Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Ineligible people

13. Using registry data to study ivacaftor
Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Ineligible people
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

14. What are we trying to estimate?
𝑌 Outcome of interest (FEV1%)
𝑋 Ivacaftor use (0 or 1)
Counterfactual outcomes
𝑌 𝑋=1
Outcome had a person received ivacaftor
𝑌 𝑋=0 Outcome had a person NOT received ivacaftor
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

15. What are we trying to estimate?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝑋=1
𝑋 = 1 − 𝐸 𝑌 𝑋=0
𝑋 = 1
𝑌 Outcome of interest (FEV1%)
𝑋 Ivacaftor use (0 or 1)
Counterfactual outcomes
𝑌 𝑋=1
Outcome had a person received ivacaftor
𝑌 𝑋=0 Outcome had a person NOT received ivacaftor
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

16. What are we trying to estimate?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝑋=1
𝑋 = 1 − 𝐸 𝑌 𝑋=0
𝑋 = 1
𝑌 Outcome of interest (FEV1%)
𝑋 Ivacaftor use (0 or 1)
Counterfactual outcomes
𝑌 𝑋=1
Outcome had a person received ivacaftor
𝑌 𝑋=0 Outcome had a person NOT received ivacaftor
= 𝐸 𝑌 𝑋=1 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

17. What are we trying to estimate?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝑋=1
𝑋 = 1 − 𝐸 𝑌 𝑋=0
𝑋 = 1
𝑌 Outcome of interest (FEV1%)
𝑋 Ivacaftor use (0 or 1)
Counterfactual outcomes
𝑌 𝑋=1
Outcome had a person received ivacaftor
𝑌 𝑋=0 Outcome had a person NOT received ivacaftor
= 𝐸 𝑌 𝑋=1 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

18. What are we trying to estimate?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝑋=1
𝑋 = 1 − 𝐸 𝑌 𝑋=0
𝑋 = 1
𝑌 Outcome of interest (FEV1%)
𝑋 Ivacaftor use (0 or 1)
Counterfactual outcomes
𝑌 𝑋=1
Outcome had a person received ivacaftor
𝑌 𝑋=0 Outcome had a person NOT received ivacaftor
= 𝐸 𝑌 𝑋=1 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝐺 = 1, 𝑃 = 1 ???
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

19. Naïve treatment effects
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑁𝑇𝐸Time−period = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
`Naïve’ treatment effects (NTE)
Post+elig vs pre+elig
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

20. Naïve treatment effects
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑁𝑇𝐸Time−period = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
𝑁𝑇𝐸Genotype = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
`Naïve’ treatment effects (NTE)
Post+elig vs pre+elig
Post+elig vs post+inelig
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

21. Naïve treatment effects
`Naïve’ treatment effects (NTE)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
• Under what conditions do the NTEs correspond to the CTE?
• How can we assess potential bias in the NTEs if those conditions are not met?
• We use causal diagrams to investigate
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
𝑁𝑇𝐸Time−period = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
𝑁𝑇𝐸Genotype = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
Post+elig vs pre+elig
Post+elig vs post+inelig

22. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Which scenario are we in….?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌𝑋
𝑃
𝐺
𝐻
𝑌𝑋
𝑃
𝐺
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

23. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Directed acyclic graph (DAG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

24. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Directed acyclic graph (DAG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

25. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
[Richardson & Robins 2013]
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

26. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌 𝑥=0 ⊥ 𝐺, 𝑃
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1 = 𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 for any 𝑔, 𝑝
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

27. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌 𝑥=0 ⊥ 𝐺, 𝑃
= 𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 0
= 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
[pre+elig]
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1 = 𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 for any 𝑔, 𝑝
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

28. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌 𝑥=0 ⊥ 𝐺, 𝑃
= 𝐸 𝑌 𝑋=0 𝐺 = 0, 𝑃 = 1
= 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1 = 𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 for any 𝑔, 𝑝
[post+inelig]
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

29. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌 𝑥=0 ⊥ 𝐺, 𝑃
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1 = 𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 for any 𝑔, 𝑝
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
Post+elig vs pre+elig
Post+elig vs post+inelig

30. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
`Naïve’ treatment effects (NTE)
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
Post+elig vs pre+elig
Post+elig vs post+inelig
𝑁𝑇𝐸Time−period = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 1, 𝑃 = 0
𝑁𝑇𝐸Genotype = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝐺 = 0, 𝑃 = 1
Post+elig vs pre+elig
Post+elig vs post+inelig

31. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐻 𝐻
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

32. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌 𝑥=0 ⊥ 𝐺, 𝑃|𝐻
𝐻 𝐻
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1, 𝐻 = 𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝, 𝐻 for any 𝑔, 𝑝
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

33. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 = ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1) [standardization]
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

34. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 = ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 𝑔, 𝑃 = 𝑝, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
[standardization]
𝑌 𝑥=0
⊥ 𝐺, 𝑃|𝐻
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

35. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 = ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 𝑔, 𝑃 = 𝑝, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 0, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
Using pre+elig
= ෍
ℎ
𝐸 𝑌 𝐺 = 1, 𝑃 = 0, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
[standardization]
𝑌 𝑥=0
⊥ 𝐺, 𝑃|𝐻
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

36. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 = ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 𝑔, 𝑃 = 𝑝, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝑋=0
𝐺 = 0, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
= ෍
ℎ
𝐸 𝑌 𝐺 = 0, 𝑃 = 1, 𝐻 = ℎ Pr(𝐻 = ℎ|𝐺 = 1, 𝑃 = 1)
[standardization]
𝑌 𝑥=0
⊥ 𝐺, 𝑃|𝐻
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
Using post+inelig

37. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

38. Estimating the CTE
𝑌𝑋
𝑃
𝐺
𝑌 𝑥=0𝑋 | 𝑥 = 0
𝑃
𝐺
Directed acyclic graph (DAG) Single world intervention graph (SWIG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
𝑌 𝑋=0
is NOT independent of 𝐺, 𝑃

39. Negative control outcomes
Lipsitch M,TchetgenTchetgen E, CohenT. Negative Controls: ATool for Detecting
Confounding and Bias in Observational Studies. Epidemiology. 2010;21(3):383–8.
Negative control outcome
𝑌𝐴
𝐿
𝑈

40. Negative control outcomes
Lipsitch M,TchetgenTchetgen E, CohenT. Negative Controls: ATool for Detecting
Confounding and Bias in Observational Studies. Epidemiology. 2010;21(3):383–8.
Negative control outcome
𝑌𝐴
𝐿
𝑈
𝑌 𝑁𝐸𝐺
The set of common causes of 𝐴 and 𝑌 is the
same as the set of common causes of 𝐴
and 𝑌 𝑁𝐸𝐺
If we repeat out analysis replacing 𝑌 with
𝑌 𝑁𝐸𝐺and find a ‘null’ result then this suggests
no bias due to unobserved confounding in the
main analysis

41. Negative control outcomes
SoferT, Richardson D, Colicino E, Schwartz J,TchetgenTchetgen E. On negative
outcome control of unobserved confounding as a generalization of difference-in-
differences. Stat Sci. 2016;31(3):348–61.
𝑌 𝑃𝑅𝐸 𝐴
𝐿
𝑈
𝑌 𝑃𝑂𝑆𝑇

42. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Directed acyclic graph (DAG)
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
𝑌 𝑃𝑅𝐸 𝑋
𝐺
𝑌 𝑃𝑂𝑆𝑇
• Genotype (𝐺) is not an unmeasured confounder
• It is an ‘uncontrollable’ confounder because of it’s deterministic association with 𝑋

43. Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Non-eligible people
Using negative control outcomes

44. Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Non-eligible people
Using negative control outcomes

45. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)

46. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)
𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 1)
post+elig – post+inelig

47. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)

48. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)
𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 = 𝛼 + 𝛽𝑔 + 𝛾𝑝
Consider the following linear model….

49. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)
𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 = 𝛼 + 𝛽𝑔 + 𝛾𝑝
Consider the following linear model….
𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 0 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 0)
Key assumption:
no 𝒈 × 𝒑 interaction

50. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)
𝐸 𝑌 𝑋=0 𝐺 = 𝑔, 𝑃 = 𝑝 = 𝛼 + 𝛽𝑔 + 𝛾𝑝
Consider the following linear model….
𝐸 𝑌 𝐺 = 1, 𝑃 = 0 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)
Key assumption:
no 𝒈 × 𝒑 interaction

51. Estimating the CTE
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0|𝐺 = 0, 𝑃 = 1)
We can write this as a “difference in differences”
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
− 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌 𝑋=0
|𝐺 = 0, 𝑃 = 1)
𝑁𝐶𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 1)
− 𝐸 𝑌 𝐺 = 1, 𝑃 = 0 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)
Naïve treatment effect
Negative control effect
Negative-control-corrected treatment effect

52. Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Non-eligible people
Using negative control outcomes
𝑁𝐶𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 1)
− 𝐸 𝑌 𝐺 = 1, 𝑃 = 0 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)
Negative-control-corrected treatment effect

53. Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Non-eligible people
Using negative control outcomes
𝑁𝐶𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 1, 𝑃 = 0)
− 𝐸 𝑌 𝐺 = 0, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)
Negative-control-corrected treatment effect

54. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
Two negative control corrected treatment effects (NCCTE)
NCCTEgenotype = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 1)
− 𝐸 𝑌 𝐺 = 1, 𝑃 = 0 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)
NCCTEtime−period = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 1, 𝑃 = 0)
− 𝐸 𝑌 𝐺 = 0, 𝑃 = 1 − 𝐸(𝑌|𝐺 = 0, 𝑃 = 0)

55. Estimating the CTE
𝑌𝑋
𝑃
𝐺
Which scenario are we in….?
Causal treatment effect (CTE)
𝐶𝑇𝐸 = 𝐸 𝑌 𝐺 = 1, 𝑃 = 1 − 𝐸 𝑌 𝑋=0
𝐺 = 1, 𝑃 = 1
𝑌𝑋
𝑃
𝐺
𝐻
𝑌𝑋
𝑃
𝐺
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
CTE can be estimated using
naïve treatment effects (NTE)
CTE can be estimated using adjusted
naïve treatment effects (NTE)
Negative control corrected
treatment effect (NCCTE) can
be used

56. Application: UK CF Registry
Pre-ivacaftor era
2007-2011
Post-ivacaftor era
2012-2016
Eligible people
Non-eligible people
N=437
N=7378
N=397
N=6382
• We estimated naïve treatment effects (NTE), negative-control effects (NCE) and
negative control-corrected effects (NCCTE)
• There are two versions of each: time-period comparison, genotype comparison

57. Analysis: Naïve treatment effect
Post+elig: 𝑋 = 1
2013 2015
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
Generalized estimating equations fitted to
estimate two effects:
- Step change effect: the initial impact of
treatment on FEV1%
- The ‘slope change’ effect: the impact of
treatment on the slope of decline
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
2012 2014 2016
Post+inelig: 𝑋 = 0
2013 2015
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
2012 2014 2016
Genotype comparison

58. Analysis: Negative control effect
Pre+elig: we set 𝑋 = 1
2009 2011
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
Generalized estimating equations fitted to
estimate two effects:
- Step change effect: the initial impact of
treatment on FEV1%
- The ‘slope change’ effect: the impact of
treatment on the slope of decline
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
2008 2010 2012
Pre+inelig: 𝑋 = 0
2009 2011
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
2008 2010 2012
Genotype comparison

59. Analysis: Naïve treatment effect
Post+elig: 𝑋 = 1
2013 2015
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
2012 2014 2016
Time-period comparison
Pre+elig: 𝑋 = 0
2009 2011
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
2008 2010 2012

60. Analysis: Negative control effect
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1
Pre+inelig: 𝑋 = 0
2009 2011
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
2008 2010 2012
Time-period comparison
Post+inelig: we set 𝑋 = 1
2013 2015
𝑌1 𝑌2 𝑌3 𝑌4
Outcome:
FEV1%
2012 2014 2016

61. Results
Step change effect in FEV1% Slope change effect in FEV1%
Time period
comparison
Genotype
comparison
NTE
NCE
NCCTE
NTE
NCE
NCCTE
pre+elig
𝐺 = 1, 𝑃 = 0
pre+inelig
𝐺 = 0, 𝑃 = 0
post+elig
𝐺 = 1, 𝑃 = 1
post+inelig
𝐺 = 0, 𝑃 = 1

62. Second outcome: days on IV antibiotics
We performed a similar analysis for the ‘count’ outcome: number of days of using
IV antibiotics over the course of 1 year
𝐶𝑇𝐸 =
𝐸 𝑌 𝑋=1
𝐺 = 1, 𝑃 = 1
𝐸 𝑌 𝑋=0 𝐺 = 1, 𝑃 = 1
Analysis used a negative binomial model
Causal treatment effect

63. Second outcome: days on IV antibiotics
Time period
comparison
Genotype
comparison
NTE
NCE
NCCTE
NTE
NCE
NCCTE
Year 1 effect Year 2 effect Year 3 effect

64. Discussion
• Naive treatment effect estimates are valid only under strong assumptions
• Negative control outcomes can be used to obtain unbiased estimates of the causal
treatment effect under weaker assumptions
• It also works in other cases when there are unmeasured variables affecting 𝐻 and 𝑌
𝐺
𝑃
𝑌𝑋
𝐻
𝑈
𝐺
𝑃
𝑌𝑋
𝐻
𝑈

65. Further work
• I am currently working on extending this to estimate the effect of ivacaftor on
survival, with the aim of then estimating it’s potential impact on life expectancy
• A new CF treatment Kaftrio was recently approved in the UK – these methods can be
applied to estimate it’s ‘real world’ impact

66. UK Research &
Innovation Future
Leaders Fellowship
Cystic Fibrosis Trust
Strategic Research Centre
Grant
FundingSimon Newsome
LSHTM,UK
Novartis PharmaAG, Switzerland
Rhian Daniel
Cardiff University, UK
Diana Bilton, Siobhan Carr
Imperial College , UK
Royal Brompton and Harefield NHS Foundation
Trust, UK