Your SlideShare is downloading. ×
  • Like
Using primary care databases to evaluate drug benefits and harms: are the results replicable and valid?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Using primary care databases to evaluate drug benefits and harms: are the results replicable and valid?

  • 148 views
Published

Databases of electronic medical records and in particular primary care databases (PCDs) are increasingly used in research. The largest PCDs contain full data on all primary care consultations by …

Databases of electronic medical records and in particular primary care databases (PCDs) are increasingly used in research. The largest PCDs contain full data on all primary care consultations by millions of patients over two or more decades. They provide a means for investigating important healthcare questions which cannot be practically addressed in a Randomised Controlled Trial. However, concerns remain about the validity of studies based on data from PCDs. Most work around validity has attempted to confirm individual data values within a dataset. We take a different approach and instead replicate published PCD studies in a second, independent, PCD. Agreement of results then implies that the conclusions drawn are independent of the data source (though this doesn’t rule out that such as confounding by indication are commonly influencing both).
We replicated two previous PCD studies using the Clinical Practice Research Datalink (CPRD). The first was a retrospective cohort study of the effect of Beta-blocker therapy on survival in cancer patients using DIN-LINK. The second was a nested case-control analysis of the effects of Statins on mortality of patients with ischaemic heart disease using QRESEARCH.
Our analyses produced several important quantitative differences compared to the original studies, altering conclusions. These could not be fully explained by either demographic differences in the patient samples or structural differences between the datasets. Our study highlights both the caution that needs to be applied when assessing the findings from analysis of just a single database and the difficulties in performing replications of existing PCD studies.

Published in Health & Medicine
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
148
On SlideShare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Using primary care databases to evaluate drug benefits and harms: are the results valid and replicable? David A. Springate, University of Manchester Centres for Primary Care/Biostatistics
  • 2. Outline 1. Primary Care Database (PCD) study validity 2. PCD Replications – Statins and Ischaemic heart disease – β-blockers and Cancer 3. Lessons to be learned
  • 3. PCD studies are all the rage. . . Number of UK PCD publications is rapidly increasing 1990 1995 2000 2005 2010 050100150 PCD articles in pubmed year Numberofarticles There is global interest in UK PCD research Institutions affiliated with UK PCD publications xx x x x xxxxxx x x x x xx x x x xx x xx x xxx xx xxx x xx xxxxxx xx xx x xx x x x xx x x x xx x x xxxxxx x x x x x x xx x xxx x xxxxx xxx xxx x x x x xx xxx xx x xx x x xxx xx x x x x x x x x x x xx xx x xxx x x x x x x x x xx xx x xx xxxx x x x x xx x xx x xxx x xx xx xx x x xx x xxxxxx x x x xx x xxx x x xxx x x x x xxxxx x xx x xx xxxxxx xx xx x x x x xx x xxx x x xx xx x xxxxx x xxxxx x xx xxx x x x xx xx xxx x x x x xx x xx xx x x x x xx xx xxxxxx x x x xx x x x x x x x xx x x x x x xx x x x xxx x xxxxx x x x xxxx x x xxxxx xx xx x x xxxxxxxx xxx x xxxxx x x xx xxx x x xx xxxx x x xxx xx x xx xxxxx x xx x x xx x x x xx x x x xxx x xx x xxxx xx xxx xx x xx x xx x xxxx xxx x x xxx x x xxxx x x x x x x x x x x x x x x xxxxxx xxxx xx x xxx x x x x x x x x xx x x x x x xxx x x x xx x xxx x x x x x x x xx x x x x x x xx xx xxxx x x x x xxx x x xx xxx x x xxx x x x xx xxxxxxxxx xx xx x xxxx xx x xxxx x x xx x x x x xx x xxx x xx xxxxxx xx x xx x x x xxx x x x xxxxx xx xx x x x x x x x x x x x xxxxx x xx xxx x xxx x x x x x x x xx x x x x x xxx xx x xxxxx x x xx xx x x x xxxxxxx xx x x xxxx xx xx x x x x xxxx xx xx xxx xxx x xx xx x xxx x x x x x xxx x x x x xxxx x x x x xxxx xxx xxxxxxxx x xx xx xx x xxxx x x x xxxx x x x xx xxxx xx x xx xxx xxx x x x xxx xxxx xxxx x xx x x x x xx x x x x xx xxx x x x x x x x x xxxxxxxx x xxx xx x xxx x xx xxxx xx x xxxxxxxx xxxxx x xx xx x xxxxxxx x x xx xxx x x xx x xx xx x x xx x x x xxx x x x xxx x x xx xx xx xxx x x x xx x x xxx x x x x xx x x xxx x xx xxxxxxxx x x x x x x xx x xxxxxx x x xxxx xxx x xxx x x x x xx x x xx x x x x x x x x x x x x x x xx x xx xx x x x xx x x x xxx x xxx x xx xx x x x x x xx xx xx x x xx x xxxxxxx x xxxxxxxxxxx xxxxxxxxxxx xxx x x x xxxx x xxxx xxxxxxxxxxxxxxxxxx xxxxx
  • 4. Uses of Primary care databases. . . • Prevalence / incidence studies • Associations between conditions • Harms and risks of treatments • Comparative effectiveness • RCT comparisions / replications (and replacements?)
  • 5. BUT, There are still concerns about the validity of PCD-based studies. . . Threat Refs Data quality Herrett 2009, Khan 2010, Jordan 2004 Data completeness Marston 2010, Delaney 2007, Collins 2010 Confounding Tannen 2008, Lewis 2007 Clinical coding www.ClinicalCodes.org
  • 6. PCD Replications “Non-reproducible single occurrences are of no significance to science.” —– Karl Popper (1959) An approach to validity that asks whether flaws and differences in the data make any difference to the ultimate conclusions rather than looking at validity and completeness of the underlying individual data http://xkcd.com/242
  • 7. Replicating studies in another, independent PCD • Agreement implies that conclusions are not dependent on data source • Some factors could influence both (such as confounding by indication) • First completely independent PCD replication (but see Vinogradova (BMJ 2013): relationship between bisphosphate exposure and cancer in QResearch and CPRD)
  • 8. Replications were performed in CPRD • Largest UK primary care database (CPRD-GOLD) • ˜ 14 million patients • ˜ 650 practices across the UK • Uses the Vision GP computer system
  • 9. Criteria for replication 1. Effectiveness studies 2. Different GP computer system from CPRD (not Vision) 3. No practice overlap with CPRD 4. Representative coverage 5. Primary Care Database (Not integrated primary/secondary/pharmacy)
  • 10. PCD replications Hippisley-Cox and Coupland (2006) Effect of statins on the mortality of patients with ischaemic heart disease: population based cohort study with nested case-control analysis. Heart 92:752-758 (QResearch) Shah, Carey et al. (2011) Does β-adrenoceptor blocker therapy improve cancer survival? Findings from a population-based retrospective cohort study Br J Clin Pharmacol 72:157-161 (DIN-LINK) We then compare • summary statistics • mortality rates • model coefficients and standard errors
  • 11. Effect of statins on the mortality of patients with ischaemic heart disease (QResearch) Objective To measure the effect of statins on mortality for community based patients with IHD Design Cohort survival analysis and nested case-control Setting 1.18 million patients in 89 practices Subjects Patients with first diagnosis of IHD between January 1996 and December 2003 Outcomes - Cohort: Adjusted hazard ratios (+/- 95%CI) for all-cause mortality - Case-control: Odds ratio (+/-95%CI) for current use of statins, previous use and duration of use
  • 12. Effect of statins on the mortality of patients with ischaemic heart disease Summary statistics Measure Analysis CPRD Qresearch Number of practices Cohort 661 89 Number of patients Cohort 91589 13029 Cases 15591 2266 Controls 62356 9064 Median age Cases 80 80 Controls 79 80 Percent female Cases 45.5% 44.3% Controls 45.5% 44.3% Median followup (months) Cases 22.1 20.3 Controls 22.5 21 Percent on statins Cases 17% 19.6% Controls 23.6 25.4%
  • 13. Mortality rates for patients with IHD Age Comorbidity Sex 0 100 200 300 400 Total0−4445−5455−6465−7475−8485−94 95_plus N o diabetes D iabetes N o hypertension H ypertensionN o C C F C C F Fem ale M ale Patient group Mortalityrate(per1000personyears) PCD CPRD QResearch
  • 14. Survival analyses for patients on Statins QResearch Adjusted HR 0.47 (0.41 to 0.53) CPRD 0 2 4 6 8 Time since diagnosis of IHD (Years) Survival 0.000.250.500.751.00 Adjusted HR 0.43 (0.40 to 0.46)
  • 15. Case-control analysis: Odds ratios for effects of Statins on mortality in IHD patients Odds are relative to patients not on statins. Dotted line represents 1:1 odds Previously on statins Currently on statins 0.0 0.5 1.0 1.5 2.0 AllStatins Atorvastatin C erivastatinFluvastatinPravastatinSim vastatin AllStatins Atorvastatin C erivastatinFluvastatinPravastatinSim vastatin Statin type Oddsratio PCD CPRD Qresearch
  • 16. Misleading pooled odds ratios Combining effects of e.g. two drugs of the same BNF chapter Group Y N Odds ratio Drug 1 Cases 600 200 3/3 = 1 Drug 1 Controls 75 25 Drug 2 Cases 10 30 0.333/1 = 0.333 Drug 2 Controls 30 30 Pooled Cases 610 230 2.65/1.91 = 1.39 Pooled Controls 105 55
  • 17. Case-control analysis: Adjusted OR for duration of use of statins on survival Odds are relative to patients not on statins. Dotted line represents 1:1 odds 0.00 0.25 0.50 0.75 1.00 0−12 13−24 25−36 37−48 49−60 >60 Duration (months) AdjustedOddsratio PCD CPRD Qresearch
  • 18. Summary — Statins study 1. Strikingly similar results in the two studies, despite different GP computer systems (Vision vs EMIS) 2. As expected, narrower confidence intervals due to larger study 3. Original study was well designed (Matching, appropriate analyses etc.) 4. Given the results, pooling of “all statins” is questionable
  • 19. Does β-adrenoceptor blocker therapy improve cancer survival?(DIN-LINK) Objective To examine the effect of β-blocker treatment on cancer survival Design Survival analyses for 9 cancer types Setting 3462 cancer patients on β-blocker or other antihypertensive therapy Subjects Patients 40-85 with first cancer diagnosis between 1997 and 2006 Outcomes - Adjusted hazard ratios (+/- 95%CI) for all-cause mortality in each cancer type - Pooled hazard ratio (random effects)
  • 20. Comparison of summary statistics Cohort size: CPRD 11316 DIN-LINK 3462 BPLM = Blood pressure lowering medicines Deaths Alive at 1 year Diagnosed 1997−2001 Diagnosed 2002−2006 0 25 50 75 100 Beta−blockersO therBPLM Beta−blockersO therBPLM Beta−blockersO therBPLM Beta−blockersO therBPLM Drug combination Percentageofpatients PCD CPRD DIN−LINK
  • 21. Comparison of patient samples by cancer site BPLM = Blood pressure lowering medicines 0 10 20 30 40 breast colon lungoesophagus ovarian pancreas prostate renal stom ach Cancer site Percentageofpatients PCD CPRD DIN−LINK
  • 22. Survival analyses: comparison of hazard ratios Cancer type breast colon lung oesophagus ovarian pancreas prostate renal stomach Overall Hazard ratio 1.09 (0.8, 1.49) 1 (0.77, 1.3) 1.12 (0.89, 1.41) 1.05 (0.69, 1.6) 1.14 (0.63, 2.06) 1.88 (1.09, 3.25) 1.54 (1.13, 2.09) 1.14 (0.52, 2.52) 1.44 (0.76, 2.74) 1.18 (1.04, 1.33) % weight 14.85 20.93 25.7 8.17 4.27 4.91 15.19 2.37 3.61 100 0 1 2 3 4 all Beta−blockers vs controls DIN−LINK Cancer type breast colon lung oesophagus ovarian pancreas prostate renal stomach Overall Hazard ratio 1.19 (1.03, 1.37) 0.85 (0.74, 0.97) 1.04 (0.91, 1.19) 1.27 (1.01, 1.59) 1.05 (0.74, 1.5) 0.94 (0.74, 1.21) 1.03 (0.92, 1.15) 0.46 (0.26, 0.83) 1.03 (0.78, 1.36) 1.01 (0.91, 1.13) % weight 14.87 15.11 15.36 10.7 6.32 9.76 16.27 2.89 8.71 100 0 1 2 3 4 all Beta−blockers vs controls CPRD
  • 23. Summary — β-blocker study 1. Different individual cancer HR’s and overall conclusions 2. Important differences in some cohort statistics 3. Differences remain after correcting to give the same patient:practice 4. Differences remain after reducing the size of the CPRD study 5. Databases appear to be demographically similar (Carey et al. 2004)
  • 24. Summary — β-blocker study 1. Different individual cancer HR’s and overall conclusions 2. Important differences in some cohort statistics 3. Differences remain after correcting to give the same patient:practice 4. Differences remain after reducing the size of the CPRD study 5. Databases appear to be demographically similar (Carey et al. 2004) WHY?
  • 25. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn
  • 26. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn • Artifact of differences in GP computer systems? DIN-LINK uses Torex/iSoft systems (See Kontopantelis et al 2013)
  • 27. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn • Artifact of differences in GP computer systems? DIN-LINK uses Torex/iSoft systems (See Kontopantelis et al 2013) • Analysis methods? – No matching - potential confounding – No clustering by practice – Limited control for covariates – Is meta-analysis the most appropriate method (Assumes independence)? • Data quality?
  • 28. Conclusions / recommendations These replications add to the evidence that PCD results are valid
  • 29. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard!
  • 30. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard! – Methods details are inadequate for replication – Clinical codes not provided with the original article – Relies on active cooperation of authors of original studies – Even then, ambiguity can remain
  • 31. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard! – Methods details are inadequate for replication – Clinical codes not provided with the original article – Relies on active cooperation of authors of original studies – Even then, ambiguity can remain • Publish full methods (in online appendix?) • Publish full clinical code lists... ClinicalCodes.org
  • 32. Thanks. . . Research team David Reeves Evan Kontopantelis Ivan Olier Darren Ashcroft Authors of the original studies Iain Carey (St. Georges University, London) Carol Coupland (University of Nottingham) Contact: david.springate@manchester.ac.uk