Ellen Nolte & others: International benchmarking


Published on

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ellen Nolte & others: International benchmarking

  1. 1. Nuffield Trust workshop “Investing in health: benchmarking health systems” 17 January 2006International benchmarking:Measuring the effectiveness ofhealth systemsEllen NolteMartin McKeeChris Bain*European Centre on Health of Societies in TransitionLondon School of Hygiene and Tropical Medicine* School of Population Health, University of Queensland, Australia
  2. 2. International comparisons ofhealth care systems OECD “Measuring health care, 1960-1983 expenditure, costs and performance” (1985) OECD Health Data Set (from 1993) WHO Health System Performance Assessment Framework (HSPA; 1998) World Health Report 2000 (WHR 2000)
  3. 3. International comparisons ofhealth care systems Commonwealth Fund International Working Group on Quality Indicators (1999) OECD Health Care Quality Indicator Project (2001) Danish National Indicator Project (2000) National Performance Indicator framework NL (2003) European Union Benchmarking health monitoring programme (12/2001) Working Party on Health Systems (11/2003)
  4. 4. Difficulties… What is a health (care) system – or: what do we want a health (care) system to do/be? Are we measuring what is important, not just what is measurable? How to interpret aggregate data / composite indices? Attribution of health outcomes to health systems…
  5. 5. World Health Report 2000 Comparison of performance of 191 health systems Dimensions Health attainment (level & distribution) Responsiveness (level & distribution) Fair financing DALE (disability-adjusted life expectancy): Summary measure Attribution to health systems?
  6. 6. Mortality (SDR) from coronary heart Attribution of disease, men <65, 1998/2000, UK health outcomes to health Shetland Islands systems? 20.6 – 41.1 41.2 – 50.7 50.8 – 57.7 57.8 – 68.9 69.0 – 136.7 Risk factors (smoking, diet …) Socioeconomic status Health care … London boroughsSource: British Heart Foundation; Coronary Heart Disease Statistics (2003)
  7. 7. Framework for analysing health systems Environment Other Systems Educational System Health of population Patients Need, demand, access Health Health Process Care Outcome Outcome Structure & Organisation Financial Resources Health Care SystemBusse & Wismar 2002
  8. 8. Florence NightingaleThe concept of avoidable mortality
  9. 9. ‘Avoidable’ mortality (1) Rutstein et al. “unnecessary, untimely deaths” (1976) Conditions from which, in the presence of timely and effective medical care, premature death should not occur Single case of death (illness/disability): Why did it happen? Rate: not every single case preventable/ manageable reduction of incidence
  10. 10. ‘Avoidable’ mortality (2) immunisation, e.g. measles early detection, e.g. cervical cancer medical treatment, e.g. hypertension surgery, e.g. appendicitis
  11. 11. Changes in life expectancy(0-75) in EU countries, 1980-1998, men Portugal Austria Finland Germany west France Italy LE 1980 United Kingdom 1980-89 Denmark 1989-98 Spain Greece Netherlands Sweden 60 65 70 75 Life expectancy(0-75) (years)Source: Nolte & McKee 2004
  12. 12. Changes in life expectancy(0-75) in EU countries, 1980-1998, women Portugal Austria United Kingdom Denmark Germany west LE 1980 Italy Greece 1980-89 France 1989-98 Spain Finland Netherlands Sweden 60 65 70 75 Life expectancy(0-75) (years)Source: Nolte & McKee 2004
  13. 13. Contribution of amenable mortality to changes in life expectancy(0-75), men Spain Greece Portugal Italy Austria United Kingdom 1980-89 Germany, west 1990-98 Finland France Sweden Netherlands Denmark 0 20 40 60 80 100 % contribution to changes in life expectancy (0-75)Source: Nolte & McKee 2004
  14. 14. Contribution of amenable mortality to changes in life expectancy(0-75), women Greece Portugal Netherlands Finland United Kingdom Spain 1980-89 Denmark 1990-98 Italy Austria Sweden Germany, west France 0 20 40 60 80 100 % contribution to changes in life expectancy (0-75)Source: Nolte & McKee 2004
  15. 15. SDR amenable causes (per 100,000) Po 0 50 100 150 200 250 300 rt u U ga ni Au l te s d Ki tria ng do G m er m I an ta ly y w es Sp t a G in reSource: Nolte & McKee 2004 ec Fi e nl an Fr d N et anc he e rla D nds en m a Sw rk ed men en SDR amenable causes (per 100,000) U ni Po 0 50 100 150 200 250 300 t e rt d u Ki ga ng l do Au m st G ria re G ec er e m Ita an y ly w es Sp t D ai e n N nm et he ark rla n Fi ds nl an Age standardised death rates(0-74) Fr d an Sw ce ed women en from amenable causes, 1980 & 1998 1998 1980
  16. 16. ‘Avoidable’ Mortality Amenable (or treatable) mortality Deaths from causes sensitive to health care (primary & hospital care, collective health interventions eg screening) selected cancers (breast, colorectal, testes, cervix), diabetes <50, hypertension/stroke, surgical conditions, maternal mortality, perinatal conditions etc. Preventable mortality Deaths from causes sensitive to public health policies Lung cancer, liver cirrhosis, transport injuries
  17. 17. Age-& cause-specific contributions todifferences in male life expectancy(0-75)between Sweden & USA, 2000 0.4 Total difference: 2.30 years Years of life expectancy 0.3 preventable 0.2 amenable other 0.1 0.0 -0.1 0 1-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 Age
  18. 18. Age-& cause-specific contributions todifferences in female life expectancy(0-75)between Sweden & USA, 2000 0.4 Total difference: 1.46 years Years of life expectancy 0.3 preventable amenable 0.2 other 0.1 0.0 -0.1 0 1-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 Age
  19. 19. ‘Avoidable’ mortality in selected countries, 2000 men women USA USAGermany E&W E&W Germany Sweden Sweden 0 20 40 60 80 100 0 20 40 60 80 100 SDR 0-74 SDR 0-74 Amenable causes Liver cirrhosis Lung cancer Transport injuries
  20. 20. Conclusions (I) Improvements in access to health care had measurable impact in many EU countries in 1980s & 1990s Gains achieved reflect starting point with scope of improvement highest in countries with initially high rates Useful as screening tool to identify areas for further study
  21. 21. How do countries compare? Different models of health care provision Differences at different levels Approach: ‘probe disorders’ or ‘tracer conditions’ that capture certain elements of the health care system Discrete and identifiable health problem Has a functional impact Natural history of condition varies with utilisation and effectiveness of health care Sufficiently high prevalence
  22. 22. Diabetes as tracer condition Deaths (<45) considered ‘avoidable’ by timely and effective health care Optimal management requires co-ordinated inputs from range of health professionals incl. primary care & specialists access to essential medicines & monitoring equipment active participation of informed patients Can provide important insights into primary and specialist care, and into systems for communicating among them
  23. 23. Study design Outcome measure: Mortality-to-incidence ratio commonly used in cancer epidemiology as a crude indicator of cancer survival or “case fatality” may be interpreted as an indicator of the overall quality of health care Age-standardized incidence rates for ages 0-14 years (WHO DiaMond study, 1990-1994) Age-standardised death rates from diabetes for ages 0-39 (WHO mortality database, 1994- 1998) Study population: 29 industrialised countries
  24. 24. SIR 0-14 Fi nl 0 5 10 15 20 25 30 35 40 an Ca d na da De UK nm Au ark st ra Po lia rtu ga Sp l a E s in to ni Au a st ri incidence (SIR0-14) Fr a an Bu ce l Li gar th i un a an ia Is ra el La tv ia Ja pa n 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 SDR 0-39 Diabetes mortality (SDR0-39) andSIR0-14 SDR0-39
  25. 25. Diabetes: Mortality-incidence ratio Russia Latvia Lithuania Japan Romania Bulgaria Estonia Poland Hungary USA Denmark Slovakia Japan Israel Portugal United States Finland Slovenia Norway Finland Germany Austria Israel 30-39 Australia 20-29 Netherlands New Zeal United Kingdom France Canada Sweden 0 500 1000 1500 2000 2500 UK Spain Death rate (100,000 person yrs) Italy Greece 0 0.05 0.1 0.15 0.2 0.25 SDR-SIR ratio
  26. 26. Sensitivity analysis Ratio of national SDR Ratio of M/I ratios (1994-98)Scenario US vs. US vs. vs. UK vs. UK Canada Canada(i) as reported 2.6 2.0 3.3 3.2(US incidence: 14.8/100,000)(ii) US death rateExcess: 10% 2.4 1.8 3.0 2.9Excess: 20% 2.1 1.6 2.7 2.6Excess: 50% 1.3 1.0 1.7 1.6(iii) Increase US incidence rate toa. highest regional rate - - 2.8 2.7(17.8/100,000)b. upper 95 CI of highest - - 2.4 2.4regional rate (20.3/100,000)(iv) (iii a) + 20% mortality excess - - 1.9 1.9
  27. 27. Next steps M/I ratio only an indicator of potential differences in health system performance & should stimulate detailed assessments to confirm whether the apparent variations are real and identify the reasons Scrutinise data Understand immediate causes of death e.g. ~50% of deaths in Estonia & Latvia due to acute complications of diabetes compared to only 22% in Finland (Podar et al. 2000) Understand processes of care e.g. patients with diabetes in US less likely than those in Hungary to receive patient education, see an ophthalmologist or diabetologist, or to perform self-monitoring of blood glucose – but were also less likely to experience severe hypoglycaemia (Tabak et al. 2000)
  28. 28. Conclusions (2) M/I ratio for diabetes provides means of differentiating countries that appear to provide differing quality of care to people with diabetes and by extension to other chronic diseases Further work is required to develop a battery of performance indicators that capture other aspects of health system performance instruments that can be used for detailed health system diagnosis once indicators suggest the presence of a problem
  29. 29. Conclusions (3)International comparisons of health (care) systems havefocused on what can most readily be measured, notwhat is necessarily importantWhile indicating the existence of a possible problemthey provide few insights in how to respondTracer conditions offer approach to overcome some ofthese limitationsThis study is an attempt to show how to shift theagenda on performance assessment to disorders suchas chronic disease that are critically important