Nicholas Mays | Evaluating current market reforms in the English NHS


Published on

Professor Nicholas Mays, Professor of Health Policy at the London School of Hygiene and Tropical Medicine in the UK, presented to the HARC network in January 2008 on evaluating current market reforms in the UK National Health Service.

HARC stands for the Hospital Alliance for Research Collaboration. HARC is a collaborative network of researchers, health managers, clinicians and policy makers based in NSW, Australia managed by the Sax Institute.

HARC Forums bring members of the HARC network together to discuss the latest research and analysis about important issues facing our hospitals.

For more information visit

Published in: Health & Medicine
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Nicholas Mays | Evaluating current market reforms in the English NHS

  1. 1. Evaluating current market reforms in the English NHS Nicholas Mays Professor of Health Policy Department of Public Health & Policy, London School of Hygiene & Tropical Medicine, University of London Hospital Alliance for Research Collaboration (HARC) forum, Sydney, 29 January 2008
  2. 2. Outline• What do we mean by ‘reforms’ & ‘evaluation’?• NHS reforms since 1991, nature of evaluation & findings• 2002 reforms – reintroduction of market• Health Reforms Evaluation Programme – rationale, features, governance, process, progress• Challenges ahead
  3. 3. What do we mean by ‘reforms’ andsystem level changes?• Major changes to system incentives, governance, organisation, payment or financing
  4. 4. What is ‘evaluation’?• Ultimately normative, depending on value placed on particular effects by different actors• Usually research to see whether a policy achieved objectives set by its proponents and/or its effects on wider system goals – e.g. equity, efficiency, responsiveness, acceptability, humanity, sustainability, economy – range of goals means that onlookers can usually find some aspect to confirm their prejudices
  5. 5. What are the particular challenges ofsystem level policy ‘evaluation’?• ‘Reforms’ are seldom fixed, tend to be ‘emergent’• Difficult to define focus & disentangle effects from previous, overlapping & subsequent changes• Context matters• ‘Reforms’ tend to be politically & intellectually controversial – Public tends to have a negative view unrelated to performance• ‘Reforms’ are rarely introduced with an eye to evaluation• ‘Reforms’ tend to have multiple, broad objectives• Measurement is problematic because no simple ‘bottom line’ (e.g. productivity in public services)
  6. 6. English NHS ‘reforms’, 1991-Three main phases:• 1991-1997 – Predominantly ‘internal market’, supply side competition• 1997-2002 – ‘Command and control’, targets, performance management, investment in return for ‘modernisation’• 2002-2007 – Market for NHS services, including private & Third Sector – Gradual shift towards a ‘self-improving’ NHS
  7. 7. % waiting > 12 months England &Wales: 1999 - 2003 Criticism of Welsh NHS25 performance from Welsh Assembly2015 England Wales10 5 0 1999 2000 2001 2002 2003 2004 2005
  8. 8. % waiting > 12 months England &Wales: 1999 - 2005252015 England Wales10 5 0 1999 2000 2001 2002 2003 2004 2005
  9. 9. Trends in NSF clinical targets 1.17a Managing acute m yocardial infarctions, England and W ales 2002-5 99% 100% 100% 96% 98% 93% % hospitals achieving NSF targets in England & Wales 90% 90% 89% 83% 80% 83% 78% 70% 71% 71% 71% 60% aspirin 50% statins 45% 40% beta-blockers 30% thrombolysis (DTN 30) 24% 20% 10% 0% 2002 2003 2004 2005
  10. 10. 0 100 200 300 400 500 600 700 800 900 Croydon HA Wolverhampton HA Dorset HA Lambeth, Southwark & Lewisham HA Herefordshire HA Southampton & SW Hampshire HA Brent & Harrow HA Morecambe Bay HA Shropshire HA Nottingham HA Southern Derbyshire HA North Cheshire HA Kensington, Chelsea & Westminster HA Oxfordshire HA Suffolk HA East London & City HA South Humber HA East Sussex HA Hillingdon HA Hertfordshire HA Solihull HASource: Department of Health Bexley, Greenwich & Bromley HA Source: Department of Health East Riding & Hull HA Stockport HA Kingston & Richmond HA Leeds HA Avon HA Ealing, Hammersmith & Hounslow HA Somerset HA Cambridgeshire HA Norfolk HA South Essex HA West Kent HA Bedfordshire HA Calderdale & Kirklees HA West Sussex HA North & East Devon HA Northumberland HA Redbridge & Waltham Forest HA North Staffordshire HA Isle of Wight, Portsmouth & SE Hampshire HA Camden & Islington HA South Lancashire HA Wakefield HA Dudley HA Sefton HA Salford & Trafford HA North Essex HA Buckinghamshire HA Wiltshire HA Leicestershire HA Walsall HA Sheffield HA Cornwall & Isles of Scilly HA Barnet, Enfield & Haringey HA East Kent HA County Durham & Darlington HA South and West Devon HA East Lancashire HA Barking & Havering HA North Cumbria HA Rotherham HA Lincolnshire HA North & Mid Hampshire HA Northamptonshire HA Gateshead & South Tyneside HA West Pennine HA (rate per 100,000 operations, Feb 2002) Newcastle & North Tyneside HA Doncaster HA North Yorkshire HA St Helens & Knowsley HA Coventry HA Bury & Rochdale HA Manchester HA Variation in clinical outcomes Berkshire HA Gloucestershire HA Wirral HA Tees HA Merton, Sutton & Wandsworth HA North Nottinghamshire HA Sunderland HA Age and sex standardised death rates within 30 days of elective surgery Warwickshire HA West Surrey HA Birmingham HA South Staffordshire HA Liverpool HA North West Lancashire HA Bradford HA South Cheshire HA Barnsley HA Sandwell HA North Derbyshire HA East Surrey HA Wigan & Bolton HA Worcestershire HA
  11. 11. The re-invented NHS market in England, 2002- Money following the patients, rewarding the best and most efficient providers, giving others the incentive to improve (transactional reforms) Better care More diverse providers, with More choice and a much Better patient more freedom to innovate and stronger voice for patients improve services experience (demand-side reforms) Better value for (supply-side reforms) money A framework of system management, regulation and decision making which guarantees safety and quality, fairness, equity and value for money (system management reforms)
  12. 12. Evaluation and impact ofreintroduction of competition, 2002-• Implemented over time• Some piloting of reforms (e.g. London patient choice pilot showed patients would take up choice)• Limited signs so far of increased output or quality directly related to market reforms• Revealed excess hospital provision in some parts• Tension between competition (electives) & collaboration (chronic disease care)• Importance of regulation emerging
  13. 13. The Health Reforms Evaluation Programme (HREP)
  14. 14. DH Rationale for HREP• Extent and potential significance of changes• Desire to be able to adjust reform implementation in light of better understanding about how reforms are being implemented (mainly ‘formative’ evaluation)• Desire to leave a legacy of knowledge for future policy development on impact (‘summative’ evaluation)
  15. 15. Features of HREP• Independent, commissioned, peer reviewed research• Subject to usual DH research contract – expectation of publication of findings• Coordinated from outside the DH (NM), but part of long established DH PRP• £3.5m over 3.5 years including coordination costs• Aim: a coherent, ‘managed’ programme, not stand- alone projects, contributing to policy development
  16. 16. The aspiration: evaluation guides policy adaptation 3. Respond 1. Drive appropriately and support effective including by im plem entation of adjusting policy health reform 2. Understand quickly and accurately, to an appropriate degree of detail, what is happening and why - locally and nationally
  17. 17. Governance and organisation• ‘Sponsor’ – DH’s DG, Policy & Strategy• Oversight/QA – Reforms Evaluation Advisory Group of leading UK/international health service researchers & key policy-makers, chaired by coordinator (NM)• Commissioning group – 2 UK external experts, DH policy ‘leads’, chaired by NM• Day-to-day management – NM working with DH research liaison officer and 2 officials from PSU, Policy & Strategy Directorate
  18. 18. Role of the independent scientificcoordinator• Undertake periodic reviews integrating published research with internal analysis from DH & other agencies• Gap analysis and specification of new research• Chair the commissioning panel and direct peer review process• Coordinate delivery of the programme, encourage cross-fertilisation between projects, develop overviews of the findings, and brief DH and other stakeholders on findings & their policy implications• Chair REAP
  19. 19. Elements in HREP • Mechanism-specific projects • Whole system studies of ‘local health economies’ • Systematic reviews
  20. 20. Innovative aspects of HREP I• Successful proposals negotiated to finalise the research• Attempt to keep researchers aware of shifts in policy thinking at an early stage• Linking primary research to evidence syntheses• Attempt to synthesise published & ‘grey’ literature with internal analyses from DH, PMSU, etc. – e.g. on extent of reform implementation
  21. 21. Innovative aspects of HREP II• Exploitation of new routine datasets (e.g. Healthcare Commission’s quality of care assessments)• Use of maps to present some findings• Projects to include some ‘Reform Demonstration Systems’ to get early insights into likely changes• Encouragement to compare England with other parts of UK – to take advantage of ‘natural experiments’
  22. 22. Progress so far: projects funded• How patients choose and how providers respond – lead, Anna Dixon (Kings Fund)• Competition under fixed prices – lead, Carol Propper (University of Bristol)• Provider diversity in the NHS: Impact on quality and innovation – lead, Will Bartlett (University of Bristol)• Comparative case studies on the impact of the health reforms in England – lead, Martin Powell (University of Birmingham)• Effects of choice and market reform on inequalities of access to health care – Lead, Richard Cookson (University of York)
  23. 23. Characterising the projects I• Multi-disciplinary, though no active clinicians involved• Variety of perspectives – some strongly economic, others more sociological, organisational• Generally using ‘mixed’ methods• Focus on impact and process – e.g. ‘context-mechanism-outcome’ configurations (Pawson & Tilley, 1997)• Routine data (inc. GIS) and primary data collection• Mix of ‘whole system’ and mechanism-specific studies• National and local level studies
  24. 24. Wide range of impact measuresacross projects (‘tracer’ conditions)• Death rates after AMI, aneurysm, stroke• Revascularisation and colorectal cancer surgery rates• Financial status of trusts and staffing indicators• Inequality in age/sex standardised use rate ratios between top and bottom quintiles• Patients’ experiences of quality• Readmission rates/transfers to residential care
  25. 25. Characterising the projects III• No bids won by management consultancies though not excluded• Experienced teams with individual track records including prior collaborations• Budgets of ~ £500k• Projects already benefiting from one another• Some projects have their own advisory, interactive panels (e.g. of managers)
  26. 26. Challenges ahead I• Willingness & ability of DH policy leads to share early thinking with research teams• Building mutual trust and relationships, especially when policy officials turn over• Timely access to routine NHS information• Obtaining & using ‘grey’ literature from within government• Avoiding early feed back of findings prejudicing later analysis/conclusions
  27. 27. Challenges ahead II• Managing unrealistic expectations – Most realistic product of evaluation is enlightenment, not simple answers, evaluation cannot look at everything• Managing ‘bad news’ from the research• Change of government and/or abandonment of key policies• Ability of coordinator to ‘steer’ researchers• Timing of findings to make a difference – risk of Buxton’s Law ‘It’s too soon, until it’s too late’
  28. 28. Further information on HREP andrelated research and evaluation••• NIHR