Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons from MESAS: Monitoring and Evaluating Scotland's Alcohol Strategy (MESAS)


Published on

Clare Beeston (NHS Scotland) - NSAPAG 22 May 2019

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Lessons from MESAS: Monitoring and Evaluating Scotland's Alcohol Strategy (MESAS)

  1. 1. Lesson from MESAS: Monitoring and Evaluating Scotland's Alcohol Strategy (MESAS) North South Alcohol Policy Advisory Group. 22 May 2019 Clare Beeston. Public Health Intelligence Principal
  2. 2. Research grants For academic Sector Other funding bodies. e.g. NIHR Alcohol public health in Scotland Alcohol research network 3rd sector advocacy organisations MESAS
  3. 3. Evidence-informed, outcome focused. 1. Reduced consumption. MUP; multi-buy discount ban 2. Supporting families and communities. Support ↓ drink driving BAC 3. Positive attitudes, positive choices. Marketing and display restrictions 4. Improved treatment and support. Brief Interventions; investment in services Commitment to evaluation
  4. 4. Minimum Unit Pricing • Sunset clause – parliamentary vote before 31 April 2024 on MUP’s future • Review clause – report on MUP’s impact before then • Impact on producers, licence holders and 5 licensing objectives. • By age, gender, socio-economic and drinking status where possible. • Primary purpose of our evaluation
  5. 5. The MESAS approach • Theory-based – assessing contribution • Evaluability assessment process to determine appropriate evaluation studies and monitoring indicators • Feasible, robust, proportionate to value of information • Portfolio of connected, quantitative and qualitative studies to evidence both change and mechanisms. • Individual study and annual reports • Interrogation and interpretation of monitoring data
  6. 6. Start of decline predates the policy. External factors important 1) Falling incomes pre- recession 2) Vulnerable cohort
  7. 7. Measuring consumption “The absence of high-quality alcohol consumption data, particularly on a regional and national basis, is a profound handicap to further study.” Leon & McCambridge, 2006 Duty data Retail sales data Purchase data Survey data
  8. 8. Per adult consumption: Self-report and sales. Scotland and England & Wales. 1994-2009 Self-report accounts for 50% of what is sold Higher sales in Scotland than E & W Driven by higher off-trade sales Driven by higher sales of spirits Especially vodka – in the 30-45ppu price bands
  9. 9. Validity and reliability of alcohol retail sales data
  10. 10. The MESAS approach • Theory-based • Evaluability assessment process to determine appropriate evaluation studies and monitoring indicators • Feasible, robust, proportionate to value of information • Portfolio of connected studies – together providing plausible narrative. Quant and Qual • Individual study reports and annual report • Interrogation and interpretation of monitoring data - To the science
  11. 11. SCIENCE SCOPE SALIENCE STANDING The MESAS approach - The full picture Adapted from ideas on understanding an PH organisations fortunes The politics of institutionalising preventative health Boswell, Cairney and Denny 2019
  12. 12. Scope Remit Resources MoA To inform the vote
  13. 13. Salience Political alignment Media controversy Too cold Too hotJust right
  14. 14. “The system will be experimental, but that is a factor catered for by its provisions for review and sunset clause. It is a significant factor in favour of upholding the proposed minimum pricing régime.” Increasing the temperature… Vote will be informed by evaluation, political will, public and stakeholder attitudes (to MUP and to the evaluation)
  15. 15. Potential to get hotter…
  16. 16. • It is too early to assess the impact of MUP on alcohol consumption in Scotland. • We will focus on pure alcohol volume at total and category level. • We will include comparisons with other areas that don’t have MUP. …Just right
  17. 17. Standing Formal arrangement Informal reputation • MoA - decision making rests with NHSHS • Governance structure: Governance Board + Evaluation Advisory Groups • Transparency • Transparency • Stakeholder engagement • Honesty • Communication appropriate to role Protecting the integrity of MESAS and the evaluation. Success: The evaluation informs the sunset vote. To do this, it must be accepted by MSPs and Parliament to be a credible and robust source of evidence and one that recognises the Parliament’s responsibility for a broad range of outcomes.
  18. 18. To what extent has MUP contributed to reducing alcohol-related health and social harms in Scotland? Are some people and business affected positively and negatively more than others? The Evaluation of MUPQuestions Approach Theory-based. Mixed methods portfolio of studies. Delivery Portfolio of studies (to date): 12 MESAS-Funded: In-house and commissioned 6 Separately-funded: 4 different lead organisations, involving 10+ different UK and international institutions
  19. 19. Influence of factors external to the strategy e.g: other influences on alcohol price and/or disposable income MUP implemented Compliance Price change. No alcohol <50ppu Reduced Purchasing (in off-trade) Reduced consumption Reduced harm Product & marketing changes Change in social norms and attitudes to MUP Economic impact on alcohol industry Displacement of spending Substitution: non- beverage alcohol or drugs Impact on demand for services
  20. 20. Summary • Pay attention to science, scope, salience, and standing • Be clear on your primary purpose (scope) – everything else should come from this • Robust (and comprehensive) science, guided by a theoretical framework to improve interpretation • Communication that keeps salience ‘just right’ • Processes to protect standing – with all the relevant stakeholders.
  21. 21. Thanks for listening @MESAS_NHS
  22. 22. Supplementary slides
  23. 23. A theory- based approach Reasonable to assume contribution if: • There is a plausible ‘theory of change’ that shows how the activities link to the intended outcomes; • the activities were implemented in a way likely to achieve the outcomes; • Evidence is gathered that demonstrates the sequence of expected results is being realised; and • External factors influencing outcomes have been assessed and accounted for (Mayne 2008)
  24. 24. Litres pure alcohol sold per adult (16+), Scotland and England & Wales 1994-2017 CSR Multi-buy discount ban Higher sales in Scotland, Driven by higher off-trade sales
  25. 25. 0.00 0.05 0.10 0.15 0.20 0.25 0.30 <10 10- 14.9 15- 19.9 20- 24.9 25- 29.9 30- 34.9 35- 39.9 40- 44.9 45- 49.9 50- 54.9 55- 59.9 60- 64.9 65- 69.9 70- 74.9 75- 79.9 80- 84.9 ≥85 Price band (ppu) Lperadult(off-trade) Scotland England & Wales Price distribution (L per adult) of alcohol sold as vodka off-trade in Scotland and England & Wales, 2011 Driven by higher sales of vodka at the lower end of the price distribution
  26. 26. 0 5 10 15 20 25 30 <15 15-19.9 20-24.9 25-29.9 30-34.9 35-39.9 40-44.9 45-49.9 50-54.9 55-59.9 60-64.9 65-69.9 70-74.9 75-79.9 80-84.9 ≥85 %ofalloff-tradealcoholsold(Lpurealcohol) Pence per unit of alcohol 2009 2010 2011 2012 2013 2014 2015 2016 2017 Price distribution (%) of pure alcohol sold off-trade in Scotland, 2009–2017 <50ppu 2009 –2017 77%; 72%; 66%; 60%; 53%; 52% 51%; 51%; 47%
  27. 27. Self-reported weekly consumption, by income, Scotland 2015-16 Source: Scottish Health Survey Highest income decide most likely to exceed guidelines. - paradox But of those that exceed the guidelines, consumption is heaviest in the income quintiles – partial explanation
  28. 28. Consumption distribution and the alcohol harm paradox Source: Scottish Health Survey (2012/13 combined) The heaviest drinkers in the lowest income quintile drink far more than heaviest drinkers in other quintiles - ? Partial explanation of the alcohol paradox
  29. 29. (General acute) Alcohol-related hospitalisation rates Scotland, 1981/82 - 2017/18 Stay=burden Patients ≈prevalence New patients ≈ incidence of a-r hospitalisation
  30. 30. Start of decline predates the policy. External factors important 1) Falling incomes pre- recession 2) Vulnerable cohort