The PPT presents some reflections on outcomes reporting at the government policy level, using the example of gambling harm minimization in NZ. The reader should access the subsequent Outcomes Monitoring Report.

1. More does not mean better:
Lessons from the New Zealand Ministry of
Health experience in Gambling Harm
Reduction Outcomes Reporting
Presentation for International Gambling Conference,
February 2018. Auckland, New Zealand.
By Dr John Wren, Senior Research Advisor, Addictions Team, Systems
Outcomes, Ministry of Health
Sean-Paul Kearns, Data Analyst, Addictions Team, Ministry of Health

2. We will
Provide some context on the
history of outcomes reporting
in New Zealand
Outline the contents of the
2013 New Zealand Gambling
Baseline Report
Reflect on the experience of
producing the report and
subsequent updates
Summarise what we have learnt
Outline how we have
applied the lessons in 2017
Update Report

3. Context – New Zealand Legislative Framework for
Gambling
Under the Gambling Act 2003, the
Ministry of Health is ‘the Department
responsible’ with the funding and
coordination of problem gambling
services. It assumed responsibility for
this role on 1 July 2004.
Since 1 July 2016, the Ministry’s work is guided by its
Strategy to Prevent and Minimise Gambling Harm 2016/17
to 2018/19

4. Origins of push for Outcomes Reporting
• The 2003 legislation includes requirements for ‘independent scientific
research associated with gambling’ and for ‘evaluation’
• Ministry promotion of adopting Good Public Health Practice in the design,
delivery and monitoring of public health programmes…
2006 MOH Guide on Logic
Models published
http://www.health.govt.nz/publication/guide-
developing-public-health-programmes
2007 MOH Guide on Outcomes
Monitoring published
http://www.health.govt.nz/publication/how-monitor-
population-health-outcomes-guidelines-developing-
monitoring-framework

5. Development of Gambling Outcomes Reporting
2007 Outcomes
Framework Advisory
document
commissioned 2009 Outcomes Framework
Discussion document issued
2010 Outcomes Framework
incorporated into 2011-
2016 Strategic Plan for
Preventing and Minimising
Gambling Harm

6. In 2012 preparation of a Baseline Report was
commissioned to be released end 2013

7. Intent of the Outcomes Reporting…is to help answer
questions such as
 How do we know the programme of work is reducing harm? Are we
measuring the outcomes adequately?
 What components are missing from the programme?
 Was resourcing adequate for each component of the programme and for the
programme as a whole?
 Was the intervention mix effective? Was it based on evidence? Does it
reflect the principles of the 1986 Ottawa Charter for Health Promotion?
 Do new interventions need to be developed to have a comprehensive range
of interventions available for each component of the programme?
 Which parts of the programme are working? Which parts of the programme
are not working? Do resources need to be refocused?

8. What was in the Baseline Report? A lot of detail
Achievement of goal: ‘Government, the gambling sector,
communities and families/whānau working together to
prevent and minimise gambling harm, and to reduce
related health inequities’
65 Outcomes
Indicators
10 Key Indicators
11 Objectives

9. What was in the Baseline Report? Complexity in layout and content

10. What was in the Baseline Report: Complexity

11. Evolution in systems thinking, from this 2007 Framework

12. To this framework - 2013

13. B+
Individual
Propensity towards
Gambling Activity
& Ease of Access
Participation in
Gambling
Activity
Experience of Harm
or benefit
to individual,
family/whanau /
significant others
(PGSI Score)
Enjoyment
benefit
Perceived
probability of
Financial benefit
Type of gambling
activity
Gambler’s Propensity
to Seek Support /
Treatment
Support /
Treatment
Provided
Gambling Addiction and Harm Loop
Treatment
Loop
B Balancing Loop / Positive or
Negative Feedback
R Reinforcing Loop / Positive or
Negative Feedback
- Variables move in opposite
direction to each other
+ Variables move in same direction
to each other
Delay / Damping effect
Variables outside of
considerationDirection of influence
Advertising / New
Product
enticement
Access to
attractive
services
Influencers of behaviour
and points for Public
Health Intervention
Potential for recidivism
Probability
of harm
To this in 2017: Dynamic Causal Loop Model of Cause and Effect
Click mouse to start animation
Presence of other Addiction
co-morbidity / Mood &
Anxiety Disorder
R+

14. R+
B+
Individual
Propensity towards
Gambling Activity
& Ease of Access
Participation
in Gambling
Activity
Experience of Harm
or benefit
to individual,
family/whanau /
significant others
(PGSI Score)
Enjoyment
benefitPerceived
probability of
Financial benefit
Type of gambling
activity
Gambler’s Propensity
to Seek Support /
Treatment
Support /
Treatment
Provided
Gambling Addiction and Harm Loop
Treatment
Loop
B Balancing Loop / Positive or
Negative Feedback
R Reinforcing Loop / Positive or
Negative Feedback
- Variables move in opposite
direction to each other
+ Variables move in same direction
to each other
Delay / Damping effect
Variables outside of
considerationDirection of influence
Advertising and
New Product
enticement
Access to
attractive
services
Influencers of behaviour
and points for Public
Health Intervention
Potential for recidivism
Probability
of harm
Breaking the Causal Loop – Points of Intervention at 2017
Financial
Capability
(Sorted Whānau)
HP Advertising
/ Regulation
Regulation
Education about
alternatives /
addictions
HP Education /
Advertising
Evidence based
customer focussed
service design –
Awareness raising
of services
Upskilling of
workforce
Promotion of
service standards
Click mouse to start
animation
Early & Clinical
Orientated
Interventions
Presence of other
Addiction co-morbidity /
Mood & Anxiety Disorder

15. 15
Our reflections on monitoring progress
• Impressive range of indicators - but not all were
measurable
• 65 Indicators is a lot of work, requiring a lot of resources
– do we really need that many indicators?
• Report 250 pages long, but no clear statement on
progress achieving goals of
• ‘preventing and minimising harm’
• ‘health inequities’ being reduced? (note report
talks about ‘inequalities’ – this is not the same
thing as ‘inequity’

16. 16
Our reflections on monitoring progress
• Baseline report has a lot of detail, which makes it quite
dense to read
• Drowning in data - is the level of detail presented
required?
• Is the focus right – have we reported outcomes in the
right way – i.e. do we know what progress has been
achieved? What has changed?
• Need to sharpen the Logic Flow – to understand what is
important and why – Move to Causal Loop Models?

17. Recommendations
Not
necessary to
report
everything
It is
‘necessary’
to have a
clear and
meaningful
logic
structure
Know:
who the
primary
audience is
what the
primary
purpose is
Tell the
‘story’
Involving key
stakeholders
is a good
idea:
Requires
leadership
Just because
you can, doesn’t
mean you
should
Reject the
mantra of ‘facts
speaking for
themselves’
An Outcomes report that
implements these reflections
is being developed now

18. Applying our reflections: One example. The Strategic Goal: Has there
been a reduction in harm since 2010?
Gambling participation has fallen…

19. …however problem gambling levels
relatively unchanged in recent years
The Strategic Goal: Has there been a reduction in harm since 2010?

20. Any
questions /
comments?

21. Supplementary Slides:
Examples Objective 1: Reducing Inequality / Inequity
NGS 2012-2015 Pattern of Harm

22. First strategic objective: What about inequality /inequity?
Inequality = observed different experience of gambling (e.g. participation,
harm, service utilisation) between population groups of interest (e.g.
Ethnicity, Gender, Social Deprivation)
Inequality may / may not = inequity
Inequity = observed level of harm experienced or services utilised by
population groups of interest are at levels different from that expected given
the level of health need or participation in harmful activity in the
population groups
Concept of ‘Inequity’ often associated with ‘Fairness’
Presence of an inequity may not indicate ‘unfairness’ – depends upon the cause
of the inequity and the size of the inequity
Disparity = observed substantive difference between population groups of
interest
Key terms:

23. First strategic objective: What about inequalities/inequity?
Gambling Inequalities Continue to Exist: Odds Ratios – HLS 2016
Ethnicity
Predictors of low risk
gambling
Predictors of moderate-
risk/problem gambling
Predictors of any level
of risky gambling
Predictor for
participants who agreed
on the statement of
someone close
gambling more than
intended
Māori 1.93* 4.7*** 2.61*** 2.84***
Pacific 3.54*** 2.4 3.21*** 1.96**
Asian 2.07 9.5* 3.24* 0.26**
European/Other Reference Reference Reference Reference
Ethnicity
Predictors of being
impacted by
someone else’s
gambling
Predictors of going
without because of
someone's gambling in
the household
Predictors for those
who contact gambling
problem services
Predictors for
respondents who have
seen/heard a gambling
harm advertisement
Māori 2.15*** 3.48*** 3.14** 0.99
Pacific 0.88 1.33 1.21 0.62**
Asian 0.7 0.63 0.31 0.31***
European/Other Reference Reference Reference Reference
Under-utilisation at a level indicating an inequity and
disparity given harm levels in top row
Substantive inequality between ethnic groups
* p < 0.05, ** p < 0.01, *** p < 0.001

24. Inequality: Class 4 venues by Area Deprivation
9.33% 9.18% 9.55% 12.32% 12.62%
38.34% 37.91% 38.30% 37.68% 37.95%
52.33% 52.91% 52.15% 50.00% 49.43%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Jan-12 Jan-13 Jan-14 Jan-15 Jan-16
Low (1-3) Mid (4-7) High (8-10)

25. First strategic objective: Reducing Inequality
• Some evidence of harm
reduction within ethnic group
overtime
• No evidence of relative reduction
in inequality over time between
ethnicities
• This result is consistent with
other NZ health experience
DRAFT

26. First strategic objective: Reducing Inequality
• Some evidence of harm
reduction within ethnic group
overtime
• No evidence of relative reduction
in inequality over time between
ethnicities
• This result is consistent with
other NZ health experience
DRAFT

27. NGS Pattern of Harm 2012-2015 Similar to HLS