Presentation given by Teresa Jones at the Workshop exploring Qualitative and Mixed Methods in Research Evaluation and Policy 2015 (QMM2015)

1. Exploring stakeholders’
views in the context of
collaborative, public health
research:
a mixed methods
approach
Teresa Jones on behalf of
the EQUIPT consortium
Please treat this presentation as
confidential as the work is still in
progress.

2. Overview
• Summary
• Background to the EQUIPT project
Aims of the UK mixed methods study
• Methods used
• Results – Qualitative & Quantitative
• What would we have missed by not using a
mixed methods approach?
• Conclusions
08 October 2015 T Jones Brunel University London 2

3. Summary
EQUIPT - The development of a viable, practical, decision-support
aid for use by decision makers across five European countries and
investigation of its transferability beyond those countries. The
inclusion of stakeholders’ views is considered to be a very
important aspect of the project.
Mixed methods study – to explore the additional information
obtained by including both quantitative and qualitative findings
from the UK stakeholder interviews
Funded by FP7 from the European Commission

4. Background:
the EQUIPT project
• Development of a decision-support aid to
inform decisions on tobacco control spending
• For use initially in five European countries –
Germany, Hungary, Netherlands, Spain & UK
• To investigate the transferability of economic
evaluations beyond those five countries to
other Central and Eastern European countries
08 October 2015 4

5. Background:
aims of the mixed methods study
• To investigate the UK stakeholders’ needs for, and
views of, the proposed decision aid
• To conduct a quantitative analysis to provide the
overall picture of stakeholders’ perspectives and
also those of the UK stakeholders
• To collect qualitative data to help understand the
context for UK stakeholders
• Merging of results to provide enriched detail to
help understanding and enable creation of more
effective, country specific, bespoke decision aid.
08 October 2015 5

6. Method
08 October 2015 6
Quantitative & Qualitative data
collection via semi-structured face to
face interview
Quantitative data analysis Qualitative data analysis
Quantitative results Qualitative results
Merge results
Excel
SPSS
Coding
Excel
Interpretation
Double Data entry
Excel
Transcription of
audio files

7. Methods (1)
• Introduction of stakeholders to decision aid via
custom-built video
• Collection of views via questionnaire survey
• Countries: Germany, Hungary, Netherlands, Spain
& UK
• Stakeholders (purposive sample):
– decision makers;
– purchasers of services/pharma products;
– professionals/service deliverers;
– evidence generators;
– advocates of health promotion
08 October 2015 7

8. Method (2):
Questionnaire survey
• Specifically developed by EQUIPT team
members from all 5 countries
• Initially developed in English and then
translated into the language of each country
for the survey
• Conducted generally face to face, otherwise
by Skype or telephone
• Conducted by native speaker in each country
08 October 2015 8

9. Method (3):
Questionnaire survey(contd)
Question types
Quantitative
• 7-point Likert scale
(1=strongly disagree,
7=strongly agree)
08 October 2015 9
Survey method
• Interviewee’s place of
work
• Face to face
• Paper based
• Audio recorded
Qualitative
• Open questions at the
end of each section

10. Method (4):
Questionnaire survey (contd)
Questions included:
• Application of health economics concepts from a
decision-making point of view
• Needs assessment
• Risk perception
• Advantages & disadvantages of the decision aid
• Social support
• Self-efficacy
• Intention to use the decision aid
• Availability of smoking cessation interventions
08 October 2015 10

11. Method (5):
Data collection
Quantitative
• Total for 5 countries & the UK
• Double data-entry – Excel
• Analysis – SPSS
Qualitative
• The UK
• Transcription of UK audio recordings - Word
• Transfer of relevant text extracts - Excel
08 October 2015 11

12. Results (1):
Quantitative - Stakeholders
Total (5 countries) UK
Number of stakeholders 93 14
Role of stakeholders
- Decision makers 29 9
- Purchasers of
services/pharma
7 2
-Professional/service
providers
18 1
- Evidence generators 15 1
- Advocates of health
promotion
14 1
Intenders 81% 79%
08 October 2015 12

13. Results (2)
Q1a – Who would support you in using the Tobacco
ROI tool?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 5.26(1.98) 6.26(2.09)
Intenders: mean (SD) *5.53(0.92) 6.30(1.48)
Non-intenders (SD) *3.28(1.30) 6.08(1.08)
08 October 2015 13
*statistically significant difference

14. Results (2)
Q1b – Who else would support you in using the
Tobacco ROI tool?
Qualitative (open question)
08 October 2015 14
Government organisations:
• NHS;
• NICE;
• local authority directors of public
health;
• local politicians;
• local authority elected members;
• Local Government Association ;
health service mangers;
• health & social, care board
Research:
• Wider research community;
Voluntary/advocacy:
• advocacy organisations;
• ASH;
• voluntary sector;
• Smoke Free board
Other:
• some employer organisations eg
CBI, Federations of Small
Businesses;
• the public;
• an education setting;
• patient interest groups;
• patient client organisations

15. Results (3)
Q2a – I would encounter resistance using the
Tobacco ROI tool
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 2.93(2.08) 2.92(2.27)
Intenders: mean (SD) 2.85(2.06) 3.18(2.32)
Non-intenders (SD) 3.53(2.00) 1.50(0.71)
08 October 2015 15

16. Results (3)
Q2b – Who else would not support you in using
the Tobacco ROI tool?
Qualitative (open question)
08 October 2015 16
Government organisation:
• local government associations;
• NHS beyond public health;
• people with other health priorities
eg obesity or alcohol;
• those with commissioning
priorities;
• some council members concerned
about the nanny state;
• some public health staff who are
more qualitatively stronger;
• possibly too much based on
numbers and costs for the public
health arena
Others:
• Retailers;
• licensed traders;
• smokers

17. Results (4)
Q3a – How confident are you about using
the Tobacco ROI tool?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 5.28(1.98) 5.42(1.53)
Intenders: mean (SD) 5.33(0.80) 5.48(1.47)
Non-intenders (SD) 5.11(0.71) 5.19(1.47)
08 October 2015 17

18. Results (4)
Q3b– What other difficulties would using such a
tool have for you?
Qualitative (open question)
08 October 2015 18
Current model:
• details of how the modelling has
been put together;
• data entry;
• confidence in and access to the
service level data;
• mismatch of data;
• clarity about the underlying data
and processing;
Updating:
• certainty that the most up to
date data is used;
• inclusion of new intervention
effects;
• capacity for the inclusion of local
data;
• introduction of e-cigarettes;
Other:
• Time;
• relevance to my role;
• a contact;
• targeting of a subset of the
population

19. What would we have missed by not
using a mixed methods approach?
Quantitative only
• Specific details on support from
organisations/groups
• Some organisations have been listed as
providing support and also resistance
• Specific detail on concerns about data quality,
updating of the decision aid, etc.
08 October 2015 19

20. What would we have missed by not
using a mixed methods approach?
Qualitative only
• Opinions from a broader group
• Non-intenders (all 5 countries) had less support
• But no significant difference in the level of
resistance between intenders/non-intenders
• Overall and in the UK, stakeholders were
confident about using the decision aid
• No significant difference between intenders/non-
intenders
08 October 2015 20

21. Preliminary conclusions
Mixed methods analysis:
• Beneficial to understanding the variations in
contexts & needs of stakeholders within UK
• Provides comparisons of stakeholder views
across 5 countries
• Enables development of a more effective aid
• Provides valuable information for
transferability
08 October 2015 21

22. Next steps
• Further integration of UK data
• Potentially explore data for Germany, Hungary,
Netherlands and Spain using similar methods
• Combination of data from all 5 countries to
identify similarities and differences
• Use combined data to further inform
development of the decision aid
08 October 2015 22

23. Results (5a)
Q4 – The smoking epidemic is not severe in my
country?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 1.85(1.25) 2.50(1.65)
Intenders: mean (SD) *1.69(1.11) 2.18(1.41)
Non-intenders (SD) *2.75(1.61) 3.67(2.31)
08 October 2015 Presentation Title 23
*statistically significant difference

24. • The epidemic is severe for certain elements of
our community
• It is severe in terms of health impact
compared to other areas of health impact
• With 100,000 deaths/year, I’d call it severe
• It is much less severe than some other
countries
08 October 2015 Presentation Title 24
Results (5b)
Q4 – The smoking epidemic is not severe in my
country?
Qualitative (open question)

25. Results (6)
Q5a – Which of these advantages would the tool
have for you?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 5.62(1.61) 5.46(1.41)
Intenders: mean (SD) *5.80(0.85) 5.64(1.26)
Non-intenders (SD) *4.60(1.08) 4.81(1.75)
08 October 2015 Presentation Title 25
*statistically significant difference

26. Current aid:
• Powerful;
• tangible outputs;
• can tailor it;
• quick;
• possibility of consistent use
across national/regional/local
services;
• play around and model it to
see what you can get on a fixed
budget (previously not
available);
08 October 2015 Presentation Title 26
Results (6a)
Q5b – What other advantages, apart from those
listed, would the tool have for you?
Qualitative (open question)
• explicit;
• provision of data for input
into papers to enable me to
extend tobacco control work;
• useful for some communities
& subsections;
• external legitimacy for
investment;
• enable more work with
smokers within council

27. Suggested progressions:
• application for patients with eg
COPD;
• illustration of where short-term
returns might be made;
• looking at broader issues in
tobacco control;
08 October 2015 Presentation Title 27
Results (6b)
Q5b – What other advantages, apart from those
listed, would the tool have for you?
Qualitative (open question)
• continuous update with new
data on interventions/costs;
• potential modelling of
commissioning scenarios;
• use as a performance
management tool

28. Results (7)
Q6a – Which of these disadvantages would the
tool have for you??
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 2.90(1.73) 2.79(1.64)
Intenders: mean (SD) *2.74(1.07) 2.58(1.45)
Non-intenders (SD) *3.56(1.06) 3.81(1.72)
08 October 2015 Presentation Title 28
*statistically significant difference

29. Results (7a)
Q6b– What other disadvantages, apart from
those listed, would such a tool have for you??
Qualitative (open question)
08 October 2015 Presentation Title 29
Data:
• one must have faith in the
numbers put in and their
derivation;
• validity of the aid;
• bias due to evidence on
pharmaceutical interventions
being much stronger than eg
social/media type;
Short term benefits:
• LAs need to constantly consider
immediate social care issues;
• politicians interested in short
term returns (eg election
cycles);
• raise expectations of real cash
savings instantly;

30. Results (7b)
Q6b – What other disadvantages, apart from
those listed, would such a tool have for you??
Qualitative (open question)
08 October 2015 Presentation Title 30
Broader context:
• potential investment across
lifestyle areas not included;
• multiple outcomes beyond health
are not included;
• benefit is to the NHS but
commissioners are local
government therefore needs
health & social care cost impacts;
• examines only narrow part of
tobacco control;
Other:
• Compatibility with other
systems & management
framework;
• must be user friendly & easy to
use;
• returns are insufficient.