Presentation given by Teresa Jones at the 2015 Workshop Exploring Qualitative and Mixed Methods in Research Evaluation and Policy (QMM2015)

1. Exploring stakeholders’
views in the context of
collaborative, public health
research:
a mixed methods
approach
Teresa Jones on behalf of
the EQUIPT consortium
Please treat this presentation as
confidential as the work is still in
progress.

2. Overview
• Summary
• Background to the EQUIPT project
Aims of the UK mixed methods study
• Methods used
• Results – Qualitative & Quantitative
• What would we have missed by not using a
mixed methods approach?
• Conclusions
08 October 2015 T Jones Brunel University London 2

3. Summary
EQUIPT - The development of a viable, practical, decision-support
aid for use by decision makers across five European countries and
investigation of its transferability beyond those countries. The
inclusion of stakeholders’ views is considered to be a very
important aspect of the project.
Mixed methods study – to explore the additional information
obtained by including both quantitative and qualitative findings
from the UK stakeholder interviews
Funding: We have received funding from the European Community’s
Seventh Framework Programme under grant agreement No. 602270
(EQUIPT)
Ethics approval was received from Brunel University London Research
Ethics Committee

4. Background:
the EQUIPT project
• Development of a decision-support aid to
inform decisions on tobacco control spending
• For use initially in five European countries –
Germany, Hungary, Netherlands, Spain & UK
• To investigate the transferability of economic
evaluations beyond those five countries to
other Central and Eastern European countries
08 October 2015 4

5. Background:
aims of the mixed methods study
• To investigate the UK stakeholders’ needs for, and
views of, the proposed decision aid
• To conduct a quantitative analysis to provide the
overall picture of stakeholders’ perspectives and
also those of the UK stakeholders
• To collect qualitative data to help understand the
context for UK stakeholders
• Merging of results to provide enriched detail to
help understanding and enable creation of more
effective, country specific, bespoke decision aid.
08 October 2015 5

6. Method
08 October 2015 6
Quantitative & Qualitative data
collection via semi-structured face to
face interview
Quantitative data analysis Qualitative data analysis
Quantitative results Qualitative results
Merge results
Excel
SPSS
Coding
Excel
Interpretation
Double Data entry
Excel
Transcription of
audio files

7. Methods (1)
• Introduction of stakeholders to decision aid via
custom-built video
• Collection of views via questionnaire survey
• Countries: Germany, Hungary, Netherlands, Spain
& UK
• Stakeholders (purposive sample):
– decision makers;
– purchasers of services/pharma products;
– professionals/service deliverers;
– evidence generators;
– advocates of health promotion
08 October 2015 7

8. Method (2):
Questionnaire survey
• Specifically developed by EQUIPT team
members from all 5 countries
• Initially developed in English and then
translated into the language of each country
for the survey
• Conducted generally face to face, otherwise
by Skype or telephone
• Conducted by native speaker in each country
08 October 2015 8

9. Method (3):
Questionnaire survey(contd)
Question types
Quantitative
• 7-point Likert scale
(1=strongly disagree,
7=strongly agree)
08 October 2015 9
Survey method
• Interviewee’s place of
work
• Face to face
• Paper based
• Audio recorded
Qualitative
• Open questions at the
end of each section

10. Method (4):
Questionnaire survey (contd)
Questions included:
• Basic knowledge in health economics
• Needs assessment
• Risk perception
• Advantages & disadvantages of the decision aid
• Social support
• Self-efficacy
• Intention to use the decision aid
• Availability of smoking cessation interventions
08 October 2015 10

11. Method (5):
Data collection
Quantitative
• Total for 5 countries & the UK
• Double data-entry – Excel
• Analysis – SPSS
Qualitative
• The UK
• Transcription of UK audio recordings - Word
• Transfer of relevant text extracts - Excel
08 October 2015 11

12. Results (1):
Quantitative - Stakeholders
Total (5 countries) UK
Number of stakeholders 93 14
Role of stakeholders
- Decision makers 29 9
- Purchasers of
services/pharma
7 2
-Professional/service
providers
18 1
- Evidence generators 15 1
- Advocates of health
promotion
14 1
Intenders 81% 79%
08 October 2015 12
EQUIPT WP1 stakeholder interview data

13. Results (2)
Q1a – Who would support you in using the Tobacco
ROI tool?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 5.26(1.98) 6.26(2.09)
Intenders: mean (SD) *5.53(0.92) 6.30(1.48)
Non-intenders (SD) *3.28(1.30) 6.08(1.08)
08 October 2015 13
*statistically significant difference
EQUIPT WP1 stakeholder interview data

14. Results (2)
Q1b – Who else would support you in using the
Tobacco ROI tool?
Qualitative (open question)
08 October 2015 14
Government organisations:
• NHS;
• NICE;
• local authority directors of public
health;
• local politicians;
• local authority elected members;
• Local Government Association ;
health service mangers;
• health & social, care board
Research:
• Wider research community;
Voluntary/advocacy:
• advocacy organisations;
• ASH;
• voluntary sector;
• Smoke Free board
Other:
• some employer organisations eg
CBI, Federations of Small
Businesses;
• the public;
• an education setting;
• patient interest groups;
• patient client organisations

15. Results (3)
Q2a – I would encounter resistance using the
Tobacco ROI tool
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 2.93(2.08) 2.92(2.27)
Intenders: mean (SD) 2.85(2.06) 3.18(2.32)
Non-intenders (SD) 3.53(2.00) 1.50(0.71)
08 October 2015 15
EQUIPT WP1 stakeholder interview data

16. Results (3)
Q2b – Who else would not support you in using
the Tobacco ROI tool?
Qualitative (open question)
08 October 2015 16
Government organisation:
• local government associations;
• NHS beyond public health;
• people with other health priorities
eg obesity or alcohol;
• those with commissioning
priorities;
• some council members concerned
about the nanny state;
• some public health staff who are
more qualitatively stronger;
• possibly too much based on
numbers and costs for the public
health arena
Others:
• Retailers;
• licensed traders;
• smokers

17. Results (4)
Q3a – How confident are you about using
the Tobacco ROI tool?
Quantitative (7-point Likert scale)
Total (5 countries) UK
Overall score: mean(SD) 5.28(1.98) 5.42(1.53)
Intenders: mean (SD) 5.33(0.80) 5.48(1.47)
Non-intenders (SD) 5.11(0.71) 5.19(1.47)
08 October 2015 17
EQUIPT WP1 stakeholder interview data

18. Results (4)
Q3b– What other difficulties would using such a
tool have for you?
Qualitative (open question)
08 October 2015 18
Current model:
• details of how the modelling has
been put together;
• data entry;
• confidence in and access to the
service level data;
• mismatch of data;
• clarity about the underlying data
and processing;
Updating:
• certainty that the most up to
date data is used;
• inclusion of new intervention
effects;
• capacity for the inclusion of local
data;
• introduction of e-cigarettes;
Other:
• Time;
• relevance to my role;
• a contact;
• targeting of a subset of the
population

19. What would we have missed by not
using a mixed methods approach?
Quantitative only
• Specific details on support from
organisations/groups
• Some organisations have been listed as
providing support and also resistance
• Specific detail on concerns about data quality,
updating of the decision aid, etc.
08 October 2015 19

20. What would we have missed by not
using a mixed methods approach?
Qualitative only
• Opinions from a broader group
• Non-intenders (all 5 countries) had less support
• But no significant difference in the level of
resistance between intenders/non-intenders
• Overall and in the UK, stakeholders were
confident about using the decision aid
• No significant difference between intenders/non-
intenders
08 October 2015 20

21. Preliminary conclusions
Mixed methods analysis:
• Beneficial to understanding the variations in
contexts & needs of stakeholders within UK
• Provides comparisons of stakeholder views
across 5 countries
• Enables development of a more effective aid
• Provides valuable information for
transferability
08 October 2015 21

22. Next steps
• Further integration of UK data
• Potentially explore data for Germany, Hungary,
Netherlands and Spain using similar methods
• Combination of data from all 5 countries to
identify similarities and differences
• Use combined data to further inform
development of the decision aid
08 October 2015 22