Evidence based advocacy

1. Evidence-based Advocacy
Daniel Ratchford, Quality Health
Zack Pemberton-Whitely, Leukaemia Care

2. Using qualitative and quantitative
data: how to structure your research
Daniel Ratchford
Quality Health – Chief Executive
daniel.ratchford@quality-health.co.uk or @danielratchford

3. Quality Health
A specialist health and social care research organisation, working for public, private and
not-for profit sectors, in the UK and overseas
 We design innovative data collection mechanisms to gather insight from patients, service
users, staff and other stakeholders
 We use our unique expertise and experience to collect, process, analyse and present
information in the most meaningful ways
 We work collaboratively with our clients to support service improvements, inform change,
influence policy, and improve outcomes
25/10/2019 3
We help our clients achieve high quality and positive outcomes
in health, social care and other services

4. Key steps in generating data for Patient Advocacy
Work out what the problem is
 E.g. Do you have policy ideas you need to back up with evidence?
 E.g. Do you want to test a theory about patient experience?
Consider the bigger picture
 What is already known?
 Is there other relevant research?
 Are other Patient Advocacy organisations planning anything similar?
Have a clear analysis strategy
 Make sure you know from the start how you will analyse the data
 What exactly do you want to show, and to whom?
 Will qualitative findings (e.g. stories) be more powerful than quantitative (e.g. numbers)?
25/10/2019 4
Be clear from the start how you want to use the data

5. Good data can help your organisation to…
Predict an outcome
 E.g. If patients’ care is changed, how will this affect their overall wellbeing?
Explain a cause or consequence
 E.g. What has caused patients to report a drop in the standards of care?
Evaluate policy
 E.g. what has been the impact on patients of a change in policy or a new procedure?
Describe service provision
 E.g. What does high quality care look like from a patient perspective?
Develop good practice
 E.g. What do patients say would improve their care?
Empower patients
 E.g. What are the best ways to enhance patients’ lives, and the lives of patients in the future?
25/10/2019 5
Work out what your problem is: don’t just run a survey
for the sake of it!

6. Some good examples of focused patient advocacy
research
European Cancer Patient Coalition
Cancer Associated Thrombosis Awareness Survey
Aims:
 To understand – and compare across countries – patients’ awareness of
CAT
 To map where, and when, patients get their information
 To identify gaps of information and support during the patient pathway
25/10/2019 6

7. Some good examples of focused patient advocacy
research
Acute Leukemia Advocates Network
ALAN Quality of Life Survey
Aims:
 To understand the quality of life of acute leukaemia patients at different
points in the patient journey
25/10/2019 7

8. Some good examples of focused patient advocacy
research
All.Can International
Survey of Waste and Inefficiency in Cancer Care
Aims:
 To gather the views of cancer patients and carers into areas of waste and
inefficiency in cancer care
 To gather examples of good practice, and areas for improvement
25/10/2019 8

9. Some good examples of focused patient advocacy
research
CML Advocates Network
CML Treatment Free Remission Survey
Aims:
 To understand the experiences of CML patients considering and
undertaking Treatment Free Remission
 To compare different patient groups
 To consider the impact of different interventions and communications in
this process
25/10/2019 9

10. Main types of research to generate data
Literature review and desk research
 Reviewing the main ideas and research relating to your area of interest
 Accessing existing relevant data sets
Quantitative research
 Questionnaires delivered on paper, on-line, in person or by phone
 Structured interviewing
Qualitative research
 In-depth interviews, in person or by phone
 Focus groups
 Workshops
25/10/2019 10
Many of the most successful research programmes
use a combination of the above

11. Main types of research to generate data – pros and cons
PROs
Literature review and desk research
 Well-respected academic approach
 Relatively cheap
 Doesn’t require external support
Quantitative research
 Best known and accepted research approach
 Gathers views from many patients
 Statistically significant results
Qualitative research
 Produces narrative rather than numbers
 Stories, quotes, and films can have more
impact in advocacy campaigns
25/10/2019 11
CONs
Literature review and desk research
 No new primary data
 Access to academic papers expensive
 Requires research skills
Quantitative research
 Can be expensive
 Sampling bias often an issue
 No depth behind the numbers
Qualitative research
 Expensive
 Usually only involves small numbers
 Difficult to argue it’s representative

12. Typical research outputs
 Data tables (from qualitative surveys)
 Written commentaries of findings
 Breakdowns
 Powerpoint presentations
 Infographics
 Thematic analyses of qualitative research
 Quotes from interviews
 Films of patient stories
 Improvement events
 Formal papers for scientific journals
 Press releases and advocacy campaigns
25/10/2019 12
You should be clear at the start about what you need,
and how it will be used

13. 25/10/2019 13
Data
Tables

14. 25/10/2019 14
Written commentaries of findings

15. 25/10/2019 15
Main worries after the end of treatment *
- by Gender
*Respondents could select all answer options that applied (tick all that apply)
Q7 / base = all respondents - What are the main worries you had after your treatment ended?
Males (n=450) Females (n=560)
Presentations: a breakdown

16. Infographics
25/10/2019 16

17. 25/10/2019 17
Thematic
analysis

18. 25/10/2019 18
Journal articles

19. 25/10/2019 19https://vimeo.com/292669223/4f1e532e76
Interviews

20. 25/10/2019 20https://www.youtube.com/watch?v=1pRtBtpPLTw
Films of patient stories

21. Some common problems…and how to avoid them
Not involving patients
 Coproduction of research design
 Patients and patient advocates on Steering Groups
 Patients involved in reviewing and testing survey materials
Not focusing on a specific issue - being too ambitious in scope
 Identify the theme/issue you’re interested in
 Keep it short
 Don’t just run a survey for the sake of it!
Not knowing what’s already been done
 Talk to the experts
 Carry out some form of literature review
25/10/2019 21

22. Some common problems…and how to avoid them
Not knowing what others are doing
 Network with other patient advocacy groups, in other countries - It’s likely
someone else is interested in the same issue
Lack of testing and piloting before fieldwork
 Don’t rush into implementation
 Test survey materials thoroughly
 Formal Cognitive Testing always identifies issues
Asking the wrong people
 Think about the sample of patients you want to hear from
 Design your research framework to generate a representative sample
 Don’t just listen to the same old voices
25/10/2019 22

23. Some common problems…and how to avoid them
Choosing the wrong delivery mechanism
 Using an external agency can be expensive and add complexity
 But carrying our research in-house risks lack of independence; and most
organisations don’t have the skills and capacity
 There will be times where both of these are right: consider carefully for each
piece of research
Not knowing how you’ll use the results
 Be clear at the start about what you need, and how it will be used
25/10/2019 23
Talk as much as possible to other Patient Advocacy
Groups about their experiences

25. How to CREATE evidence for
advocacy impact?
Zack Pemberton-Whiteley
Acute Leukemia Advocates Network – Chair
Leukaemia Care (UK) – Patient Advocacy Director
zackpw@leukaemiacare.org.uk or @ZPWLC

26. Methodology is key…
25/10/2019 26

27. Case Study: Living with Leukaemia
Leukaemia Care
Survey conducted in partnership with Quality Health

28. Living with Leukaemia: Project Aims
 The aim was to provide further evidence on UK
leukaemia patient experience
 Understanding the needs of patients with different
forms of leukaemia
 This project was designed as a follow up to NHS
England’s annual Cancer Patient Experience Survey
(CPES)
 Two iterations of the survey
• September 2016 to December 2016 – Published September 2017
• September 2017 to December 2017 – Published September 2018
25/10/2019 28

29. Living with Leukaemia: Methodology – Cohorts
Contacting patients with leukaemia who were
identified using the National Cancer Patient
Experience 2016 Survey data
25/10/2019 29
CPES
Leukaemia Care
Anonymous
Contacting leukaemia patients from Leukaemia
Care’s database. Those with consent to
contact, and who had a valid postal address,
but no email address recorded on the
Leukaemia Care database.
An anonymous online survey of the wider UK
blood cancer community. This arm also
included patients from the Leukaemia Care
database that Leukaemia Care had consent to
contact, and who had a valid email address.

30. Living with Leukaemia: Questionnaire Design
25/10/2019 30
Aims – Exploring UK leukaemia patient experience
Topics – Following the journey from diagnosis onwards
Testing – To refine the content
Comparability – between 2016 and 2017 versions

31. Living with Leukaemia: Outcome
25/10/2019 31
Response Rates – 2329 leukaemia patients
Publish the results!
Plans: Patient, Carer and Healthcare Professional

32. Case Study: Acute Leukemia Global
Quality of Life Survey
Acute Leukemia Advocates Network (ALAN)
Survey conducted in partnership with Quality Health

33. Global Quality of Life Survey (10+ Languages)
Aim: identify and measure Quality of Life issues, raise
awareness and change practice
Insights > Development > Publication
Timelines: Fieldwork from March 2019 to July 2019
ALAN Survey: Overview

34. Demographics: Including self-reported acute leukemia
diagnosis
HM-PRO
Goswami, Salek, Ionova, Oliva et al. HM-PRO: A Novel Patient-Reported Outcome Measure in
Hematological Malignancy for Use in Clinical Practice. Blood 2017 130:2176
Hypotheses Questions: Test against HM-PRO to assess how
they impact on quality of life
Follow Up Questions: To gather further information on patient
experience of each hypotheses
ALAN Survey: Methodology – Survey Layout

35. Physical symptoms
and side effects of
acute leukaemia and
treatment
Emotional impact of
acute leukaemia and
treatment
Physical and mental
health
Information to help
you understand and
manage your acute
leukaemia
Ability to perform
meaningful activities
(physical activities,
hobbies or
interactions with
friends, family and
the ‘outside world’)
Well-being and lives
of your carers,
friends or family
ALAN Survey: Methodology - Hypotheses
 Our hypothesis is that patients
reporting lower scores in each of
these areas will report a lower quality
of life score overall (measured using
the HM-PRO)
 The hypotheses were:
• designed in advance
• based on a literature search and
input from patient advocates and
clinicians
• questions designed to match
hypotheses and show statistical
significance of results

36. How to USE evidence for advocacy
impact?
Zack Pemberton-Whiteley
Acute Leukemia Advocates Network – Chair
Leukaemia Care (UK) – Patient Advocacy Director
zackpw@leukaemiacare.org.uk or @ZPWLC

37. Why use evidence? Credibility
25/10/2019 37

38. What to use?
Different evidence is needed for different situations
25/10/2019 38

39. How to use evidence…
25/10/2019 39
 Listen to the evidence
 Act on the evidence
 DO NOT use evidence just to back up
your existing point of view
“Follow the evidence wherever it
leads, and question everything”
Neil deGrasse Tyson

40. STOP: Does your evidence show what you think it does?
25/10/2019 40
Is your methodology correct?
 Did you ask the right question?
 Did you ask the right people?
 Did you ask enough people?
 Is your conclusion what the data
actually shows?
 Is it statistically valid?

41. Defining Unmet Needs
Remember to “Follow the evidence…”, USE it to…
 Identify: What are the issues to focus on
 Measure
Quantify issues (e.g. the % of patients experiencing)
Differences between particular groups. such as demographics,
cancer type, regions, time
 Solutions: Create a set of recommendations to address
 Further Questions - Is there anything you need more evidence
on? e.g. Understanding the different perspectives of patients,
carers and HCPs
25/10/2019 41

42. Organisational Strategy
25/10/2019 42
 How are you going to address the unmet needs? How can
your organisation work on the recommendations?
Questions to think about:
 What are you already addressing?
 What should you do differently?
 What do you need to do more of?
 What are you not doing?
 What can you NOT do? What should be worked on by
others?
Collect Further Evidence
 Use Key Performance Indicators (KPIs) – to measure
outcomes and impact
 Collect data and compare – e.g. performance across years

43. Developing New Services
One you have identified an unmet need to focus on…
 Designing the service – what exactly do you need to
address?
 Use evidence for your funding application
 Use feedback to refine existing services
• Use Key Performance Indicators (KPIs) – to
measure outcomes and impact
• Individual feedback from users (e.g. survey)
 Consider running a small PILOT to show it works?
25/10/2019 43
Peer to Peer
Support?
Hospital
Support
Workers?
Counselling?
Helpline?
WhatsApp?
Emotional Impact

44. Campaigns – Awareness and Early Diagnosis
Understand your audience
 Public v Patient – are you targeting a specific demographic?
 What does the audience already know?
Use Evidence
 Your own (e.g. survey)
 External (e.g. NCIN Routes to Diagnosis)
Consider your message
 What does the audience need to know?
 Where to focus? – e.g. leukaemia, blood cancer or cancer
Example: explaining how to #SpotLeukaemia
 Leukaemia is a cancer
 It affects people of all ages
 The most common symptoms experienced before diagnosis:
25/10/2019 44

45. Campaigns - Policy
Understand your audience
 Just 26% of MPs know ‘quite a lot’ or ‘a great deal’ about blood cancer.
(YouGov, 2018)
Use Evidence
 Tailor the evidence to your audience – e.g. regional
 It doesn’t have to be your own – e.g. incidence or prevalence
Consider your message
 What will have the impact with this audience?
 Where to focus? – e.g. leukaemia, blood cancer or cancer
Example: explaining why ‘Blood Cancer’ matters from their
perspective?
 The fifth most common type of adult cancer
 The most common cancer amongst children
 The third most fatal cancer
25/10/2019 45

46. Campaigns – Patient Issues
 Using evidence to develop and run campaigns to address
specific issues
 Audience – Patients? Clinical community?
 Evidence – Tailor to look at differences? Can you do a
specific survey? Or breakdown of an existing survey?
 Message – What are you trying to change?
Example: ‘Watch Wait Worry’ campaign
 We developed:
• Evidence Report – for clinicians
• Supportive guidance for CLL patients on watch and wait
• Social media campaign
 Ongoing – Poster at EHA on the emotional impact of watch
and wait for CLL
25/10/2019 46

47. Health Technology Appraisals (HTA)
25/10/2019 47
 Different HTA processes have different opportunities
for patient organisations to get involved
 In most cases the consideration of patient
perspective is qualitative, so it is difficult to
understand the impact on decision making
 You can create quantitative evidence (e.g. patient
surveys), but at present there is no mechanism for
the inclusion of such evidence
 Focus on:
• Influencing the areas that affect cost-
effectiveness
• Impact not involvement
• Explaining existing evidence (e.g. trial) and the
benefit from a patient perspective

48. Regulators – European Medicines Agency (EMA)
Example – Blinatumomab, MRD Activity
 Initial CHMP Negative Decision
 Don’t just follow the process!
 Explanation that:
• Realities of the setting – e.g. outcomes and unmet need
• Benefits of the treatment – e.g. from clinical trial
• Evidence of benefit - from the patient perspective
• Explanation of benefits not captured in the trials
 Outcome: Positive CHMP and EMA approval
25/10/2019 48