Evaluation case study
August 2021
Dr Glenn O’Neil
2
There are currently 28 million
smokers aged 24-35 in
28 Member States of the EU
In 2011, the EU launched a
€ 33 million EU wide anti-smoking
campaign targeting young people
across Europe for 3 years
I was part of the evaluation team
that evaluated the campaign in 2015
Background
3
Campaign approach
“Ex-smokers are unstoppable” campaign
emphasized the benefits of becoming an ex-
smoker; it was not a prevention campaign
The campaign focused less on negative
“shock” tactics used in anti-smoking
campaigns
4
5
1. Highlight the benefits of giving up smoking
and encourage and assist citizens to help
them quit;
2. Raise awareness of the dangers of tobacco;
3. Contribute to ensuring that new generations
live longer and healthier lives in a smoke-
free Europe
Campaign objectives
6
Activities
7
What should we evaluate?
How should we evaluate?
8
What should we evaluate?
Did the campaign reach its
objectives?
Effectiveness
Reach Awareness Attitudes Action Change
9
What should we evaluate?
What worked well?
Efficiency
Cost-efficiency Activities Future campaigns
10
Why should we evaluate?
Prove and improve
11
How should we evaluate?
Monitoring data:
web, media,
social media,
iCoach
Stakeholder
surveys and
interviews
Focus groups
and surveys of
smokers and
ex-smokers
iCoach users
interviews
Analysis of
campaign
financial reports
Evaluation matrix
KPI Methods Sources
To what extent
did the campaign
achieve its
objectives?
Evaluation question
No. of smokers
reached
No. of smokers
aware of
campaign
No. of Smokers
registering on
iCoach
No. of registered
users stopping
smoking
Which campaign
channel was the
most cost-efficient?
Cost of user
reached per
channel
Financial analysis Campaign
financial reports
and reach data
Media monitoring
Surveys
Interviews
iCoach monitoring
data
Media / social
media
Smokers
iCoach users
Etc. Etc. Etc. Etc.
13
Key findings – reach to action
19 million smokers
reached
450,000 signed
up to iCoach
65,000 reported
quitting smoking
69% 1.6% 0.2%
It cost €500 for each smoker who quit
14
Key findings – role of campaign
19 million smokers
reached
450,000 signed
up to iCoach
65,000 reported
quitting smoking
Role of:
• Nicotine
replacements
• E-cigarettes
• Self-help literature
• Already decided to
stop smoking
• Only one-third of
smokers seek
support or
assistance to stop
Role of:
• Influence of family
and friends
• Cost of smoking
15
Advantages of stopping
smoking
Financial benefits from
stopping smoking
Better physical shape from
stopping smoking
Better life from stopping
smoking
Positive messaging was
appreciated by audiences
Key findings - Messaging
Most successful with Southern
Europe, men, those already
decided to stop smoking
16
Key findings – cost per channel
Print
Digital
TV
€ 8 million
€ 7.4 million
€ 2.3 million
5.4 million
43.7 million
6.3 million
€ 1.49
€ 0.17
€ 0.37
Cost
No. of
smokers
25-34 years
reached
Cost per
smoker
reached
17
1. Highlight the benefits of giving up smoking
and encourage and assist citizens to help
them quit;
2. Raise awareness of the dangers of tobacco;
3. Contribute to ensuring that new generations
live longer and healthier lives in a smoke-
free Europe
Campaign objectives – achieved?
18
Recommendations
What would you recommend based
on these findings for future EU
campaigns on smoking and other
health issues?
19
Recommendations
1. Repeat and reuse the campaign concept,
messaging and tools
2. Adopt a more targeted approach:
Southern and Eastern European countries
and women
3. Use a mix of digital and TV channels
4. Improve objective setting and
measurement
5. Build evaluation into future campaigns
from the start
20
Conclusions
• All communicators have a role in
monitoring and evaluation
• Thinking about evaluation before you start
helps you map out activities to change
desired
• Showing the results of communications is
feasible
• Value is focusing on the outcome level –
what did your communications change?
21
Q&A
Report & contact details
oneil@owlre.com
glennoneil
www.owlre.com
@glenn_oneil
www.intelligentmeasurement. net
Contacts:
http://publications.europa.eu/resource/cellar
/4c008831-cef9-11e5-a4b5-
01aa75ed71a1.0001.01/DOC_1

Case study on evaluating a smoking cessation campaign

  • 1.
    Evaluation case study August2021 Dr Glenn O’Neil
  • 2.
    2 There are currently28 million smokers aged 24-35 in 28 Member States of the EU In 2011, the EU launched a € 33 million EU wide anti-smoking campaign targeting young people across Europe for 3 years I was part of the evaluation team that evaluated the campaign in 2015 Background
  • 3.
    3 Campaign approach “Ex-smokers areunstoppable” campaign emphasized the benefits of becoming an ex- smoker; it was not a prevention campaign The campaign focused less on negative “shock” tactics used in anti-smoking campaigns
  • 4.
  • 5.
    5 1. Highlight thebenefits of giving up smoking and encourage and assist citizens to help them quit; 2. Raise awareness of the dangers of tobacco; 3. Contribute to ensuring that new generations live longer and healthier lives in a smoke- free Europe Campaign objectives
  • 6.
  • 7.
    7 What should weevaluate? How should we evaluate?
  • 8.
    8 What should weevaluate? Did the campaign reach its objectives? Effectiveness Reach Awareness Attitudes Action Change
  • 9.
    9 What should weevaluate? What worked well? Efficiency Cost-efficiency Activities Future campaigns
  • 10.
    10 Why should weevaluate? Prove and improve
  • 11.
    11 How should weevaluate? Monitoring data: web, media, social media, iCoach Stakeholder surveys and interviews Focus groups and surveys of smokers and ex-smokers iCoach users interviews Analysis of campaign financial reports
  • 12.
    Evaluation matrix KPI MethodsSources To what extent did the campaign achieve its objectives? Evaluation question No. of smokers reached No. of smokers aware of campaign No. of Smokers registering on iCoach No. of registered users stopping smoking Which campaign channel was the most cost-efficient? Cost of user reached per channel Financial analysis Campaign financial reports and reach data Media monitoring Surveys Interviews iCoach monitoring data Media / social media Smokers iCoach users Etc. Etc. Etc. Etc.
  • 13.
    13 Key findings –reach to action 19 million smokers reached 450,000 signed up to iCoach 65,000 reported quitting smoking 69% 1.6% 0.2% It cost €500 for each smoker who quit
  • 14.
    14 Key findings –role of campaign 19 million smokers reached 450,000 signed up to iCoach 65,000 reported quitting smoking Role of: • Nicotine replacements • E-cigarettes • Self-help literature • Already decided to stop smoking • Only one-third of smokers seek support or assistance to stop Role of: • Influence of family and friends • Cost of smoking
  • 15.
    15 Advantages of stopping smoking Financialbenefits from stopping smoking Better physical shape from stopping smoking Better life from stopping smoking Positive messaging was appreciated by audiences Key findings - Messaging Most successful with Southern Europe, men, those already decided to stop smoking
  • 16.
    16 Key findings –cost per channel Print Digital TV € 8 million € 7.4 million € 2.3 million 5.4 million 43.7 million 6.3 million € 1.49 € 0.17 € 0.37 Cost No. of smokers 25-34 years reached Cost per smoker reached
  • 17.
    17 1. Highlight thebenefits of giving up smoking and encourage and assist citizens to help them quit; 2. Raise awareness of the dangers of tobacco; 3. Contribute to ensuring that new generations live longer and healthier lives in a smoke- free Europe Campaign objectives – achieved?
  • 18.
    18 Recommendations What would yourecommend based on these findings for future EU campaigns on smoking and other health issues?
  • 19.
    19 Recommendations 1. Repeat andreuse the campaign concept, messaging and tools 2. Adopt a more targeted approach: Southern and Eastern European countries and women 3. Use a mix of digital and TV channels 4. Improve objective setting and measurement 5. Build evaluation into future campaigns from the start
  • 20.
    20 Conclusions • All communicatorshave a role in monitoring and evaluation • Thinking about evaluation before you start helps you map out activities to change desired • Showing the results of communications is feasible • Value is focusing on the outcome level – what did your communications change?
  • 21.
  • 22.
    Report & contactdetails oneil@owlre.com glennoneil www.owlre.com @glenn_oneil www.intelligentmeasurement. net Contacts: http://publications.europa.eu/resource/cellar /4c008831-cef9-11e5-a4b5- 01aa75ed71a1.0001.01/DOC_1