Monitoring &
Evaluation
SOCIAL MARKETING
OCTOBER 2010
October 2010 2
Not everything that
can be counted
counts,
and not everything
that counts can be
counted.
ALBERT EINSTEIN
I...
Monitoring+
Evaluation=k
WHAT ARE WE
TALKING ABOUT?
3
Monitoring & evaluation: differences
Monitoring: conducted
as the initiative is implemented.
Evaluation: conducted
at the ...
Monitoring & evaluation answer Qs
Monitoring & Evaluation Plans
address these questions:
 Why are we doing this?
 What w...
Model of Effect
WHY DO WE USE MONIT+EVAL &
FOR WHAT ?
6
Model of Effect
Social marketing programs
are rarely as simple as:
‘We’ll tell them, they’ll learn
it, and they’ll change....
So the question is:
Is the program expected to
result in changes in…
1. individual behaviour?
2. social networks/norms?
3....
Model of Effect
Mapping out how the program is
expected to work is critical, because
it ensures that the right outcomes ar...
Model of Effect
Define how we expect change to occur.
Describe how specific product, price,
place and promotional interven...
What shall we
measure?
MEASUREMENT IN A NUTSHELL
11
What shall we measure?
There are 3 types of measures:
1. Outputs
2. Outcomes
3. Impacts
12October 2010
OUTPUTS
Outputs = quantity
Output/Process Measures:
 These are measures that quantify what we’ve done.
 Typically done by counti...
Outputs = frequencies/descriptives
Typical types:
 Numbers of materials distributed (how many)
 Reach and frequency (how...
OUTCOMES
Outcomes what measure
Relate directly to your goals and
objectives & measure things such as:
 Changes in Behavior
 Chang...
IMPACT
Impact measures
Impact measures the influence of the initiative on
health outcomes. Typically focus on outcomes like:
 Li...
How &
when
shall we
measure?
How & when will we measure?
 Quantitative:
 Surveys through telephone, mail and online, Intercept
interviews, Controlled...
Questions for you!
Which methods do you use for
Outputs?
Outcomes?
Impact?
WHEN will you measure WHAT using
WHICH METHO...
BUDGET
MONITORING & EVALUATION COSTS
23
Monitoring & evaluation Cost
Deciding what we need to do and
what we can do depends on budgets.
So, what can we do?
1st de...
LOGIC
EVALUATION MODELLED
25
Logic Model - 1
Displays the sequence of actions
that describe what the program is
and will do – how investments link
to r...
Logic model core components
1. INPUTS: resources, contributions, investments that go into
the program
2. OUTPUTS: activiti...
From the highly recommended site:
www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html#more
28October 2010
VERB It’s what you do
CASE STUDY
29
http://www.cdc.gov/youthcampaign/
Goals
 Increase knowledge and improve attitudes and beliefs
about tweens’ regular participation in physical activity.
 I...
VERB Logic Model
http://www.cdc.gov/youthcampaign/
31October 2010
VERB Channels – integrated
communication mix
Mass Media Adv Campaign
 Television commercials (Viacom, Disney, etc)
 Maga...
VERB Channels - cont’d
Public Relations
 Media relations
 Community involvement
 Policy (lobbying)
 Partnerships
Natio...
Results 2002-2003
 40% awareness goal for first year nearly doubled
 Approx. 75% of tweens surveyed aware
 Tweens under...
WEB 2.0 METRICS
FOCUS ON
35
Why is it appropriate?
Social media is about people and relationships,
something that is challenging to “measure”.
[…]
Wit...
Measuring social media success
The Trinity Approach (Kaushik, 2006),
as seen on Fichter (2008):
 Behavioral Data (how muc...
Social Media metrics
(cfr. Fichter & Wisniewski, 2009)
 Search Engines
 Focus on specific results
 Technorati/Google Bl...
LET’S GET
DOWN TO IT
HANDS ON SOCIAL MEDIA METRICS
39
October 2010 40
References
 CDC, Centers for Disease Control and Prevention. Social media tools for consumers and
partners - tools. Retri...
Useful Web 2.0 resources & tools
 Getclicky. Real Time web analytics: http://getclicky.com/
 Google Analytics: http://ww...
Upcoming SlideShare
Loading in...5
×

Social marketing monitoring & evaluation October 2010

726

Published on

Social marketing monitoring & evaluation, Social Marketing class, October 2010

Published in: Education, Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
726
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Social marketing monitoring & evaluation October 2010

  1. 1. Monitoring & Evaluation SOCIAL MARKETING OCTOBER 2010
  2. 2. October 2010 2 Not everything that can be counted counts, and not everything that counts can be counted. ALBERT EINSTEIN If you can not measure it, you can not improve it. LORD KELVIN
  3. 3. Monitoring+ Evaluation=k WHAT ARE WE TALKING ABOUT? 3
  4. 4. Monitoring & evaluation: differences Monitoring: conducted as the initiative is implemented. Evaluation: conducted at the end of the initiative. 4October 2010
  5. 5. Monitoring & evaluation answer Qs Monitoring & Evaluation Plans address these questions:  Why are we doing this?  What will we measure?  How will we measure it?  When will we measure it?  How much will it cost? 5October 2010
  6. 6. Model of Effect WHY DO WE USE MONIT+EVAL & FOR WHAT ? 6
  7. 7. Model of Effect Social marketing programs are rarely as simple as: ‘We’ll tell them, they’ll learn it, and they’ll change.’ 7October 2010
  8. 8. So the question is: Is the program expected to result in changes in… 1. individual behaviour? 2. social networks/norms? 3. institutions? 8October 2010
  9. 9. Model of Effect Mapping out how the program is expected to work is critical, because it ensures that the right outcomes are measured. DONER, 2003 9October 2010
  10. 10. Model of Effect Define how we expect change to occur. Describe how specific product, price, place and promotional interventions with priority audience segments will have an impact on given factors. 10October 2010
  11. 11. What shall we measure? MEASUREMENT IN A NUTSHELL 11
  12. 12. What shall we measure? There are 3 types of measures: 1. Outputs 2. Outcomes 3. Impacts 12October 2010
  13. 13. OUTPUTS
  14. 14. Outputs = quantity Output/Process Measures:  These are measures that quantify what we’ve done.  Typically done by counting or recording our activities. 14October 2010
  15. 15. Outputs = frequencies/descriptives Typical types:  Numbers of materials distributed (how many)  Reach and frequency (how many were exposed and how often)  Media coverage (did they pick up the story? How much space/time was allocated)  Impressions (how much would that have cost per person?)  Implementation (did we do everything? On time? On budget? )  Others? 15October 2010
  16. 16. OUTCOMES
  17. 17. Outcomes what measure Relate directly to your goals and objectives & measure things such as:  Changes in Behavior  Changes in Knowledge  Changes in Belief  Responses to Campaign Elements  Campaign Awareness  Customer Satisfaction Levels  Partnerships and Contributions Created  Policy Changes  Others? 17October 2010
  18. 18. IMPACT
  19. 19. Impact measures Impact measures the influence of the initiative on health outcomes. Typically focus on outcomes like:  Lives saved  Diseases prevented  Injuries avoided  Water quality improved  Air quality improved  Landfill reduced  Wildlife and habitats protected  Crimes prevented  Etc. 19October 2010
  20. 20. How & when shall we measure?
  21. 21. How & when will we measure?  Quantitative:  Surveys through telephone, mail and online, Intercept interviews, Controlled experiments, Content Analysis, Records, Databases, National Registries, Survey Centres  Qualitative:  Interviews, Focus Groups, Observational Research, Content Analysis 21October 2010
  22. 22. Questions for you! Which methods do you use for Outputs? Outcomes? Impact? WHEN will you measure WHAT using WHICH METHOD? 22October 2010
  23. 23. BUDGET MONITORING & EVALUATION COSTS 23
  24. 24. Monitoring & evaluation Cost Deciding what we need to do and what we can do depends on budgets. So, what can we do? 1st decide what we need to do 2nd determine if we can afford to do that  Does the benefit justify the cost?  If so, find the money  If not, revise your plan 24October 2010
  25. 25. LOGIC EVALUATION MODELLED 25
  26. 26. Logic Model - 1 Displays the sequence of actions that describe what the program is and will do – how investments link to results. 26October 2010
  27. 27. Logic model core components 1. INPUTS: resources, contributions, investments that go into the program 2. OUTPUTS: activities, services, events and products that reach people who participate or who are targeted 3. OUTCOMES: results or changes for individuals, groups, communities, organizations, communities, or systems 4. ASSUMPTIONS: the beliefs we have about the program, the people involved, and the context and the way we think the program will work 5. EXTERNAL FACTORS: the environment in which the program exists includes a variety of external factors that interact with and influence the program action. 27October 2010
  28. 28. From the highly recommended site: www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html#more 28October 2010
  29. 29. VERB It’s what you do CASE STUDY 29 http://www.cdc.gov/youthcampaign/
  30. 30. Goals  Increase knowledge and improve attitudes and beliefs about tweens’ regular participation in physical activity.  Increase parental and influencer support and encouragement of tweens’ participation in physical activity.  Heighten awareness of options and opportunities for tween participation in physical activity.  Facilitate opportunities for tweens to participate in regular physical activity.  Increase and maintain the number of tweens who regularly participate in physical activity. 30October 2010
  31. 31. VERB Logic Model http://www.cdc.gov/youthcampaign/ 31October 2010
  32. 32. VERB Channels – integrated communication mix Mass Media Adv Campaign  Television commercials (Viacom, Disney, etc)  Magazines, Newspapers  Large format print ads  Radio  Flyers, Handouts Internet Promotions 32October 2010
  33. 33. VERB Channels - cont’d Public Relations  Media relations  Community involvement  Policy (lobbying)  Partnerships National and Community Outreach  Schools  Student Groups  Recreation Centers 33October 2010
  34. 34. Results 2002-2003  40% awareness goal for first year nearly doubled  Approx. 75% of tweens surveyed aware  Tweens understand the message  Approx. 33% increase in tween activity http://pediatrics.aappublications.org/cgi/content/full/116/2/e277 34October 2010
  35. 35. WEB 2.0 METRICS FOCUS ON 35
  36. 36. Why is it appropriate? Social media is about people and relationships, something that is challenging to “measure”. […] With social media, qualitative and quantitative must be used hand in hand to tell the story. [...] The reality is that quantitative measures can only tell a small part of the social media story and qualitative measures are very important. FICHTER & WISNIEWSKI, 2009:56 36October 2010
  37. 37. Measuring social media success The Trinity Approach (Kaushik, 2006), as seen on Fichter (2008):  Behavioral Data (how much?) > Output  Outcome (so what?) > Outcome + Impact  Experience (why?) > Impact 37October 2010
  38. 38. Social Media metrics (cfr. Fichter & Wisniewski, 2009)  Search Engines  Focus on specific results  Technorati/Google Blogsearch/Liquida/OkNo/Wikio  Delicious  Twitter/Twitter Search  Google Trends/Twist  Social Mention  Web analytics  Google Analytics  Webalizer, Awstats, WP Stats, etc. 38October 2010
  39. 39. LET’S GET DOWN TO IT HANDS ON SOCIAL MEDIA METRICS 39
  40. 40. October 2010 40
  41. 41. References  CDC, Centers for Disease Control and Prevention. Social media tools for consumers and partners - tools. Retrieved 10/5/2009, 2009, from http://www.cdc.gov/SocialMedia/Tools/index.html  Doner, L. (2003). Approaches to evaluating social marketing programs. Social Marketing Quarterly, 9(3), 18‐26.  Fichter, D. (2008). Social media metrics: Making the case for making the effort. Online, 32(6), 54-57.  Fichter, D., & Wisniewski, J. (2009). Social media metrics: Tracking your impact. Online, 33(1), 54-57.  Kaushik, A. (2006). Trinity: A mindset & strategic approach. Retrieved 10/20/2009, 2009, from http://www.kaushik.net/avinash/2006/08/trinity-a-mindset-strategic-approach.html  Program Development and Evaluation Unit,University of Winsconsin. Evaluation logic model. Retrieved 10/20/2009, 2009, from http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html#more October 2010 41
  42. 42. Useful Web 2.0 resources & tools  Getclicky. Real Time web analytics: http://getclicky.com/  Google Analytics: http://www.google.com/analytics  Google Blogsearch: http://blogsearch.google.com/  Google Trends: http://www.google.com/trends  Google Documents (Forms): http://docs.google.com  Social Mention. Real-time social media search and analysis: http://www.socialmention.com/  Technorati blogsearch: http://technorati.com/search  Trendistic. See trends in Twitter: http://trendistic.com/  Twitter Search. See what's happening: http://search.twitter.com/  Survs.com: http://www.survs.com  Surveymonkey: http://www.surveymonkey.com October 2010 42

×