Agile Measurement
by #IPASocialWorks
Part of the IPA‟s Agility Adaptathon
7th May 2014
Introduction
Stephen Maher
CEO of MBA
Chairman of #IPASocialWorks
Chairman of The Marketing Society
Today…
• Provocation – what‟s hot, what‟s not (Chris Macleod)
• Inspiration – the best new cases (Stephen Maher)
• Investigation – measuring not counting (Jenny Burns)
• Experimentation – the 5 point plan (James Devon)
• Break out groups (all)
• Process review & key learning (workshop hosts)
• The next 100 days challenge (Stephen Maher)
Provocation
Chris Macleod
Marketing Director, Transport for London
IPA Client Council
What‟s hot…
Social Media measurement is the hot topic
How can we make proper business decisions
regarding social media investment?
What are the frameworks for how to make social
media activity work really well?
A complex measurement ecosystem
Social “firehose”
Measurement
& Evaluation
Website
analytics
Implicit data
& marketing
automation
Explicit,
transactional
data
Qual
& Quant
What‟s not…
We‟re mainly counting, not measuring
Lots of „something definitely happening‟
but few really robust cases.
Failure to align objectives to business KPIs
and separate effects from other activity
Measuring Social ROI is less developed and highly complex
vs.
Social is clearly important and attracting increasing budgets.
There‟s lots of dynamic data. And no legacy
It‟s relative value is not yet fully understood.
What is #IPASocialWorks for?
Add to the “Treasure
house of learning”
Robust
methodologies for
using social and
measuring its impact
A detailed guide to the
various research
techniques
– a “how to”
To provide definitive guidance
as to the roles that social media can play &
how to measure its effectiveness and ROI
A uniquely collaborative approach
The process so far
Development of
hypotheses
Desk research /
Outreach
Peer Review
by effectiveness
experts
x2
> 130 cases
explored
> 35 depth
case interviews
> 25 cases
interrogated
Inspiration
Stephen Maher
Cases uncovered so far
“Social Idea”
ROMI 61
Footfall through
airport
“Social Idea”
ROMI 2.44
Market Mix Modelling
“Customer Service”
Reduced cost of service
“Customer Service”
A known link between
CSI scores and profit
“Customer Service”
Reduced cost to serve
and increased retention
Customer Service using Twitter
850k of 1.2m followers
for real-time information
24/7, teams of 10 in 3 shifts
2-3 mins response time
Reduced cost of service
vs other channels
74% satisfaction & positive influence
on TFL reputation
Customer Service using Twitter
Objective to improve service,
brand warmth and advocacy
£2m annual saving in costs avoided
by using social channels
“Net Easy” & increased retention
For Service (making it Easy…)
#IPASocialWorks
15th October 2013
40%difference in churn: ‘easy’ vs ‘difficult’
SurvivalRate
Dayspost interaction
Cases uncovered so far
“Social Idea”
ROMI 61
Footfall through
airport
“Social Idea”
ROMI 2.44
Market Mix Modelling
“Customer Service”
Reduced cost of service
“Customer Service”
A known link between
CSI scores and profit
“Customer Service”
Reduced cost to serve
and increased retention
“Social Idea”
„Like Loan‟ campaign
$NZ 4.5m from leads
“Social Idea”
+9% to 4.27m tourists
Highest ever “spend per tourist”
“Social Idea”
TV + Facebook had 66%
greater ROI
+9% sales YOY
The Crème Egg Challenge
Determined to
bake-in the social
measurement
Facebook not only delivered 21% unique reach among
16-24‟s, but for those exposed to TV + Facebook, we saw
a significant uplift in ROI.
Facebook support
can lead to improved
TV ROI
It‟s more fun in the Philippines
+9% to 4.27m tourists
Highest ever “spend per tourist”
ASB Bank “Like Loan”
A promising case study…
“€170m worth of leads from social”
Investigation
Jenny Burns
Global Head of Social Media,
Royal Sun Alliance
Experimentation
James Devon
Planning Director, MBA
IPA Value of Creativity Group
Plan of action
Introduce the
“5 point plan”
Group work on
live Client brief
Group feedback
Plan of action
Introduce the
“5 point plan”
Group work on
live Client brief
Group feedback
Objectives / KPIs
„58% place “Likes”, “Tweets”, “Clicks” and
“CTR” in their Top 5 Marketing ROI KPIs‟
Dissolve the
KPI solos
The brand‟s overall
marketing and
business goals
Audience, time-
periods etc.
Why Social?
“…but your scientists were so preoccupied
with whether or not they could,
they didn't stop to think if they should.”
Does it work?
vs
Does it work better than other things I could do?
Management & collection of data
Management & collection of data
Management & collection of data
What data will you need to test your hypothesis?
Where will you get it from?
Plan in advance!
Channel choice
Channel choice
Who are you trying to reach?
What is the role of each platform / channel?
How do you need to change your measurement
approach by channel?
How will you compare across channels
Evaluation Design
How does it link to other activity?
e.g. Search TV
How do you know? e.g. tagging activity /
MMM / Adding social to existing studies /
A/B testing
Soft metrics –
Likes/Followers/Shares/Intentions to
purchase/Image shifts/ User
sessions/Views on You
Tube/Positive mentions /
Hard metrics - Customer
Satisfaction / Customer Effort /
Likelihood to churn / Fully resolved
issues /Qualified Leads/ Sales/
Upsell /Call Deflection / Cost
avoidance/Search increases / Paid
Member increase/ New customers
How much / what material are you
sending out?
How much has been received / seen?
How many people saw it?
Who were they?
What were the consequences of this?
Plan of action
Introduce the
“5 point plan”
Group work on
live Client brief
Group feedback
The Groups
What you should do
Receive the Client brief
Use the 5 point plan to structure a discussion
Agree how to robustly measure the activity
Group leader to present to the room
The 5 point plan
Objectives / KPIs
Why Social?
Management & collection of data
Channel choice
Evaluation design
Plan of action
Introduce the
“5 point plan”
Group work on
live Client brief
Group feedback
The next 100 days
Stephen Maher
Cases uncovered so far
“Social Idea”
ROMI 61
Footfall through
airport
“Social Idea”
ROMI 2.44
Market Mix Modelling
“Customer Service”
Reduced cost of service
“Customer Service”
A known link between
CSI scores and profit
“Customer Service”
Reduced cost to serve
and increased retention
“Social Idea”
„Like Loan‟ campaign
$NZ 4.5m from leads
“Social Idea”
+9% to 4.27m tourists
Highest ever “spend per tourist”
“Social Idea”
TV + Facebook had 66%
greater ROI
+9% sales YOY
Cases uncovered so far
“Social Idea”
ROMI 61
Footfall through
airport
“Social Idea”
ROMI 2.44
Market Mix Modelling
“Customer Service”
Reduced cost of service
“Customer Service”
A known link between
CSI scores and profit
“Customer Service”
Reduced cost to serve
and increased retention
“Social Idea”
„Like Loan‟ campaign
$NZ 4.5m from leads
“Social Idea”
+9% to 4.27m tourists
Highest ever “spend per tourist”
“Social Idea”
TV + Facebook had 66%
greater ROI
+9% sales YOY
We‟d love to hear from you if…
…you have a robust case to discuss
@IPA_Updates, IPA Facebook or
LinkedIn “IPA Effectiveness” group
Contribute to the “how to” guide
(being curated by Ray Poynter)
http://newmr.org/ipaSocialworks/
100 Days
Think about how this approach
applies to your work
Provide feedback via the IPA‟s hub at
ipa.co.uk/adapt

IPA Agility Adaptathon - #IPASocialWorks Agile Social Media Measurement

  • 1.
    Agile Measurement by #IPASocialWorks Partof the IPA‟s Agility Adaptathon 7th May 2014
  • 2.
    Introduction Stephen Maher CEO ofMBA Chairman of #IPASocialWorks Chairman of The Marketing Society
  • 3.
    Today… • Provocation –what‟s hot, what‟s not (Chris Macleod) • Inspiration – the best new cases (Stephen Maher) • Investigation – measuring not counting (Jenny Burns) • Experimentation – the 5 point plan (James Devon) • Break out groups (all) • Process review & key learning (workshop hosts) • The next 100 days challenge (Stephen Maher)
  • 4.
    Provocation Chris Macleod Marketing Director,Transport for London IPA Client Council
  • 5.
    What‟s hot… Social Mediameasurement is the hot topic How can we make proper business decisions regarding social media investment? What are the frameworks for how to make social media activity work really well?
  • 6.
    A complex measurementecosystem Social “firehose” Measurement & Evaluation Website analytics Implicit data & marketing automation Explicit, transactional data Qual & Quant
  • 7.
    What‟s not… We‟re mainlycounting, not measuring Lots of „something definitely happening‟ but few really robust cases. Failure to align objectives to business KPIs and separate effects from other activity
  • 8.
    Measuring Social ROIis less developed and highly complex vs. Social is clearly important and attracting increasing budgets. There‟s lots of dynamic data. And no legacy It‟s relative value is not yet fully understood.
  • 9.
    What is #IPASocialWorksfor? Add to the “Treasure house of learning” Robust methodologies for using social and measuring its impact A detailed guide to the various research techniques – a “how to” To provide definitive guidance as to the roles that social media can play & how to measure its effectiveness and ROI
  • 10.
  • 11.
    The process sofar Development of hypotheses Desk research / Outreach Peer Review by effectiveness experts x2 > 130 cases explored > 35 depth case interviews > 25 cases interrogated
  • 12.
  • 13.
    Cases uncovered sofar “Social Idea” ROMI 61 Footfall through airport “Social Idea” ROMI 2.44 Market Mix Modelling “Customer Service” Reduced cost of service “Customer Service” A known link between CSI scores and profit “Customer Service” Reduced cost to serve and increased retention
  • 14.
    Customer Service usingTwitter 850k of 1.2m followers for real-time information 24/7, teams of 10 in 3 shifts 2-3 mins response time Reduced cost of service vs other channels 74% satisfaction & positive influence on TFL reputation
  • 15.
    Customer Service usingTwitter Objective to improve service, brand warmth and advocacy £2m annual saving in costs avoided by using social channels “Net Easy” & increased retention For Service (making it Easy…) #IPASocialWorks 15th October 2013 40%difference in churn: ‘easy’ vs ‘difficult’ SurvivalRate Dayspost interaction
  • 16.
    Cases uncovered sofar “Social Idea” ROMI 61 Footfall through airport “Social Idea” ROMI 2.44 Market Mix Modelling “Customer Service” Reduced cost of service “Customer Service” A known link between CSI scores and profit “Customer Service” Reduced cost to serve and increased retention “Social Idea” „Like Loan‟ campaign $NZ 4.5m from leads “Social Idea” +9% to 4.27m tourists Highest ever “spend per tourist” “Social Idea” TV + Facebook had 66% greater ROI +9% sales YOY
  • 17.
    The Crème EggChallenge
  • 18.
    Determined to bake-in thesocial measurement
  • 19.
    Facebook not onlydelivered 21% unique reach among 16-24‟s, but for those exposed to TV + Facebook, we saw a significant uplift in ROI. Facebook support can lead to improved TV ROI
  • 20.
    It‟s more funin the Philippines +9% to 4.27m tourists Highest ever “spend per tourist”
  • 21.
  • 22.
    A promising casestudy… “€170m worth of leads from social”
  • 23.
    Investigation Jenny Burns Global Headof Social Media, Royal Sun Alliance
  • 24.
    Experimentation James Devon Planning Director,MBA IPA Value of Creativity Group
  • 25.
    Plan of action Introducethe “5 point plan” Group work on live Client brief Group feedback
  • 26.
    Plan of action Introducethe “5 point plan” Group work on live Client brief Group feedback
  • 27.
    Objectives / KPIs „58%place “Likes”, “Tweets”, “Clicks” and “CTR” in their Top 5 Marketing ROI KPIs‟ Dissolve the KPI solos The brand‟s overall marketing and business goals Audience, time- periods etc.
  • 28.
    Why Social? “…but yourscientists were so preoccupied with whether or not they could, they didn't stop to think if they should.” Does it work? vs Does it work better than other things I could do?
  • 29.
  • 30.
  • 31.
    Management & collectionof data What data will you need to test your hypothesis? Where will you get it from? Plan in advance!
  • 32.
  • 33.
    Channel choice Who areyou trying to reach? What is the role of each platform / channel? How do you need to change your measurement approach by channel? How will you compare across channels
  • 34.
    Evaluation Design How doesit link to other activity? e.g. Search TV How do you know? e.g. tagging activity / MMM / Adding social to existing studies / A/B testing Soft metrics – Likes/Followers/Shares/Intentions to purchase/Image shifts/ User sessions/Views on You Tube/Positive mentions / Hard metrics - Customer Satisfaction / Customer Effort / Likelihood to churn / Fully resolved issues /Qualified Leads/ Sales/ Upsell /Call Deflection / Cost avoidance/Search increases / Paid Member increase/ New customers How much / what material are you sending out? How much has been received / seen? How many people saw it? Who were they? What were the consequences of this?
  • 35.
    Plan of action Introducethe “5 point plan” Group work on live Client brief Group feedback
  • 36.
  • 37.
    What you shoulddo Receive the Client brief Use the 5 point plan to structure a discussion Agree how to robustly measure the activity Group leader to present to the room
  • 38.
    The 5 pointplan Objectives / KPIs Why Social? Management & collection of data Channel choice Evaluation design
  • 39.
    Plan of action Introducethe “5 point plan” Group work on live Client brief Group feedback
  • 40.
    The next 100days Stephen Maher
  • 41.
    Cases uncovered sofar “Social Idea” ROMI 61 Footfall through airport “Social Idea” ROMI 2.44 Market Mix Modelling “Customer Service” Reduced cost of service “Customer Service” A known link between CSI scores and profit “Customer Service” Reduced cost to serve and increased retention “Social Idea” „Like Loan‟ campaign $NZ 4.5m from leads “Social Idea” +9% to 4.27m tourists Highest ever “spend per tourist” “Social Idea” TV + Facebook had 66% greater ROI +9% sales YOY
  • 42.
    Cases uncovered sofar “Social Idea” ROMI 61 Footfall through airport “Social Idea” ROMI 2.44 Market Mix Modelling “Customer Service” Reduced cost of service “Customer Service” A known link between CSI scores and profit “Customer Service” Reduced cost to serve and increased retention “Social Idea” „Like Loan‟ campaign $NZ 4.5m from leads “Social Idea” +9% to 4.27m tourists Highest ever “spend per tourist” “Social Idea” TV + Facebook had 66% greater ROI +9% sales YOY We‟d love to hear from you if… …you have a robust case to discuss @IPA_Updates, IPA Facebook or LinkedIn “IPA Effectiveness” group Contribute to the “how to” guide (being curated by Ray Poynter) http://newmr.org/ipaSocialworks/
  • 43.
    100 Days Think abouthow this approach applies to your work Provide feedback via the IPA‟s hub at ipa.co.uk/adapt