1.
@philnottingham

2.
Boston, Massachusetts is
a lovely place, but it has a
number of problems.

3.
Chief among these
problems is…

4.
The mayor’s office had
a bright idea to fix this

5.
And it worked
incredibly well!

6.
Except there was one
problem…

7.
The app was used primarily by
Boston techies, who have a lot
more money than the majority
of the population

8.
And so the data suggested
that the potholes in the rich
parts of the city were causing
the most problems

9.
GARY KING, HARVARD UNIVERSITY
“This was an example of really bad analytics
and it’s even worse because it’s the kind of
thing that feels like it should work and does
work a little bit.”

10.
What they think they’re solving for
The most problematic potholes

11.
What they’re actually solving for
Potholes near rich people

12.
What they should be measuring
Potholes in areas with highest density of traffic

13.
When we don’t know how to
properly read and interpret
data, gathering it often causes
more problems that it solves

14.
I think this is an endemic
issue in digital marketing

15.
1 Valuing everything by ROI
3 Problems

16.
1 Valuing everything by ROI
2 Treating all metrics as KPIs
3 Problems

17.
1 Valuing everything by ROI
2 Treating all metrics as KPIs
3 Misaligning goals, channels and content
3 Problems

18.
1
Valuing everything by ROI

19.
(Gain from investment - Cost of investment)
Cost of investment
ROI =

20.
What you think you’re solving for
Efficiency & efficacy of spend

21.
So, you try to put a monetary
value on all activities

22.
The most common, and
egregious way of doing this is
with last touch attribution

23.
Session 1 Session 2 Session 3 Session 4
Last Touch Attribution

24.
When you use last-touch
attribution, this is how you
end up valuing the different
parts of your website…

25.
Product Support Blog Pricing Contact
Pretty
good!
Meh?
Rubbish
Great!
Amazing!!!

26.
MATTHEW ZOOK, UNIVERSITY OF KENTUCKY
“Once something gets measured and put into
a model then everything starts optimising
around that metric.”

27.
No-one in their right minds would read this
blog post and then buy a car insurance
policy, but when you value your content by
how it contributes to conversions… then you
end up trying to make that happen.

28.
A slightly better way to
measure value is a linear or
time decay model

29.
Session 1 Session 2 Session 3 Session 4
Linear Attribution

30.
What you think you’re solving for
Efficiency & efficacy of spend

31.
What you’re actually solving for
Conversions

32.
But this still has issues with
any activity designed for
brand building.

33.
Lets take the example of PR

34.
Earlier this year, my boss Chris
Savage was on BBC news

35.
Gut feel tells you that this is
really good PR coverage

36.
Here’s what the return looked
like in Google Analytics

37.
500 visits
0 Conversions
~$10,000

38.
Even with linear or even first
touch attribution, this looks like
incredibly poor return

39.
But… maybe we can measure
the value of this TV spot instead
as if it were a bit of TV coverage

40.
The way TV was historically
measured was via impressions
and how it contributed to brand
salience (as determined by
recall surveys)

41.
This was an ok way to measure
TV 30 years ago, when there
wasn’t many other marketing or
advertising channels available

42.
But today… there’s just too
much noise to make such data
meaningful. Brands who still
measure in this way
dramatically over value TV

43.
AN ALL TOO COMMON RESPONSE TO THIS PROBLEM
“Lets temporarily stop all other brand activities so
we can properly measure PR…”

44.
But we know that PR, or advertising,
or other brand building activity never
works in isolation. Stopping
everything necessarily reduces the
efficacy of the activity you’re trying
to measure.

45.
So we’re left with the conclusion
that…

46.
ROI is just a bad way to measure PR

47.
Think of it like a soccer team.

48.
#WistiaFest / @philnottingham

49.
“ROI” in a soccer analogy is the
number of goals scored.

50.
However… you wouldn’t measure
every player on how many goals they
score, or indeed even how they
contribute in terms of assists.

51.
Goal keepers are measured by saves,
defenders by tackles, Midfielders by
passes up the field, and strikers by
goals and assists.

52.
Yet you need a well rounded team,
with all the different players, to be
successful.

53.
If you measure everything by “ROI”,
you’re building a team of strikers.

54.
THE WORST CLIENTS OF ANY SEO AGENCY
“How much is this link worth?”

55.
Anyone in SEO who’s had this
question knows how problematic
and widespread this mentality is.

56.
What you think you’re solving for
Efficiency & efficacy of spend

57.
What you’re actually solving for
Conversions

58.
What you should be measuring
A sensible proxy for ‘value’

59.
Effectiveness
Sum (DA^2)
Efficiency
KPIs for PR
Sum (DA^2)
Cost

60.
2
Treating all metrics as KPIs

61.
Everything needs to go up. We need
to get more conversions, more
traffic, more links, lower bounce-
rate…longer time on page etc…

62.
What you think you’re solving for
Optimising breadth of value

63.
Here’s an example from a brand I
was working with recently.

64.
They appreciated the potential value
YouTube could bring to their
marketing, and so tried to optimise
their activity for all possible value

65.
Views
Retention
Shares
Subscribes
Click-Throughs
Conversions
TrueView Advertising
Shorten Video
Include Share CTA
Include Subscribe CTA
Include Link to Website
Track Click-Through Conversions

66.
What you think you’re solving for
Optimising breadth of value

67.
This created a frankenstein’s
monster of a video that had no value.

68.
What you’re actually solving for
Nothing

69.
What you should be measuring
One specific, qualified variable

70.
Here’s how we’ve tried to do this at
Wistia

71.
Effectiveness
View-through Conversions
Efficiency
KPIs for YouTube Advertising
View-through Conversions
Cost

72.
3
Misaligning goals, channels and content

73.
Here’s a strategy I was recently
handed by a global insurance
business

74.
Strategy
Industry Finance
Audience Millenials
Content Explainer Videos
Channel LinkedIn
Goal Sign-ups
Distribution Budget £1M

75.
What you think you’re solving for
Integrated Strategy

76.
Channel
ContentAudience

77.
The problem is that the strategy had
no overlap in the channel, the
audience or the content

78.
Strategy
Industry Finance
Audience Millenials
Content Explainer Videos
Channel LinkedIn
Goal Sign-ups
Distribution Budget £1M

79.
No amount of budget in the world
can make this strategy work.

80.
What you’re actually solving for
Discrete, unconnected and fragmented problems

81.
What you should be doing
Creating content on channels appropriate for
the goal and audience

82.
Goal
Audience
Channel
Content
Distribution
Metrics
KPI
What are you trying to achieve?
Who are you trying to reach?
Where can you reach them?
How can you meaningfully engage on this channel?
How can you get your content to the right people?
What can you accurately measure?
What’s a good proxy to measure the goal?
Strategy Questionnaire

83.
Here’s an example of how we
recently applied this plan to
Facebook video

84.
Goal
Audience
Channel
Content
Distribution
Metrics
KPI
Strategy Questionnaire
Brand Awareness
People working on video for social media
Facebook
Competition

85.
We created a test video to see if
there was any latent demand for this
kind of content.

86.
It worked!

87.
Goal
Audience
Channel
Content
Distribution
Metrics
KPI
Strategy Questionnaire
Brand Awareness
People working on video for social media
Facebook
Competition
Promoted Posts, Word of Mouth

88.
We then encouraged further
distribution with a bit of light
influencer marketing

89.
Goal
Audience
Channel
Content
Distribution
Metrics
KPI
Strategy Questionnaire
Brand Awareness
People working on video for social media
Facebook
Competition
Promoted Posts, Word of Mouth
Views, Audio, Retention, Likes, Comments, Shares
Page Likes

90.
Effectiveness
Page Likes
Efficiency
KPIs for Facebook Competitions
Page Likes
Cost

91.
This lead to a 30% increase in
overall page likes over the duration
of the campaign, at a cost of about
$2 per like.

92.
• Bad Data < No Data
• Measure everything, but always optimise for just one KPI
• Only use ROI for direct response & conversion activities
• Ensure alignment of content with channel and audience
• Tie everything back to the goal
Takeaways

93.
Thanks!
Please feel free to follow
me on the Twitters!
@philnottingham