With Digital's increased importance in the marketing mix, brand leaders are naturally concerned with measuring and proving its efficacy. With Digital, there is no shortage of data to evaluate as part of this process. But all that data brings with it a new challenge – identifying which metrics we should care about, and how to best measure them. This paper examines the current state of digital measurement and opines for more scientific measurement approach utilizing a test and control methodology.

1. A Perspective for Retail Marketers
By Scott Eagle, CMO – Conversant®
MEASURING WHAT MATTERS

2. INTRODUCTION
A Growing Concern
New Capabilities and New Urgency
HOW THE INDUSTRY ASSESSES DIGITAL
MARKETING PERFORMANCE TODAY
The Metrics Chosen to Measure
The Motivations for Measurement
The Channels Measured (and Not Measured)
Allocating Credit Versus Measuring Impacts
THE MORE ACCURATE AND SCIENTIFIC ALTERNATIVE
Test and Control Incrementality Measurement
Our Test and Control Process
CONCLUSION
3
4
10
12
Table of Contents
2

3. As we all know, digital spending has grown incredibly rapidly.
eMarketer reports that in 2014 global digital spending will hit
$137.5B – just over a quarter of total marketing spend. With
its increased importance in the marketing mix, brand leaders
are naturally concerned with measuring and proving its
efficacy. With Digital, there is no shortage of data to evaluate
as part of this process. But all that data brings with it a new
challenge – identifying which metrics we should care about,
and how to best measure them.
Digital ad spending worldwide, 2012-2018
INTRODUCTION
Source: eMarketer, March 2014
A GROWING CONCERN
This topic is clearly a growing concern for marketing leaders,
and one that is contributing to the growth in marketing
analytics spending at many brands. According to the
February, 2014 CMO Survey, marketing analytics spending
is expected to rise 72% in the next three years.
Increased marketer
attention is leading
many brands to
reconsider why,
what and how they
measure. Years ago,
when digital marketing
measurement began in
earnest, measurement
was seriously limited by the available technology. Now,
there is growing recognition that some of our approaches to
assessing marketing performance remain rooted to the limited
measurement capabilities of the past.
Marketing analytics spending: share of
total marketing budgets
Marketing analytics
spending is expected
to rise 72% in the next
three years.
Source: The CMO Survey, February 2014
7.1%
12.2%
Current Levels Next 3 Years
Digital Ad Spending % of Total Media Ad Spending
$104B
20.8
23.2 25.3 27.0 28.4 $119B 29.8 31.1
$154B
$171B
$187B
$204B
$137B
2012 2013 2014 2015 2016 2017 2018
3

4. NEW CAPABILITIES
AND NEW URGENCY
But times have changed. Our industry can now focus on
scientifically validated measurement methodologies.
Methodologies that reveal the causal effects of digital
marketing on what most brands are really interested
in – total brand sales. More than that – not only can we
measure more scientifically – we must if we are to deliver
the sort of results that are expected of us now and in
the future. Nearly every marketing leader is under more
pressure than ever to drive additional impact with the
resources we are allocated.
This paper assesses the state of digital marketing
measurement today and then advocates for a scientifically
derived measurement approach that addresses its current
shortcomings. We’ll review:
• What’s wrong with how the industry measures
performance today?
• The more accurate and scientific alternative –
test and control incrementality measurement.
Not only can we measure more
scientifically – we must if we are to
deliver the sort of results that are
expected of us.
HOW THE INDUSTRY
ASSESSES DIGITAL
MARKETING
PERFORMANCE TODAY
The most common approaches to
marketing measurement today have the
following characteristics in common:
THE METRICS CHOSEN TO MEASURE
Many brands substitute surrogate metrics like clicks,
visits and purchase intent rather than focusing on the
overarching KPI – incremental sales effects.
THE MOTIVATIONS FOR MEASUREMENT
Many focus primarily on justifying past actions rather than
determining the best courses of action for future success.
THE CHANNELS MEASURED
(AND NOT MEASURED)
When they DO focus on actual sales metrics, brands
usually measure online sales only rather than calculating
sales impacts across all channels.
ALLOCATING CREDIT VERSUS
MEASURING IMPACTS
Most focus on crediting sales to tactics according to
predetermined percentages rather than determining which
sales were actually caused by each program.
Let’s examine each of these in turn.
4

5. Instead of focusing on true measures of performance against
KPIs, many brands have traditionally chosen to focus on
surrogate metrics. Clicks. Impressions. Page views. Interaction
times. Why?
• Because they are easy to measure. Years ago, when
more sophisticated measurement tools and approaches
were unavailable, most brands decided that these easy-to-
measure surrogates were the best we could do – and
that they provided at least some indication of marketing
effectiveness.
• Because brands have used these measures in the past.
Precedent likely drives some of the decision to measure
certain surrogate metrics like clicks.
• Because they lack the analytics personnel necessary
to improve their measurement approaches. Again,
according to the same CMO Survey, many marketers feel
that they don’t have the right people in place to maximize
the effectiveness of their analytics investments.
METRICS
Many brands substitute surrogate metrics like clicks, visits and purchase intent
rather than focusing on the overarching KPI – incremental sales effects.
9.0%
24.5%
18.4% 18.0%
15.9%
8.2%
6.1%
Does your company have the right talent to
fully leverage marketing analytics?
Source: The CMO Survey, February 2014
Does not have
the right talent
Has the
right talent
1 2 3 4 5 6 7
5

6. Many brands focus their measurement and research efforts on
providing rationale for past programs. In fact, a 2012 eConsultancy
study done in partnership with Google revealed that the most
important reason why marketers say they measure is to justify digital
spending.
Goals for attribution
“We did this and the results were that.”
While such measurement has a valuable role for marketing
accountability, I would argue that it plays an even
more important role in helping us to optimize program
performance in the future. This is a major distinction. When
the core objective of a performance measurement system is
to demonstrate the value of past performance, accuracy can
become subordinate to the desire for big numbers.
When our primary focus is instead on program optimization,
the need for accurate measurement becomes paramount.
When the core objective of a
performance measurement system
is to demonstrate the value of past
performance, accuracy can become
subordinate to the desire for
big numbers.
JUSTIFYING DIGITAL SPENDING
UNDERSTANDING FUNNEL
AND SALES CYCLE LENGTH
TO PLAN CAMPAIGNS
CREATE MOST EFFECTIVE MEDIA
MIX BASED ON TRUE VALUE
TOWARD CONVERSION
CORRECTLY DETERMINING
AFFILIATE PAYMENTS
High Priority Medium Priority Low Priority
57% 36% 7%
47% 42% 11%
62% 28% 10%
36% 39% 25%
Source: Google/eConsultancy, 2012
0% 20% 40% 60% 80% 100%
MOTIVATIONS
Many focus primarily on justifying past actions rather than
determining the best courses of action for future success.
6

7. The industry’s emphasis on justifying past
actions versus improving future results is evident
when we examine the attribution approaches
currently in use. According to a 2013 survey
of senior retail marketing leaders underwritten
by Conversant, the most popular approach to
measurement is last click.
Does your brand use the
following approach to digital
marketing attribution?
These survey results are disconcerting in light
of the fact that marketers view last click as the
least accurate method. See this data from the
eConsultancy/Google study:
Marketers’ assessment of the
effectiveness of attribution models
There are positive signs of change, however.
Another Conversant industry survey of both
agency- and brand-side professionals revealed
that just under a third planned to adopt more
accurate methodologies.
As more brands change their approaches, I
sincerely hope that they choose a methodology
that is truly better – one that doesn’t rely on
a rules-based or arbitrary allocation of credit
to particular marketing events. Many of the
attribution platforms currently getting attention
in the marketplace simply use a more complex
formula for arbitrarily allocating credit. And
greater complexity is no guarantee of greater
accuracy.
Percent of marketers/agency
leaders who plan to change
measurement approach in 2014
MOTIVATIONS (CONT.)
68% 680= LAST CLICK
17% 170= FIRST CLICK
43% 430= LAST VIEW
47% 470= MULTI-TOUCH
38% 380= INCREMENTALITY
Source: Conversant Survey, 2013
(Numbers add up to greater than 100% because many
companies use multiple attribution methods)
Source: Conversant Survey, 2014
Yes No Don’t Know
Very Effective
Somewhat Effective
Somewhat Ineffective
Very Ineffective
Source: Google/eConsultancy, 2012
CUSTOMIZED BY CHANNEL
UNIQUE METHODOLOGY
LINEAR
FIRST CLICK
LAST CLICK
0% 20% 40% 60% 80% 100%
43% 26%
32%
29% 49% 17% 5
27% 64% 9%
23% 69% 8%
14% 61% 25%
14% 55% 18% 13%
7

8. Most retail brands are only using online sales impact to evaluate and
measure digital marketing effectiveness. Again, the primary reason why
is that it is far easier to calculate the online sales that correlate with a
marketing program than the offline sales.
Measuring online sales
impact only is fine for
marketing programs
designed to drive only
online sales, like most
affiliate marketing. But for
omni-channel marketing
strategies and programs,
more comprehensive
performance measurement
is vital.
In most categories, offline sales dwarf online sales. So, by limiting their
measurement pool, brands artificially devalue the total impact of their
digital marketing programs. Because most methodologies currently
in use focus only on online results, total economic impact is often
significantly understated.
Most retail brands are
using only online sales
impact to evaluate
and measure digital
marketing effectiveness
34% 680= ELECTRONICS & APPLIANCES
22% 440= ENTERTAINMENT & LEISURE
14% 280= APPAREL & FOOTWEAR
9% 180= CONSUMER HEALTHCARE
8% 160= BEAUTY (MASS & LUXURY)
4% 80= PERSONAL CARE
3% 60= HOME & DIY
1% 20= GROCERY
US eCommerce sales, by category:
(2012, % of total)
Source: Booz & Company, 2012
CHANNELS
When they do focus on actual sales metrics, brands usually measure online
sales only rather than calculating sales impacts across all channels.
8

9. Many marketing programs allocate credit for all sales to persons
reached by marketing programs running at the time of the sale, even
when the marketing itself was superfluous to their decisions to buy.
The problem is that most online sales that are credited to a marketing
program would have occurred anyway, without the marketing activity.
Most? Really? Yes, really! Conversant conducted a massive study across
25 leading retailers in multiple classes of trade and found that 67 to 90
percent of sales that were credited to marketing exposure based upon
cookie-centric attribution methodologies were actually NOT caused by
that exposure. In other words, the shoppers that converted received
the marketing program message, but would’ve made the conversion
even if they hadn’t received the marketing message.
Many people use attribution and measurement as interchangeable
terms, but they aren’t the same. Measurement and analytics should
be about taking scientific approaches to determining the sales
impacts of a program. Attribution is focused on crediting a sale (or a
portion of a sale) based upon a model.
Cumulative Effects
So one might ask, if standard measurement under-credits in some
ways but over-credits in others, does it “all come out in the wash?”
In other words, does our flawed methodology actually deliver a
reasonable approximation of digital’s total economic impact?
Absolutely not. Again, extensive Conversant research demonstrates
that the flawed measurement approaches in use by many retailers
do not provide a reasonable approximation of the true sales effects
of marketing programs. In addition, these flawed approaches often
don’t even reveal a correct ranking of the value marketing tactics
bring to a given business.
Most online sales that are credited to a
marketing program would have occurred
anyway, without the marketing activity.
IMPACTS
Many retailers focus on crediting sales to tactics according to predetermined
percentages rather than determining which sales were actually caused by each.
9

10. Earlier in this paper we identified some major flaws in the most
commonly used measurement methodologies. I’d like to use
the balance of this paper to outline an alternative approach that
mitigates all of these problems.
Test and Control Incrementality Measurement
At Conversant, we believe in “incrementality” measurement and use a
scientifically validated test and control methodology to provide precision
performance measurement.
The hallmark of this approach is an emphasis on “incrementality.” In
other words, what incremental revenue did a program actually cause?
This means we don’t take credit for sales that would have happened if
a shopper wasn’t exposed to a particular campaign, or take credit for
sales effects caused by tactics other than Conversant’s programs.
To suss out incrementality, we conduct constant, scientifically based
A/B testing, using statistically matched samples. One group sees
the marketing messages in the Conversant program, one sees public
service announcements (PSAs) instead. By calculating the sales made
to each group, and then subtracting the PSA “control” cell sales from
the “test” cell sales, we get a precise measure of the sales that were
actually caused by the program. Furthermore, we track revenue across
both online and offline sales channels to ensure we get an accurate
view of the program’s total impact. By assessing incremental sales
impacts across all channels including online, retail, catalog, call center,
etc., we can provide a true indicator of a program’s performance.
Track revenue across both online and
offline sales channels to ensure you get an
accurate view of total impact.
THE MORE ACCURATE AND SCIENTIFIC ALTERNATIVE
Conversant uses a scientifically validated test and control
methodology to provide precision performance measurement.
10

11. To do all this, we follow a four-step process:
1 Step 1: Create test and control
groups. In order to identify the
incremental sales driven by a
campaign, we need to devise test and
control groups. These groups are carefully
developed using a scientifically validated
method that matches both groups across
a host of variables, including interests,
shopping behaviors, demographics,
marketing exposures and more.
2 Step 2: Show the test group the
campaign. Show the control
group PSAs. Conversant shows
the test group creative from your
campaign, while the control group sees
only public service announcements. Each
group is exposed to similar amounts of
other marketing communications. The
only difference in their brand experience
is whether or not they are exposed to the
Conversant campaign.
3 Step 3: Examine the online and
offline sales to each group.
The Conversant data platform
doesn’t track only your website
sales. It integrates with your POS systems,
as well as your catalog and call center
sales systems, to calculate total revenue
for both online and offline.
4 Step 4: Calculate the true
incremental sales. Once we have
the sales figures for each group,
we calculate incremental sales by
subtracting the control group sales from the
test group sales. Because revenue from the
control group reflects sales that would have
occurred naturally, this calculation reveals
the incremental sales that were truly driven
by your campaign. We also have the ability
to use the test/control methodology to
measure key brand attributes and the positive
or negative impacts of a given campaign.
It’s critical for marketers to understand these
effects as well.
Scientifically based incrementality
measurement not only measures the past
performance of marketing tactics. It also
provides the information necessary to
optimize current and future programs –
thereby driving incremental sales growth
over time.
OUR PROCESS
Scientifically based
incrementality
measurement provides
the information necessary
to optimize current and
future programs.
Our approach to
incrementality measurement
1: CREATE
2: EXPOSE
3: MEASURE
4: CALCULATE
Test
Test
Test
Control
Control
Control
PSA
Incremental Sales
11

12. CONCLUSION
Chances are that your brand can and should make significant improvements in the way that it measures marketing
effectiveness. Most brands know that this is something they need to do. But for many, fixing the problem tends to fall
low on the priorities list – because the task can seem daunting – and the current situation doesn’t seem “that bad.”
Make no mistake. It may well be “that bad.” Your role, and that of the marketing organization, depend on providing the
best results possible. Flawed measurement leads to bad decision-making. Some may believe that instinct combined with
existing flawed measurement data can drive great results. The reality is that flawed measurement is of little or no value
in decision making. The sooner that you correct the problem, the faster you’ll be able to deliver great results for your
brands.
I welcome your comments and ideas on this topic. Please write to me at: ScottEagle@conversantmedia.com
ABOUT THE AUTHOR
Scott Eagle leads Conversant’s global marketing function, including strategy and the integration of marketing
programs across our solutions groups. An accomplished senior executive with a strong background in client-side
digital marketing and consumer brand management, Mr. Eagle has over 25 years of experience as a marketing
leader managing major Fortune 500 brands and building successful new companies. Mr. Eagle has served as CMO
for Empowered Careers, eHarmony and Claria Corporation, and he has held management positions at Concentric
Network Corporation, MFS Communications and P&G. Mr. Eagle holds a B.S. in economics from The Wharton
School, University of Pennsylvania, and serves on the board of Akademos, Inc.
ABOUT CONVERSANT, INC.
Conversant, Inc. (Nasdaq: CNVR) is the leader in personalized digital marketing. Conversant helps the world’s biggest
companies grow by creating personalized experiences that deliver higher returns for brands and greater satisfaction
for people. We offer a fully integrated personalization platform, personalized media programs and the world’s largest
affiliate marketing network - all fueled by a deep understanding of what motivates people to engage, connect and buy.
For more information, please visit: www.conversantmedia.com
Copyright 2014 Conversant, Inc. All Rights Reserved.
12