This research paper aims to challenge common beliefs about the value of online display ad clicks. The paper analyzes data from $11.5 billion in online sales tracked by Criteo to show that: 1) People who click on ads buy 3 times more frequently than non-clickers. 2) Almost half of regular online buyers click on Criteo ads. 3) A small number of users are responsible for most clicks and sales, following an exponential distribution. The paper concludes that properly targeted, relevant ads can generate valuable clicks that indicate genuine purchase intent, contrary to previous research based on untargeted banner ads.

1. RESEARCH PAPER # 1
MEASURING
THE VALUE OF USERS
WHO CLICK ON
ONLINE DISPLAY ADS
Patrick Wyatt
Head of Business Intelligence, Criteo
June 2012

2. ABSTRACT
A common belief persists among marketers: very few people click on online display ads, and those who do are
not the kind of people worth advertising to anyway. This leads to the conclusion that the efficiency of display
campaigns should be measured not with clicks, but rather by whether an ad was displayed (so-called “view-
through”).
This Criteo research paper aims to show that the bias against clicks is misguided. We have reviewed over $10B
of e-commerce sales and have learnt that the audience who clicks on timely, relevant ads is extremely valuable.
1 INTRODUCTION
A much-cited comScore research paper from 2008 entitled How Online Advertising Works: Whither the
Click? found that a small subset of people – less than 10% of all internet browsers – were responsible for more
than 80% of all clicks. Most worryingly, it produced evidence to suggest that the demographics of clickers are
skewed towards younger users aged 25 to 44 earning less than $40,000 per year. comScore concluded that
clickers were “…hardly an attractive target segment for most advertisers” 1. Taken at face value, these findings
cast significant doubt on the value of clicks and clickers. This study has been so often repeated that it has become
axiomatic within the world of online advertising.
But it is often forgotten that the original intent of that study was to discover whether clicks in a brand context
were meaningful, and that the study is based on untargeted, non-personalized banner ads. At some point, it
became accepted that clicks in any online context are not valuable.
Normally, marketers go to great lengths to get responses from customers, using the marketing channels in which
they invest. So before writing off the click, it is worth taking a second look. What we found was surprising and
does much to counteract some of the myths that surround click-based advertising.
2 DATASET
Criteo collects purchase data from the sites with which it partners in order to provide personalized display adver-
tising. For each purchase that occurs on one of our partners’ sites, we record the items bought and their prices,
and link this to an anonymous unique identifier stored in a cookie. This provides Criteo with what may be the
internet’s biggest structured database of e-commerce transactions. In 2011, Criteo recorded nearly $100BN of
online transactions, an amount that is larger than the e-Commerce revenues of either Amazon or eBay.
For this study, we looked at internet browsers exposed to a Criteo retargeting ad during the first seven days of
March 2012. This population numbered 147 million unique browsers.
We then split this population into two – those who had clicked on a Criteo banner in the previous six months,
and those who had not. For each group, we looked at the sales that these browsers had completed online
during the entire first quarter of 2012 – January 1st to March 31st, 2012. We should note that any browser who
clicked on other ads but not on a Criteo ad is therefore viewed as a “non clicker” in this study.
The total sales considered in the study amounted to $11.5 billion dollars.
2 RESEARCH PAPER # 1

3. 3 EXPLODING MYTHS ABOUT CLICKERS
MYTH 1
“People who click don’t buy”
“The demographics of clickers are skewed towards younger browsers aged 25 to 44 earning less than $40,000
per year. This is hardly an attractive target segment for most advertisers” 1
Demographics are taken by comScore as a proxy for user value. We have the luxury of being able to measure user
value directly, in terms of the online purchases of clickers and non-clickers. We see that the clicking audience is much
more valuable than the audience who does not engage with our ads.
REALITY 1
Clickers Buy 3x More Frequently than Non-Clickers
1.2
1
Sales per user during Q1
0.8
0.6
0.4 1.13
0.2
0
0.39
Clickers Non-Clickers
Figure 1. Number of tracked online sales per browser exposed to an Ad (Q1 2012)
3 RESEARCH PAPER # 1

4. MYTH 2
“Nobody clicks on ads anymore”
“Recent research conducted by comScore on behalf of Starcom and Tacoda showed that average click rates on
display ads in 2008 were less than 0.1%....The comScore research also revealed that two-thirds of Internet brow-
sers do not click on any display ads over the course of a month” 1
Our research shows very different results.
Almost half of regular buyers on our clients’ e-Commerce sites are clickers on Criteo ads. In contrast, those who
don’t buy are much less likely to click on an ad. Thus the behavior we see is what you would expect: the more
interested you are in buying, the more interested you are in the ad.
This study is able to track only whether users clicked on Criteo ads: if we were able to track clicks on all (Criteo
and non-Criteo) advertising, the proportion of regular buyers who click would be likely to be higher.
Our conclusion is that the most valuable browsers of an e-commerce site – their buyers – are engaging heavily
with online display ads.
We have wondered why comScore sees such different results. Our hypothesis is that the comScore results are
based on traditional display ads, which are not personalized and targeted to an individual browser’s interests, and
may not always be optimized to ensure that the ad is seen.
REALITY 2
Almost half of regular buyers click on Criteo ads alone.
100 %
90 %
80 %
70 %
Share of Criteo Clickers
60 %
50 %
40 %
30 %
20 %
43%
10 % 33%
0 15%
Non-Buyers Occasional Buyers Regular Buyers
Figure 2. Share of client site visitors that are clickers on Criteo ads, as a function of their
buying behavior. *
* “Occasional Buyers” are those who made one purchase in Q1 2012. “Regular Buyers” are those who made 5 or more purchases.
4 RESEARCH PAPER # 1

5. MYTH 3
“A small number of people are responsible
for a disproportionate number of clicks”
“The comScore research also revealed ….that only 16% of Internet browsers account for 80% of all clicks” 1
This statement is in fact partially true, but should not be taken as surprising, or even problematic. We see that 80%
of clicks are contributed by 50% of clickers, so much less skewed than the comScore research, but still with most
clicks coming from only half the browsers. This is actually a classic exponential distribution, which we see in many
other cases.
One compelling example is the share of online sales that come from a small number of browsers. This shows a
very similar distribution to that of clicks.
REALITY 3
20% of browsers are responsible for 50% of clicks
and sales
100 %
90 %
80 %
Share of Browsers and Sales
70 %
60 %
50 %
40 %
30 %
20 %
10 % Sales
Click
0
0% 20% 40% 60% 80% 100%
Cumulative % of Browsers
Figure 3. Shows independent Distributions of Sales and Clicks (Q1 2012)
(For the intersection of these groups, and an answer to the question “Do people who click more also buy more?” see Myth 4.)
5 RESEARCH PAPER # 1

6. MYTH 4
“People who click a lot don’t buy a lot”
A final variation on these myths is the idea that there is a small group of browsers who have nothing better to do
than to click on online ads, but who are not serious shoppers. This too is inaccurate: those who click heavily are
the same people who are buying heavily.
REALITY 4
The more browsers click, the more they buy.
100
90
Number of Sales per user (Index, Base 100)
80
70
60
50 Increasing clicks
and sales per user
40
30
20
10
0
1 6 11 16 21 26
Number of clicks per user
Figure 4. Average sales per user as a function of the number of clicks (Q1 2012)
6 RESEARCH PAPER # 1

7. 4 CONCLUSION
We have demonstrated that, within the population who have seen Criteo ads:
i) Almost half of regular buyers on e-commerce sites click
on Criteo’s online display ads
ii) These clickers buy much more, on average, than non-clickers
iii) Those who click more, buy more
Together, this data makes a compelling case that the “click” can be a genuine expression of purchase intent,
and is not, as was suggested, mainly accidental or spurious.
This is not to deny that some clicks on online display are of low value. If you show someone a badly-targeted,
poorly-timed, or irrelevant ad, then the clicks gathered will not contain any meaningful intent. If your CTR is
0.06% – fewer than 1 in 1,000 impressions are leading to a click – then it should be no surprise that the clicks
the ad is driving are inherently worth very little.
Nothing here should be taken to cast doubt on comScore’s original paper, which was based on the performance
of low-CTR, branding-oriented advertising. However, the clear message is that their lessons cannot be applied
to properly-executed performance display advertising. If your relevant and timely ad campaign shows a CTR of
0.8%, which means that almost 1 in 100 Ad impressions is sufficiently engaging to interrupt what the user was
doing on that page, it is a strong sign of genuine interest from real browsers who want to buy your products.
REFERENCES
[1] comScore, How Online Advertising Works: Whither the Click? (2008) http://www.comscore.com/layout/
set/popup/content/download/1214/13682/file/How_Online_Advertising_Works.pdf
[2] comScore, Natural Born Clickers (2009)
http://www.comscore.com/Press_Events/Press_Releases/2009/10/comScore_and_Starcom_USA_
Release_Updated_Natural_Born_Clickers_Study_Showing_50_Percent_Drop_in_Number_of_U.S._Internet_
Browsers_Who_Click_on_Display_Ads
[3] Nathan Woodman, Natural Born Clickers (2012)
http://www.clickz.com/clickz/column/2166536/natural-born-clickers
7 RESEARCH PAPER # 1