State of Ad Fraud Ad Blocking Q1 2016 Update Augustine Fou
The State of Ad Fraud
Q1 2016 Update
Augustine Fou, PhD.
acfou [at] mktsci.com
212. 203 .7239
April 2016 / Page 1marketing.scienceconsulting group, inc.
• Year-over-year, ad fraud has not improved overall; but
individual companies see improvements with greater effort
• Bots remain the key ingredient for all forms of ad fraud and
their actions are polluting analytics, making them unreliable
• Viewability standards are widespread now, but ad blocking
has emerged as the new “crisis du jour”
• Cybercriminals continue to easily avoid or defeat detection
technologies because they cheat and don’t play by any rules
• The proliferation of connected devices and IoT will increase
the capabilities of bots and opportunity for ad fraud
• 4D cybersecurity approaches are needed to fight ad fraud
April 2016 / Page 2marketing.scienceconsulting group, inc.
Ad Fraud Background Update
April 2016 / Page 3marketing.scienceconsulting group, inc.
Year-over-year fraud estimates continue to vary widely
WhiteOps/DCN (Oct 2015)WhiteOps/ANA (Dec 2015)
Average Bots 3%
Range: 2% - 7%
Average Bots 11%
Range: 3% - 37%
“there are industry average bot% and very wide ranges;
but what really matters is the bot activity on YOUR site.”
April 2016 / Page 4marketing.scienceconsulting group, inc.
Bad guys continue to focus on where the money is
• rich media
not at risk
(86% of digital spend - 2014)
April 2016 / Page 5marketing.scienceconsulting group, inc.
Two main types of fraud have the same 2 key ingredients
(includes mobile display, video ads)
1. Put up fake websites and
load tons of ads on the pages
(includes mobile search ads)
2. Use bots to repeatedly load
pages to generate fake ad
impressions (hide the true
origins to avoid detection)
1. Put up fake websites and
participate in search networks
2. Use bots to type keywords
to cause search ads to load
and then to click on the ad
to generate the CPC revenue
April 2016 / Page 6marketing.scienceconsulting group, inc.
Bots are the cause of ALL automated ad fraud
Bots are made from 1) malware
compromised PCs or 2) headless
browsers (no screen) in datacenters.
April 2016 / Page 7marketing.scienceconsulting group, inc.
“Ad fraud and ad blocking are
now related because of bots”
April 2016 / Page 8marketing.scienceconsulting group, inc.
Bots of all kinds do all kinds of fraud
April 2016 / Page 9marketing.scienceconsulting group, inc.
Different kinds of bots create different kinds of fraud
Malware (on PCs)Botnets (from datacenters)
April 2016 / Page 10marketing.scienceconsulting group, inc.
Bad guys’ (advanced) bots are not on any industry list
in the wild
bad guys’ bots
Dstillery, Oct 9, 2014_
“findings from two independent third parties,
Integral Ad Science and White Ops”
Rocket Fuel, Sep 22, 2014
“Forensiq results confirmed that ... only 3.72% of
impressions categorized as high risk.”
2 - 3%
comScore, Sep 26, 2014
“most campaigns have far less; more in the
2% to 3% range.”
detect based on
industry bot list
“not on any list”
disguised as normal browsers –
Internet Explorer; constantly
adapting to avoid detection
April 2016 / Page 11marketing.scienceconsulting group, inc.
Advanced bots need to be detected differently
• found page via search
• observed events (mouse click
“Honest” (declared) bots
• search engine crawlers
• declare user agent honestly
• observed to be 1 – 5% of
• come from data centers
• malware compromised PCs
• deliberately disguised user
agent as normal browsers
April 2016 / Page 12marketing.scienceconsulting group, inc.
Any device with a chip and internet can be used as a bot
Traffic cameras turned into
botnet (Engadget, Oct 2015)
April 2016 / Page 13marketing.scienceconsulting group, inc.
Fraud Activities Mess Up Measurement
April 2016 / Page 14marketing.scienceconsulting group, inc.
Bad guys hide fraud by passing fake parameters
Click thru URL
passing fake source
April 2016 / Page 15marketing.scienceconsulting group, inc.
Bad guys fake KPIs, trick measurement systems
Bad guys have higher CTR Bad guys have higher viewability
Bad guys stack
ads above the
fold to fake
Good guys have to
array ads on the
page – e.g. lower
April 2016 / Page 16marketing.scienceconsulting group, inc.
Bad guys’ bots earn more money, more efficiently
Cash out sitesCookie collecting
Source: DataXu/DoubleVerify Webinar, April 2015
April 2016 / Page 17marketing.scienceconsulting group, inc.
Bad guys’ bots can fake quantity and quality metrics
click on links
load webpages tune bounce rate
April 2016 / Page 18marketing.scienceconsulting group, inc.
Turn off 1 ad network to cut out 90% bot traffic (red)
• Bot volumes are flat across (red line at top; green volume bars at
• Human traffic patterns are messy and irregular (blue line at top)
• Dark red area decreased after we turned off a single bad ad network
• Share blacklists through central data repository to increase impact
April 2016 / Page 19marketing.scienceconsulting group, inc.
Google analytics view of traffic from single fraud domain
Despite losing all of the traffic from these fake/fraud sites, there was no
change to the number of pledges, during the same period of time.
goal events – no change
“ … because bots don’t make donations!”
April 2016 / Page 20marketing.scienceconsulting group, inc.
AppNexus example – cleaned up 92% of impressions
Increased CPM prices
volume by 92%
“AppNexus aggressively detected and discarded fraudulent
inventory; good for them and good for those who buy from
them. This should be an example for publishers.”
April 2016 / Page 21marketing.scienceconsulting group, inc.
What normal (human) traffic patterns look like (5 + 2)
• Notice the normal 5+2 pattern of weekday versus weekend traffic
• Human traffic is lowest between 2a – 5a , because humans sleep
• Bot volumes are now seen to be roughly tuned to human pattern
5 weekdays 5 weekdays
(lower volume on 2 weekend days)
April 2016 / Page 22marketing.scienceconsulting group, inc.
End of month traffic and impressions fulfillment
end of February
April 2016 / Page 23marketing.scienceconsulting group, inc.
Bad acting mobile apps and .xyz domains (abnormal CTRs)
.xyz domainssuspicious mobile apps
April 2016 / Page 24marketing.scienceconsulting group, inc.
Connection with Ad Blocking
April 2016 / Page 25marketing.scienceconsulting group, inc.
Ad blocking impacts both CPM and CPC ads
Ad blocking ONNo ad blocking
April 2016 / Page 26marketing.scienceconsulting group, inc.
Humans use adblock, fraud bots don’t block ads
“We have long used ad block detection as a signal
for human users because humans use adblock.”
“Fraud bots that want the ads to load don’t block ads.”
Note that we focus on the % of ads that successfully loaded, not blocked.
80 – 85% loaded is a good rule of thumb
April 2016 / Page 27marketing.scienceconsulting group, inc.
University websites (no ads)
April 2016 / Page 28marketing.scienceconsulting group, inc.
Tech and Gaming (tech savvy users)
April 2016 / Page 29marketing.scienceconsulting group, inc.
Ad supported content sites see highest ad blocking rates
April 2016 / Page 30marketing.scienceconsulting group, inc.
April 2016 / Page 31marketing.scienceconsulting group, inc.
Debunk ALL assumptions and discard all averages
• Ad Fraud / Bots
• Ad Blocking
• Bots are low; list-based detection catches very little
• Bots are malware on PCs; more and more from data centers
• Bots are simple (cant fake clicks, mouse, scrolling); they do
• Ad blocking is lower in mobile; mobile is less measurable
• Ad blocking is higher among younger users; not necessarily
• Ad blocking is still in the single digits; look at “ads served”
• Higher CTRs are better; not if the clicks are fake
• Higher viewability is better; not if the sites fake viewability
• Data is reliable; not so if you don’t know bot/blocking levels
April 2016 / Page 32marketing.scienceconsulting group, inc.
About the Author
April 2016 / Page 33marketing.scienceconsulting group, inc.
Dr. Augustine Fou – Recognized Expert on Ad Fraud
SPEAKING ENGAGEMENTS / PANELS
4A’s Webinar on Ad Fraud
Digital Ad Fraud Podcast
Programmatic Ad Fraud Webinar
AdCouncil Webinar on Ad Fraud
TelX Marketplace Live
ARF Audience Measurement
IAB Webinar on Ad Fraud / Botnets
April 2016 / Page 34marketing.scienceconsulting group, inc.
Harvard Business Review – October 2015
Hunting the Bots
Fou, a prodigy who earned a Ph.D. from MIT at
23, belongs to the generation that witnessed
the rise of digital marketers, having crafted his
trade at American Express, one of the most
successful American consumer brands, and at
Omnicom, one of the largest global advertising
agencies. Eventually stepping away from
corporate life, Fou started his own practice,
focusing on digital marketing fraud
Fou’s experiment proved that fake traffic is
unproductive traffic. The fake visitors inflated
the traffic statistics but contributed nothing to
conversions, which stayed steady even after the
traffic plummeted (bottom chart). Fake traffic is
generated by “bad-guy bots.” A bot is computer
code that runs automated tasks.