Fraud Fighting Techniques by Augustine Fou PhD


Published on

Despite growing awareness and industry initiatives to curb rampant online ad fraud, it is still a huge problem that most advertisers and their media buying agencies have not solved.

Published in: Technology, Business

Fraud Fighting Techniques by Augustine Fou PhD

  1. 1. Fighting Rampant Online Ad Fraud Dr. Augustine Fou October 2013 -1- Augustine Fou
  2. 2. Ad Waste from Fraud Display Ad Fraud Video Ad Fraud 480 billion 20.1 billion display ad impressions /mo video ad impressions /mo 29% 40% Source: Solve Media 2013 Source: Vindico, 2013 confirmed bot traffic $1 - $3.50 ~$10 cost per thousand cost per thousand $2-6 billion $1 billion wasted ad spend (annualized) -2- estimated fake views wasted ad spend (annualized) Augustine Fou
  3. 3. $9.5 Billion Ad Waste in 2013 Source: Solve Media August 2013 -3- Augustine Fou
  4. 4. Industry Players Nielsen/IAB IntegralAds (AdSafe) DoubleVerify Industry working group to define ad viewability. Verify ad placement against blacklist of known fraudulent websites. Ad placement, behavioral compliance, fraud detection Viewable rates (of display ads) ranged from 14% to 79%. PageFair Viewable rates (of display ads) ranged from 14% to 79%. Ad blocking detection Solve Media Algo detection of bot-like activity and other malware. Using CAPTCHAs to detect humans versus bots. Botnet Costing Display Advertisers $6 Million per month. Feb 2013 Global Bot Traffic on Pace to Waste Up to $9.5 billion in 2013 Ad Budgets. Sep 2013 -4- WhiteOps Realtime bot detection algorithms Augustine Fou
  5. 5. Fake Mobile Ad Traffic While “mobile” is more likely to be human, there is no guarantee! -5- Augustine Fou
  6. 6. Traffic Firehose On/Off Source: Alexa Legit human traffic does not change rapidly; but bot traffic (firehose) can be rapidly turned off and directed to other sites. -6- Augustine Fou
  7. 7. Detecting Fraudulent Sites • Rapid, large, unexplained changes in traffic • Increase in traffic without corresponding increase in search volume (how human users find content sites); under-indexing traffic % from search • Large discrepancy between comScore traffic numbers and Quantcast/Alexa counts (hard to fake actual installed toolbars) -7- Augustine Fou
  8. 8. Fraud Fighting Techniques • Blacklisting fake websites by URL/domain • Whitelisting legit publisher sites by domain • Detecting bot traffic and known botnet IP addresses • Scanning sites for malware (Google hosts list) • Bot flag on bid records • Real time rejection of fraudulent ad impressions • Revised traffic numbers based on audience overlap -8- Augustine Fou
  9. 9. Defining Viewability Source: May 2, 2013 “Even if industry standards define viewability, bad guys can still display dozens of „viewable‟ ads in hidden iframes or with pixel opacity set to zero (invisible).” -9- Augustine Fou
  10. 10. Blacklisting Sites “For every site that is added to a black list, bad guys can put up many more to continue to commit ad fraud (because they don‟t have to play by the rules).” - 10 - Augustine Fou
  11. 11. Bot Detection Source: March 2013 “It was commonly assumed that mouse movement and scrolling were human activities; now sophisticated bots can also fake mouse movement and page scrolls.” - 11 - Augustine Fou
  12. 12. CAPTCHAs • 43% deemed “suspicious” • 29% “confirmed bot traffic” “Startup called Vicarious automatically solves CAPTCHAs.” Oct 2013 Source: Solve Media via Marketing Charts April 26, 2013 “It was commonly assumed that solving CAPTCHAs could only be done by humans; not any more.” - 12 - Augustine Fou
  13. 13. Assumptions No Longer Valid 1. bots can’t fake mouse movements and webpage scrolling - they can easily now 2. captchas can only be solved by humans - bots can solve them too now 3. it requires malware infected computers to commit ad fraud - bad guys can set up hundreds of thousands of server instances to simulate users without having to infect any computers with malware 4. malware can be caught by virus software when installed - some malware does not need to be installed, they are carried along with the code of a toolbar, plugin, extension, etc. or can be asynchronously introduced later via updates (especially when user has permitted auto-updates) 5. if a correct bid record is passed it should be a human user - bots can easily send fake information to simulate being a user (e.g. cookies, referrer, search query, characteristics of the computer and browser, etc.) 6. fraudulent traffic is a small portion of legitimate human traffic -- bot traffic is far larger than actual human traffic. This comes from both “legitimate” sources like Google crawlers or “site up status checkers” and “illicit” sources like server-side scripts, browser side scripts, browser extensions and plugins, and other malware. - 13 - Augustine Fou
  14. 14. Related Articles Bad Guys Happily Rob Display Advertisers By: Augustine Fou, July 23, 2012 Everything Fake (Display Ad Fraud, Search Click Fraud) By: Augustine Fou, April 2013 Blacklisting vs Whitelisting By: Augustine Fou, October 2013 Fake Profiles on Facebook By: Augustine Fou, July 2013 How Display Ad Fraud Works By: Augustine Fou, May 2013 - 14 - Augustine Fou
  15. 15. Dr. Augustine Fou – Digital Consigliere “I advise clients on optimizing advertising across all channels. Using insights and fast-feedback loops from digital, we can not only target brand ads better, but we can shift towards more detailed measurement and ROI.” FORMER CHIEF DIGITAL OFFICER, HCG (OMNICOM) MCKINSEY CONSULTANT CLIENT SIDE / AGENCY SIDE EXPERIENCE PROFESSOR AND COLUMNIST ENTREPRENEUR / SMALL BUSINESS OWNER PHD MATERIALS SCIENCE (MIT '95) AT AGE 23 ClickZ Articles: Slideshares: LinkedIn: - 15 - @acfou Augustine Fou