Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Analysis of TAG "certified against fraud" program


Published on

Many companies felt pressured to get TAG certification (certified against fraud). What does TAG certification mean and what does it not mean? Does TAG certification actually reduce fraud, or is it that companies that already filter for IVT have lower fraud than ones that do not filter for IVT? For folks thinking about getting TAG certification, here are some considerations to think through.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

An Analysis of TAG "certified against fraud" program

  1. 1. September 2019 / Page 0marketing.scienceconsulting group, inc. An Analysis of TAG Certified Against Fraud September 2019 Augustine Fou, PhD. acfou [at] 212. 203 .7239
  2. 2. September 2019 / Page 1marketing.scienceconsulting group, inc. Who am I? (Ad Fraud Researcher) I am a digital marketer of 23 years. I see rampant amounts of fraud in my industry, the largest it’s ever been. I observe that certain industry efforts, that started with the good intention of fighting fraud, are having the opposite effect – because inaccuracies and misrepresentations are causing a false sense of security, and not actually reducing fraud. My goal here is to provide an objective assessment of what one such “certification” program is, and is not, so buyers (marketers) and sellers (publishers) can accurately weigh their decision whether to continue to pay for said certification, or not. I am a researcher and consultant. I have my own technology that collects data from the digital campaigns that I audit at no cost for clients. I show all the data – i.e. not “black box” – so they can decide whether they agree with my assessments or not. Some clients then hire me to teach them how to look at data and analytics to find fraud and reduce it themselves.
  3. 3. September 2019 / Page 2marketing.scienceconsulting group, inc. What is TAG? “Trustworthy Accountability Group” - TAG “Trustworthy Accountability Group (TAG) is a first-of-its-kind cross- industry accountability program to create transparency in the business relationships and transactions that undergird the digital ad industry, while continuing to enable innovation. A joint marketing- media industry program, TAG was created with a focus on four core areas: eliminating fraudulent digital advertising traffic, combating malware, fighting ad-supported Internet piracy to promote brand integrity, and promoting brand safety through greater transparency. TAG was created by the American Association of Advertising Agencies (4A’s), Association of National Advertisers (ANA), and Interactive Advertising Bureau (IAB) and works collaboratively with companies throughout the digital ad supply chain.”
  4. 4. September 2019 / Page 3marketing.scienceconsulting group, inc. What is this deck about? To clarify what this deck covers and its objective • This deck is NOT about the MRC or their standards. Standards like those of G-IVT and S-IVT are necessary and useful; they set a floor for quality and serve as guideposts that parties can shoot for. • This deck is NOT about the IAB Tech Lab or their protocols, like ads.txt and app-ads.txt. Those protocols are useful tools for companies intending to do the right thing. They represent concrete steps companies can take to try to reduce ad fraud. • This deck IS about TAG’s “Certified Against Fraud” (CAF) certification program. I will present observations by me and others who work in the adtech trenches every day, about what TAG CAF is and is not. • The intent is to enable those who decide on whether to get TAG certified to cut through the hype and make a more informed decision.
  5. 5. September 2019 / Page 4marketing.scienceconsulting group, inc. Agenda We will ask a series of questions that someone in the position of deciding whether to get the TAG “Certified Against Fraud” certification should consider.
  6. 6. September 2019 / Page 5marketing.scienceconsulting group, inc. Does TAG provide Accountability?
  7. 7. September 2019 / Page 6marketing.scienceconsulting group, inc. Majority Certified by Self-Attestation Yellow highlight shows the companies certified through self-attestation
  8. 8. September 2019 / Page 7marketing.scienceconsulting group, inc. Commentary by Adops Folks Reddit AdOps group discussions/questions about TAG Certification
  9. 9. September 2019 / Page 8marketing.scienceconsulting group, inc. Commentary by Adops Folks Reddit AdOps group discussions/questions about TAG Certification
  10. 10. September 2019 / Page 9marketing.scienceconsulting group, inc. Commentary by Adops Folks Reddit AdOps group discussions/questions about TAG Certification
  11. 11. September 2019 / Page 10marketing.scienceconsulting group, inc. Does TAG provide Accountability? No.
  12. 12. September 2019 / Page 11marketing.scienceconsulting group, inc. Does TAG CAF cause reduction in ad fraud?
  13. 13. September 2019 / Page 12marketing.scienceconsulting group, inc. Has TAG shown “cause and effect”? Does TAG CAF certification CAUSE a reduction of ad fraud? Or is it simply that companies that already use IVT filters have lower IVT than ones that do not use IVT filters? It’s the latter.
  14. 14. September 2019 / Page 13marketing.scienceconsulting group, inc. NOT TAG certified, but clean all along Good publishers were clean all along because of good business practices
  15. 15. September 2019 / Page 14marketing.scienceconsulting group, inc. TAG Certified, but committing ad fraud TAG “Certified Against Fraud” but actively committing ad fraud “the code used by NMG is designed to interfere with the ability of third-party measurement systems to determine how much of a digital ad was viewable during a browsing session. This code manipulated data to ensure that otherwise unviewable ads showed up in measurement systems as valid impressions, which resulted in payment being made for the ad.”
  16. 16. September 2019 / Page 15marketing.scienceconsulting group, inc. Does TAG CAF cause reduction in ad fraud? No.
  17. 17. September 2019 / Page 16marketing.scienceconsulting group, inc. Does TAG analyze data correctly?
  18. 18. September 2019 / Page 17marketing.scienceconsulting group, inc. 2017 Fraud Benchmark Study TAG claimed certifications reduced fraud, using invalid comparison dy/TAG%20Fraud%20Benchmark%20Study.pdf?hsLang=en-us
  19. 19. September 2019 / Page 18marketing.scienceconsulting group, inc. TAG claimed large reduction TAG willfully chose an invalid comparison to show large decrease “Given that TAG Certified Channels deliver an 83% reduction in IVT as compared to general industry fraud rates, it follows that working with TAG Certified companies creates an opportunity to reduce fraud and boost overall trust in the industry.” “These campaigns ran from July 1 through October 31, 2017.” “Specifically, we looked at over 6.5 billion impressions that had been measured by either MOAT, DoubleVerify or Integral Ad Science, prior to reaching us.” (this means they were filtered) “We then compared our TAG Certified baseline to a combination of the publicly reported marketplace fraud data collected by DoubleVerify, Integral Ad Science, and White Ops and documented in reports they made publically available. The various vendor studies identified a blended IVT rate in display and video, of 12.03% in U.S.-based display traffic.” -83% TAG willfully chose an INVALID comparison, to claim an 83% “reduction” (which they did not cause) display+video display display
  20. 20. September 2019 / Page 19marketing.scienceconsulting group, inc. Valid comparison was 1.48% vs 1.8% The filtered IVT rate of 1.8%, same time period, same IVT vendor 1.8% fraud The VALID comparison would be the filtered IVT rate for desktop display (1.8%) versus 1.48% - a 17% difference. There is still NO cause and effect (TAG Certification did not cause it) -83% display+video display display
  21. 21. September 2019 / Page 20marketing.scienceconsulting group, inc. Valid vs invalid comparison The data was publicly available from the same IVT companies 1.48% IVT The VALID comparison would be the filtered IVT rate for desktop display (1.8%) versus 1.48% - a 17% difference, not the 83% reduction claimed. There is still NO cause and effect (TAG Certification did not cause reduction) 1.48% IVT1.8% IVT = -17% Measured IVT rate In TAG channels (filtered, desktop, display – IAS MQR H2 2017, U.S.) 8.83% = -83% TAG Claimed VALID comparison
  22. 22. September 2019 / Page 21marketing.scienceconsulting group, inc. 2018 Fraud Benchmark Study TAG commissioned another report, to claim “reduction” in fraud
  23. 23. September 2019 / Page 22marketing.scienceconsulting group, inc. TAG again claimed large reduction TAG again chose an invalid comparison, rather than the valid one “These campaigns included display media and video on desktop, mobile web and in-app. These campaigns ran from January 1, 2018 through August 31, 2018.” “We then compiled an average of these numbers in order to compare it to the rate of fraud we measured in TAG Certified Channels. We make no claim as to the validity the third-party tests since we have not seen the sources or the methodologies used for assessing fraud rates (e.g. percentage of traffic measured, types of fraud measured, etc.). Therefore, our comparison should be considered direction.”
  24. 24. September 2019 / Page 23marketing.scienceconsulting group, inc. Valid comparison was different The data was publicly available for same time period, and market desktop mobile 41/Tag2018fraudbenchmarkstudy.pdf
  25. 25. September 2019 / Page 24marketing.scienceconsulting group, inc. Valid vs invalid comparison TAG chose the invalid comparison rather than the valid comparison 1.68% IVT The VALID comparison would be the filtered IVT rate for desktop+mobile and display+video. This shows a range of 27% reduction to more than 2X INCREASE in fraud in TAG channels. 1.68% IVT = -27% to +240% (IAS MQR H1 2018, U.S.) 10.43% = -84% TAG Claimed display video mobile desktop 1.8% 2.3% 0.8% 0.7% VALID comparison Measured IVT rate In TAG channels
  26. 26. September 2019 / Page 25marketing.scienceconsulting group, inc. 2019 EU Fraud Benchmark Study TAG commissioned a third report
  27. 27. September 2019 / Page 26marketing.scienceconsulting group, inc. TAG compared to “non-optimized” TAG chose an invalid comparison using a blended average, why? “These campaigns ran from January 1, 2018 through November 30, 2018” “In order to create a comparison of TAG-Certified Channels versus the industry standard, the study used Integral Ad Science global insights report and Impact (Forensiq) European fraud rates that were supplied to the research team. Fraud rates were based on display media for desktop and mobile web.” “The amount of fraud (both SIVT and GIVT) found in TAG-Certified Channels across multiple inventory types is 0.53%. The overall blended rate we will use for comparison is 8.99%, which represents that TAG partners have 94.1% cleaner traffic than partners who work outside of this certification.”
  28. 28. September 2019 / Page 27marketing.scienceconsulting group, inc. Public data from same time period TAG cited an invalid comparison using an unfounded blended rate U_Fraud%20Study_2019.pdf
  29. 29. September 2019 / Page 28marketing.scienceconsulting group, inc. TAG chose invalid comparison TAG chose an invalid blended rate to show huge difference 0.53% IVT The VALID comparison would be the filtered IVT rate for desktop+mobile display+video (see grid) versus 0.53% - a -52% to +6% difference, not the 94% reduction claimed. There is still NO cause and effect 8.99% = -94% TAG Claimed 0.53% IVT = -52% to +6% (IAS MQR H2 2018, U.K.) display video mobile desktop 1.0% 1.1% 0.5% 0.5% VALID comparison Measured IVT rate In TAG channels
  30. 30. September 2019 / Page 29marketing.scienceconsulting group, inc. 2019 Germany Benchmark Study TAG commissioned 4th report, in Germany fs/2848641/TAG-WhitePaper- Germany-003.pdf
  31. 31. September 2019 / Page 30marketing.scienceconsulting group, inc. TAG chose invalid comparison TAG claimed a reduction, but valid comparison revealed higher fraud 2.05% IVT The VALID comparison would be the filtered IVT rate for desktop+mobile display+video (see grid) versus 2.05% 14.21% = -86% TAG Claimed 2.05% IVT = +70% to +242% (IAS MQR H1 2019, Germany) display video mobile desktop 1.2% 1.0% 0.6% VALID comparison TAG channels had HIGHER FRAUD IN ALL CASES Measured IVT rate In TAG channels
  32. 32. September 2019 / Page 31marketing.scienceconsulting group, inc. Does TAG analyze data correctly? No.
  33. 33. September 2019 / Page 32marketing.scienceconsulting group, inc. Does TAG “Certified Against Fraud” Catch Fraud?
  34. 34. September 2019 / Page 33marketing.scienceconsulting group, inc. TAG Certified Against Fraud Means? Certified Against Fraud only requires GIVT detection/filtration Source: AL%20(Version%201.0).pdf “G-IVT covers ‘honest bots’ like search engine crawlers that say their name honestly in the user agent. Virtually NONE of the bots used for ad fraud are caught this way, because fraud bots actively disguise themselves to trick detection. By only requiring G-IVT detection and filtration the certification ensures the majority of fraud is missed.”
  35. 35. September 2019 / Page 34marketing.scienceconsulting group, inc. G-IVT doesn’t catch all fraud IVT detection severely under-reports ad fraud 1.3% + 57% = 58% bot fraud site/app fraud overall fraud IVT detection sees this IVT detection misses this “flashlight apps that load ads and webpages in the background are not detected as IVT; so IVT detection is missing many other, current forms of ad fraud.”
  36. 36. September 2019 / Page 35marketing.scienceconsulting group, inc. Examples of fraud not detected as IVT everything-2019-update/10 fraud-is-rampant-beyond-belief-102771113/23 origination-super-networks-v-redacted-84576544 replayed-deviceids-to-defeat-fraud-detection
  37. 37. September 2019 / Page 36marketing.scienceconsulting group, inc. Continuous new discoveries of ad fraud All of these were found, documented by others, investigative reporters Source:
  38. 38. September 2019 / Page 37marketing.scienceconsulting group, inc. Certification did not catch fraud TAG “Certified Against Fraud” companies committing fraud YouAppi sued by Uber for faked app installs and-now-ad-tech/ Newsweek caught using malicious code to alter viewability measurements fraud Adconion (now Amobee) Hijacked IP Addresses for Spamming for-spamming/#more-48805 sued by FTC for fake ads
  39. 39. September 2019 / Page 38marketing.scienceconsulting group, inc. Does TAG “Certified Against Fraud” Catch Fraud? No.
  40. 40. September 2019 / Page 39marketing.scienceconsulting group, inc. Other Considerations
  41. 41. September 2019 / Page 40marketing.scienceconsulting group, inc. What “certified” means, or not “Certified Against Fraud” • Registered business w/ address • Have a TAG compliance officer • Attend training annually • Comply with G-IVT guidelines • Employ domain, datacenter, app threat filtering • Disclose traffic sourcing • Implement Payment ID • Honor ads.txt files Questions / Considerations • Does this prove no ad fraud? • Does “compliance” mean no fraud? • Bad guys also know the rules • Does G-IVT see/catch all fraud? • Does this filter/catch more advanced fraud, mobile app fraud? • Do bad guys disclose honestly? • Fraudsters also need to get paid • Does an honor system work here? Source:
  42. 42. September 2019 / Page 41marketing.scienceconsulting group, inc. These approaches do not see or catch fraud Items in TAG’s checklist are/have been easily bypassed Domain filtering Fraudsters use domain spoofing to get around this, pretend to be good sites Fraudsters use residential proxies to disguise traffic, so it doesn’t appear to be data center IVT detection doesn’t work well in mobile apps and most of the fraud is not detected, so it is not filtered TAG Certified Newsweek did not disclose traffic sourcing until after being caught for using malicious code to trick viewability measurements malicious-code-ad-fraud Datacenter filtering App threat filtering Disclose traffic sourcing G-IVT filtering Real fraud bots are not G-IVT because they do not say their name honestly, like googlebot
  43. 43. September 2019 / Page 42marketing.scienceconsulting group, inc. Can TAG tell if a company lied to them? Even if TAG audited them, what fraud could they see? Simple questions to consider for yourself: • Does TAG have any technology to collect data? • Does TAG have data to use as a “truth set” to know what is fraud and what is not fraud? • Does TAG have the people with ad ops, analytics, and fraud analysis experience? • If TAG audited a “certified” company, would they know what to look for, where to look for it, and what it means if they see it?
  44. 44. September 2019 / Page 43marketing.scienceconsulting group, inc. Marketers solved fraud themselves Examples of how businesses solved fraud without TAG certification businesses-beat-ad-fraud-dr-augustine-fou-/ beat-ad-fraud-cybersecurity-ad-fraud-researcher/ “This was the story of how one marketer went on a journey to defeat ad fraud, by taking steps on her own. This marketer never used popular fraud detection tech; she never needed it. Her vigilance and detailed analysis, along with plain old common sense led her to find the fraud and remove it from her campaigns.” “Other small and medium-sized businesses that run their own digital campaigns also need them to perform well. And a key to that is knowing how to avoid ad fraud and thus avoid getting ripped off.”
  45. 45. September 2019 / Page 44marketing.scienceconsulting group, inc. Why pay for TAG certification? Publishers/Sellers • If TAG certification does not prove or disprove fraud, why should you pay for TAG certification? • If you already have G-IVT/S-IVT detection/filtration in place and other best practices, can you show buyers those anti-fraud measures and supporting data so they can see you had low fraud all along? • If you need help, I will help you. Advertisers / Buyers • If TAG certification does not prove or disprove fraud, what is your motivation for forcing publishers to pay for TAG certification? • You can just ask if a seller has G-IVT and S-IVT filtration in place; ask for detailed data to you can check for signs of fraud. This will help you actually detect/reduce ad fraud. • Consider buying from good publishers that have been clean all along, whether or not they have TAG certification.
  46. 46. September 2019 / Page 45marketing.scienceconsulting group, inc. Call to Action If you see something, say something … to TAG and to me. If you see a TAG Certified vendor committing fraud, document it and “say something” – NOTIFY TAG and notify me. 20Non- Compliance%20&%20Appeal%20FINAL%20January%202018.pdf?t=1516314600832 “All complaints, notices and briefs set forth in this process shall be submitted to”
  47. 47. September 2019 / Page 46marketing.scienceconsulting group, inc. Recap • Across 4 studies, written and distributed from 2017 to 2019, TAG repeatedly chose to cite invalid comparisons, instead of valid ones from the same data, to prove value in their self-attested certifications • TAG CEO Mike Zaneis repeatedly made claims that the certifications caused massive reductions (e.g. “600% reduction”) in ad fraud when the certification does not reduce fraud – e.g. NOT certified publishers were clean all along; CERTIFIED publisher was caught for ad fraud • At least four TAG Certified Against Fraud members have now been exposed for committing ad fraud, so far. • Citing IVT rates is not an effective measure of fraud because it misses other (larger) forms of fraud that are not IVT; checklist of filtration methods are ineffective because fraudsters have already defeated all these measures.
  48. 48. September 2019 / Page 47marketing.scienceconsulting group, inc. Related Articles/Decks marketers-anti-adfraud-playbook rs-playbook-questions-to-ask-verification-vendors marketers-anti-adfraud-playbook What Marketers Can Do Themselves Why Detection Tech is Limited fraud-detection-doesnt-work What the Industry Wants You To Believe narrative-dr-augustine-fou-ad-fraud-historian/ Myths, Misconceptions, Conflicts misconceptions-allow-ad-fraud-flourish-dr- augustine-fou-/ QVI1eeQbm20ktZcKt- tSxajf3BGmhmKIoJV48dgY2Oo/edit#gid=71 6446669 History of Ad Fraud