This document discusses two big problems in digital advertising and real-time bidding (RTB): data leakage and market problems. It describes how the current RTB process broadcasts personal data about users to hundreds of companies, allowing mass data collection and reuse. This supports untrustworthy websites and enables bot fraud. It also explains how most of an advertiser's budget is extracted by middlemen in the programmatic supply chain rather than going to publishers.
11. @johnnyryan
CNIL regulator caught it with
68 million illegal RTB records.
Example
Vectaury: a small DSP/DMP/
trading desk in France. €3.5M
annual turnover in 2017 (though
subsequently won a €20M
investment).
DSP
14. @johnnyryan
Is 68 million
just 30%?
Then this small company
was sent personal data
¼ BILLION times via RTB
(in just one year)
15. Ad server SSP
Step 2.
Ad server
selects an SSP
Step 3.
SSP selects an
exchange
website.com
Ad server
javascript
SSP
javascript
Step 1.
User requests
webpage
Ad exchange
This is the current process of
real-time bidding that is used in
online behavioural advertising.
Channel of data leakage
Legend
Money
DATA LEAKAGE
IN ONLINE
ADVERTISING
@johnnyryan
16. Ad server SSP
Step 2.
Ad server
selects an SSP
Step 3.
SSP selects an
exchange
Step 7.
DSP serves
agency creative
Step 8.
Assets load
from CDN
Step 9.
Agency ad server
loads verification
vendor
MARKETERS
website.com
AD
Winningbid
Ad server
javascript
SSP
javascript
DMP
DMP
DMP DMP
DSP
DSP
DSP
DSP
DSP
DSP
javascript
Ad server
javascript
Step 6.
Exchange serves
winning bid
Verification
javascript
Agency
ad server
Verification
vendor
Winning DSP
Step 1.
User requests
webpage
Ad exchange
Step 4.
Exchange sends
bid requests to
hundreds of
partners
Step 5.
Exchange lets
some DMPs/
DSPs to refresh
cookie sync
CDN
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
This is the current process of
real-time bidding that is used in
online behavioural advertising.
DATA LEAKAGE
IN ONLINE
ADVERTISING
Channel of data leakage
Legend
Money
@johnnyryan
20. @johnnyryanThe website this specific person is currently viewing
Various ID codes that identify this
specific person, and can tie them to
existing profiles
Distinctive characteristics of this specific person
This specific person’s IP address
Distinctive information about
this specific person’s device
Distinctive information about this specific
person’s device
This young woman’s GPS coordinates!
24. @johnnyryan
Index Exchange 50 billion
1. “Tour IX’s Amsterdam and Frankfurt Data Centers”, Index Exchange, 2 July 2018 (URL: https://www.indexexchange.com/tour-ix-amsterdam-frankfurt-data-centers/).
2. "OpenX Ad Exchange", OpenX (URL: https://www.openx.com/uk_en/products/ad-exchange/).
3. “Buyers”, Rubicon Project (URL: https://rubiconproject.com/buyers/).
4. "How PubMatic Is Learning Machine Learning", PubMatic, 25 January 2019 (URL: https://pubmatic.com/blog/learning-machine-learning/)
5. "Maximize yield with Oath's publisher offerings", Oath, 3 April 2018 (URL: https://www.oath.com/insights/maximize-yield-with-oath-s-publisher-offerings/)
6. 500 Billion / 29.6 = 18.6 billion impressions per day. Using AppNexus 1:11.5 ratio, this is 214 auctions per day. 500+ impressions figure cited in “Optimize your mobile
strategy”, Smaato (URL: https://www.smaato.com/).
7. “Transacting at a peak of 11.4 billion daily impressions, our marketplace handles more traffic each day than Visa, Nasdaq, and the NYSE combined” at https://
www.appnexus.com/sell. Note that in 2017, AppNexus said in “AppNexus Scales with DriveScale”, 2017 (URL: http://go.drivescale.com/rs/451-ESR-800/images/
DRV_Case_Study_AppNexus-final.v1.pdf) that 10.7 billion "impressions transacted" came as a result of running 123 billion auctions. The impressions transacted to
auctions ratio appears to be roughly 1:11.5. Therefore, the 11.4 daily impressions reported in 2018 equates to 131 billion auctions per day.
8. DoubleClick.Net Usage Statistics (URL: https://trends.builtwith.com/ads/DoubleClick.Net).
Real-time bidding bid requests per day
OpenX 60 billion2
Rubicon Project Unknown, 1 billion people’s devices3
PubMatic 70 billion4
Oath/AOL 90 billion5
AppNexus 131 billion6
Smaato 214 billion7
Google Unknown, live on 8.4 million websites8
1
Index Exchange 50 billion
The biggest
31. @johnnyryan
GDPR, Article 5 (1)
(f) processed in a manner that ensures
appropriate security of the personal data,
including protection against unauthorised or
unlawful processing and against accidental
loss, destruction or damage, using
appropriate technical or organisational
measures (‘integrity and confidentiality’).
32. @johnnyryan
GDPR, Article 5 (1)
(f) processed in a manner that ensures
appropriate security of the personal data,
including protection against unauthorised or
unlawful processing and against accidental
loss, destruction or damage, using
appropriate technical or organisational
measures (‘integrity and confidentiality’).
33. @johnnyryan
Publishers recognize there is no technical
way to limit the way data is used after the
data is received by a vendor for decisioning/
bidding on/after delivery of an ad…
“
”
there is no technical
way to limit the way data is used after
Surfacing thousands of vendors with broad
rights to use data w/out tailoring those
rights may be too many vendors/permissions
“
”
thousands of vendors
“pubvendors.json v1.0: Transparency & Consent Framework”,
IAB, May 2018
34. @johnnyryan
The MO may adopt procedures for
periodically reviewing and verifying a
Vendor’s compliance with the Policies.
“Transparency & Consent Framework Policies, 2019-08-21.3”
IAB, August 2019
“
”
may adopt
Management Organisation (the IAB)
35. @johnnyryan
Buyer will regularly monitor your
compliance with this obligation, and
immediately notify Google in writing if
Buyer can no longer meet … this obligation...
“
”
“
”
must not: (i) use callout data ... to create
user lists or profile users; (ii) associate
callout data ... with third party data...
Buyer will
“Authorized Buyers Programme Guidelines”,
Google, August 2018
36. @johnnyryan
GDPR, Article 5 (1)
(f) processed in a manner that ensures
appropriate security of the personal data,
including protection against unauthorised or
unlawful processing and against accidental
loss, destruction or damage, using
appropriate technical or organisational
measures (‘integrity and confidentiality’).
39. @johnnyryanHow RTB data leakage supports untrustworthy websites
The Daily Bugle
///
Step 3.
100s of companies in the ad
auction can now re-identify
John as a Daily Bugle reader
Step 1.
User “John” visits
The Daily Bugle
Step 2.
Bid request
broadcasts personal
data about John
John
40. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
to John
How RTB data leakage supports untrustworthy websites
The Daily Bugle
///
Step 3.
100s of companies in the ad
auction can now re-identify
John as a Daily Bugle reader
Step 1.
User “John” visits
The Daily Bugle
€1 advertisement
Step 2.
Bid request
broadcasts personal
data about John
John
41. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
to John
How RTB data leakage supports untrustworthy websites
The Daily Bugle
Step 5.
Later, John visits a
low quality website
Step 3.
100s of companies in the ad
auction can now re-identify
John as a Daily Bugle reader
Step 1.
User “John” visits
The Daily Bugle
€1 advertisement
De5troyTru5t.com
///
Step 2.
Bid request
broadcasts personal
data about John
John
42. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
to John
How RTB data leakage supports untrustworthy websites
The Daily Bugle
Step 5.
Later, John visits a
low quality website
Step 6.
Bid request
announces John is
here
Step 3.
100s of companies in the ad
auction can now re-identify
John as a Daily Bugle reader
Step 1.
User “John” visits
The Daily Bugle
€1 advertisement
De5troyTru5t.com
///
Step 2.
Bid request
broadcasts personal
data about John
John
43. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
to John
Step 7.
De5troyTru5t.com is paid
€0.01 to show ad to John
How RTB data leakage supports untrustworthy websites
The Daily Bugle
Step 5.
Later, John visits a
low quality website
Step 6.
Bid request
announces John is
here
Step 3.
100s of companies in the ad
auction can now re-identify
John as a Daily Bugle reader
Step 1.
User “John” visits
The Daily Bugle
€1 advertisement
De5troyTru5t.com
€0.01 advertisement
///
Step 2.
Bid request
broadcasts personal
data about John
Worthy sites lose their unique audience, and feed
a business model for the bottom of the Web.
John
44. @johnnyryan
The Daily Bugle
Step 1.
A bot masquerading
as a human visits
The Daily Bugle
Step 2.
Bid request
broadcasts personal
data about Bot///
Fake
How RTB enables to steal from publishers and
advertisers.
fraudsters
45. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
The Daily Bugle
Step 5.
Later, an
untrustworthy website
buts bot traffic
Step 6.
Bid request
announces Bot is
here
Step 3.
100s of companies in the ad
auction can now re-identify
Bot as a Daily Bugle reader
Step 1.
A bot masquerading
as a human visits
The Daily Bugle
€1 advertisement
De5troyTru5t.com
Step 2.
Bid request
broadcasts personal
data about Bot
Bot
///
Fake
///
Fake
How RTB enables to steal from publishers and
advertisers.
fraudsters
46. @johnnyryan
Step 4.
The Daily Bugle is
paid €1 to show ad
Step 7.
De5troyTru5t.com is paid
€0.01 to show ad to Bot
The Daily Bugle
Step 5.
Later, an
untrustworthy website
buts bot traffic
Step 6.
Bid request
announces Bot is
here
Step 3.
100s of companies in the ad
auction can now re-identify
Bot as a Daily Bugle reader
Step 1.
A bot masquerading
as a human visits
The Daily Bugle
€1 advertisement
De5troyTru5t.com
€0.01 advertisement
Step 2.
Bid request
broadcasts personal
data about Bot
Bot
///
Fake
///
Fake
How RTB enables to steal from publishers and
advertisers.
fraudsters
47. @johnnyryan
$ ///
VisitorSiteSupply-side
platform (SSP)
Demand-side
platform (DSP)
Data management
platform (DMP)
Marketer Ad Exchange
Serve page
Request page
Request bid
Request segment
Request bid
Cookie to SSP
Deliver ad
Sync
Deliver segment
Sync
Ad request
Store data
“Demand side” “Supply side”
(one or many)
(10s or 100s or 1000s?)
DSPDMP SSP
48. @johnnyryan
Buyer Seller
Extracts 70-55% of
buyer’s media budget.
Distribution
Marketer
$ DMP DSP Ad Exchange SSP
Site
Unique audience
commodified and
arbitraged.
Untrustworthy sites
business model
enabled.
Bot fraud boosted.
70% figure from the Guardian
and Rubicon case in 2017. 55%
figure from “The Programmatic
Supply Chain: Deconstructing the
Anatomy of a Programmatic
CPM”, IAB, March 2016.
MARKET OVERVIEW (NOW)
PERSONAL DATA IN IAB / GOOGLE RTB
Victims of massive
fraud.
2019 estimates range from $5.7B
(ANA) - $42B (Juniper Research).
49. NPO (publisher) revenue increase, after removing all 3rd party
ad tracking in 2020
Jan. Feb. March April May
COVID-19MARKETSHOCK
June
Data from Ster, NPO’s saleshouse.
+79%
+62%
+27%
+9%
+17%
+25%
+50%
+75%
July Aug.
+17%
+21%
+77%
+54%
Sept.
50. @johnnyryan
1. Sales data from Ster, NPO’s saleshouse.
2. Category rank from NOBO, via Ster. Highest category shown where a property is in more than one category.
NPO property Impressions sold1 Reach rank in category2
nos.nl 186% 3rd (News)
blauwbloed.eo.nl 171% 1st (Royals)
nporadio1.nl 198% 3rd (Music)
kro-ncrv.nl 183% 1st (Dating)
avrotros.nl 112% 8th (Entertainment)
funx.nl 180% 4th (Music)
vpro.nl 192% 19th (Music)
nporadio2.nl 196% 5th (Music)
home.bnnvara.nl 192% 18th (Entertainment)
wnl.nl 189% 38th (News)
nporadio4.nl 199% 12th (News)
3fm.nl 194% 13th (Music)
bvn.nl 197% 20th (Entertainment)
powned.tv 188% 48th (News)
omroepmax.nl 192% 8th (Opinion)
First half of 2020: sales per NPO site
Small
Big
51. @johnnyryan
Extracts much lower %
of buyer’s media budget.
Unique audience
become immune to
commodification and
arbitrage.
No opportunity for
untrustworthy sites.
Bot fraud reduced.
Bot fraud opportunity
reduced.
MARKET OVERVIEW (POST-FIX)
NON-PERSONAL DATA IN IAB / GOOGLE RTB
Marketer
$ DMP DSP Ad Exchange SSP
Site
Buyer SellerDistribution
63. @johnnyryan
There must be a way for an individual to
prevent information about him that was
obtained for one purpose from being
used or made available for other
purposes without his consent.
“
”
Report to Sec. Caspar W. Weinberger. Advisory Committee
on Automated Personal Data Systems, July 1973.
64. @johnnyryan
-GDPR, Article 5 (1) (b)
(b)collected for specified, explicit and legitimate purposes
and not further processed in a manner that is
incompatible with those purposes; further processing for
archiving purposes in the public interest, scientific or
historical research purposes or statistical purposes shall,
in accordance with Article 89(1), not be considered to be
incompatible with the initial purposes
Personal data shall be:
68. @johnnyryan
1 O 1 O1
1 O
1 O 1 O1
1 O 1 O1
1 O 1 O1
Put a company’s
data under the
microscope.
69. @johnnyryan
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 1
1 O 1 O 11 O 1 O 1
1. An organization
collects some personal
data. It is lawful.
2. The organization has
many purposes that it
wants to use the data for.
3. The organization has an internal
data free-for-all.
70. @johnnyryan
1 O 1 O 1
1. An organization
collects some personal
data. It is lawful.
2. The organization has
many purposes that it
wants to use the data for.
3. The organization has an internal
data free-for-all.
4. But this is vulnerable to
enforcement of GDPR Article 5(1)b.
71. @johnnyryan
1 O 1 O 1
Many purposes.
But few lawful
data.
@johnnyryan
82. @johnnyryan
Data through Google’s own
properties (all purposes)
Data collected on other companies’
properties (all purposes)
All data used across all Google
businesses, in all markets
83. @johnnyryan
Data through Google’s own
properties (all purposes)
Data collected on other companies’
properties (all purposes)
All data used across all Google
businesses, in all markets
89. @johnnyryan
Many companies trading personal data
without any control
One big company cross-using personal
data beyond intended purpose, and
bundling consents.
RTB external
data free-for-all
Big tech’s internal
data free-for-all
90. @johnnyryan
4
Big tech operates an internal
data free-for-all. It cross-uses
personal data from its many
disparate services for its
advertising business.
This has created a big tech
monopoly.
Failure to enforce GDPR
Article 5(1)b against big tech’s
internal data free-for-all could
then let big tech envelop the
whole RTB market.
Enforcement of GDPR Article
5(1)f would stop the external
data free-for-all between
thousands of companies in the
“real-time bidding” (RTB)
market.
1
2
3
Robust enforcement of GDPR
Article 5(1)b against big tech’s
internal data free-for-all
would allow publishers to
compete with big tech fairly.
OR
4
The market requires both internal
& external GDPR enforcement
91. @johnnyryan
1. Cross-context adtech steals your
audiences, drives down ad prices, and
lets Tech companies + Junk websites
arbitrage you.
2. Data protection law - if enforced - is big
tech kryptonite.
3. Publishers should lobby for data
protection laws and robust enforcement.