Machine Learning for Performance Advertising

Copyright © 2015 Criteo
Machine Learning for Performance Advertising
Grenoble Data Science

Machine Learning for Performance Advertising
Eustache Diemert, Staff Research Scientist @ Criteo Research
Grenoble Data Science Meetup – Oct. 2017

Part I : Introduction to Performance Advertising

Performance Advertising ?
4
• Advertisers want sales
• short-term
• measurable impact
• Not interested in
• brand awareness
• marketing pressure
• segments (e.g. socio-demo)

Programatic Advertising Scenario
56€
Promo !

User 123456789

User 123456789
For Sale

User 123456789
For sale
0,10€
0,15€
Advertiser 1
Advertiser 2

User 123456789
For Sale
0,10€
0,15€
~30 ms

Winning
Advertiser

Performance Advertising Setup
12
Advertiser
Publisher
1. User visits a publisher webpage
2. Bidders recieve real-time auction
3. Winner displays ad for advertiser
4. User converts on advertiser website
(click / sale / lead)
CPA
CPM
Bidder

Performance Advertising Metrics
13
• Ideally: number of sales « generated » by advertising for a given
budget
• But difficult/costly to measure and optimize
• E.g. incrementality A/B test
• (also sales amount, margin etc)
• Practically: number of sales attributed to advertising
• Commonly: last click attribution
• (also multi-touch, data driven etc)

Real-time bidding for performance
advertising
Key question : how much should we bid in the auction ?

A little bit of game theory: 2nd price auctions
Sealed, 1 turn auction, winner pays the second highest bid
Value = 1€
bid= 0,75€
bid= 1,1€
Value = 1€
bid= 0,75€
bid= 1,1€
Competition:
0,5€
Competition:
1,5€
Case 1
Case 2
Value = 1€
bid= 0,75€
bid= 1,1€
Competition:
1,05€
Case 3
Value = 1€
bid= 0,75€
bid= 1,1€
Competition:
0,8€
Case 4

Auction games
• Second-price auctions
• Dominant strategy: bid the expected gain (« truthful auction »)
• An overbid means you are losing money
• An underbid means you are losing potential revenue
• Also: non-second price
• Floors (hard/soft/dynamic)

Baseline Bidding Policy
17
• Under 2nd price auction hypothesis, dominant strategy is to bid
expected value
𝑏𝑖𝑑∗
= 𝐶𝑃𝐴 × 𝑝𝑆𝑎𝑙𝑒
« Probability of post-click
attributed conversion »
« Value of a conversion »

Baseline Bidding Policy
18
• Under 2nd price auction hypothesis, dominant strategy is to bid
expected value
𝑏𝑖𝑑∗
= 𝐶𝑃𝐴 × 𝑝𝑆𝑎𝑙𝑒
« Probability of post-click
« Value of a conversion »
Model quality/calibration impacts
revenue

§ What can we use to predict clicks & sales?
§ User behavior on advertizer’s website
§ time since last visit
§ engagement level
§ last product seen, etc..
§ user fatigue: nb displays in last x days
Data features
§ Publisher:
§ publisher_id
§ url
§ display format
§ Campaign:
§ vertical_id: travel, classified, cars, etc
§ average ctr

Learn on huge volumes of data
10 000 displays

10 000 displays
leads to
50 clicks

10 000 displays
leads to
50 clicks
leads to
1 sale

Sizing of our prediction problems
§ Class unbalance: 0.5 / 100
§ N samples: 109
§ N raw variables: 102
§ N encoded features: 107

Which algo to solve our problems?
Structured data
• Lots of info in the data
• High predictability
• Highly structured info
Unstructured data
• Poor predictability
• Signal dominated by noise
• Highly unstructured info

§ Predict: P(Sales) = P(Click) P(Sales | Click)
§ P(Sales) ~ Bernoulli
§ Use (regularized) logistic regression
P(Y=1 | X) = 1/ (1+e-wTx)
§ Outputs a score in [0,1], interpreted as a probability
§ Negative log likelihood:
NLLH (y, p) = – y log p – (1 – y) log (1 – p)
• Convex Optimization, using (cheap) 1+st order methods (SGD, L-BFGS, SAG, …)
Optimizing for sales

§ Vanilla Logistic Regression uses binary features only
§ Standard representation of categorical features: “one-hot” encoding
For instance, site feature
§ Dimensionality equal to the number of different values -- can be very large
§ Hashing to reduce dimensionality (made popular by John Langford in VW)
Hashing trick
cnn.com news.yahoo.com
0 0 01 0 0 0
h : string ! [0 . . . 2b
1]

§ Outer product between two features; similar to a polynomial kernel of degree 2
§ Large number of values hashing trick.
§ Example: between site and advertiser,
Feature is 1 site=finance.yahoo.com & advertiser=bank of america
Quadratic features
Publisher network
Publisher
Site
Url
Advertiser network
Ad
Campaign
Advertiser
,

Part II : Attribution Model for Bidding Performance
Joint work with Julien Meynet, Pierre Galland, Damien Lefortier
published at AdKDD & TargetAd workshop (KDD 2017)

Outline
• The problem: bidding in display advertising
• Model:
• Attribution model
• Attribution aware bidder
• Impact on offline evaluation metrics
• Experience & results

« Post-click attributed conversions »?
30
Display ad
impression
Paid search
click
Display ad
click
Email
open $$$
• Last-click is the de facto attribution model…
… but advertisers are moving towards “better” attribution models:
• Rule-based, uniform, linear, etc..
• Data driven: regression, shapley value, etc..
• But what is the impact from a bidder’s perspective?
• What is the optimal bidding strategy right after a click?

Attribution Probability Through Time Matters
32
Attributionprobability
givenconversion

Attribution Model
33
• How can we model probability of getting the attribution given there
will be a conversion?
• 𝑆: Post click conversion
• 𝐴: Attributed conversion
• 𝑋: Contextual features
• Δ: Delay click/conversion
𝑃𝑟 𝐴 = 1 𝑆 = 1, 𝑋 = 𝑥, Δ = δ) = 𝑒9: ; <
,
𝜆 𝑥 ≥ 0Tapez une équation ici.

Conversion Modeling
34
• Baseline solution:
• 0/1 prediction problem ⟹ Logistic Regression
• Large scale / latency constraint ⟹ Hashing trick
𝑏𝑖𝑑∗
= 𝐶𝑃𝐴 × 𝑝𝑆𝑎𝑙𝑒 « Probability of post-click
But what are positives / negatives?

From Attribution Model to an Attribution Aware Bidder
35
𝐴PQ 0 0 1
𝐴RS 1 3⁄ 1 3⁄ 1 3⁄
𝐴VQ 1 0 0
𝐴WPP 1 1 1
𝐴WX 0.6 0.1 0.3
Cast the problem
as an internal
attribution
problem

Attribution Aware Bidder: An Intuitive View
36
AB: previous click gives us the
attribution, only bid « marginal value »
LCB: user is engaged, go for last-clickbidvalue
t
New display opportunity

Attribution Aware Bidder
37
• Baseline Last-click Bidder (LCB)
• Attribution-aware Bidder (AB):
𝛿[: time elapsed since last click
𝑏𝑖𝑑 = 𝐶𝑃𝐴 × 𝑃𝑟 𝐴PQ = 1 𝑋 = 𝑥) Tapez une équation ici.
𝑏𝑖𝑑 = 𝐶𝑃𝐴 × 𝑃𝑟 𝐴WPP = 1 𝑋 = 𝑥) 1 − 𝑒9: ; <b , Tapez une équatio
Bid proportionally to the marginal contribution of the display

Impact on the
offline evaluation
metrics

Offline Evaluation of Bidders
39
• Utility metric on logged
feedbacks:
• Expected Utility: add uncertainty
on the cost distribution:
𝑐 ~ Γ 𝛼 = 𝛽𝑐h + 1, 𝛽
𝑈 𝑝k
= l(𝑎h 𝑣h − 𝑐h)𝕀(𝑝k
h
𝑣h > 𝑐h)
h∈s
Tapez une équation ici.
𝑝k
h
𝑣h
𝑐h

Attribution Aware Expected Utility*
40
• Inject attribution function in the Utility:
𝐴𝑈 𝑝k
, 𝑎 = l(𝑎(𝑥h)𝑣h − 𝑐h)𝕀(𝑝k
h
𝑣h > 𝑐h)
h∈s
Tapez une équation ici.
Internal attribution function:
• can be last-click, first click, etc..
• can be the proposed attribution
model
* Evaluation of the proposed metric would require a
proper offline / online correlation analysis

Offline Evaluation - Dataset
42
Log sampled from 30 days of Criteo traffic
• Anonymized
• Each line is an impression with:
• Timestamp
• Price paid
• Contextual features (user, advertiser, publisher)
• Click*, click position*, click number*
• Conversion*, conversion value*
• Attribution label (conversion was attributed to Criteo)
• 16M displays, 5M clicks, 800k conversions
Will be available at http://research.criteo.com/ soon

Attribution Rates vs Time
43
Decay of attribution rate after a click
> 40% of conversions have
more than one click in the
preceding 30d

Offline Evaluation – Impact on Bid Profiles
44
Post-click bid profiles for 3 bidders:
• Last-Click Bidder (LCB)
• First-Click Bidder (FCB)
• Attribution Bidder (𝐴𝐵)
All models are learn using
regularized logistic regression
+ hashing trick

Offline Evaluation – Bidders Comparision
45
Results for 3 bidders on the Attribution Aware Expected Utility
𝐿𝐶𝐵 𝐹𝐶𝐵 𝐴𝐵
Win Rate 0.94 0.90 0.89
𝑈W
∗
, 𝛽 = 1000 2852 ± 43 2888 ± 43 𝟑𝟑𝟗𝟔 ± 𝟓𝟑
• We limit user over exposure after a click
• We get closer to lift-based bidding
• We can reinvest budget on more profitable campaigns / more
incremental ads

Online result
46
We tested online a simple modification of baseline through A/B
testing:
Δ𝑂𝐸𝐶
(long term)
Revenue
(short term)
Advertiser
ROI
User ad
exposure
+𝟓. 𝟓%
world wide
negative positive lower
𝑏𝑖𝑑ˆ‰Šˆ = 𝑏𝑖𝑑‹‰Œ × 𝐴 1 − 𝐵𝑒9:<b Tapez une équation ici.

Work in progress & Next steps
• Better attribution modeling
• Exponential decay is naive: build a better model (e.g travel
partners have different attribution schemes)
• Model both conversion lift and attribution lift
• Delayed feedback in both cases
• Derive a robust (counterfactual) offline metric

Questions?
References
Simple and Scalable Response Prediction for
Display Advertising, O. Chapelle, E. Manavoglu,
and R. Rosales, ACM TIST, 2013.
Offline Evaluation of Response Prediction in
Online Advertising Auctions, O. Chapelle,
WWW’15
Attribution Modeling Increases Efficiency of
Bidding in Display Advertising, E. Diemert, J.
Meynet, P. Galland, D; Lefortier KDD’17 TargetAd
workshop best paper finalist
http://labs.criteo.com
Ø Articles on dev & science at Criteo
http://research.criteo.com
Ø Conference reports & cutting edge science ;)
e.diemert@criteo.com

Machine Learning for Performance Advertising

Recommended

Recommended

More Related Content

Similar to Machine Learning for Performance Advertising

Similar to Machine Learning for Performance Advertising (20)

Recently uploaded

Recently uploaded (20)

Machine Learning for Performance Advertising