Asko Relas: Machine Learning for conversion optimization – How to be relevant at scale

How to be relevant at
scale
Machine Learning for conversion optimization
31.10.2018Etunimi Sukunimi1Asko Relas

How do we do
conversion optimization
of personalized offers and content
across channels
in an automated manner?

Picking the best converting content
X% Y% Z%
Dreams to fit
any lifestyle
An immersive environment to enhance your
imagination about coziness
Daydreaming
is a state of
mind
Feel at home
on the go
Which variant of a teaser should we show to which customer?

Multi-armed bandit
Which bandit should I play?
Start with random experimentation, and as experience is gained, focus more on the one that
gives the most wins.
X% Y% Z%

Sounds a bit like A/B-testing...
B
A
Testing
(Learning)
C
Traffic
A/B/C test

B
A
Testing
(Learning)
C
Using the best
variant
(Earning)
C
Traffic
A/B/C test

B
A
Testing
(Learning)
C
Using the best
variant
(Earning)
C
Traffic
A/B/C test
Earning while learning
A
C
B
Traffic
Multi-armed bandit

B
A
Testing
(Learning)
C
Using the best
variant
(Earning)
C
Exploit the best converting variant increasingly
Traffic
A/B/C test
Earning while learning
A
C
B
Traffic
Multi-armed bandit

When to use each approach
A/B testing:
• Experiments where the intent is to find the best variant, and to stick with it for some time
• Eg. making decisions on user interface design
Bandit:
• Short tests, where we don’t have enough time to run an A/B test
• Think about news headlines: By the time an A/B test finishes, the news may be old
• Continuous optimization, where focus is on maintaining high conversion rate
• Reacting to for instance changes in behavior over the course of a year

Deep dive
Multi-armed bandits in practice

A conversions B conversions C conversions p %
0 0 0 100
Exploration vs. exploitation
How to balance between learning and earning:
• When deciding which variant to show, with probability p pick a random variant,
otherwise pick the best variant so far
• Start with p = 100% and decrease it as we gain experience

0 0 0 100
1 0 0 99

0 0 0 100
1 0 0 99
1 2 3 95

0 0 0 100
1 0 0 99
1 2 3 95
2 3 7 90

0 0 0 100
1 0 0 99
1 2 3 95
2 3 7 90
4 5 14 80

0 0 0 100
1 0 0 99
1 2 3 95
2 3 7 90
4 5 14 80
7 12 65 40

0 0 0 100
1 0 0 99
1 2 3 95
2 3 7 90
4 5 14 80
7 12 65 40
12 23 162 10

X1%
Dreams to fit
any lifestyle
Z1%
Feel at home
on the go
Y1%
Daydreaming
is a state of
mind
But different users behave differently...

X1%
Dreams to fit
any lifestyle
Z1%
Feel at home
on the go
Y1%
Daydreaming
is a state of
mind
X2%
Dreams to fit
any lifestyle
Z2%
Feel at home
on the go
Y2%
Daydreaming
is a state of
mind

X1%
Dreams to fit
any lifestyle
Z1%
Feel at home
on the go
Y1%
Daydreaming
is a state of
mind
X2%
Dreams to fit
any lifestyle
Z2%
Feel at home
on the go
Y2%
Daydreaming
is a state of
mind
X3%
Dreams to fit
any lifestyle
Z3%
Feel at home
on the go
Y3%
Daydreaming
is a state of
mind

Use contextual data as input
• Look at the context in which the past conversions occurred
• Depends on the business domain
• Demographics (Age group, Geography, Family, ...)
• Temporal (Time of day, Day of week, Time of year)
• Products / services bought
• Channel, Device
• Once we have collected data, we can identify the attibutes that actually do
correlate

Mapping user context to best content
User context
1
0
0
1
0
Age: 18-25
Age: 26-35
Age: 36-45
Channel: web
Channel: mobile
Other attributes...

Variants to pick from
Content A
Content B
Content C
Dreams to fit
any lifestyle
An immersive environment to enhance your imagination
about coziness
Daydreaming
is a state of
mind
about coziness
Feel at home
on the go
User context
1
0
0
1
0
Age: 18-25
Age: 26-35
Age: 36-45
Channel: web
Channel: mobile
Other attributes...

Variants to pick from
Content A
Content B
Content C
Dreams to fit
any lifestyle
about coziness
Daydreaming
is a state of
mind
about coziness
Feel at home
on the go
User context
1
0
0
1
0
Age: 18-25
Age: 26-35
Age: 36-45
Channel: web
Channel: mobile
Neural net for mapping a user context
to a piece of content
0.45
0.56
0.82
Other attributes...

Combining with a product
recommender system

The whole setup
Feedback
Product promotion(s)
Recomms
Calculate product
recomms for each user
Behavioral
data
CRM user
profile
Choose best content for
each recommendation

vs.
The whole setup
Feedback
Recomms
Calculate product
Behavioral
data
CRM user
profile
each recommendation

vs.
The whole setup
Feedback
Recomms
Calculate product
Behavioral
data
CRM user
profile
each recommendation
Dreams to fit
any lifestyle
about coziness
Daydreaming
is a state of
mind
about coziness
Feel at home
on the go

Summary
• A/B testing everything at scale is time consuming, costly, and slow
• Pick your battles and let a machine do the bulk work
• Earn While You Learn
• Data collection is a cost, which bandits attempt to minimize
• Some DXPs already provide bandits out of the box
• We used a neural net for the contextual bandit, but that is not the only possible approach
• Any gradient-descent based algorithm will allow incremental (read: fast) learning
• Random forests etc. require less feature engineering, but are slower learners due to
their batch orientation

31.10.2018Etunimi Sukunimi31
Thank you!

Asko Relas: Machine Learning for conversion optimization – How to be relevant at scale

More Related Content

Similar to Asko Relas: Machine Learning for conversion optimization – How to be relevant at scale

More from Loihde Advisory

Recently uploaded

Asko Relas: Machine Learning for conversion optimization – How to be relevant at scale