Slides of the keynote of Gui Liberali at Conversion Hotel 2016: http://www.conversionhotel.com - Morphing Websites

1. 11/21/16
1
Truth, Experimentation, and
Personalization
Website Morphing
Gui Liberali
Erasmus University
Truth and Personalization: Imagine 3
Banner Ads and Hypothetical CTRs
Joe Smith comes to our website, we give him the highest-conversion banner:
Banner Informational, conversion of 0.16
Numbers are illustrative, not actual data
Joe Smith actually is a person, so we learn his true segments and give him the
highest-conversion banner. Banner emotional, Conversion of 0.26
1 st
truth: segment
belonging matters
How Does Truth Matter for Personalization?
Imagine that Joe does not live in a box
Numbers are illustrative, not actual data
Why the Truth Matters for Personalization
2
ndtruth:w
e
m
ay
not
know
the
truth
about
conversion
untilw
e
explore
and
learn
Numbers are illustrative, not actual data
Increasing Conversion Requires Learning both Truths
1. We are not binary - we all have a bit of many segments.
• How do we learn to which segments we belong?
2: What are the true conversion rates ? How do I learn them?
Learning the truth requires good tools.
How good are the tools we use in A/B testing, funnel testing and webdesign?
Summary so far ...a 120-year+ Old Offline Problem
Cosmopolitan’s circulation in 1892: 75,000 monthly copies
Visual optimization was already an issue.
BUT in the 19th century they
could only deliver and print one
cover per month
So, a 100 years ago,
best-on-average was
good enough

2. 11/21/16
2
The tests used in old A/B testing are also
more than a century old
• Hypothesis testing is from 1770
• Correlations are from 1904
• t-test (to find best-on-average) was created in 1908
Which conversion methods do you use in your website?
From :
q 19th century
q 20th century
q 21st century
1. How to Find the Optimal Website Design?
• Is there such a thing as an “average person” ? Or average
person in the segment?
• Nothing wrong with the tests, but with what we do with
them.
• Do you want to learn best creative on average or do
you want to increase conversion?
• Then stop testing creatives. Test methods of deciding
who gets what creative!
Is there Anything Wrong with Old Tests ?
1. How to Find the Optimal Website Design?
Here is one method.
Morphing
One size does not fit all
(a) General content, large-load, graphical morph (b) Focused, small-load, verbal morph
Visual
Technical
Content
More
Content
Audio
Less
Content
General
Content
Website designs as Morphs
Appeals to Verbal
Cognitive Style
Appeals to Visual
Cognitive Style
We showed online sales
of BT group’s broadband
plans to increase up to
20% with morphing
Appeals to Holistic, impulsive cognitive
style
Appeals to Analytical, deliberative
cognitive style
One size does not fit all
Banners as Morphs
We increased CTR on
cnet.com between 97%
and 103% by using
morphing

3. 11/21/16
3
Morphing: real-time adaptation of the
firm’s actions to each consumer
Cognitive-style, segment, position in
the purchase funnel, ..
Website design, banner, advertising
campaign, media copy per channel,
promotion, price, product bundle, product
recommendation …
Consumer
latent
variable
Marketing
instrument
to be adapted
(morphed)
x
Morphing - Evidences from the Field
Morphing Banner Advertising (2014)
• Morphing almost doubled click-through rates for context-
matched banners (83%, p<0.01 and 97%, p=0.028) lifts,
respectively, for banners and for consumers
• Was live on CNET.com (8.4M visitors/day) for a month
• Extends behavioral targeting to avoid local max solutions
Website Morphing (2009)
• Online sales of BT group’s broadband plans can be
increased up to 20% by adapting the website to
match cognitive styles using our adaptive learning
algorithm
Website Morphing 2.0 (2014)
• NPV reward improvement of 69% over the NPV of
the original algorithm; corresponds to $17.5 million
for BT group if implemented
• Proof-of-Feasibility on Suruga Bank
• Generalized version of morphing, including when to
morph, switching costs, multiple morphs
Actual data
Morphing How Does it Work?
Choice descriptor
Priors on distribu6on of
styles
Updated user
style descriptor On-line
Op6miza6on
Pgm: probability of
purchase for individual in
style g given morph m
Banner Ads -> 1 2 3 4
Cogni6ve
Style
Impulsive
Delibera6ve 0.1 0.11 0.2 0.13
Illustra6ve example
User
Op6mal morph for current user
Website
User makes a
purchase or
quit
Bayesian
update
Numbers are illustrative, not actual data
Test algorithms instead of creatives.
Testing
Test and control approach
Control is traditional randomized design: person comes to your website and
you give a random morph (creative)
Test is your new method: user comes to your website and the algorithm
decides which morph (creative) to give to her
Learning the truth about users
and conversion will allow you
to higher conversion via
personalization
Bottom line
• Stop learning (A/B) then earning (roll-out) -> Start earning while
learning! (Exploration/exploitation bandits)
• Stop learning the best creative -> Start learning which creative
is best to which user
• Stop using aggregate data -> Start using individual-level data
and reports
References
• Hauser JR, Urban GL, Liberali G, Braun M (2009) Website morphing. Marketing
Sci. 28(2):202–224.
• Urban G., Liberali G., MacDonald E., Bordley R., Hauser J. (2014) Morphing
Banner Advertising, Marketing Science, 33(1): 27-46.
• Hauser J., Liberali G., Urban G. (Summer 2014) Website Morphing 2.0: Switching
Costs, Partial Exposure, Random Exit, and When to Morph. Management Science.

4. 11/21/16
4
Thank you!
Contact: liberali@rsm.nl