29. 29
“The philosophy is you have to
build your company to be a
big experimental engine and it has to
start right at the beginning.”
Jeff Holden
Uber Chief Product Officer,
2016
41. 41
YOUR DATA
WAREHOUSE
CUSTOM QUERIES
DATA VISUALIZATION
SELECT optimizely.variation_id AS variation,
COUNT(optimizely.visitor_id) AS high_value_visitors,
SUM(optimizely.revenue) AS total_revenue,
SUM(ltv.ltv_to_date) AS ltv_to_date
FROM optimizely_schema AS optimizely
LEFT JOIN master_ltv_table AS ltv
ON optimizely.visitor_id = ltv.customer_id
WHERE optimizely.experiment_id = 64538596
AND optimizely.revenue > 1500
AND date(optimizely.timestamp) > ‘2018-07-01
GROUP BY variation
ORDER BY day DESC;
GROUP BY “variation”
ORDER BY “day” DESC;
42. 42
Access your results
data from data
warehouse
Analyze with flexibility
via custom SQL
queries
Build dashboards using
your favorite data
visualization tools
Enhanced Data Export
COMING SOON
69. 69
Send experiment context to
New Relic APM
Measure performance by
variation or experiment
Optimize the performance of
server-side tests
ANNOUNCING
New Relic APM Integration
80. 80
“The differentiator for us is
ensuring that we experiment
our way to a better customer
experience.”
Abdul Mullick
Head of Digital Transformation, Sky
89. 89
Match concepts,
not rigid rules
Use machine learning to
build dynamic segments
Adapt in real time, with
no “cold start”
Target based on
predictive intent
Adaptive Audiences
COMING SOON
90. LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
PGA Championship
Tiger Woods
91. LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
PGA
Tiger Woods
Championship
Ferrari edges out Mercedes after last
year’s strong showing in the world
championship.
This Year’s Top Formula 1 Teams
Ferrari
Formula 1
championship.
Mercedes
92. LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
Ferrari edges out Mercedes after last
year’s strong showing in the world
championship.
This Year’s Top Formula 1 Teams
Ferrari
Mercedes
Formula 1
world championship.
93. LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
Ferrari edges out Mercedes after last
year’s strong showing in the world
championship.
This Year’s Top Formula 1 Teams
Republicans and Democrats can each find
reasons for hope in the latest polls for
2018’s midterm elections.
Latest Senate Polls Show Tight Race
94. LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
Ferrari edges out Mercedes after last
year’s strong showing in the world
championship.
This Year’s Top Formula 1 Teams
Republicans and Democrats can each find
reasons for hope in the latest polls for
2018’s midterm elections.
Latest Senate Polls Show Tight Race
Marine biologists announced the
discovery of a new species living in the
deepest reaches of the Mariana Trench
New Tropical Fish Species Discovered
95. Politics
Interest
Sports
Fans
Science
Geeks
Kids
LISTEN: PGA Championship Podcast
Tiger Woods made a big splash Thursday
at the BMW Championship. Can he keep
up the momentum?
Ferrari edges out Mercedes after last
year’s strong showing in the world
championship.
This Year’s Top Formula 1 Teams
Republicans and Democrats can each find
reasons for hope in the latest polls for
2018’s midterm elections.
Latest Senate Polls Show Tight Race
Marine biologists announced the
discovery of a new species living in the
deepest reaches of the Mariana Trench
New Tropical Fish Species Discovered
96.
97.
98. The Ultimate
Sports Package
Enjoy live action across 23 channels in stunning
HD, including Premier League, full Olympics
coverage, and the climax to the 2018 Formula 1®
season - all from the comfort of your couch.
Available on TV, mobile apps, and tablet.
$59
ENJOY THIS SPECIAL OFFER
ADD ON TODAY
$20a month
99. $89
SIGN ON TODAY
Entertain the
Whole Family
Keep the children entertained with 11 live
channels and over 5,000 episodes on demand
in our new Kids plan. It also comes with the Cool
Kids app, packed with games and music, with
content updated every month.
GET STARTED
$60a month
106. 106
How could we answer more questions on
the website, before you call in?
OPTIMIZATION MANAGER
What if you could check your balance
through our Android app?
MOBILE DEVELOPER
If you’ve already been through online
diagnostics, could we pick up where you
left off on the call?
PRODUCT OWNER
107. 107
SALES MANAGER
How might we use support
calls to upsell new services?
How could we answer more questions
on the website, before you call in?
OPTIMIZATION MANAGER
Why don’t we A/B test our hours?
Would 24-hour support be worth it?
SUPPORT DIRECTOR TECHNICAL SUPPORT ADVISER
Could we test a different script for
responding to billing issues?
What if you could check your balance
through our Android app?
MOBILE DEVELOPER
What if we offered video tips for
setting up interactive TVs?
SATELLITE TECHNICIAN
If you’ve already been through online
diagnostics, could we pick up where you
left off on the call?
PRODUCT OWNER
108. 108
What if we played better music
while you’re waiting on hold?
UX DESIGNER
What if we tested male vs. female voices
on the interactive voice response?
OPTIMIZATION MANAGER
What if you could check your balance
through our Android app?
MOBILE DEVELOPER
If you’ve already been through online
diagnostics, could we pick up where you
left off on the call?
PRODUCT OWNER
Could we answer this with a
chatbot or a voice interface?
DIGITAL STRATEGIST
Why don’t we offer priority
support for high value customers?
SUPPORT DIRECTOR TECHNICAL SUPPORT ADVISER
How could we offer better support to
deaf or hard-of-hearing customers?