Join Optimizely’s Co-founder Pete Koomen for an in-depth discussion about new product developments for measuring the impact of your experiments on the metrics that matter most to your business. In this session you’ll learn: - How Optimizely measures the impact that experiments have on your business - New tools that Optimizely is developing for data scientists and analysts to perform custom and advanced analysis - How your team can get more accurate experiment results faster with Optimizely's data platform

1. Pete Koomen
Co-founder, Optimizely
Metrics that matter
Measure Experiment Impact with
Optimizely’s Data Platform
Logan Linn
Staff Software Engineer, Optimizely

2. 4
Agenda 1. Optimizely Data Lab Overview
2. Live Demo
1. Q&A

3. 5
▪ This is a technical presentation
▪ Optimizely Data Lab is work in progress
▪ Request early access at optimizely.com/data
A Few Notes

4. 6
1. A portable experiment analysis environment for ad-hoc
analysis and building experiment reports with first and
third party data
2. A set of Python experiment analysis libraries
3. Enriched events, a queryable, event-level experiment
dataset
A powerful new toolkit for analysts and data scientists who
want to uncover deeper insights from experiments
Optimizely Data Lab

5. 7
Who is it for?
Data scientists, analysts, engineers and technical product managers who want
to do more with experiment results than what is possible with Optimizely’s
results page today.
For example, customers can:
▪ connect experiment data from third-party data sources
▪ work with metrics and segments that can’t be defined in Optimizely’s UI
▪ use their choice of statistical models
▪ perform custom experiment analyses
▪ build custom experiment reports

6. Demo

7. Experiment Lab

8. Optimizely Data Lab Container
Open in Lab
Report Template
Jupyter Notebook
Notebook
Hub
Time Series Data
API (realtime)
Experiment
Analysis Libraries
Enriched Event Data
Experiment Config
APIs

9. 11
Enriched Events Dataset
timestamp: 09/10/2019 13:50
event_id: 1243254356
visitor_id: ouid_434562346
revenue: 2700
value: 2
user_attributes: {
market: us,
lang: en,
vip: true
}
event_tags: {
product_type: outwear,
order_id: 432564236
}
Enriched
Events
dataset
Enriched
Data Export
experiment_id: 543543632
variation_id: 654645676
session_id: sid_213564367
“purchased” event
Event API

10. Demo

11. 13
▪ We used ODL to ask and answer a series of increasingly
important questions.
▪ We measured the ROI of our experiment on metrics that
our business cares about and trusts.
▪ Best of all, we developed a report template that can be
run on any future Optimizely experiment.
What did we do?

12. Q&A
Request early access at
optimizely.com/data

13. Thank you!

14. Up Next…
▪ Lunch now being served in the Expo Hall
▪ Head to the blue tables during lunch for Industry Roundtables
▪ Lunchtime hands-on feature flags workshop in Yerba Buena 5-6
at 1pm