Scaling experiments from 1 to 100+ with Optimizely

•Download as PPTX, PDF•

0 likes•264 views

Are you struggling to go from 1 to 10 or from 100 to 1000 experiments? Optimizely Full Stack is a powerful tool, but knowing best practices for implementation is key to 10x-ing your experimentation program. In this workshop, we'll describe best practices like installing and wrapping the SDK, strategies for code footprint, and fitting a product experimentation culture into your team's process. Come learn how to best leverage Full Stack to scale your product experimentation program.

Technology

2
Scaling Fullstack
Going from 1 to 100+ experiments
Asa Schachar
Engineering Manager
Ali Rizvi
Staff Software Engineer
Thilo Richter
Solutions Architect Manager

Simple Experimentation Setup
Application
SDK
Datafile
A
B
Remote
Configuration
Analytics

Application Backend
app.optimizely.com
Analytics
Backend
Frontend Application
app.optimizely.com
Your App
SDK
Datafile
Optimizely User
Your Users
Events
A
B

Application Backend
app.optimizely.com
Analytics
Backend
Frontend Application
app.optimizely.com

Application Backend
app.optimizely.com
Analytics
Backend
Frontend Application
app.optimizely.com
SDKSDK
SDK

Installing SDKs
Application Backend
app.optimizely.com
Analytics
Backend
Frontend Application
app.optimizely.com
SDKSDK
SDK

Datafile Polling
Application
SDK
Datafile
A
B
5 min
CDN

Datafile Polling
Application
SDK
Datafile
A
B
Remote
Configuration
Analytics
CDN
5 min

Example #1
SDK Stateless
Bucketing Logic
app thread #1
Analytics Backend
Request #1SDK
SDK app thread #2 Request #2
SDK app thread #3 Request #3

Example #1
Analytics Backend
app thread #1 Request #1
app thread #2 Request #2
Wrappe
r
Stateful, Thread-safe,
Singleton Wrapper
Wrapper
SDK
SDK Stateless Bucketing
Logic

100%
100%
Local
(local computer)
Staging
(staging website)
Production
(production website)
Rollback
23
0% 0%
Environments

Challenges of 10-100
Experiments
1. Centralized Metrics Tracking
2.

Scaling experiments from 1 to 100+ with Optimizely

What's hot

Code-to-Cloud Visibility: An Essential Framework for DevOps SuccessJadeCampbell13

Engineering OperationsCybera Inc.

LilaPapiernikResume.DeveloperIILila Papiernik

DevOps Powered by Splunk Hands-OnSplunk

Discovering and Fixing Dependency Vulnerabilities for Kubernetes apps with Sn...Codefresh

ATAGTR2017 Security Testing / IoT Testing in Real WorldAgile Testing Alliance

Why monitoring is an analytics problemPhillip Liu

ATAGTR2017 Machine Learning telepathy for Shift Right approach of testingAgile Testing Alliance

Enterprise Ready Test Execution Platform for Mobile AppsVijayan Srinivasan

Spool manager pipe weld shop Yakup Bozkurt

Software Supply Chain Observability with Grafeas and KritisAysylu Greenberg

ATAGTR2017 Protractor Cucumber BDD ApproachAgile Testing Alliance

ATAGTR2017 CDC Tests - Integration Tests cant be made simpler than this!Agile Testing Alliance

ATAGTR2017 Analytics TestingAgile Testing Alliance

IBM Index Conference - 10 steps to build token based API SecuritySenthilkumar Gopal

ATAGTR2017 HikeRunner: Load Test FrameworkAgile Testing Alliance

Shhh!: Secret Management Practices for Infrastructure as Code Akond Rahman

Turning Cloud Metrics into ResultsInfluxData

OSMC 2013 | The future of Nagios by Andreas EricssonNETWAYS

Characterizing Defective Configuration Scripts Used for Continuous DeploymentAkond Rahman

What's hot (20)

Code-to-Cloud Visibility: An Essential Framework for DevOps Success

Engineering Operations

LilaPapiernikResume.DeveloperII

DevOps Powered by Splunk Hands-On

Discovering and Fixing Dependency Vulnerabilities for Kubernetes apps with Sn...

ATAGTR2017 Security Testing / IoT Testing in Real World

Why monitoring is an analytics problem

ATAGTR2017 Machine Learning telepathy for Shift Right approach of testing

Enterprise Ready Test Execution Platform for Mobile Apps

Spool manager pipe weld shop

Software Supply Chain Observability with Grafeas and Kritis

ATAGTR2017 Protractor Cucumber BDD Approach

ATAGTR2017 CDC Tests - Integration Tests cant be made simpler than this!

ATAGTR2017 Analytics Testing

IBM Index Conference - 10 steps to build token based API Security

ATAGTR2017 HikeRunner: Load Test Framework

Shhh!: Secret Management Practices for Infrastructure as Code

Turning Cloud Metrics into Results

OSMC 2013 | The future of Nagios by Andreas Ericsson

Characterizing Defective Configuration Scripts Used for Continuous Deployment

Similar to Scaling experiments from 1 to 100+ with Optimizely

DevOps Powered by SplunkSplunk

Platform 4.0 Meetup Launch EventFokke Zandbergen

Dynatrace: Going beyond APM and soaring to the futureDynatrace

Predix Builder RoadshowPredix

Data-Driven Software Engineering for Agile TeamsTechWell

Technical Architect on Embedded System.Prasad Roy Raju

Big Data And HTML5 (DevCon TLV 2012)Ido Green

Breaking the 2 Pizza Paradox with your Platform as an ApplicationMark Rendell

Optimizely Agent: Scaling Resilient Feature DeliveryOptimizely

Oracle APEX, Low Code for Data Driving AppsFranco Ucci

Paypal teradata gimel_thrift_serverAnisha Nainani

Release Readiness Validation with Keptn for Austrian Online Banking SoftwareAndreas Grabner

Sharepoint 2013 applied architecture from the field (v2)Tihomir Ignatov

Cirrculum_Vitaevemulakonda

What's new in android 2018 (dev fest)Shady Selim

GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...James Anderson

SoftwareQualityControlProfessional_12Yrs_SreekrishnaHPanditSreekrishna Pandit

AWS Startup Insights SingaporeAmazon Web Services

Resume ram-krishnaRam Krishna | BitHostIn

Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...IDEAS - Int'l Data Engineering and Science Association

Similar to Scaling experiments from 1 to 100+ with Optimizely (20)

DevOps Powered by Splunk

Platform 4.0 Meetup Launch Event

Dynatrace: Going beyond APM and soaring to the future

Predix Builder Roadshow

Data-Driven Software Engineering for Agile Teams

Technical Architect on Embedded System.

Big Data And HTML5 (DevCon TLV 2012)

Breaking the 2 Pizza Paradox with your Platform as an Application

Optimizely Agent: Scaling Resilient Feature Delivery

Oracle APEX, Low Code for Data Driving Apps

Paypal teradata gimel_thrift_server

Release Readiness Validation with Keptn for Austrian Online Banking Software

Sharepoint 2013 applied architecture from the field (v2)

Cirrculum_Vitae

What's new in android 2018 (dev fest)

GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...

SoftwareQualityControlProfessional_12Yrs_SreekrishnaHPandit

AWS Startup Insights Singapore

Resume ram-krishna

Microsoft Cognitive Service, Tap into the Power of Machine Learning with Easy...

Recently uploaded

Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge

Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson

Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes

Understanding the Laravel MVC ArchitecturePixlogix Infotech

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxnull - The Open Security Community

New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada

Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard

08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls

DMCC Future of Trade Web3 - Special EditionDubai Multi Commodity Centre

Pigging Solutions Piggable Sweeping ElbowsPigging Solutions

Unlocking the Potential of the Cloud for IBM Power SystemsPrecisely

The transition to renewables in India.pdfCompetition Advisory Services (India) LLP

Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK

Artificial intelligence in the post-deep learning eraDeakin University

"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays

Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55

Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software

Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited

Install Stable Diffusion in windows machinePadma Pradeep

Recently uploaded (20)

Designing IA for AI - Information Architecture Conference 2024

Are Multi-Cloud and Serverless Good or Bad?

Enhancing Worker Digital Experience: A Hands-on Workshop for Partners

Understanding the Laravel MVC Architecture

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx

New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024

Maximizing Board Effectiveness 2024 Webinar.pptx

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

DMCC Future of Trade Web3 - Special Edition

Pigging Solutions Piggable Sweeping Elbows

Unlocking the Potential of the Cloud for IBM Power Systems

The transition to renewables in India.pdf

Unblocking The Main Thread Solving ANRs and Frozen Frames

Artificial intelligence in the post-deep learning era

"Federated learning: out of reach no matter how close",Oleksandr Lapshyn

Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation

Unleash Your Potential - Namagunga Girls Coding Club

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365

Install Stable Diffusion in windows machine

Scaling experiments from 1 to 100+ with Optimizely

2. 2 Scaling Fullstack Going from 1 to 100+ experiments Asa Schachar Engineering Manager Ali Rizvi Staff Software Engineer Thilo Richter Solutions Architect Manager

3. 1 to 10 Experiments

5. Simple Experimentation Setup Application SDK Datafile A B Remote Configuration Analytics

6. Application Backend app.optimizely.com Analytics Backend Frontend Application app.optimizely.com Your App SDK Datafile Optimizely User Your Users Events A B

7. Application Backend app.optimizely.com Analytics Backend Frontend Application app.optimizely.com

8. Application Backend app.optimizely.com Analytics Backend Frontend Application app.optimizely.com SDKSDK SDK

9. Installing SDKs Application Backend app.optimizely.com Analytics Backend Frontend Application app.optimizely.com SDKSDK SDK

10. Datafile Polling Application SDK Datafile A B 5 min CDN

11. Datafile Polling Application SDK Datafile A B Remote Configuration Analytics CDN 5 min

12. 10 to 100 Experiments

13.

14. Example #1 SDK Stateless Bucketing Logic app thread #1 Analytics Backend Request #1SDK SDK app thread #2 Request #2 SDK app thread #3 Request #3

15. Example #1 Analytics Backend app thread #1 Request #1 app thread #2 Request #2 Wrappe r Stateful, Thread-safe, Singleton Wrapper Wrapper SDK SDK Stateless Bucketing Logic

16. Example #2

17. Example #2

18. Example #2

19. Example #2

20. 100+ Experiments

21.

22.

23. 100% 100% Local (local computer) Staging (staging website) Production (production website) Rollback 23 0% 0% Environments

24.

25. What are your challenges?

26. Challenges of 1-10 Experiments 1. 2.

27. Challenges of 10-100 Experiments 1. Centralized Metrics Tracking 2.