Chris Stein is a lead consultant who helps enterprises determine ROI forecasts, pricing strategies, timelines, and resource needs. He has experience as a product manager for fintech SaaS solutions and has managed core product development. Some of his past roles include leading BPO and change management projects, spearheading agile project management processes, and inspiring customers to build experimentation as a strategic part of their business at Optimizely. Chris provides strategic consulting in digital experience optimization, program structures, and experiment-driven product development for enterprise customers.

1. CHRIS STEIN
Lead Consultant, Value & Strategy
How to Drive Revenue in an
Increasingly Online World

2. Lead Consultant
Value Consultant helping enterprises determine
ROI forecast, pricing, timeline, and resources
Experienced Product Manager
Managed core product development of Fintech
SaaS solution
Selected experience
Customers
Education & Certification
University of Amsterdam (expected)
Master of Business Administration (MBA)
University of Amsterdam
BSc. Business Technologies & Entrepreneurship
Lead Consultant, Optimizely
Inspire enterprise customers to build
experimentation as a strategic arm of their
business.
Product Manager, Equiniti plc
Managed SaaS platform implementations for
several top 10 global banking corporations
Associate, Fiducia Management
Consultants
Led BPO and Change Management projects in
Hong Kong and Greater China
Enterprise Program Build
Spearheaded the establishment of agile
project management governance and
reporting processes
Lean Six Sigma – Green Belt
UNC Plus Delta
Chris Stein
Lead Consultant,
Strategy & Value
Areas of expertise & industry
Chris has a proven track record as Product
Manager, Management Consultant, and Leader
of high-performing teams. At Optimizely, he
provides strategic consulting in digital
experience optimization, program structures,
and experiment-driven product development for
enterprise and Fortune 500 businesses.

3. ● Customize the widgets on your page
to your preference
● This webinar is recorded and you will
receive the link with the slides in the
next few days
● We will have time for questions!
Please submit them in the Q&A box
on the left side of the screen
Housekeeping

4. 1. Optimizely - Digital Lab
2. The Value of Experimentation
3. Kickstarting your Program & Use Cases
4. Q&A
Agenda

5. World’s Largest Digital Laboratory

6. The only solution built for your entire team: engineers, product
managers, growth marketing, and data analysts.
What
We Do
World’s #1 digital laboratory - Optimizely customers have
run over 1.8M experiments on our platform since 2010.
Remove Guesswork and act with confidence
Leader in Progressive Delivery
& Experimentation
Next gen “Test and Learn” platform for enterprise-wide
digital experience optimization
Built for the whole team

7. Number of Variations
Creating a minimum of 4-5 variations
can increase win rates by 40-70%
Metrics Setting
Measuring micro conversions can
increase learning rate by >100%
Instrumenting Analytics
Using analytics to inform hypotheses and
priorities can increase win rates by 40%
Code Complexity
Making more meaningful changes
can increase win rates by >100%
Source: Optimizely Global Benchmark, 100K experiments, n=1000
● Analysis of over 1,000 companies
and more than 100K experiments
● Identification of best practices for
experimentation
Global Optimization
Benchmark
Optimizely is the World’s Leader in Experimentation Best Practices
Key qualities from 100K experiments, measuring the high performing results in terms of driving revenue

8. Category leaders choose Optimizely
Retail B2B Media High Tech
Financial
Services
Consumer
Technology

9. The Value of Experimentation

10. Surge in digital: customer behaviour is shifting
Source: McKinsey & Company: October 26, 2020, https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/a-global-view-of-how-consumer-behavior-is-changing-amid-covid-19
Percentage of customers who have tried new shopping behaviours
since the outbreak of COVID-19
Intent to continue

11. Source: McKinsey & Company: October 14, 2020 https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/these-eight-charts-show-how-covid-19-has-
changed-b2b-sales-forever
Most seller interactions have moved to Remote or Digital

12. Source: McKinsey & Company: October 14, 2020 https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/these-eight-charts-show-how-covid-19-has-
changed-b2b-sales-forever
9 out of 10 companies expect to remain primarily digital

13. “There are known knowns; there are things we know we
know.
We also know there are known unknowns; that is to say
we know there are some things we do not know.
But there are also unknown unknowns — there are things
we do not know we don’t know.”
-- Donald Rumsfeld, US Secretary of State for Defense, 2002

14. “There are known knowns; there are things we know we
know.
We also know there are known unknowns; that is to say
we know there are some things we do not know.
But there are also unknown unknowns — there are things
we do not know we don’t know.”
-- Donald Rumsfeld, US Secretary of State for Defense, 2002
“Experimentation is the QA of User Behaviour”

15. 15
Extended social distancing, some
behaviors are lasting but diminished
What’s the Next Normal: User behavior likely to vary by industry
TODAY
GRADUAL RECOVERY
RAPID RECOVERY
INTO “NEXT NORMAL”
Rapid return to norms and user behavior
of pre-2020; no extended changes
Permanent changes to user behavior
with new patterns emerging

16. 16
What’s the Next Normal: User behavior likely to vary by industry
TODAY
GRADUAL RECOVERY
RAPID RECOVERY
INTO “NEXT NORMAL”
Rapid return to norms and user behavior
of pre-2020; no extended changes
Extended social distancing, some
behaviors are lasting but diminished
Permanent changes to user behavior
with new patterns emerging
React. Still keeping
norms of pre-COVID.
UX changes. New
products/features.
Retention & blue
sky research.

17. Critical pillars to address changing consumer behavior
“Even the simplest IT projects require change management”
OPTION 01
Test & Learn
Culture
Be critical, humble,
and responsive
OPTION 02
Analytics &
Data Trust
Foundational
requirement to
become data driven
instead of opinion
led
Resourcing
& Skill Sets
Empower your
teams
Operational
Strategy
Build for scale
Customer Centricity
Source: McKinsey Global survey of business building, August 2020, n=300
Source: Optimizely Global Benchmark, 100K experiments, n=1000

18. Step
01
Step
02
Step
03
Analytics-led approach
to problem solving
- Identify to value drivers
- Distill findings
- Avoid paralysis
Enable rapid test-&-learn
processes
- Operational model
- Staggered rollout
- Build a practice
Seek Executive
sponsorship early
- Define the vision
- Unlock resources
- Communication
Three critical steps to kickstart your program
How to get started and take experimentation to the next level?
Source: Harvard Business School, February 2020, The Effects of Hierarchy on Learning and Performance in Business Experimentation

19. Kickstarting your Program

20. 21
Growing experimentation maturity builds business value
Experiments
Per Month
Source of
Test Ideas
Expert
Advisory
6-100s consistently
Submission process
across whole business
Strategic Program
Advisory
2-5 every month
Marketing & Product
Technical consulting
Experimentation
Strategy
Critical to Product and
Business planning
Foundational strategy for
specific products
Business buy-in Culture of experimentation
(evidence-based & customer centric)
Evangelism
(Teams)
0-1, ad-hoc
Manual analysis (CRO)
Basic training
Ad-hoc testing
Sporadic
(Individuals)
Executive Commitment
Executive Sponsor &
Steering Meetings
Limited visibility
Not clear
“Tactical” “Growth” “Strategic”

21. Align teams
Consider team needs and ensure cross-team
collaboration for long-term success
Reduce time to first value
Focus on the simple use cases to achieve
quick learnings while building out your
strategy
Define your value metrics
Consider your end users when defining your
experimentation strategy
Experimentation at

22. Strategic buy-in
Senior business leaders help promote a culture of
experimentation
Build vs. Buy
Augment existing tools to enable best coverage for the
respective use cases
Enablement
Tell them what you will tell, tell them, and then tell
what you told them
Building for Growth

23. E13n & Feature Management
Be nimble by using MVP/MVE first, or by
pursuing the Riskiest Assumption Test (RAT)
Source of insights
Steer discussion towards what you are trying
to solve before you entertain how to solve it
Rituals and Artefacts
Raise awareness, set standard practices,
and establish supportive communities
Experimentation at
Farfetch eBook: https://www.farfetchtechblog.com/en/blog/post/experimentation-at-farfetch-an-introduction/

24. 1. Customer behaviour is volatile
2. Embrace the unknown unknowns
3. It’s a process, not a project
Key Takeaways

25. Q&A

26. Thank you!