SlideShare a Scribd company logo
Overcoming the Challenges of
Experimentation on a Service
Oriented Architecture
Lupin Campos & Stephen Schwahn

September 2020
Hi! We are Lupin and Steph :)
Lupin Campos
Director of Product Growth
Tech Startup Founder (Funnely)
Software Development Company Founder (Leanbirds & Condiminds)
Former Venture Partner @ 500Startups
Steph Schwahn
Growth Team Tech Lead
Tech Startup Founder (Crescendo)
Mobile App Development Company Founder (Plutonium Apps)
Georgia Tech Alumni, graduated with high honors.
So far:
- 1 complete backend refactor 

- 2 frontend refactors

- 36 Registration flow experiments (web + fullstack)

- 40% Improvement on Registration Conversion Rate
1Let’s Get Into
Context
A quick note on who we are, what we do, and the
challenges that we found in the way.
This is our registration Process...
Phone
Number
Role
Verification
Code
Email &
Password
Child Info
Personal
Info
Sync
Bank Acc
(Website)
….And here are the services behind it.
A simplified version, you don’t want to see the entire thing. Trust us.
1Service Oriented Architectures with
Optimizely
(the wrong way)
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack on Frontend
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack on Frontend
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
Variant: Variant 1
Variable: FlowId = 6
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'
30s 40s
400s 200s
5s
Greenlight Financial Technology, Inc.
SOA with Optimizely Fullstack
Web
App
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
User_id: '1234_4321'
30s 40s
400s 200s
5s
1Service Oriented Architectures with
Optimizely
(our way)
Greenlight Financial Technology, Inc.
Solution: SOA with Optimizely Agent
Marketing
Site
Mobile
App Edge
Service
1
Edge
Service
2
Nested
Service
1
Nested
Service
2
Helper
Service
1
Backend
Feature or Experiment
Key: 'featureTest’
User_id: '1234_4321'
Optimizely
Agent
30s
Web
App
Greenlight Financial Technology, Inc.
In Practice….
Client 1
Client 2
Client n
.
.
.
Ambassador
(API Gateway)
Pod (3-30 running based on traffic)
Optimizely
Service
(Node.js)
Optimizely
Agentlocalhost:8080
Datafile sync
(every 30s)
localhost:3000
Exposes Optimizely Service
on service port 80
Service
grpc:80
pod-ip:3000
NodeJS Container Go Container (sidecar)
Datafile
pod-ip:3000pod-ip:3000
1Let’s talk about “User ID”
And how do we pass it everywhere!
Greenlight Financial Technology, Inc.
Used for bucketing
MurmurHash(“user_1234”)
Rollout to 60%
Greenlight Financial Technology, Inc.
Used for bucketing
MurmurHash(“user_5678”)
Rollout to 60%
Greenlight Financial Technology, Inc.
What if we don’t have the user id?
MurmurHash(“steph@greenlight.me”)
?
Rollout to 60%
Greenlight Financial Technology, Inc.
Introducing the Experimentation ID!
MurmurHash(“exp_user_1234”)
Rollout to 60%
1Introducing
Experimentation-ID
Service
How we keep a consistent user id regardless of
context
Greenlight Financial Technology, Inc.
Experimentation
-ID service
Greenlight Financial Technology, Inc.
Optimizely Service orchestrates
Greenlight Financial Technology, Inc.
New API Contract!
export type Context = UserIdContext | EmailContext | AnonymousIdContext | RegistrationIdContext | DeviceInfoContext
export type EmailContext = {
email: string
}
export type AnonymousIdContext = {
anonymousId: string
}
export type DeviceInfoContext = {
deviceOs: string,
deviceToken: string
}
export type UserIdContext = {
userId: string
}
export type RegistrationIdContext = {
registrationId: string
}
export function isFeatureEnabled(featureKey: string, context: Context, attributes?: Attributes)
export function getEnabledFeatures(context: Context, attributes?: Attributes)
...
1Truly Full Stack
Experimentation
Through the ENTIRE user journey, from the
marketing landing page, through registration
services, and all the way to fundamental service
primitives
Thank You
greenlightcard.com #greenlight
Science of delivering it right

More Related Content

What's hot

Opticon 2017 How Developers Can Take Experimentation
Opticon 2017 How Developers Can Take ExperimentationOpticon 2017 How Developers Can Take Experimentation
Opticon 2017 How Developers Can Take Experimentation
Optimizely
 
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
Optimizely
 
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance EdgeAhead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Optimizely
 
Evolving Experimentation from CRO to Product Development
Evolving Experimentation from CRO to Product DevelopmentEvolving Experimentation from CRO to Product Development
Evolving Experimentation from CRO to Product Development
Optimizely
 
[Webinar] How Vivid Seats Ensures Experimentation Success
[Webinar] How Vivid Seats Ensures Experimentation Success [Webinar] How Vivid Seats Ensures Experimentation Success
[Webinar] How Vivid Seats Ensures Experimentation Success
Optimizely
 
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
South Tyrol Free Software Conference
 
ATAGTR2017 What Lies Beneath Robotics Process Automation
ATAGTR2017 What Lies Beneath Robotics Process AutomationATAGTR2017 What Lies Beneath Robotics Process Automation
ATAGTR2017 What Lies Beneath Robotics Process Automation
Agile Testing Alliance
 
Streamlining Automation Scripts and Test Data Management
Streamlining Automation Scripts and Test Data ManagementStreamlining Automation Scripts and Test Data Management
Streamlining Automation Scripts and Test Data Management
QASymphony
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation
Optimizely
 
Evolve or Die: Healthcare IT Testing | QASymphony Webinar
Evolve or Die: Healthcare IT Testing | QASymphony WebinarEvolve or Die: Healthcare IT Testing | QASymphony Webinar
Evolve or Die: Healthcare IT Testing | QASymphony Webinar
QASymphony
 
6 Strategies to Uncover Growth Opportunities for Your Business
 6 Strategies to Uncover Growth Opportunities for Your Business 6 Strategies to Uncover Growth Opportunities for Your Business
6 Strategies to Uncover Growth Opportunities for Your Business
Atlassian
 
Taking Your Product Development to the Next Level with Full Stack
Taking Your Product Development to the Next Level with Full StackTaking Your Product Development to the Next Level with Full Stack
Taking Your Product Development to the Next Level with Full Stack
Optimizely
 
Thinking Beyond HPQC ALM
Thinking Beyond HPQC ALMThinking Beyond HPQC ALM
Thinking Beyond HPQC ALM
QASymphony
 
Supercharging Optimizely Performance by Moving Decisions to the Edge
Supercharging Optimizely Performance by Moving Decisions to the EdgeSupercharging Optimizely Performance by Moving Decisions to the Edge
Supercharging Optimizely Performance by Moving Decisions to the Edge
Optimizely
 
Take Your Experimentation Program to the Next Level
Take Your Experimentation Program to the Next LevelTake Your Experimentation Program to the Next Level
Take Your Experimentation Program to the Next Level
Optimizely
 
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
Roger Snook
 
Delivering Excellent Digital Quality to Your Customers
Delivering Excellent Digital Quality to Your CustomersDelivering Excellent Digital Quality to Your Customers
Delivering Excellent Digital Quality to Your Customers
Applause
 
Tackling Strategic Engineering Challenges
Tackling Strategic Engineering ChallengesTackling Strategic Engineering Challenges
Tackling Strategic Engineering Challenges
Applause
 
DevOps: A Value Proposition
DevOps: A Value PropositionDevOps: A Value Proposition
DevOps: A Value Proposition
Nicole Forsgren
 
DevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
DevOps Beyond the Buzzwords: Culture, Tools, & Straight TalkDevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
DevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
Mark Heckler
 

What's hot (20)

Opticon 2017 How Developers Can Take Experimentation
Opticon 2017 How Developers Can Take ExperimentationOpticon 2017 How Developers Can Take Experimentation
Opticon 2017 How Developers Can Take Experimentation
 
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
 
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance EdgeAhead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
 
Evolving Experimentation from CRO to Product Development
Evolving Experimentation from CRO to Product DevelopmentEvolving Experimentation from CRO to Product Development
Evolving Experimentation from CRO to Product Development
 
[Webinar] How Vivid Seats Ensures Experimentation Success
[Webinar] How Vivid Seats Ensures Experimentation Success [Webinar] How Vivid Seats Ensures Experimentation Success
[Webinar] How Vivid Seats Ensures Experimentation Success
 
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
SFScon21 - Paolo d’Incau - Going to production in a few months – How we did it!
 
ATAGTR2017 What Lies Beneath Robotics Process Automation
ATAGTR2017 What Lies Beneath Robotics Process AutomationATAGTR2017 What Lies Beneath Robotics Process Automation
ATAGTR2017 What Lies Beneath Robotics Process Automation
 
Streamlining Automation Scripts and Test Data Management
Streamlining Automation Scripts and Test Data ManagementStreamlining Automation Scripts and Test Data Management
Streamlining Automation Scripts and Test Data Management
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation
 
Evolve or Die: Healthcare IT Testing | QASymphony Webinar
Evolve or Die: Healthcare IT Testing | QASymphony WebinarEvolve or Die: Healthcare IT Testing | QASymphony Webinar
Evolve or Die: Healthcare IT Testing | QASymphony Webinar
 
6 Strategies to Uncover Growth Opportunities for Your Business
 6 Strategies to Uncover Growth Opportunities for Your Business 6 Strategies to Uncover Growth Opportunities for Your Business
6 Strategies to Uncover Growth Opportunities for Your Business
 
Taking Your Product Development to the Next Level with Full Stack
Taking Your Product Development to the Next Level with Full StackTaking Your Product Development to the Next Level with Full Stack
Taking Your Product Development to the Next Level with Full Stack
 
Thinking Beyond HPQC ALM
Thinking Beyond HPQC ALMThinking Beyond HPQC ALM
Thinking Beyond HPQC ALM
 
Supercharging Optimizely Performance by Moving Decisions to the Edge
Supercharging Optimizely Performance by Moving Decisions to the EdgeSupercharging Optimizely Performance by Moving Decisions to the Edge
Supercharging Optimizely Performance by Moving Decisions to the Edge
 
Take Your Experimentation Program to the Next Level
Take Your Experimentation Program to the Next LevelTake Your Experimentation Program to the Next Level
Take Your Experimentation Program to the Next Level
 
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
Mobile DevOps: Rapid Response to Feedback and continuous delivery in a mobile...
 
Delivering Excellent Digital Quality to Your Customers
Delivering Excellent Digital Quality to Your CustomersDelivering Excellent Digital Quality to Your Customers
Delivering Excellent Digital Quality to Your Customers
 
Tackling Strategic Engineering Challenges
Tackling Strategic Engineering ChallengesTackling Strategic Engineering Challenges
Tackling Strategic Engineering Challenges
 
DevOps: A Value Proposition
DevOps: A Value PropositionDevOps: A Value Proposition
DevOps: A Value Proposition
 
DevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
DevOps Beyond the Buzzwords: Culture, Tools, & Straight TalkDevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
DevOps Beyond the Buzzwords: Culture, Tools, & Straight Talk
 

Similar to Overcoming the Challenges of Experimentation on a Service Oriented Architecture

2018-10-25 Group RPA - AA Deck.pptx
2018-10-25 Group RPA - AA Deck.pptx2018-10-25 Group RPA - AA Deck.pptx
2018-10-25 Group RPA - AA Deck.pptx
saubhagya ranjan
 
Astin Profile
Astin ProfileAstin Profile
Astin Profile
Sharnenda Dey
 
Astin Profile
Astin ProfileAstin Profile
Ideamart nsbm outreach 2015
Ideamart nsbm outreach 2015Ideamart nsbm outreach 2015
Ideamart nsbm outreach 2015
Shafraz Rahim
 
Softjourn ticketing experience
Softjourn ticketing experienceSoftjourn ticketing experience
Softjourn ticketing experience
Natalie Senyk
 
For Engineers
For EngineersFor Engineers
For Engineers
Jonathan Peters
 
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
apidays
 
Online Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
Online Banking 2.0 webinar, October 28, 2010 by Jouk PleiterOnline Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
Online Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
Backbase
 
Future Of Work - GetLinks Service Introduction 2020
Future Of Work - GetLinks Service Introduction 2020Future Of Work - GetLinks Service Introduction 2020
Future Of Work - GetLinks Service Introduction 2020
Djoann Fal
 
20091203 Presentatie Eurostar V02
20091203 Presentatie Eurostar V0220091203 Presentatie Eurostar V02
20091203 Presentatie Eurostar V02
Henri Haarmans
 
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunitiesThe Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
OpenSpan
 
Globsync Technology IT & WEB Services
Globsync Technology IT & WEB ServicesGlobsync Technology IT & WEB Services
Globsync Technology IT & WEB Services
Saurav Misra
 
Actminds Outsourcing Summit 07
Actminds Outsourcing Summit 07Actminds Outsourcing Summit 07
Actminds Outsourcing Summit 07
cnetto
 
Deliver agile flow presentation (1)
Deliver agile   flow presentation (1)Deliver agile   flow presentation (1)
Deliver agile flow presentation (1)
James Urquhart
 
Cloud computing: Stan Freck
Cloud computing: Stan FreckCloud computing: Stan Freck
Cloud computing: Stan Freck
Lisa Malone
 
The Bigger Picture: New Opportunities for the Modern Enterprise
The Bigger Picture: New Opportunities for the Modern EnterpriseThe Bigger Picture: New Opportunities for the Modern Enterprise
The Bigger Picture: New Opportunities for the Modern Enterprise
Inside Analysis
 
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
e-Xpert Solutions SA
 
DevOps as Digital Transformation
DevOps as Digital TransformationDevOps as Digital Transformation
DevOps as Digital Transformation
Peter Muryshkin
 
Industrial Presentaion
Industrial Presentaion Industrial Presentaion
Industrial Presentaion
Hasitha Ediriweera
 
SigmaTec Solutions
SigmaTec SolutionsSigmaTec Solutions
SigmaTec Solutions
asalman
 

Similar to Overcoming the Challenges of Experimentation on a Service Oriented Architecture (20)

2018-10-25 Group RPA - AA Deck.pptx
2018-10-25 Group RPA - AA Deck.pptx2018-10-25 Group RPA - AA Deck.pptx
2018-10-25 Group RPA - AA Deck.pptx
 
Astin Profile
Astin ProfileAstin Profile
Astin Profile
 
Astin Profile
Astin ProfileAstin Profile
Astin Profile
 
Ideamart nsbm outreach 2015
Ideamart nsbm outreach 2015Ideamart nsbm outreach 2015
Ideamart nsbm outreach 2015
 
Softjourn ticketing experience
Softjourn ticketing experienceSoftjourn ticketing experience
Softjourn ticketing experience
 
For Engineers
For EngineersFor Engineers
For Engineers
 
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
Apidays Paris 2023 - How to use NoCode as a Microservice, Benjamin Buléon and...
 
Online Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
Online Banking 2.0 webinar, October 28, 2010 by Jouk PleiterOnline Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
Online Banking 2.0 webinar, October 28, 2010 by Jouk Pleiter
 
Future Of Work - GetLinks Service Introduction 2020
Future Of Work - GetLinks Service Introduction 2020Future Of Work - GetLinks Service Introduction 2020
Future Of Work - GetLinks Service Introduction 2020
 
20091203 Presentatie Eurostar V02
20091203 Presentatie Eurostar V0220091203 Presentatie Eurostar V02
20091203 Presentatie Eurostar V02
 
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunitiesThe Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
The Light Bulb Moment – Learning to-identify-robotic-automation-opportunities
 
Globsync Technology IT & WEB Services
Globsync Technology IT & WEB ServicesGlobsync Technology IT & WEB Services
Globsync Technology IT & WEB Services
 
Actminds Outsourcing Summit 07
Actminds Outsourcing Summit 07Actminds Outsourcing Summit 07
Actminds Outsourcing Summit 07
 
Deliver agile flow presentation (1)
Deliver agile   flow presentation (1)Deliver agile   flow presentation (1)
Deliver agile flow presentation (1)
 
Cloud computing: Stan Freck
Cloud computing: Stan FreckCloud computing: Stan Freck
Cloud computing: Stan Freck
 
The Bigger Picture: New Opportunities for the Modern Enterprise
The Bigger Picture: New Opportunities for the Modern EnterpriseThe Bigger Picture: New Opportunities for the Modern Enterprise
The Bigger Picture: New Opportunities for the Modern Enterprise
 
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
Fédération d’identité : des concepts Théoriques aux études de cas d’implément...
 
DevOps as Digital Transformation
DevOps as Digital TransformationDevOps as Digital Transformation
DevOps as Digital Transformation
 
Industrial Presentaion
Industrial Presentaion Industrial Presentaion
Industrial Presentaion
 
SigmaTec Solutions
SigmaTec SolutionsSigmaTec Solutions
SigmaTec Solutions
 

More from Optimizely

Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
Optimizely
 
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
Optimizely
 
The Science of Getting Testing Right
The Science of Getting Testing RightThe Science of Getting Testing Right
The Science of Getting Testing Right
Optimizely
 
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
Optimizely
 
Zillow + Optimizely: Building the Bridge to $20 Billion Revenue
Zillow + Optimizely: Building the Bridge to $20 Billion RevenueZillow + Optimizely: Building the Bridge to $20 Billion Revenue
Zillow + Optimizely: Building the Bridge to $20 Billion Revenue
Optimizely
 
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
Optimizely
 
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
Optimizely
 
Building an Experiment Pipeline for GitHub’s New Free Team Offering
Building an Experiment Pipeline for GitHub’s New Free Team OfferingBuilding an Experiment Pipeline for GitHub’s New Free Team Offering
Building an Experiment Pipeline for GitHub’s New Free Team Offering
Optimizely
 
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product StrategyMaking Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Optimizely
 
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives RevenueKick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
Optimizely
 
Experimentation through Clients' Eyes
Experimentation through Clients' EyesExperimentation through Clients' Eyes
Experimentation through Clients' Eyes
Optimizely
 
The Future of Software Development
The Future of Software DevelopmentThe Future of Software Development
The Future of Software Development
Optimizely
 
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
Optimizely
 
Run High Impact Experimentation with High-quality Customer Discovery
Run High Impact Experimentation with High-quality Customer DiscoveryRun High Impact Experimentation with High-quality Customer Discovery
Run High Impact Experimentation with High-quality Customer Discovery
Optimizely
 
Using Empathy to Build Custom Solutions at Scale
Using Empathy to Build Custom Solutions at ScaleUsing Empathy to Build Custom Solutions at Scale
Using Empathy to Build Custom Solutions at Scale
Optimizely
 
How to find data insights that will drive a 10X impact
How to find data insights that will drive a 10X impact How to find data insights that will drive a 10X impact
How to find data insights that will drive a 10X impact
Optimizely
 
Targeted Rollouts: How to Release Features to Multiple Audiences
Targeted Rollouts: How to Release Features to Multiple AudiencesTargeted Rollouts: How to Release Features to Multiple Audiences
Targeted Rollouts: How to Release Features to Multiple Audiences
Optimizely
 
Deploying Fearlessly in a Continuous World
Deploying Fearlessly in a Continuous WorldDeploying Fearlessly in a Continuous World
Deploying Fearlessly in a Continuous World
Optimizely
 
Detecting incorrectly implemented experiments
Detecting incorrectly implemented experimentsDetecting incorrectly implemented experiments
Detecting incorrectly implemented experiments
Optimizely
 
Move Fast in the Age of Uncertainty
Move Fast in the Age of UncertaintyMove Fast in the Age of Uncertainty
Move Fast in the Age of Uncertainty
Optimizely
 

More from Optimizely (20)

Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
 
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...
 
The Science of Getting Testing Right
The Science of Getting Testing RightThe Science of Getting Testing Right
The Science of Getting Testing Right
 
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...
 
Zillow + Optimizely: Building the Bridge to $20 Billion Revenue
Zillow + Optimizely: Building the Bridge to $20 Billion RevenueZillow + Optimizely: Building the Bridge to $20 Billion Revenue
Zillow + Optimizely: Building the Bridge to $20 Billion Revenue
 
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...
 
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...
 
Building an Experiment Pipeline for GitHub’s New Free Team Offering
Building an Experiment Pipeline for GitHub’s New Free Team OfferingBuilding an Experiment Pipeline for GitHub’s New Free Team Offering
Building an Experiment Pipeline for GitHub’s New Free Team Offering
 
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product StrategyMaking Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product Strategy
 
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives RevenueKick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue
 
Experimentation through Clients' Eyes
Experimentation through Clients' EyesExperimentation through Clients' Eyes
Experimentation through Clients' Eyes
 
The Future of Software Development
The Future of Software DevelopmentThe Future of Software Development
The Future of Software Development
 
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...
 
Run High Impact Experimentation with High-quality Customer Discovery
Run High Impact Experimentation with High-quality Customer DiscoveryRun High Impact Experimentation with High-quality Customer Discovery
Run High Impact Experimentation with High-quality Customer Discovery
 
Using Empathy to Build Custom Solutions at Scale
Using Empathy to Build Custom Solutions at ScaleUsing Empathy to Build Custom Solutions at Scale
Using Empathy to Build Custom Solutions at Scale
 
How to find data insights that will drive a 10X impact
How to find data insights that will drive a 10X impact How to find data insights that will drive a 10X impact
How to find data insights that will drive a 10X impact
 
Targeted Rollouts: How to Release Features to Multiple Audiences
Targeted Rollouts: How to Release Features to Multiple AudiencesTargeted Rollouts: How to Release Features to Multiple Audiences
Targeted Rollouts: How to Release Features to Multiple Audiences
 
Deploying Fearlessly in a Continuous World
Deploying Fearlessly in a Continuous WorldDeploying Fearlessly in a Continuous World
Deploying Fearlessly in a Continuous World
 
Detecting incorrectly implemented experiments
Detecting incorrectly implemented experimentsDetecting incorrectly implemented experiments
Detecting incorrectly implemented experiments
 
Move Fast in the Age of Uncertainty
Move Fast in the Age of UncertaintyMove Fast in the Age of Uncertainty
Move Fast in the Age of Uncertainty
 

Recently uploaded

03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching
quanhoangd129
 
07. Ruby String Slides - Ruby Core Teaching
07. Ruby String Slides - Ruby Core Teaching07. Ruby String Slides - Ruby Core Teaching
07. Ruby String Slides - Ruby Core Teaching
quanhoangd129
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
DEMONDUOS
 
How to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at ScaleHow to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at Scale
Anchore
 
Fantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdfFantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdf
6m9p7qnjj8
 
05. Ruby Control Structures - Ruby Core Teaching
05. Ruby Control Structures - Ruby Core Teaching05. Ruby Control Structures - Ruby Core Teaching
05. Ruby Control Structures - Ruby Core Teaching
quanhoangd129
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
aadhiyaeliza
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
quanhoangd129
 
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdfApplitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools
 
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databasesBATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATbern
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
Zyple Software
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
norina2645
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
bahubalikumar09988
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
Daniel Zivkovic
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
87tomato
 
DSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
DSD-INT 2024 Rainfall nowcasting – now and then - UijlenhoetDSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
DSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
Deltares
 
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
kalichargn70th171
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
Nextskill Technologies
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
TaskSprint | Employee Efficiency Software
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
revolutionary575
 

Recently uploaded (20)

03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching
 
07. Ruby String Slides - Ruby Core Teaching
07. Ruby String Slides - Ruby Core Teaching07. Ruby String Slides - Ruby Core Teaching
07. Ruby String Slides - Ruby Core Teaching
 
Authentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptxAuthentication Review-June -2024 AP & TS.pptx
Authentication Review-June -2024 AP & TS.pptx
 
How to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at ScaleHow to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at Scale
 
Fantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdfFantastic Design Patterns and Where to use them No Notes.pdf
Fantastic Design Patterns and Where to use them No Notes.pdf
 
05. Ruby Control Structures - Ruby Core Teaching
05. Ruby Control Structures - Ruby Core Teaching05. Ruby Control Structures - Ruby Core Teaching
05. Ruby Control Structures - Ruby Core Teaching
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
 
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdfApplitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdf
 
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databasesBATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databases
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
 
DSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
DSD-INT 2024 Rainfall nowcasting – now and then - UijlenhoetDSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
DSD-INT 2024 Rainfall nowcasting – now and then - Uijlenhoet
 
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdfA Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
A Step-by-Step Guide to Selecting the Right Automated Software Testing Tools.pdf
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
 
How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024How To Fill Timesheet in TaskSprint: Quick Guide 2024
How To Fill Timesheet in TaskSprint: Quick Guide 2024
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
 

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

  • 1. Overcoming the Challenges of Experimentation on a Service Oriented Architecture Lupin Campos & Stephen Schwahn September 2020
  • 2. Hi! We are Lupin and Steph :) Lupin Campos Director of Product Growth Tech Startup Founder (Funnely) Software Development Company Founder (Leanbirds & Condiminds) Former Venture Partner @ 500Startups Steph Schwahn Growth Team Tech Lead Tech Startup Founder (Crescendo) Mobile App Development Company Founder (Plutonium Apps) Georgia Tech Alumni, graduated with high honors.
  • 3. So far: - 1 complete backend refactor - 2 frontend refactors - 36 Registration flow experiments (web + fullstack) - 40% Improvement on Registration Conversion Rate
  • 4. 1Let’s Get Into Context A quick note on who we are, what we do, and the challenges that we found in the way.
  • 5. This is our registration Process... Phone Number Role Verification Code Email & Password Child Info Personal Info Sync Bank Acc (Website)
  • 6. ….And here are the services behind it. A simplified version, you don’t want to see the entire thing. Trust us.
  • 7. 1Service Oriented Architectures with Optimizely (the wrong way)
  • 8. Greenlight Financial Technology, Inc. SOA with Optimizely Fullstack on Frontend Web App Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend
  • 9. Greenlight Financial Technology, Inc. SOA with Optimizely Fullstack on Frontend Web App Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend Variant: Variant 1 Variable: FlowId = 6
  • 10. Greenlight Financial Technology, Inc. SOA with Optimizely Fullstack Web App Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend User_id: '1234_4321'
  • 11. Greenlight Financial Technology, Inc. SOA with Optimizely Fullstack Web App Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend User_id: '1234_4321' 30s 40s 400s 200s 5s
  • 12. Greenlight Financial Technology, Inc. SOA with Optimizely Fullstack Web App Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend User_id: '1234_4321' 30s 40s 400s 200s 5s
  • 13. 1Service Oriented Architectures with Optimizely (our way)
  • 14. Greenlight Financial Technology, Inc. Solution: SOA with Optimizely Agent Marketing Site Mobile App Edge Service 1 Edge Service 2 Nested Service 1 Nested Service 2 Helper Service 1 Backend Feature or Experiment Key: 'featureTest’ User_id: '1234_4321' Optimizely Agent 30s Web App
  • 15. Greenlight Financial Technology, Inc. In Practice…. Client 1 Client 2 Client n . . . Ambassador (API Gateway) Pod (3-30 running based on traffic) Optimizely Service (Node.js) Optimizely Agentlocalhost:8080 Datafile sync (every 30s) localhost:3000 Exposes Optimizely Service on service port 80 Service grpc:80 pod-ip:3000 NodeJS Container Go Container (sidecar) Datafile pod-ip:3000pod-ip:3000
  • 16. 1Let’s talk about “User ID” And how do we pass it everywhere!
  • 17. Greenlight Financial Technology, Inc. Used for bucketing MurmurHash(“user_1234”) Rollout to 60%
  • 18. Greenlight Financial Technology, Inc. Used for bucketing MurmurHash(“user_5678”) Rollout to 60%
  • 19. Greenlight Financial Technology, Inc. What if we don’t have the user id? MurmurHash(“steph@greenlight.me”) ? Rollout to 60%
  • 20. Greenlight Financial Technology, Inc. Introducing the Experimentation ID! MurmurHash(“exp_user_1234”) Rollout to 60%
  • 21. 1Introducing Experimentation-ID Service How we keep a consistent user id regardless of context
  • 22. Greenlight Financial Technology, Inc. Experimentation -ID service
  • 23. Greenlight Financial Technology, Inc. Optimizely Service orchestrates
  • 24. Greenlight Financial Technology, Inc. New API Contract! export type Context = UserIdContext | EmailContext | AnonymousIdContext | RegistrationIdContext | DeviceInfoContext export type EmailContext = { email: string } export type AnonymousIdContext = { anonymousId: string } export type DeviceInfoContext = { deviceOs: string, deviceToken: string } export type UserIdContext = { userId: string } export type RegistrationIdContext = { registrationId: string } export function isFeatureEnabled(featureKey: string, context: Context, attributes?: Attributes) export function getEnabledFeatures(context: Context, attributes?: Attributes) ...
  • 25. 1Truly Full Stack Experimentation Through the ENTIRE user journey, from the marketing landing page, through registration services, and all the way to fundamental service primitives