The document discusses A/B testing methods using NGINX, highlighting its importance and potential impacts on key metrics. It details various deployment strategies such as blue-green deployment and canary releases, along with specific methods for implementing A/B testing. Additionally, it provides resources for further reading and demos related to A/B testing with NGINX.

1. Apply A/B Testing with
NGINX Routing Policy
Mr.Supachai Jaturaprom (Tum)
Pre-sales Engineer
CCIE#65006
Email: supachai@vstecs.co.th
VSTECS (Thailand) Co.,Ltd.
NGINX Community Thailand.
VSTECS (Thailand) Co.,Ltd. 1

2. What is A/B Testing?
https://www.orderhive.com/knowledge-center/ab-testing
VSTECS (Thailand) Co.,Ltd. 2

3. The Importance of A/B Testing
https://exp-platform.com/Documents/2014%20experimentersRulesOfThumb.pdf
“ Small Changes can have a Big Impact to Key Metrics ”
VSTECS (Thailand) Co.,Ltd. 3

4. +450M
sites run NGINX
WE HAVE TREMENDOUS INSIGHT INTO APPLICATION PATTERNS
#1
Source: Netcraft June 2020 Web Server Survey
40%
Source: NGINX User survey 2017, 2018, 2019
of NGINX deployments
are as an API gateway

5. A/B
Testing
VSTECS (Thailand) Co.,Ltd. 5

6. VSTECS (Thailand) Co.,Ltd. 6
Method for Deploy Production ?
Blue-Green Deployment
Canary Release
Feature Toggles
Experiment toggle
Permission toggle
……..

7. Method-1 : Apply Blue-Green with NGINX
0 %
100 %
100 %
VSTECS (Thailand) Co.,Ltd. 7
All Users
NGINX

8. 95 %
5 %
100 %
VSTECS (Thailand) Co.,Ltd. 8
All Users
NGINX
Method-2 : Apply Canary Release with NGINX

9. 90 %
10 %
100 %
VSTECS (Thailand) Co.,Ltd. 9
Group B
NGINX
Method-3 : Apply A/B Testing with NGINX
Group A
- Client IP Address
- Token (URI / Header)
- Argument more….
- NGINX Variables ($request_id, ….)
Client classification

10. uid=111
uid=222
100 %
VSTECS (Thailand) Co.,Ltd. 10
Group B (102)
NGINX
Method-4 : Apply A/B Testing with NGINX (JWT)
Group A (101)
- Authentication
- JWT Claim content routing
Insert JWT in
- URI Argument
- Cookie

11. Live Demo: Apply A/B Testing with NGINX
A/B testing with random client spliting.
A/B testing with sticky route.
Dynamic A/B Testing with multi-domain
VSTECS (Thailand) Co.,Ltd. 11
A/B Testing with JWT

12. Resources
Git Repository of A/B testing with NGINX
• https://github.com/supachai-j/ab-testing-with-nginx
• Demo-1: A/B testing with random client spliting.
• Demo-2: A/B testing with sticky route.
• Demo-3: Dynamic A/B Testing with Multi-domain.
• Demo-4: A/B Testing with JWT
Getting Started with NGINX Guides
• https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/
• https://www.nginx.com/blog/performing-a-b-testing-nginx-plus
• https://www.nginx.com/blog/dynamic-a-b-testing-with-nginx-plus
• https://www.nginx.com/blog/authentication-content-based-routing-jwts-nginx-plus
VSTECS (Thailand) Co.,Ltd. 12
NGINX Community Thailand.