This document covers multiple techniques to do "Testing in Production (TiP)". Example Blue Green, Various Canary, A/B Testing, Chaos testing and Shadowing.

1. Testing in Production (TiP)
~ Gir

2. There are tons of testing methodologies are available to qualify production grade build.
Still, Why do we need another testing method that too in Production?
● contains impact to smaller deployment or users group
● confirms the quality of build with production data
● addresses environment specific issues, if any
● make sure monitoring, DR planning and coverage
● method to reduce, control and contain Business impact...
Why?

3. Blue/Green
How?,
At any time, only one of the environments is
live, with the live environment serving all
production traffic. For this example, Blue is
currently live and Green is idle or vice versa
Covers,
Blue-green deployment is a technique that
reduces downtime and risk by running two
identical production environments called Blue
and Green.
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
v1v2

4. Canary (Controlled Exposure)
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
10%
Users
90%
Users
...
v1v2
How?,
Roll-in new deployment in controlled manner,
starting with exposing smaller percentage of
user and increase exposure, if successful else
rollback. For example route 10% of users to
V2 and 90% of users to V1.
Covers,
Controlled Exposure will contain the impact
with in small percentage of user group.

5. Canary (Traffic Shaping / Shifting)
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
10%
Traffic
90%
Traffic
...
v1v2
How?,
Roll-in new deployment in controlled manner,
starting with exposing smaller percentage of
traffic and increase exposure, if successful
else rollback. For example route 10% of traffic
to V2 and 90% of traffic to V1.
Covers,
Controlled Exposure will contain the impact
with in small percentage of traffic (requests).

6. Canary (Gradual Release)
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
10%
Components
...
90%
Components
v1v2
How?,
Roll-in new deployment in controlled manner,
starting with exposing smaller percentage of
capacity and increase exposure, if successful
else rollback. For example deploy 10% of
server capacity with V2 while 90% with V1.
Covers,
Controlled Exposure will contain the impact
with in small percentage of deployment and
traffic that it intakes.

7. A/B Testing
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
50%
Users
50%
Users
...
1b1a
Conversion
Rate 20%
Conversion
Rate 65%
How?,
Deploy 2 versions of UI services, side by side
on infrastructure at equal capacity and direct
50% of users traffic to Version 1a while
remaining to 1b and measure the acceptance
rate to choose the best. Its implementation of
statistics “two-sample hypothesis testing”.
Covers,
Helps to identify one of 2 UI or Applications
with higher user conversion rate.

8. Chaos Testing
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
...
v2v2
Web Server
Application
Server
Database
UI Service
Application
Service
Database
...
v2
How?,
Failure is unavoidable, why not deliberately
introduce it to ensure your systems and
processes can deal with the failure? Can be
done by breaking components, services, data
or communication channel randomly.
Covers,
Helps to identify failure impacts at services
and components level and plan for DR policy

9. Shadowing Pre-Production
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Web Server
Application
Server
Database
UI Service
Application
Service
Database
Router
Users
v1v2
Quality
Check
shadow
traffic
How?,
Deploy production parallel application or
services on shadow environment with new
build (v2) and wiretap production data traffic,
copy into shadow service to simulate and
qualify.
Covers,
Qualify a build with production parallel data.

10. AnswersQuestions