Quality assurance in distributed continuous delivery

Quality Assurance
in
Distributed Continuous Delivery
By
Jin Ming｜@金明i
Continuous Delivery Lead, ThoughtWorks

Wednesday, August 8, 12

Text
Background: Top One Property Listing Site in OZ


Business Overview

Seller Consumer

Agency


Architecture Overview


How to Do QA?


Quality is Value to Some Person.
---Gerald Weinberg


Challenges We’re Facing


Legacy Systems


Cross Systems


Distributed Development


Also, We Want to Be


We Need “Total Quality”


Test Quadrants
Automat Business Facing
ed
Manual
& Manual

Functional Tests
Exploratory Testing
Examples
Scenarios
Story Tests
Usability Testing
Prototypes
UAT(User Acceptance Testing)
Simulations
Alpha / Beta
...
...
Q2
Q3
Critique
Supporting
Team Q1
Q4
Product

Unit Tests
Performance & Load Testing
Component Tests
Security Testing
...
...

Automat
Tools
ed
Technology Facing

Functional Test Boundaries


We Need to Ensure End-to-End


We Need to Ensure Single System


Units Inside Single System


Automatic Those Tests


Test Pyramid

Efﬁciency Scope
Cost End-to-End Test End-User
Simulation
System Test
Scope
End-User Unit Test Efﬁciency
Simulation Cost


Capybara
Functional Test Samples
Selenium

WebDriver

Performance Test Samples


Deployment Validation


Test Plan


Vision & High-
Level Stories

$

Iteration $

Release $

$

Test Plan
Release Test
Plan
Iteration Test
Plan
Daily Test
Plan


Iterations
Stabilize (opt)

I0 I1 I2 I3 I4 … In Pre-production Production

Acceptance Exploratory, UAT PVT
Test Analysis
Testing Automation
I1
I1 I1
iIities
operations
&Compliance
Acceptance Exploratory,
Test Analysis
Testing Automation Data
I2
I2 I2 Migration

Deployment
Acceptance Exploratory,
Test Analysis
Testing Automation
I3
I3 I3 Full
Regression
Test

Automated Regression Test
Integration End-To-End Integration End-To-End
Test Test Test Test
Performance Profiling Test

Load/Stress Load/Stress


Good Enough?


It’s Just Validation.
Where is Veriﬁcation?


Customer Domain Expert QA Developer

BDD

Stories Behaviour Application

Exercise
Matches to
expectations against


Co-Education


Quality Assurance Activities
Business
Analytics

Test
Showcase
Strategy

Risk
Exploratory
Assessmen
Test
t
Agile Quality
Assurance

Common
Test
Vision on
Analytics
Quality

Process
Test
Improveme
Framework
nt


Build Quality In


Shared Responsibility


Communication & Collaboration


QA is NOT JUST Testing


Eradicating Non-Determinism in Tests


Visualizing Quality


?
? ?
? ?
?

Quality assurance in distributed continuous delivery

More Related Content

Similar to Quality assurance in distributed continuous delivery

Recently uploaded

Quality assurance in distributed continuous delivery