Test Data Management and Its Role in DevOps

T11

Test
Data
Management

5/11/17
11:15

Test
Data
Management
and
Its
Role
in

DevOps

Presented
by:

Sunil
Sehgal

TechArcis
Solutions,
Inc.

Brought
to
you
by:

350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073

888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/

Sunil
Sehgal

A
regular
contributor
to
StickyMinds.com
and
other
TechWell
publications,
Sunil

Sehgal
is
an
expert
in
quality
assurance
and
testing
solutions,
with
a
focus
on

transformation
initiatives
that
bring
year-‐after-‐year
cost
and
quality
advantages.
At

TechArcis,
Sunil
leads
the
global
strategy,
innovation
initiatives,
and
new
solutions

for
the
company.
A
thought
leader
in
quality
engineering
and
an
early
adopter
of

DevTestOps,
Sunil
is
very
active
in
creating
innovative
automation
solutions
for
true

implementation
of
agile
and
DevOps.
His
unique
business
and
technology

background
-‐
and
hands-‐on
experience
across
a
range
of
business
sectors
-‐
are

valuable
assets
for
his
clients
and
TechArcis.
Reach
Sunil
at

sunil.sehgal@techarcis.com.

© TechArcis Confidential and Proprietary
Quality Assurance I Testing Transformation I Outsourcing
11th May’ 2017
Sunil Sehgal
Managing Partner – Global Testing Practice
O: 770-415-4815 x 101
M: 678-361-4357
sunil.sehgal@techarcis.com
Test Data Management and Its Role in DevOps, Continuous
Integration and Delivery
Quality Engineering driven by Innovations and Automation

2
Agenda
DevOps Movement
Introduction to Continuous Delivery
TDM becomes a pre-requisite to the success of DevOps, CI / CD
Today’s Test Data Challenges
Top considerations for a TDM Approach
Practical Tips to Get Started
The Data Gap
Key Takeaways
References: DevOps Guide, TechWell, Delphix, CA, K2View, Other Industry materials

3
Applications & API Economy - Three Factors in Play
Testing has to change as well
Dev-test -opsDev-ops
And Dev-test-ops is not possible without Test Automation
“CONTINUOUS”
deployment of Code
and Quality Solutions
“DISRUPTION”
(Technology & Business Models)
DEMAND FOR “AUTOMATION”
• Test Automation
• TDM Automation

4
Thriving in “Disruption”
Quality and Convenience Drives “Customer Success”
Product Innovation and Development
is Changing
Software Development is also changing and
embracing Automation
Agile and DevOps is the new norm
But, how to drive “Continuous” Quality at
Speed in this Disruption?

5
DevOps Movement
One of the hallmarks of high performers in any field is that they always “accelerate
from the rest of the herd.” In other words, the best always get better.

6
DevTestOps - Understanding “Continuous” terminology
Process of delivering software updates to users on a nearly constant basis.
Code can be rapidly and safely deployed to production by delivering every
change to a production-like environment
Continuous deployment is the next step of continuous delivery
Continuous Integration Continuous Integration means the constant integration of changes to an
application at all stages of the delivery chain.

7
Understanding “Continuous” paradigm
Post
Prod
Tests
Local Dev
Environment
Developer
Branch
CI
Server
QA/
Staging
Environment
Mainline
App Store
and Play
store
CI
Server
Local Dev
Environment
Developer
Branch
QA/
Staging
Environment
Code
Revision
QA
Code
Revision QA
Immediate
Deploy
Detects
Branch
Change
Delivery to
QA/Staging
Detects
PASS
QA
PASS
Merge
to Main
Bad
ReleaseCI Build
FailedQA
FailedCI Build
FailedCode
Review
Failed
Code
Commit
Delivery to
QA/Staging
Code
Commit

8
“Test Automation” at speed of Dev
User creates
story
Defines
acceptance
criteria
Dev
Test
Dev team codes the
User Story
Apply action and validation
steps in automation
components
Story available for
test
Mapping script with code
Run test
FAIL
Report to dev. team
Code Check-in
PASS
Add to sprint suite
Trigger CI
Update result in
JIRA
Discuss Requirements,
Application flow, Test
Points
Example In-Sprint Workflow
1
Day-Zero Test automation for
in-Sprint User stories is the
only way Agile Testing can be
scaled 2
3Always tie test automation
to source code branch that
is being developed.
Test automation consistency
and maintenance is hard to
achieve in real world

9
Even organizations with cutting-edge DevOps practices
are finding that standup and reproducibility constraints
still apply to data.
The Data Gap

10
1. Industry best practices for TDM
2. What others are doing / challenges faced?
3. TDM Solutions building / Key points to focus on.
We will focus on three important points…..

11
Rise of “TDM (automated)”
Once viewed as a
back-office function,
Test Data
Management
(TDM) has emerged as
a critical business
enabler for enterprise
agility, security, and
cost efficiency.
Adoption of DevOps
and other
agile methodologies is
giving rise
to New TDM
approaches, tools,
and strategies that
solve data
delivery challenges.

12
The Current State of Test Data Management
Who’s responsible for TDM in an organization?

13
How long does it take to deliver a test dataset?
Today’s Test Data Challenges
It takes 3.5 days and 3.8 people,
on average, to fulfill a request
for a new environment.
At 1 out of every 5 organizations,
it takes over a week.

14
What percentage of defects are data-related?
Development teams lack high-fidelity data

15
How Organizations manage their Data today – Lets
understand the Four S’s of Test Data Management
Sub setting
Strategies
Synthetic Data
Shared Test Data
Environments
Standalone Masking
Solutions
• This emerged to
overcome
limitations in
copying and
moving full,
production-sized
datasets.
• Smaller, more
portable datasets
can serve as
substitutes for
complete ones
but in practice,
Subsets fail to
adequately
embody the
breadth of real
world conditions
• This is an alternative
approach in which
algorithmically
generated test data
substitutes for data
derived from production
sources
• Synthetic data
circumvents the security
issues involved in
distributing “real data”
which contains
potentially sensitive
information but
sometimes fail to cover
all the data
permutations attendant
in production sources
• The inability to
provision dedicated QA
environments to
individual testers leads
to share test datasets
among –projects and
teams.
• Sharing provides
efficiency benefits by
giving multiple teams
immediate, concurrent
access to a common
data environment but
in practice, conflicts
occur when more than
one stakeholder
contends for the same
resources at the same
time
• Need to ensure that
sensitive information
is protected when
delivered to non-
production
environments,
including those used
for testing
• Given the
shortcomings of
synthetic data, as
well as those of
measures including
access control and
encryption, data
masking has become
a de facto standard
for securing test data

16
The Issue ? No single technology exists that fulfills
all TDM requirements.
• Four solutions typify test data management today: Subsetting, Synthetic Data, Shared
Environments, and Standalone Masking.
• These approaches have reached widespread adoption and are often used in combination
with one another.

17
What traditional TDM methods are you using?

18
What percentage of testing requirements are met
using subsets?

19
Why does your organization use subsets?

20
Building a Comprehensive TDM Toolset
Teams must……
Build an
Integrated TDM
Solution
That provides
all the Data
Types
Required to
meet a diverse
set of Testing
needs
Successful TDM approach should aim to provide the appropriate types of test data,
weighing the pros and cons of each.

21
Various Data Types : Pros and Cons
Production Data
provides the most
complete test
coverage
comes at the
expense of agility
and storage costs
For some
applications, it can
also mean
exposing sensitive
data.
ProsCons
Subsets of
Production data
significantly more
agile than full copies
provide some
savings on hardware,
CPU, and licensing
costs
difficult to achieve
sufficient test
coverage
Masked Production
data
makes it possible for
development teams to
use real data without
introducing unsafe levels
of risk
masking processes can
elongate environment
provisioning
masking requires
staging environments
with additional storage
and staff to ensure
referential integrity after
data is transformed
Synthetic data
circumvents security
issues
the space savings are
limited
creating test data is
also prone to human
error and requires an in-
depth understanding of
data relationships

22
Top Considerations for a Test Data Management Approach
• Automation
• Toolset Integration
• Self Service
Data Delivery:
reducing the time to deliver test
data to a developer or tester
• Data Age
• Data Accuracy
• Data Size
Data Quality :
meeting requirements for high-
fidelity test data
• Complete Solution
• No need for development expertise
• Integrated masking and distribution
Data Security:
minimizing security risks without
compromising speed
• Data consolidation
• Data archiving
• Environment utilization
Infrastructure Costs:
lowering the costs of storing and
archiving test data

23
Testing in a traditional scenario vs. with an optimized TDM approach
Taking an Optimized Approach
As companies adopt more iterative release methodologies to become more agile, data is
the fastest, easiest way to accelerate test cycles.
A well-orchestrated approach to TDM has the potential to transform the overall
application development process by slashing wait-times for data.

24
Incorporating Data Virtualization in TDM
• One technology that transforms how
companies manage data for their critical
application projects is data virtualization.
o With data virtualization, teams can mask and
deliver data 100 times faster while using 10
times less space.
• In addition, data virtualization integrates
with existing TDM and DevOps tools to
provide...
o Automated, self-service access to data, enabling
development teams to roll out projects at twice
the speed.

25
In order to deliver on the promise of DevOps, CI / CD and hit
continuous release targets for even the largest, most complex and
integrated applications, organizations need solutions…
1. That provide the same flexibility for data as for code bases.
2. The same automation and repeatability for data as for
configurations.
TDM becomes a pre-requisite to the success of
DevOps and CI / CD

26
Practical Tips to Get Started

27
Better TDM with Data as a Service (Daas):
Bring Quality, Speed, and Security to Test Data
• Capturing data—including ongoing changes— in production systems
• Versioning and managing data across the full application lifecycle
• Delivering data to non-production systems, including test environments
DaaS platforms bring the benefits of virtualization to application data by:
• Bookmark and share test data with teammates
• Refresh test data from the latest version production
• Reset test data back to prior point in time
• Branch data for performance and A/B testing
Since DaaS systems capture and version production data over a window of time, testers can:

28
DevOps isn’t just about self-sufficiency. It’s also about sophisticated collaboration. Without a
DaaS solution, often data can be the bottleneck to efficient collaboration.
Bookmark and Share
With legacy solutions, data is locked with the user instead of traveling with code
With DaaS, free data movement accelerates test-fix cycles.

29
With DaaS, test environment setup takes minutes instead of hours, and is the
minority of test cycle time.
Refresh and Reset

30
Data branching from production and non-production keeps data and code in sync.
Provision and Branch

31
Key Takeaways

32
Continuous Delivery needs all systems to be fast
Developer 1
Developer 2
Developer 3
A System is developed
• Instant
Configuration
• Up-to-date
snapshots of
complex
environments
• Share and use
environments in
parallel
• Powerful cloud,
infrastructure, on
demand
Applications are deployed
in production
• Continuous Deployment
• Environment Management
• Software is fully tested first
time round and defects are
detected.
Testing – Starts immediately,
in parallel
• Gold Copy data covers every test
that needs to be run, including
future and negative scenarios
• It is available instantly, and is
delivered into unlimited, isolated
environments
• It is available across versions and
releases in parallel, and is fully
versioned
• It is absolutely no sensitive
information.
100% 100% 100%
Test Environment Test Data Continuous Code
Deployment

33
Continuous Testing with “Automated Test Data”
mirroring the production data
Metric
Watch
Local Dev
Environment
Developer
Branch
CI
Server
QA/
Staging
Environment
Mainline
App Store
and Play
store
CI
Server
Local Dev
Environment
Developer
Branch
QA/
Staging
Environment
Code
Revision
QA
Code
Revision QA
Immediate
Deploy
Detects
Branch
Change
Delivery
to Staging
Detects
PASS QA
PASS
Merge
to Main
Bad
ReleaseCI Build
FailedQA Failed
CI Build
FailedCode
Review
Failed
Code
Commit
Delivery
to Staging
Code
Commit
Automated Test Data for
debugging and Unit tests
Automated Test Data for Integration,
Regression and E2E Tests (Production Data)
Automated Test as per
End user conditions
(Production Data)

34
Key takeaways
2
Automated TDM is key to deliver on the promise of DevOps, CI / CD
and hit continuous release targets
Global teams requires access of TDM infrastructure anytime, anywhere
Agile and DevOps is the new norm
Drive “Continuous” Quality at Speed in this disruption1
3
Teams must build an integrated solution that provides all the data
types required to meet a diverse set of testing needs.
DevOps success depends on effective use of Automated TDM solution
Prod
Better TDM with Data as a Service: Bring Quality, Speed, and Security to Test Data
With DaaS, we can leverage the same capabilities for managing the data that the
code talks to.
4

Thank You
Your Trusted Quality Assurance Partner
Quality Assurance I Testing Transformation I Outsourcing
www.techarcis.com
We are a Technology Company with 100%
specialization in QA and Testing

Test Data Management and Its Role in DevOps

More Related Content

What's hot

Similar to Test Data Management and Its Role in DevOps

More from TechWell

Recently uploaded

Test Data Management and Its Role in DevOps