In the past few years we’ve used cloud technologies to improve pre-production flexibility and solve many problems that previously prevented us from delivering high quality apps to production. However, one problem consistently prevents full test coverage prior to deployment—the lack of comprehensive test data. As we try to get faster and leaner in our agile development processes, the problem with data becomes even more difficult to solve. Robert Kelman describes the evolution of DIRECTV’s (now AT&T) Agile Test Data Management program. He explains the five critical elements—centralized team, automation tools and processes, aggressive environment refresh, test data organization, and innovation through experimentation—of their data management program, which supports both development and testing efforts. Robert examines the details and examples of these five components, and demonstrates the relationships between them.

1. AW13
Session
6/8/16 4:15 PM
Five Critical Elements for Successful
Agile Data Management
Presented by:
Robert Kelman
DIRECTV
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888---268---8770 ·· 904---278---0524 - info@techwell.com - http://www.techwell.com/

2. Robert Kelman
DIRECTV
Robert Kelman is director of tools and architecture for DIRECTV (now part of the
AT&T family) software quality engineering organization. With twenty years of
experience delivering enterprise software, Robb has developed a passion for
continuous improvement and innovation to drive quality. For the past ten years
he has lead a team of solution engineers who develop and manage tools that
increase flexibility and automation to ensure all enterprise applications are
deployed with the highest level of quality. Robb has a Lean-Six Sigma Green Belt
certification. Follow him on Twitter @rkelman.

3. 5/30/16
1
Robb Kelman
Twitter:
@rkelman
5 Critical Elements
Successful Agile Data
Mgmt Program
¡ Background
¡ 5 Critical Elements
§ Sophisticated Central Team
§ Integration w/ Tools & Automation
§ Data Patterns & Classification
§ Flexible Refresh Approach
§ Experimentation & Innovation
¡ Tying the elements together
¡ Q & A
OUTLINE

4. 5/30/16
2
BACKGROUND
¡ Started with Test Environments & Architecture
¡ 2007 added reference data (shared data)
¡ 2008 added transactional data (application specific)
¡ 2010 added Development Support
¡ 2015 Started aggressive migration to self-service
§ This shifted structure & organization of the team
BACKGROUND
Architecture Test Data
Tools

5. 5/30/16
3
¡ Why a central team?
§ Initially all testers owned test data….
§ Which meant…
§ Nobody owned test data
¡ Central team also facilitated:
§ Standards, consistency, repeatability
§ Easier collaboration
§ Re-use; eliminating duplication and waste
§ Data Control; Eliminated Data Collisions between teams
¡ Build Projects around motivated individuals...
CENTRAL TEAM
¡ The Team is not:
§ Role specific:
§ DBAs
§ Testers
§ Data Entry
§ Monopoly
§ “Not my problem”
§ Not just data
¡ The Team is:
§ Collaborative:
§ Dev & Test Partners
§ Tech Problem Solvers
§ Sophisticated
§ Many different hats
§ Developer
§ Test Automation
§ Architect
§ DBA
§ File Management
CENTRAL TEAM

6. 5/30/16
4
TOOLS & AUTOMATION
¡ Data Management MUST be automated
TOOLS & AUTOMATION

7. 5/30/16
5
¡ Data Management MUST be automated
TOOLS & AUTOMATION
¡ Data Management MUST be automated
§ Automated with bulk flexibility
§ Ideally set-up with API & Tool integration
§ If not with API & Tool Integration with Self-Service
Capability
¡ Data Management must be aligned with
Tools and Test Automation
§ Automation and Tools Experts with Data Mgmt Goals
§ Data Mgmt Team with Tools Expertise
¡ ...Give them the environment they need…
TOOLS & AUTOMATION

8. 5/30/16
6
¡ Tools are not:
§ Individual automation
§ Just another stand-alone
tool
§ (only) Automated Testing
¡ Tools are:
§ Focused on Bulk Set-up
§ Data Management is a
Quality Opportunity
§ Leverage Implicit Testing
§ Should use designed
flows
§ Include:
§ Metrics & Reporting
§ File & DB Loads
§ Data Virtualization
§ Service Virtualization
TOOLS & AUTOMATION
DATA PATTERNS

9. 5/30/16
7
DATA PATTERNS
¡ Not all data follows the same rules
§ Different Applications require different data
§ Some data objects require special attention
§ Some applications require complete synchronization
§ Some data is managed by business
§ Data Classification & Organization is about:
§ Understanding the context of user stories
§ Understanding data supporting those scenarios
¡ Continuous attention to technical excellence
and good design enhances agility.
¡ Patterns are not:
§ Rules about how data
flows in production
§ Rules that cannot
change with new
Architecture or
information
§ “This is the way we’ve
always done it.”
¡ Patterns are:
§ Tools that provide data
capabilities & flexibility
§ Guidelines of what is
needed for user stories
§ Especially at a technical
& integration level
§ Related to “data objects”
§ For example:
§ Customer
§ Offer
§ Processes (Installation)
§ Dealer; Installer
DATA PATTERNS

10. 5/30/16
8
FLEXIBLE REFRESH
¡ The Refresh is foundation of overall dataset
§ The most complicated task of the Data Management Team
§ This cannot be managed as self-service
§ Requires Centralized Team to Coordinate
§ Can be practiced using table top exercises
§ This is where compliance requirements need to be met
§ Requires powerful automation
§ Effort and downtime would be unmanageable otherwise
§ Sub-refreshes require data patterns/classification
§ Not just data subsets, but application subsets
¡ Reflect on how to become more effective,
then tune and adjust accordingly
FLEXIBLE REFRESH

11. 5/30/16
9
¡ Innovation is not:
§ Blank check to break
environments
§ Unchecked risk taking
¡ Innovation Is:
§ Smart, Scientific
approach to eliminating
data constraints
§ Planning & Contingency
§ An approach to
understand system limits
§ A way to get out of:
“This is the way we’ve
always done it”
§ Always tied to:
§ Tools
§ Refresh
§ Classification
FLEXIBLE REFRESH
EXPERIMENTATION & INNOVATION

12. 5/30/16
10
¡ Data Mgmt, like all Dev needs to evolve
§ Especially as environments become more flexible
§ As flexible development processes are
§ Data Management will need to become as flexible
§ Most of our work is done in test/dev environments
§ Most mistakes can be corrected
§ Often we miss operational aspects of exploratory testing
§ Dev/Test Environments are inherently constrained
§ We must find (easy) ways around those constraints
¡ Responding to change…
over following a plan
EXPERIMENTATION & INNOVATION
¡ Innovation is not:
§ Blank check to break
environments
§ Unchecked risk taking
¡ Innovation Is:
§ Smart, Scientific
approach to eliminating
data constraints
§ Planning & Contingency
§ An approach to
understand system limits
§ A way to get out of:
“This is the way we’ve
always done it”
§ Always tied to:
§ Tools
§ Refresh
§ Classification
EXPERIMENTATION & INNOVATION

13. 5/30/16
11
¡ Data Mgmt is foundational to Quality
§ Needs to:
§ Have a balanced approach to simulating production
§ Be adaptable
§ Be Automated; Bulk-centric
§ Smart – The Right data
In the right place
By the right time
TYING THE 5 TOGETHER
¡ Thank you
QUESTIONS?