Model-Based Testing for Achieving Maximum CD Pipeline Acceleration

World®
’16
Model-Based Testing for Achieving
Maximum CD Pipeline Acceleration
Alex Martins - CTO Continuous Quality – CA Technologies
Ben Johnson-Ward - ARD Jedi Master – CA Technologies
DO5X29S
DEVOPS

2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation

Abstract
Most successful DevOps teams have learned how to work together seamlessly.
Then, they get to a point in their journey where they look for a way to achieve
maximum software development lifecycle acceleration and they think about
continuous delivery. One of the things most organizations that adopt continuous
delivery are overlooking is the acceleration of the entire application lifecycle, not
just the build-test-deploy-operate cycle. What about accelerating the
requirements gathering process? These are still communicated through the
written language—the same way it’s been done across different teams for the past
30 years: Written in Word documents, Excel spreadsheets or in requirements
management tools. Using this completely manual process leads to ambiguity and
introduces approximately 50-percent of defects in the application code.
This panel of experts from QA organizations and QA products will explore the
different ways that model-based testing has brought maximum acceleration into
the continuous delivery pipeline for many different organizations.
Alex Martins
CA Technologies
CTO Continuous
Quality

Agenda
CHALLENGES OF ACHIEVING CONTINUOUS DELIVERY
HOW OTHERS DESIGN THEIR PROJECTS
CLOSING
LEVERAGING DESIGN TECHNIQUES INTO SOFTWARE DESIGN
HOW IT WORKS
Q & A
1
2
3
4
5
6

Developers Release Team Operations
TESTBUILD DEPLOY
Testers
OPERATE
Gap in CD Pipeline Acceleration Initiatives
Performance
Testing Deploy to Production
Customer
Experience
Code | Integrate | Build Functional & UA Testing
idea
Extreme acceleration possible through better tools and
infrastructure that support the cultural change on the team
§ We typically focus on accelerating
code-to-deployment activities
§ The Intake process usually remains a
bottleneck and mostly manual
§ Mostly manual to go from Idea to
Code (for TDD) and to Test Cases and
Test Automation scripts
§ How to automate the CD Pipeline
Intake process?
Requirement
Test Cases
Test Data
Virtual
Services
Test
Automation

Core Challenges to Achieving Continuous Delivery
Requirements
Development
QA / Testing
Operations
Release
70% of all
testing is still
manual.5
63% of testers admit they
can’t test across all the different
mobile devices and OS versions.6
50% of time
spent looking for
test data.7
79% of teams face
prohibitive restrictions, time
limits or access fees on needed
3rd party services.3
X X X
64% of total defect cost originate in
the requirements analysis and design
phase.1
50% of developers
time is spent fixing and finding defects.3
?
80% of teams experience delays in development and QA
due to unavailable dependencies.4
?
57% are dissatisfied
with the time it takes to
deploy new features7
!
20% of companies are
performing full release automation
today.8
5 Perfecto Mobile 2014 Benchmark Survey, Why Mobile Apps Fail 2014
6 Source: Bloor Research, 2014
7 Forrester Global Modern Service Delivery Benchmark Online Survey 2014
8 The Forrester Wave™: Application Release Automation 2015
9 Ankit Oberoi. 5 Reasons Visitors Leave Your Website. Website Magazine 2014
10 EMA™ White Paper APM Essentials: Navigating the Maze of End User Experience
Solutions Prepared for CA Technologies 2016
40% of users will abandon your app if it
takes more than three seconds to load.9
70% of delays in application delivery are attributed to
new and changing requirements.2
50% of companies are managing cloud services with homegrown tools such as
scripts, macros, etc. Even fewer have mobile-specific monitoring capabilities.10
? ?
1 Hyderabad Business School GITAM University Quality Flaws: Issues and Challenges in Software Development 2012
2 Forrester Brief: Software Requirements Practices Are Ripe For Disruption 2014
3 Study conducted by the Judge Business School at Cambridge University 2013
3 Voke Market Snapshot Service Virtualization 2015
4 Bloor Report Automated test case generation 2014

Clarity and Vision Throughout the Sprint – Goal
BUSINESS ANALYST PROGRAMMER TESTERUSER
The closer the vision means the user gets a quality product.
There are fewer bugs and the product is delivered faster.

Let’s look at how others design
their projects

Designing a House
§ Customer describes his dream house to the architect:
§ We are about to have our first baby and I want to build a
beautiful house for my family where we can all live
comfortably
§ I need the house to have 2 bedrooms, 1 kitchen, 1 living
space, 1 dining room, 1 TV room and 1 bathroom
§ The house needs to feel spacious and warm

Generates LOTS of Questions
§ How many Sq ft should the house have?
§ How many floors?
§ How big each room should be?
§ Where should each room be positioned?
§ What kind of light fixtures should we use?
§ What does beautiful or comfortable mean?
§ Endless questions...

A Picture Is Worth a Thousand Words
§ Outside of the IT world, Architecture can mean:
– A general term to describe buildings and other physical structures.[1]
– The art and science of designing buildings and (some) non-building
structures.[1]
– The style of design and method of construction of buildings and other
physical structures.[1]
§ The customer draws an initial sketch which is then further
refined with the Architect in multiple iterations until the
customer is satisfied and accepts it
[1] Shorter Oxford English Dictionary (1993), Oxford, ISBN 0 19 860575 7

Persona 1 – The Home Owner

Persona 2 – The Architect

Persona 3 – The Interior Designer

Persona 4 – The Electrical Engineer

Other Personas in the Value Stream
Plumbing
Home Automation
design
Sound design
Exterior design
Landscaping
And more…
Questions
§ Could we really design a house
by just using words?
§ If so, would it be faster?
§ How to track the impact across
the layers if the home owner
changes his mind through the
design process?

Contractor 1 – Designing a Plant
CAD Software

CAD Software

Computer-Aided Design Software
§ Assists in accelerating design, but most importantly:
§ Enables multiple personas to work on the same design
§ Enables each persona to work on a separate layer of the
design
§ Keeps all layers together with full traceability
§ Automatically identifies the impact of any changes made to
one layer and prompts the layer owner for an action

Civil Engineering and Software Engineering
The Best From Both Worlds
Civil
Engineering
Design
Construction
Software
Engineering
Design
Construction
§ Multiple Proven
Practices
§ Change Friendly
§ Software
Assisted
§ Change Friendly
§ Historically
Proven Practice
§ Hard and Costly
to Change
§ Manual Design
§ Hard and Costly
to Change

Aligning to Agile Software Design
Building the Architectural Runway
Civil Engineering Design
Software Engineering Construction
§ Multiple Proven
Practices
§ Change Friendly
§ Software
Assisted
§ Change Friendly
§ SW Design Patterns originated from Civil Engineering and Architectural Design, by Christopher Alexander in 1977
§ Model-Driven Visualization of Requirements – also borrowed from Civil Engineering
§ Service Virtualization concepts inspired by practices adopted in Civil and Military Engineering when working with
multiple contractors – understand the interface specs and build to that spec. Flexibility to change implementation

How do we leverage these proven design
techniques into software design?

The Flight Booking Path
§ As a passenger
I want to search for domestic flights
So that I can buy the one that matches my need
A Simple Epic
§ Acceptance Criteria
– I can complete the purchase in
no more than 6 screens
– I can see a list of all flights
available based on my criteria
– I can choose my seats
– I will not see any promotions
§ Definition of Done
– User able to navigate from
search to purchase completion
screen

Generates LOTS of Questions
§ Do you want to view the departure and return flights as a
combined pair, or do you want to select each independently?
§ When should the user select seats? After selecting the
departure and return flights or just before payment?
§ Does the user need to be logged on to do the search? If not,
will the user have the opportunity to logon at some point?
§ Endless questions…

A Picture Is Worth a Thousand Words
§ In the IT world:
– A model in systems engineering and software engineering is a structured
representation of the functions (activities, actions, processes, operations)
within the modeled system or subject area.[1]
– The purposes of the model are to describe the functions and processes, assist
with discovery of information needs, help identify opportunities, and establish
a basis for determining product and service costs[2]
§ The Product Owner draws an initial sketch which is then further refined
with the Modeler in multiple iterations until the Product Owner is satisfied
and accepts it
[1] FIPS Publication 183 released of IDEFØ December 1993 by the Computer Systems Laboratory of the National Institute of Standards and
Technology (NIST).[2] Reader's Guide to IDEF0 Function Models

Persona 1 – The Product Owner

Persona 2 – The (CAD) Modeler
§ Streamlines understanding across all teams in the value
stream by removing all ambiguity from the scope
§ Foundational layer others will build on top of
(single source of truth)

Persona 3 – The Data Engineer

Persona 4 – The Services Engineer

Persona 4 – The Test Automation Engineer

Other Personas in the Value Stream
UX Designer
Performance
Engineer
Environment
Manager
Release
Manager
Operations
Engineer
And more…
Questions
§ Could we really design an
application based on
written language?
– Yes, but how did that work
for us so far?
§ Can we take CD to the next
level of Speed and Quality?

§ Business Flows based
on User Stories
§ Alternate Scenarios
§ Negative Scenarios
§ Data Requirements
§ Data Parameters
§ Test Datamart
connection
Test Data
Model
Layered approach to Continuous Delivery
§ External Interfaces
Requirements
§ API Definitions
§ Service Virtualization
Services
§ Framework linkage
§ Keyword linkage
§ Actions on the app
Test Automation
Overlaying all Layers Roles and Benefits
§ Performance SLAs
§ Load Testing
Scenarios
§ User Profiles
Performance Testing
••Draws and maintains business process flows
••Baselines understanding across the team
••Removes all ambiguities
Modeler
(PO/BA/Tester)
••Find and reserve data automatically
••Match test data to each test case
Test Data
Engineer
••No dependency on external interfaces
••Accelerate Code Development
Services
Engineer
••Automatic creation of scripts
••Automatic impact analysis
Test
Automation
Engineer
••Automatic generation of load scenarios
••Automatic creation of scripts
Performance
Engineer

Let’s take a look at how it actually works…

Questions?

Recommended Sessions
SESSION # TITLE DATE/TIME
DO5X47S 'Shift-Right' - Rapid Evolution with DesignOps 11/16/2016 at 3:45 pm
DO5X27S Whats New in CA Agile Requirements Designer 11/17/2016 at 12:45 pm
DO5X40S Case Study: Why Rabobank Was Inspired 11/17/2016 at 3:45 pm

Must See Demos
Use Model
Based Testing
CA Agile
Requirements
Designer - 515 -
DevOps Theatre 5
DevOps 5 Theatre
Achieve Max
Test Coverage
CA Agile
Requirements
Designer - 516
DevOps Theatre 5
Theatre
Improve Data
Compliance
CA Test Data
Manager
512 - DevOps
Theatre 5
Deliver Test
Data Faster
CA Test Data
Manager
511 - DevOps
Theatre 5

Stay connected at communities.ca.com
Thank you.

Model-Based Testing for Achieving Maximum CD Pipeline Acceleration

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Model-Based Testing for Achieving Maximum CD Pipeline Acceleration

Similar to Model-Based Testing for Achieving Maximum CD Pipeline Acceleration (20)

More from CA Technologies

More from CA Technologies (20)

Recently uploaded

Recently uploaded (20)

Model-Based Testing for Achieving Maximum CD Pipeline Acceleration