Requirements are King – Better Requirements = Better Software

World®
’16
Requirements are King – Better
Requirements = Better Software
Benjamin Johnson-Ward – Senior Consultant, Presales – CA Technologies
DO5X23E
DEVOPS

2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
For Informational Purposes Only
Terms of this Presentation
© 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.

Abstract
Countless research and case studies have long pointed to a cold, hard truth: the requirements, not
developers or testers, introduce the vast majority of defects that wreak havoc in production. Rather than
solve this issue, the development practices that have arisen around “Agile” have introduced a new
challenge, as testers have to piece together and react to a constant barrage of disparate user stories and
change requests.
What’s more, the requirements remain static, so that testers have to manually convert them into tests.
This is a slow, unsystematic process, which creates defects when the ambiguous requirements are
misinterpreted.
In order for testing and development to keep up with constantly changing user needs, requirements
must be captured in a format which minimizes miscommunication, can react to change, and enables
subsequent test assets to be derived automatically.
§ This session will set out how “active” flowchart modelling can:
§ Provide a format already familiar to business analysts to capture user needs
§ Provide a format for testers to generate optimized test cases, test scripts, data and expected results
automatically.
§ Automatically reflect changes made to the requirements in the tests assets
In this requirements-driven approach, testing thereby becomes a largely automated comparison of how a
system should function to the system that has been delivered.
Benjamin
Johnson-
Ward
CA Technologies
Senior Consultant,
Presales

Agenda
WHAT IS A REQUIREMENT?
WHY FOCUS ON REQUIREMENTS?
Q&A
HOW ABOUT SOFTWARE ENGINEERING?
HOW ABOUT REQUIREMENTS ENGINEERING?
INTRODUCING THE REQUIREMENT-DRIVEN APPROACH
1
2
3
4
5
6

What is a Requirement?

Definition: Requirement
Requirement: singular documented physical and functional need that a particular design, product
or process must be able to perform.
ISO/IEC/IEEE 29148:2011
Wikipedia
Requirement: statement which translates or expresses a need and its associated constraints and
conditions.

What is a Requirement?
Erik Simmons, nuCognitive, “21st Century Requirements”, 11/08/11
A requirement is a statement of:
1. What a system must do (functional requirement)
2. How well the system must do what it does (quality or performance requirement)
3. A known resource or design limitation (constraint)
More generally,
A requirement is anything that drives a design choice

The Purpose of Requirements
The purpose of requirements is to help establish a clear,
common, and coherent understanding of what the system
must accomplish.
So why focus on requirements?
Erik Simmons, nuCognitive, “21st Century Requirements”, 11/08/11
Clear
All statements are
unambiguous, complete,
and concise
Common
All stakeholders share the
same understanding
Coherent
All statements are
consistent and form a
logical whole

Why Focus on Requirements?

What are the Main Software Challenges You are Facing?
Pre-webinar survey, “Testing Imperatives in the World of Agile Development and Continuous Delivery”, Huw Price & Jonathon Wright, 28/07/15
0% 10% 20% 30% 40% 50% 60% 70%
Time/resources in test data compliance (PII)
Defects stemming from ambiguous requirements
Testing innefficiencies leading to higher cost
Lack of test coverage creating defects/rework
Difficulty finding the right data for a particular test
Manual testing leading to project delays

VAGUENESS
Examples of Ambiguity and Incompleteness
Poor Scoping
Ambiguous Reference
Omission Acronyms
Aliasing
Sentence
Structure
Dangling
Else
Incomplete
Glossary
Implicit Constraints
I.E. or E.g.?
Lexical Ambiguity
Polysemy
Passive
Sentences

What Makes a Good Requirement?
§ complete
§ consistent
§ correct
§ modifiable
§ ranked
§ traceable
§ unambiguous
§ understandable
§ testable
NASA, Automated Requirements Measurement Tool, 1999, http://satc.gsfc.nasa.gov/tools/arm (archive.org)

What is the Problem with Requirements?
§ Though a plethora of techniques
exist, most are written in ambiguous
natural language
§ The requirements are “static” –
they offer no way to derive tests
directly from them…
§ … no way to update tests when the
requirements change – this has to
be done manually
Bender RBT, 2009

User
The end user knows
what they want
The SDC: What’s Supposed to Happen
Business Analyst
The analyst specifies
what that is
Developer
The programmer
writes the code
Tester
The tester tests
the program

The Fundamental Issue With This Model
§ At each stage that information is passed on and transformed
to another form, there is a natural degradation in quality
§ The typical SDLC has several of these “hops”:
1. Requirements are converted to code
2. Requirements are converted to test cases, to be applied to the code
3. Test cases are converted into automated tests
4. Data has to be found or created, based on the requirements, for
development and testing
Information Hops: Enterprise-Wide “Chinese Whispers”
The impact on quality is created by uncertainty. Some (inherent) uncertainty is
unavoidable. Some is avoidable, and should be avoided.

“Rumsfeld Matrix” – Avoidable vs Unavoidable
Uncertainty
§ Information in the SDLC can be broken down into:
– Known Knowns
– Unknown Knowns
– Known Unknowns
– Unknown Unknowns
Knowns Unknowns
Known Information Avoidable Uncertainty à
Observable Defects
Unknown Avoidable Uncertaintyà
Observable Defects
Unavoidable Uncertainty
In an ideal world, testers will be provided with all the information to discover
observable defects. Whatever previously unknowable information they discover
will add to the “known knowns”

Unambiguous, Incomplete Requirements
§ Poor quality requirements introduce uncertainty at the
beginning, which is then perpetuated throughout the SDLC
§ A system can be build around misunderstanding, so that:
– At least 56% of defects stem from ambiguity in requirements
1
– some
place this as high as 59%
2
or even 65%
3
– 64% of total defect cost
3
originate in the requirements analysis and
design phase – some place this as high as 80%
1
– 40-50% of project costs are expended in rework
4
– On average, only 69% of desired functionality is actually delivered
5
Introduce avoidable uncertainty at the very beginning
1 – Bender RBT, 2009
2 – IT Univeristy, Copenhagen, 2001
3 - Hyderabad Business School, 2012
4 – Critical-Logic, 2014
5 – Standish Group’s Chaos Manifesto 2014

Unambiguous, Incomplete Requirements
An example ambiguity review:

Ambiguous Natural Language
§ “Time flies like an arrow; fruit flies like an apple”
§ “Buffalo buffalo Buffalo buffalo buffalo buffalo
Buffalo buffalo”
– A grammatically correct sentence, which means “In the city of Buffalo,
bison who are bullied by bison, are themselves bullying bison”
§ We saw her duck
Sentence structure, homonyms, polysemy, lexical ambiguity, etc.

The Dangling Else
§ IF A THEN IF B THEN C ELSE D
– IF A THEN (IF B THEN (C) ELSE D)
– IF A THEN (IF B THEN (C)) ELSE D
§ IF RAIN THEN IF WINDY THEN COAT ELSE T-Shirt
– If it is raining, then wear a coat if it is also windy. If it is raining but not
also windy, wear a T-Shirt.
– If it is raining, and if it is windy, then wear a coat. Otherwise wear a T-
Shirt
Incomplete system logic stemming from syntactical ambiguity

Though a Plethora of Techniques Exist, Most Are
Written in Ambiguous Natural Language

What Are the Challenges With Requirements?
1 – Bender RBT, 2009
2 – IT University, Copenhagen, 2001
3 - Hyderabad Business School, 2012
4 – Critical-Logic, 2014
5 – Standish Group’s Chaos Manifesto 2014
§ A system can be build around misunderstanding, so that:
– At least 56% of defects stem from ambiguity in requirements
1
– some
place this as high as 59%
2
or even 65%
3
– 64% of total defect cost
3
originate in the requirements analysis and
design phase – some place this as high as 80%
1
– 40-50% of project costs are expended in rework
4
– On average, only 69% of desired functionality is actually delivered
5

The Dangling Else
§ IF A THEN IF B THEN C ELSE D
– IF A THEN (IF B THEN (C) ELSE D)
– IF A THEN (IF B THEN (C)) ELSE D
§ IF RAIN THEN IF WINDY THEN COAT ELSE T-Shirt
– If it is raining, then wear a coat if it is also windy. If it is raining but
not also windy, wear a T-Shirt.
– If it is raining, and if it is windy, then wear a coat. Otherwise wear
a T-Shirt
Incomplete system logic stemming from syntactical ambiguity

The Missing Else
A common menace when using Behaviour-Driven Development, Gherkin and RSPEC
GIVEN Current Account
And GIVEN negative balance
WHEN customer attempts payment
THEN go into overdraft

Requirements
As many as 56% of
defects are avoided
in development
§ Test cases can be Auto-
generated and re-generated
from requirements
§ They are optimized to provide
100% coverage in fewer tests
§ Changes can be quickly and
easily made
§ Requirements and
test cases are linked
to data with 100%
coverage, including
negative and future
scenarios § Automated tests
are generated from
test cases
§ Are updated if
requirements change
§ Exhaustively test
application
§ Pallet-based UI
Automation
Requirements are modelled
to a complete, unambiguous,
“Active” Flowchart
Reduce Uncertainty – Clear Requirements and Fewer “Hops”
Code Test Cases Test Data Automation

A Possible Solution: Model-Based Testing
1. Mapping requirements to an unambiguous, active flowchart
– Coders know what needs to be coded
2. Auto-generate test cases and Automated tests –
no more “Hops”
– Execute these tests against the code, testing that it matches
the functionality specified in the requirements
3. Update the model as the requirements change, or as testing
uncovers further, previously “unknown” information
Test a system against an agreed upon, clear definition of what the functionality should be

How About Software Engineering?

What is Engineering?
Engineering is the application of heuristics under uncertainty to
cause the best possible change within the available resources
(Billy Vaughn Koen*)
§ A heuristic is anything that provides a plausible aid or direction in the
solution of a problem
§ Use of heuristics does not guarantee a solution to a problem, and different
heuristics can offer conflicting advice for any given situation
§ Heuristics work by exploiting the structure of an environment to simply
decision making and narrow the solution search space
Erik Simmons, nuCognitive, 11/09/16 adapted from “a discussion of the method”, Billy Vaughn Koen Oxford University Press 2003

How Does This Apply to Software Engineering (SDLC)?
Simple example of lack of clarity and vision within the software development lifecycle
Business Analyst Programmer TesterUser
The User Knows what they want
The Analyst specifies what that is
The Programmer writes the code
The Tester tests the program
Clarity & Vision

How About Within Lean & Agile?
Dean Leffingwell, Scaled Agile Framework Foundations (v4.0.6), “Embracing Lean-Agile values”, 2016
House of Lean Agile Manifesto
LEADERSHIP
Respectfor
peopleandculture
Flow
Innovation
Relentless
improvement
VALUE
Value in the sustainably
shortest lead time
That is, while there is value in the items on the
right, we value the items on the left more.
We are uncovering better ways of developing software by doing it and helping
others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

How About Requirements Engineering?

Definition: Requirement Engineering
ISO/IEC/IEEE 29148:2011
Wikipedia
Refers to the process of defining, documenting and maintaining requirements to the sub-fields of
systems engineering and software engineering concerned with this process.
Interdisciplinary function that mediates between the domains of the acquirer and supplier to establish and
maintain the requirements to be met by the system, software or service of interest. Requirements engineering
is concerned with discovering, eliciting, developing, analysing, determining verification methods, validating,
communicating, documenting, and managing requirements.

What is Requirements Engineering?
Requirements engineering is the use of heuristics for discovering,
documenting, and maintaining a set of requirements for a system or service
At the highest level, requirements engineering seeks to:
§ Define the necessary and sufficient system scope and features to
enable system development at an acceptable level of risk
§ Enable requirements-driven architecture, design, construction,
and verification activities
§ Provide information about system development progress and status
§ Provide information to organizational learning and continuous
improvement efforts
Erik Simmons, nuCognitive, 11/09/16 adapted from “a discussion of the method”, Billy Vaughn Koen, Oxford University Press 2003

How to Manage Requirements Engineering?
To Generate
Test Cases
Design and provision
Test Data Virtualisation
To Manage
Change in
Test Cases
To Generate
Automation
Tests
To Estimate
Complexity
Populate
Story Boards
& Backlogs
To Build Better
Requirements
To Improve
Existing
Test Cases
Design and provision
Service Virtualization
Impact and
Differences
built by
CA Agile
Requirements
Designer
Native Support

Introducing the Requirement-driven
Approach

DesignOps is the New Delivery Paradigm:
From Ideation to Design, Development and Testing

Model Requirements as an “Active” Flowchart
§ A formal model that is accessible to the
business who already use VISIO, BPM, etc.
§ Which is also a mathematically precise
model of a system, so that it eliminates
ambiguity and incompleteness
§ It can be used by testers and developers –
it brings the end-user, business and IT
into close alignment

Testing With the “Active” Flowchart
§ Testers can overlay the
flowchart with all the
functional logic and data
involved in a system
§ Tests can therefore be
automatically derived
from it

Generating Test Cases
§ Exhaustively test a model – extract all possible routes from
start -> end
§ Each route / path becomes
a test case
§ Number of routes grows
exponentially with every
added decision
32 nodes + 62 edges = 1,073,741,824 possible routes
2145 Paths

Optimization / Coverage
§ Coverage, in most cases, is related to how much functionality
is being covered in a test case.
§ Traverse the model by satisfying coverage constraints:
19 PathsEdge Pairs
9 PathsIn Out Edges
5 PathsEdges
3 PathsNodes

Generate Test Data
Not Ready for Testing!
CA Test Data Manager + Required data characteristics
Provision fit for purpose data anytime and every time
Provision data with or without access to production systems
Ready for Testing!
§ Automatically profile data, model it, and accurately measure its coverage
§ Generate rich synthetic data which provides 100% coverage
§ Cover every outlier, unexpected result, boundary condition and negative path

Provide Full Traceability with Requirements
§ Know what needs to be re-tested and when the integrity of a
system is at risk… “If I change this, what will I break?”
§ The impact of a change made
to an individual component is
identified system wide
§ The impact on test cases and
user stories up and down a
system can also be identified
automatically

But All of This Still Depends on the Requirements
Being Right!
§ Yes, you can reap huge rewards from maintaining one very
high quality artefact, rather than thousands.
§ No, no silver bullet, models can be wrong, people can still
make mistakes
§ The idea behind MBT and ARD is to make those potential
mistakes more obvious and easier to fix once discovered
§ And of course leverage the model, whilst acknowledging the
residual risk.

Demo

Fun Example: Pokémon Go
Pokémon Go, Huw Price, 22/07/16, (www.grid-tools-downloads.com/huw/POKEMONGO.zip)

Summary: Heuristics, Risk and
Requirements Engineering
§ Value is key to delivering software which matters to users:
– a solution must be frequently and iteratively measured for value
§ Poor Requirements degrade quality but also interrupt the value stream:
– Testers cannot test rigorously or quickly against customer needs
§ Requirements can also therefore be key to improving the delivery rate of
valuable software
§ Enabling requirements-driven design, construction and verification:
– Model requirements as “active” flowcharts
– Test from the flowchart with auto-generation of tests/scripts
– Generate test data to match the test cases
– Update tests when the requirements change

Questions?

Recommended Sessions
SESSION # TITLE DATE/TIME
DO5T05T
TechTalk: Going Full Circle - Linking Code to Tests to
Requirements and Back Around Again
11/16/2016 at 12:45 pm
DO5X27S What's New in CA Agile Requirements Designer? 11/17/2016 at 12:45 pm
DO5X40S Case Study: Why Rabobank Was Inspired 11/17/2016 at 3:45 pm

Must See Demos
Use Model
Based Testing
CA ARD
DevOps Theatre 5
DevOps 5 Theatre
Achieve Max
Test Coverage
CA ARD
DevOps Theatre 5
Improve Data
Compliance
CA Test Data
Manager
DevOps Theatre 5
Deliver Test
Data Faster
CA Test Data
Manager
DevOps Theatre 5

Stay connected at communities.ca.com
Thank you.

Requirements are King – Better Requirements = Better Software

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Requirements are King – Better Requirements = Better Software

Similar to Requirements are King – Better Requirements = Better Software (20)

More from CA Technologies

More from CA Technologies (20)

Recently uploaded

Recently uploaded (20)

Requirements are King – Better Requirements = Better Software