Testaus 2014: Paul Gerrard - The Changing Role of Testers'

© 2012 Gerrard Consulting Limited
Version 2.0
Page 1
Intelligent Testing and Assurance
@paul_gerrard
Paul Gerrard
paul@gerrardconsulting.com
gerrardconsulting.com
The Changing Roles of Testers
A personal view
Programme Chair 2014
Helping clients transform their testing through
INNOVATION, COACHING and LEADERSHIP
Our CLIENTS
– Want to be agile rather than follow Agile dogma
– Have a pragmatic approach and are focused on delivery
– Want a solution that fits, not a badly fitting suit.

Version 2.0
Page 2
Theme
• Pressure to Change
• Software Development Patterns
• Three Innovations that Drive Change
• A New Model for Testing
• We are in Transition
• Leading Change – Killing Clichés.
Intelligent Definition and Assurance Slide 3
Pressure to Change
I meet dozens of Test Managers
through the Test Management Forum
http://uktmf.com

Version 2.0
Page 3
Every organisation seems to be
evolving (and quickly)
• 'Traditional' companies are evolving hybrid
Structured-Agile approaches
• Some companies offshore/outsource testing
– Others are bringing it in-house again
• Some are 'shifting-left'
– BDD,TDD or doing story-driven development
– Moving to Continuous Delivery and DevOps
• Embrace change and get promoted 
Have you been Agiled?
Agile is mainstream.
Why aren't you Agile?

Version 2.0
Page 4
Structured versus Agile approaches
Waterfall/Structured
Agile
‘Pure’ Agile
High Structure/Formality
• A one-dimensional view of the range of approaches
• Varying formality, but ALL require discipline
• But this one-dimensional view is too simplistic
• Emerging out of the Agile approach are (what I call)
factory approaches
• Lean, Kanban, Continuous Delivery blah blah…
No
man’s
land?
Process Formality
Wikipedia – “Agile Methods”
A confusing state of affairs

Version 2.0
Page 5
There are three patterns, not two
• Some ‘new’ approaches are labelled Agile but
they aren’t!
• Continuous-Delivery approach is appropriate
for rapid-delivery into online/mobile systems
• CD advocates often have an Agile background,
so you might think CD is Agile
• Not so.
Software Development
Patterns
A Model

Version 2.0
Page 6
Three development patterns
Structured
Agile Continuous
Autonomous
Key pattern characteristics
Characteristic Summary
Structure Structure of the project team
Pace/cadence What drives the rate of decisions making
Leadership How is the team managed/directed?
Definition In what format is requirements knowledge captured?
Testing How is testing performed? (Excludes BDD, TDD)
Auto. Test When are automated tests created? Who leads?
Measurement What/how is project measurement performed?
Governance What form does governance take?
All models are WRONG, some are USEFUL
Needless to say, I’ll be generalising quite a lot 

Version 2.0
Page 7
Profiles of the three patterns
Characteristic Structured Agile Continuous
Structure Managed Autonomous Production Cell
Pace/cadence Business decision Team decision Feedback
Leadership Project Managed Guided Research Line Managed
Definition Fixed spec Dynamic spec Live Specs
Testing Scripted Exploratory Automated
Auto. Test Retrospective Developer led Pervasive
Measurement Pervasive Avoided Analytics
Governance Bureaucratic Trust-based Electronic
Intelligent Definition and Assurance Slide 133 patterns – probably more. What about testing?
Three Innovations That
Are Driving Change
Being agile
Shift Left
Analytics

Version 2.0
Page 8
#1
Agile doesn’t work
but being agile might
Agile is no longer innovative.
Challenging Agile and being agile
might be
Is continuous delivery for you?
Agile doesn’t work?
• Well of course, it works for a lot of teams
• But larger/corporate organisations trying to
‘go Agile’ have difficulties
• Mostly a cultural problem
– Requirements take months to elicit and agree
– Complex stakeholder relationships
– Stakeholders with ‘all the knowledge’ can’t be
spared to be the on-site customer/product owner.

Version 2.0
Page 9
Pragmatism rules (again)
• Pragmatic companies use the things that work
and discard those that don’t
• Let's be honest: Most are doing some form of
hybrid approach:
– Embedding testers into their dev teams
– Breaking large requirements into stories
– Development of very small work packages
• TDD, BDD,ATDD, SBE today…
• Continuous Delivery, DevOps… tomorrow?
#2
Shift Left
Developers in test
Testing (testers?) everywhere

Version 2.0
Page 10
A tester job spec. (May 2014)
• Skills & Requirements
– Thorough C# knowledge including Linq
– Good .Net experience including charting and winforms
applications.
– Experience building and maintaining automated test systems
– Ability to interact with the developers to anticipate future
needs
– Ability to influence to ensure good test procedures are
developed and implemented
• Desirable Skills
– Some knowledge of Java for mobile testing
– An interest in photography.
TestingBDD/Test-Driven)
Trusted
Requirements +
Stories
Shift left (redistributed testing)
Functional
Requirements
Build (+little testing) but
lots of rework
Testing (lots of
checking + bugs)
Trusted
Requirements +
Stories
Testing
SAVED
TIME
BDD/Test-Driven
Who is going to do all this?
BAs, Developers or Testers?

Version 2.0
Page 11
From traditional delivery…
… to Continuous Delivery
#3
Test Analytics
Analytics from production
Analytics from test
An opportunity for testing

Version 2.0
Page 12
Analysis Insight Decision
Development
and Testing
Production
Testing
Introducing Test Analytics
“Introducing Test Analytics”, Paul Gerrard:
http://gerrardconsulting.com/index.php?q=node/630
Julian Harty, "Symbiotic Relationships betweenTesting and SMAC":
http://blog.bettersoftwaretesting.com/2013/07/symbiotic-relationships-between-
testing-and-social-mobile-analytics-cloud-computing/
“The capture, integration and analysis of test and
production monitoring data to inform business and
software development decision-making”.
Data silos for test analytics
(an illustration)
Stakeholder Requirements Assurance (Production)
Application
Monitoring
Production
Environment
Monitoring
 Stakeholders
 Business Goals
and measures
 Stakeholder
Involvement/
Engagement
 Risk
 Requirements
 Stories/Feature
Descriptions
 Glossary of terms
andTerm Usage
 Processes
 Process Paths
(Workflows)
 ManualTests
 GeneratedTest
Code (unit,
integration,
system level)
 Application
Instrumentation
 AutomatedTest
Execution History
 Test Status
 Feature Code
Change History
 Application
Process Flows
 User Accesses/
Activities
 Feature Calls,
Response times
 Feature use cases
(data usage)
 Application
Alerts/Failures
 Database Accesses
 Database Content
 Production
failures
 System assets
 Resource Usage
Log
 Performance data
 System Events
 System Alerts/
Failures/Incidents
 Outages

Version 2.0
Page 13
A simple example
(implementing a dictionary)
• A list of sentences, complete with source references that
contain important glossary phrases
• How is the same phrase is used in different contexts?
– Is its use consistent?
– If you have an agreed definition, are anomalies obvious?
– Identify requirements and features that do not mention the key
concept – but should do?
• Reference/coverage for any and every concept you define
in your glossary:
– Requirements
– Stories
– Test cases
– Test execution status and history.
A New Model for
Testing
This is a first attempt. Don't throw stones.Yet.
I'm working on a refined version (with others).
Want to help/review?

Version 2.0
Page 14
ALLTesting is
Exploratory
You explore sources of knowledge ...
... to build test models ...
... that inform your testing.
The exploration cycle
ExplorationDefinitions
(specs)
Oracles
Sources
Design
Models
Test
Models
Enquiry
Challenge
Sources:
People, documents,
experience, system under test
Heuristic
Test Models:
Can be documented
or mental models
Enables Prediction
System
under test

Version 2.0
Page 15
The testing cycle
Testing
System
Under Test
Refines model
Informs tests Apply tests
Interpret
outcome
Test
Models
Refine the
System
Logs
Problems
Apply
changes
More explore
Report
What if our source of knowledge is
mostly the system?
• Exploratory testing is the approach to use
when there is little documentation
• Our ‘source of knowledge’ is also the ‘system
under test’
• We need to merge our source of knowledge
and system under test into one...

Version 2.0
Page 16
"Exploratory" testing
Slide 32
Testing System
Under Test
Refine
Model
Apply tests
Interpret
outcome
Test
Models
Refine the
System
Log
Problems
Apply
changes
Exploration
Enquiry
Challenge
Heuristic
Enables
Prediction
Inform
tests
Report
Narrative
• All testing is "explore and test (or check)"
• A human OR tool can ‘apply the tests’ but a
human creates the model
• ‘Apply test’ means inputs can be sent and
outcomes compared in some way
– (Define ‘apply the tests’ as checking if you want)
– But interpretation is always required
• Humans interpret and act on outcomes
• 'Report' probably includes problem reporting.

Version 2.0
Page 17
What does this mean for testing?
• What is important?
– The thought processes: Explore and Test/Check
– Modelling and test design
– Observation, critical thinking, communication
– Transitions: Exploring  Testing/Checking
• Less important – logistics
– The stages of testing
– Documentation of testing, incident tracking
– Test design techniques
• Assurance covers both testing and logistics.
We are in Transition

Version 2.0
Page 18
What is Agile really,
really about?
COLLABORATION
and
RAPID FEEDBACK
The trend(s) of testing
• Testing isn’t a stage at the end, it’s a part of all
creative activities
– All artefacts are tested, by someone, in some way
• Front-loaded testing (test early, test often)
– But it’s not always called testing
• Test-Driven approach isn’t about test, its about
design+risk – devs beginning to embrace it
• Behaviour-driven approach connects
requirements, software design and testing.

Version 2.0
Page 19
Leading Change
Here are the clichés you need to kill.
Here are the soft spots for change.
Developers write code;
testers test because
developers can’t
No they don’t

Version 2.0
Page 20
Goal: End Manual
Feature Checking
Let the tools do it and
free people to do testing please!
Testing doesn’t just
apply to existing
software, at the end
We test systems, not software.
We test early, we test often but it
might not be called testing

Version 2.0
Page 21
Testing is about
measuring achievement,
not quality
Align systems delivery with project
goals and risks and you’ve got a seat
on the board
Testers don’t own
testing anymore
Be a TestMaster
"Assurance through the lifecycle"

Version 2.0
Page 22
New Model Testing
changes what testers
need to learn
Certification teaches logistics and
bureaucracy.
We need to learn and teach
modelling and critical thinking.
The phase after
development is
REWORK, not Testing
Think about it.

Version 2.0
Page 23
The phase after development is
REWORK, notTesting
• The "Test Phase" should not exist
• Testing must align with development, not
compete with it or rescue it
• Integrate test with code change and remove
the safety net
• Managers point the finger at and put pressure
on development, not testing
• We need a new model of testing.
@paul_gerrard
Paul Gerrard
paul@gerrardconsulting.com
gerrardconsulting.com
The Changing Roles of Testers
A personal view
Programme Chair 2014

Testaus 2014: Paul Gerrard - The Changing Role of Testers'

More Related Content

What's hot

Viewers also liked

Similar to Testaus 2014: Paul Gerrard - The Changing Role of Testers'

More from Tieturi Oy

Recently uploaded

Testaus 2014: Paul Gerrard - The Changing Role of Testers'