Gap assessment Continuous Testing

Gap Survey, Assessment and Analysis
For
Continuous Testing
How to identify and analyze Gaps in Continuous Testing
practices.
Gap Assessment – Continuous Testing Copyright © 2021 by Marc Hornbeek All rights reserved.

WHY?
Gap analysis provides valuable
inputs for formulating a strategy
and roadmap to improve a topic.
2
• Gap Survey is discovery tool used to collect
information about the current state of
practices for a topic.
• Gap Assessment is a process to determine
differences (Gaps) between current practices
and recommended practices for a topic.
• Gap Assessment Workshop is a meeting in
which the Gap Assessment is validated prior
to Gap Analysis.
• Gap Analysis is the process of determining
priorities for reducing Gaps found during a
Gap Assessment.
• Gap Assessment Tool is software that helps
perform Gap Surveys, Gap Assessments and
Gap Analysis.
www.engineeringdevops.com
http://mybook.to/engineeringdevops
Gap Assessment Copyright © 2021 by Marc Hornbeek All rights reserved.

3
1. Pick a
Topic that
you want to
analyze.
3. Enter practices
into the
Gap Assessment
Tool.
4. Determine
who needs to
be surveyed.
5. Gap
Survey
Collect data
with Survey
6.
Perform Gap
Assessment
7.
Perform
Gap
Analysis
2. Determine
Practices for
the Topic.
1. 2. 3. 4. 5. 6. 7.
Gap Assessment Copyright © 2021 by Marc Hornbeek All rights reserved.

• “Continuous Testing” is essential for DevOps value streams.
• To get the maximum value from Continuous Testing it is important to use
recommended practices, inclusive of people, process and technologies.
• A Gap Assessment is a great way to efficiently evaluate an organization’s
practices for Continuous Testing and determine a strategy for
improvement.
• The Gap Assessment Tool greatly speeds up the Gap Survey, Gap
Assessment and Gap Analysis process.
• What follow is an example of the Gap Assessment tool being used for the
topic “Continuous Testing” for an organization.
Step 1: Topic
Gap Assessment for Continuous Testing
Gap Assessment - Continuous Testing Copyright © 2021 by Marc Hornbeek All rights reserved.

The 15 practice category pillars
are:
• Continuous testing tenets
• Leadership
• Culture
• Test strategy and plans
• Test management
• Test automation
• Test tools
• Test infrastructure mngt.
• Test scripts
• Test results
•• Security tests
• Pre-flight testing
• Integration testing
• Pre-production testing
• Test in production
Step 2: Determine Practices
for Continuous Testing Gap
Assessment
For an example Continuous Testing assessment, I used the
practices under 15 practices categories described in three
blogs that I published as a three-part series:
https://devops.com/continuous-testing-practices-part-1/

Step 3: Enter Continuous Testing
Practices Into the Gap Assessment Tool.
A Gap Assessment Tool, pre-
loaded with a sample of
Continuous Testing practices in
a file called “Continuous Testing
Assessment” can be
downloaded for free from one
of the resource pages found on
www.engineeringdevops.com
Edit the practices categories
and add/delete practices
from each category if you
prefer to make changes.

Step 4: Determine who needs to provide input.
• Business leaders – because they influence culture and budgets for Continuous Testing.
• Developers – because they need to design in accordance with Continuous Testing practices.
• Project owners – because they influence product work priorities.
• QA testers - because they need to test in accordance with Continuous Testing practices.
• Ops - because Continuous Testing practices affect operations.
• Security – because Continuous Testing affect security practices.
For the Gap Analysis to be comprehensive, people in roles that are affected by
container practices need to be surveyed, or at least represented to ensure their
perspectives are included. The following are example roles that typically are included.
The survey can be conducted for an individual application, a group of applications, or
all the applications in the enterprise. However, it is important that the Gap
Assessment and Gap Analysis be performed on the organization segment that is being
targeted for improvement.

Step 5: Gap Survey
A Gap Survey should allow each surveyed person to enter an “Importance Level”
score, a “Practice Level” score and Comments. These are all essential for the Gap
Assessment and Gap Analysis.
Survey
(I)
Importance
(P)
Practice Level
Continuous Testing (CT) Practices
How important is this practice
to the organization? 0=not
relevant, 1=not important,
2=nice to have, 3=important,
4=very important, 5=critical
To what extent does the
organization practice this?
0=not sure, 1=Rarely, if ever;
2= Sometimes; 3=Most of the
time; 4=Always; 5=We are
really good at this. Unsure
enter NA.
Comments
Continuous Test Tenets
The Continuous Testing principle "Shift Left" is practiced in which tests are conducted as
early in the pipeline as possible.
The Continuous Testing principle "Fail Early" is practiced in which tests are arranged so
that the most likely problems are found in the early stages of the pipeline.
The Continuous Testing principle "Fail Often" is practiced in which tests are run frequently
and with many different conditions such as different variations of operating systems,
production topologies, browsers and production versions of services.

Step 6: Gap Assessment
The Gap Assessment process requires all the practices scores collected from the
surveys to be collected and assembled to calculate an aggregate set of scores. Gap
scores are calculated using a formula that weights each Practice Level score with the
corresponding Important Level score. A visual representation helps to identify
practices areas and individual practices that have the highest, most important Gaps.

Step 6b: Gap Assessment Workshop
No matter how well written the practices and score definitions are, it is not
unusual for some people to misunderstand and to enter scores that they would
not have otherwise.
For this reason, it is important to ensure that the data collected is validated before
conducting the Gap Analysis.
The preferred approach is to conduct a Gap Assessment Workshop with key
representatives from each role that participated in the survey.
During the workshop, the scores for each practice that have a high deviation
between survey responses are discussed and if necessary adjusted.

Step 7: Gap Analysis
The Gap Analysis process involves
extracting the high GAP practices
and tagging and ranking each of
them against solution categories
that are determined by a
consultant or expert.
The Gap Analysis indicates where
solution strategies and
implementation roadmaps need to
be focused to reduce the most
important gaps.
ContinuousTesting(CT)Practices
(I)
Importance
(P)
PracticeLevel
(G)
GAP
Leadership
How important isthis
practiceto the
organization? 0=not
relevant,1=not important,
2=niceto have,
3=important, 4=very
important,5=critical
To what extent doesthe
organization practicethis?
0=not sure,1=Rarely,if
ever; 2= Sometimes;
3=Most ofthetime;
4=Always;5=Wearereally
good at this. Unsure enter
NA.
Computed result is1-15.
=ʄ([(I),(P)] A score7.5or
higher indicatesan
important GAP. Process Technology People
Teststrategiesandplansarecreatedforeachproductorserviceandreviewedbycross-
functionalteamincludingdevelopment,QA,OpsandSecurityteams. 5 1 15
1
Teststrategiesandplansidentifythetypeoftests,andtheextentoftestingactivitiesfor
eachstageinthevaluestream. 5 1 15
1
Schedulesforalltesttasksareidentifiedandtracked. 5 1 15 1
Testautomationrequirementsaredefined. 5 1 15 1
Testtoolsoftwareversionsaremaintainedinasourceversionmanagementsystem. 5 1 15 1 1
Techniquesusedtoacceleratethetimeforexecutingtestsareusedsuchas verticaland
horizontalscalingoftestcampaigns.
5 1 15
1
Leadersestablishadefinitionfortestcoverage. 4 1 12 1 1
Leadersclarifyhoweveryone’srolesfitwiththeCTstrategy. 4 1 12 1
LeadersestablishincentivesforCTperformance. 4 1 12 1
Automaedtestcasesaredesignedtorunindependentofeachother. 4 1 12 1 1
Teststrategiesandplansidentifya targetfortestautomationandmanualtestingforeach
typeoftesting. Forexample100%ofalltestsintheintegrationstageand85%oftestin
Pre-Productionstageisatypicalgoal.
5 2 10
1
Allaspectsoftestmanagement, suchastestplanning,testguidelines,testcodeversion
management,andversionmanagementoftestresults,arehandledbytoolsthatarewell
integratedintothecontinuousdeliverypipelineandanytestframeworkbeingusedfor
testautomation.
5 2 10
1

Marc Hornbeek
a.k.a. DevOps_the_Gray esq.
CEO and Principal Consultant
Engineering DevOps Consulting
Author – Engineering DevOps
mhornbeek@engineeringdevops.com
Learn More

Gap assessment Continuous Testing

In this document

More Related Content

What's hot

Similar to Gap assessment Continuous Testing

More from Marc Hornbeek

Recently uploaded

Gap assessment Continuous Testing