This is the base material used during a half day workshop at expoQA 17 June 2019. Peter Marshall runs over the necessary technical, organisational, and improvement practices required to deliver high quality software. Deep dives into Continuous delivery, devops, organisational structures, agile and digital transformation.

1. How embracing a devops
culture, continuous testing,
and a user-centric approach
to product development
can help deliver products
that are aligned with your
end users’ quality
expectations
Taking a
technical and
holistic approach
to software
quality

2. CTO, Agile Consultant &
Technical Adviser
20+ years working in
commercial software
development
twitter
@petemar5hall
Pete Marshall
mail
info@leansoftwareservices.com

3. Super Hero Top Trump Cards
Name, Alias, Super Power, Values, Contribution,
Fears and Hopes?

4. Gain deep insight into how thinking about the end user first
pushes us towards optimising and improving our development
workflow to improve quality.
Understand how operability, good communication, a well
structured organisation and modern development practices
contribute to our holistic approach to quality
Learning Outcomes

5. On to the problem
domain

6. Who is working in an agile environment?
who is doing continuous delivery?
Who has a devops culture / team / role?
Who is really satisfying their end users on a regular basis?

7. Our end users
really drive the
necessary
quality of our
software

8. continuously
evolving product
great UI
constantly
available
features
work
the right
features
Users

9. Sources: Applied Software Measurement: Global Analysis of
Productivity and Quality by Capers Jones, & IBM The Future of Testing
Most forms of testing find fewer than 30% of all bugs
almost 40% of testers have seen an increase in the frequency of release
Productivity and quality are directly coupled: projects with
high quality have high productivity and vice versa

10. How do we meet
our end users
expectations of
quality?

11. rewind a little..
we have been trying to solve
this problem for a long time!

13. What does
this really
mean???

14. the test and QA
headache is
compounded
further by the need
to release faster
and more often
Amazon is on record as making changes
to production every 11.6 seconds on
average in May of 2011.
Facebook releases to production twice a
day.
Many Google services see releases
multiple times a week, and almost
everything in Google is developed on
mainline.
Etsy push unto 70 changes a day to its
monolithic software!

15. the presence of a
tester can sometimes
introduce more risk
than they mitigate..

16. Only 21% of
issues detected in
software projects
are feature
related.

17. What is an
holistic approach
to software
quality?

18. Key Concepts
…continuous
testing…Operability
Good CommsUser centric
- Team ->

19. Using Radars

20. The idea is that a person passes through three stages of gaining knowledge:
Shu: In this beginning stage the student follows the teachings of one master
precisely. They concentrates on how to do the task, without worrying too much
about the underlying theory. If there are multiple variations on how to do the task,
they concentrates on just the one way their master teaches them.
Ha: At this point the student begins to branch out. With the basic practices working
they now starts to learn the underlying principles and theory behind the technique.
They also starts learning from other masters and integrates that learning into their
practice
Ri: Now the student isn't learning from other people, but from their own practice.
They creates his own approaches and adapts what they learned to their own
particular circumstances.
SHU HA RI

21. Organisation
DevOps Adoption
Continuous Testing
Comms & Process
Monitoring & Ops
User-centric
Shu
Ha
Ri

22. Organisation
DevOps Adoption
Continuous Testing
Comms & Process
Monitoring & Ops
User-centric
1
2
3

23. User Centric
Approach

24. Continuous
Testing
Actually includes, but not limited to:
Automated testing
Automated management of production and
non production environments
Automated application feedback through
monitoring
Automated evaluation of business objectives
Evaluating product concepts
Testing
engaging in XP practices
organisational change
…not just about test automation! (ignore wikipedia)

25. The DevOps movement is built
around a group of people who
believe that the application of a
combination of appropriate
technology and attitude can
revolutionize the world of software
development and delivery.
Stephen Nelson-Smith - 2010
its about:
Automating building, testing, and
deployment
Bringing consistency to the way we
use and create technical assets
Providing teams with the tools
required to achieve technical
excellence
DevOps
aka - operability &
good communication

26. Team structures

27. The feedback cycle
Ideas/
Concept/
Reqs
Delivery
Pipeline
User
feedback
Monitoring

28. Build Agent & Devel
The delivery pipeline
Pre commit tests
Code review
PO/UX review
Branch / Merge
Build
Unit tests
Contract tests
Component tests
Coverage
Static analysis
Component
performance
End to end
component
tests
Exploratory Test
performance
testing
Story sign off /
Demos
NFRs
Analytics
Monitoring
Production tests
canary or blue /
green release
CI Environments
Developer
Local
Pre-Production Production
Virtualised Full stack environments
Commit Auto-Deploy Promote Release
Build Pipeline:
There can be multiple pipelines, usually per system, should execute in minutes
Commit:
The commit can include changes to the infrastructure, database, or application code.
Outputs:
Software, infrastructure and configuration
Environment manifests
Release notes

29. Engineering
Practices

30. automate the
build
Reduce general coding errors, create a single source of
truth, ensure basic component integration

31. vs.script your configuration!
say NO to hand crafted configuration files!

32. automate the
infrastructure
Remove configuration issues by having a repeatable
mechanism for creating environments.

33. vs.automate the testing
continuous testing does away
with the concept of
regression testing.
..an effective software delivery process
requires no regression testing stages. It
becomes integral to everything we do..

34. automate deployment
say NO to sneaker net
say NO manually to copying files
say NO to manually pulling and pushing servers in and out of load balancers
deploying software is one of the
most mission critical aspects of
what we do, let a machine do it!

35. Deployment strategies
that reduce quality risk,
and bring value, all with
zero downtime

36. Blue/Green
Deployment

38. A/B Testing &
Beta Releasing

39. Canary
Releasing

40. Sequential
Deployment
1 Remove Server #1 from Load balanced Cluster (Server #2 remains operational)
2 Update Server #1 with new code
3 Push Server #1 back into load balanced cluster
4 Repeat step 1,2,3 with server #2
Given a load balanced cluster having 2 web
servers, a sequential deploy will follow the
pattern below to provide zero downtime
SIMPLE!

41. monitor
One of the greatest and
overlooked tools in the arsenal of
the tester is monitoring.
Realtime access to
application logs
Realtime access to
error logs
Realtime access to
infrastructure logs

42. Monitoring Tools

43. “A good, easily accessible
event & error monitoring
system is essential for anyone
releasing software. Build an
early warning system into your
environment!”

44. “Understanding your
systems steady state allows
you to react quickly to any
deviation”

45. Create a
single view
of quality

46. smaller,
more
frequent
deliveries

47. using data to
focus quality
efforts

48. Customer driven data
Collected from:

49. NPS (Net Promoter Score)

50. Application Output
Response Times
Network Latency Query times
stack traces
Transaction times
Memory
consumption
CPU

51. Delivery Data

52. Cycle Time

53. Lead Time

54. Boomerangs

55. Applying
the holistic
approach

56. Measure

57. Tools - Planning/Priorities
Aggregate Marginal Gains
Pareto Principle
Low Hanging Fruit
Pain Point Priorities
10 Days Only

58. Tools - Doing
Top Down vs Bottom Up
Iterative Approach
Continuous Improvement

59. Tools - Inspect
Retrospectives
Improvement Groups
Radar Review
AS IS / TO BE

60. Wrap up

61. Thank
you!
TWITTER
@petemar5hall
Agile Consultant & CTO
Peter Marshall
fiveMAIL
info@leansoftwareservices.com
LINKEDIN
https://www.linkedin.com/in/
petedmarshall