Simon White, Marks and Spencer Group DevOps Manager discusses the disconnect between traditional SQA & Agile approaches and how DevOps can be perceived as the ‘mature Agile’ model.

1. Agile, DevOps & Test

2. Background (me)
• 10 years process control & automation
• too many years Project & Programme management
• 10 years Release, Deployment, Implementation
• quick fact – don’t like ice cream

3. Synopsis
In no particular order, let’s look at
• the disconnect between traditional SQA & Agile
approaches
• how DevOps can be perceived as the ‘mature Agile’
model

4. Wikipedia says
“Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and
methods used to ensure quality. The methods by which this is accomplished are many and varied, and may
include ensuring conformance to one or more standards, such as ISO 9000 or a model such as CMMI.
SQA encompasses the entire software development process, which includes processes such as requirements
definition, software design, coding, source code control, code reviews, software configuration management,
testing, release management, and product integration. SQA is organized into goals, commitments, abilities,
activities, measurements, and verifications.”

5. And the agile
manifesto
says…
• customer satisfaction by early and continuous delivery of useful software
• welcome changing requirements, even late in development
• working software is delivered frequently (weeks rather than months)
• close, daily cooperation between business people and developers
• projects are built around motivated individuals, who should be trusted
• face-to-face conversation is the best form of communication (co-location)
• working software is the principal measure of progress
• sustainable development, able to maintain a constant pace
• continuous attention to technical excellence and good design
• simplicity—the art of maximizing the amount of work not done—is essential
• self-organising teams
• regular adaptation to changing circumstance

6. And the agile
manifesto
says…
• customer satisfaction by early and continuous delivery of useful software
• welcome changing requirements, even late in development
• working software is delivered frequently (weeks rather than months)
• close, daily cooperation between business people and developers
• projects are built around motivated individuals, who should be trusted
• face-to-face conversation is the best form of communication (co-location)
• working software is the principal measure of progress
• sustainable development, able to maintain a constant pace
• continuous attention to technical excellence and good design
• simplicity—the art of maximizing the amount of work not done—is essential
• self-organising teams
• regular adaptation to changing circumstance

7. And people think
DevOps is…..
of course…………..

8. Back to earth….
DevOps is about
• bridging the gap between software development and operations
• collaboration – we all want an easy life, right?
• creating capabilities – do it first then help others to do it
• lean delivery – optimum automation
• rapid feedback – are we doing what we said?
• continuous innovation – it’s never good enough
DevOps IS NOT just about technology
• DevOps is about how you use technology to improve process, change culture and drive organisational
change

9. DevOps
Enables……
DevOps enables
• agile ways of working (but works with Waterfall too!!)
• consistent delivery of code
• process visibility
• software configuration management
• lean governance
By applying
• clearly defined principles
• lean ways of working
• automation
• continuous innovation

10. DevOps
Principles
• everything in source control – have a defined branch/merge strategy
• everything as code
• everything is uniquely identified
• build binaries once
• control key environments
• just enough governance and process – delivered through automation
• Security, Test & Quality built in
• path to Production is defined and managed
• we care about process and principles as much as tools and technology
• toolset must be fully integrated
• measure everything
• feedback fast- fail fast, fail cheap!
• automate everything – minimise human intervention – CD is the end game!

11. Engineering
pre-requisites
• understand your stories before you start coding
• don’t bite off more than you can chew
• developers own the code
• design for automation – build, test, deploy
• have a defined branch/merge strategy
• no tests = no code –GIVE ME TEST AUTOMATION!!
• don’t break the build – execute unit & integration tests
• make comments meaningful – reference JIRA tasks
• build once, deploy often – only deploy from artefact store

12. Then we can
build….

13. And we all live
happily ever
after

14. A brief
history…..
• Formed in 2012 as part of MCFP
• .com Delivery Function supporting
• ~ 30 Product Teams
• 11 AT&T environments
• ~ 400 AWS instances
• 35 .com components
• Defined & Implemented Group DevOps model

15. What we’ve
achieved in .com
• DevOps toolset & automation provides the only mechanism to build & deliver code to controlled
environments
• >600,000 builds with security & quality scanning built in
• >1,400 automated build/deploy jobs
• >20,000 automated deployments
• >4,000 self-serve deployments
• 2 continuous delivery microservices – lost count of the number of deploys!

16. When we
started
• Security were sceptical

17. When we
started
• Security were sceptical
• Engineers accumulated piles of tech debt
• ‘no time to write unit tests…..’
• ‘we’ve got loads of manual testers paid for…………’
• ‘…..and build/deploy engineers’
• Config management – what config management?
• ‘I need to fix code in the environment…..’
• ‘…don’t have time to do test automation….’
• ‘…but I need to know that I’ve got defects immediately so I can fix ‘em’
• ‘…..and the release note tells you everything you need to know…’

18. Yup, really..
• Security were sceptical
• Engineers accumulated piles of tech debt
• ‘no time to write unit tests…..’
• ‘we’ve got loads of manual testers paid for…………’
• ‘…..and build/deploy engineers’
• Config management – what config management?
• ‘I need to fix code in the environment…..’
• ‘…don’t have time to do test automation….’
• ‘…but I need to know that I’ve got defects immediately so I can fix ‘em’
• ‘…..and the release note tells you everything you need to know…’
• Senior Management didn’t get it

19. The sell
Build a PoC – AWS is awesome for this
• Make Security your best friend – we hooked HP Fortify into the build process
• Automate the build – show engineers the reduction in effort
• Build in quality scans, run unit tests (??!!??) – rapid feedback
• Automate the deploy – bring the engineers along!
Senior Management don’t get it, so demonstrate
• significantly reduced deployment times
| Jun 2012 (Shopping, no automation) – 1.5hrs build, 14hrs deploy
| Mar 2013 (Shopping, some automation) – build 30 mins, 4hrs deploy with outage
They bought in!
| FYI - Mar 2015 (Shopping full automation) – 15 mins build, 2 hours deploy, zero downtime

20. Then scale…
• build to your principles – there will be battles, don’t compromise!
• work with the Engineering teams – use tangible benefits to change their mind
• raise need for test automation – need de-prioritised
• demonstrate lean governance through tooling
• answer the who, what, when, how, why questions
• demonstrate ‘build once, deploy often
• metrics identify bottlenecks – in our case, lack of test automation
• raise need for test automation – get a few nods but no action
• give the teams ‘self serve’ capabilities
• demonstrate Continuous Delivery PoC
• raise the need for test automation

21. And watch the
rush…
BDD
TDD
Embedded Testers
Devs in Test
What’s our test coverage?
What tools?
In honesty, we don’t mind. We just want the experts to give us robust automated test suites that we can hook into our
delivery pipe and distribute the results
Auto PT
Baseline

22. And watch the
rush…
BDD
TDD
Embedded TestersDevs in Test
What’s our test coverage?
What tools?
Auto PT
Baseline
You can tell which developer?
Corp Governance want to look at your SCM
process…..
Automated Change process…..?

23. Not
forgetting…

24. Not
forgetting…

25. Tools deliver
scm

26. Jenkins
CI tool
Zabbix
AWS monitoring
Sonar
Code analysis
Nexus
Artefact/binary
management
Ansible
Orchestration/
CM
Fortufy
Security scanning
SVN
Code Repo
GITLab
Code Repo
MongoDB
NoSQL DBMS
Apimon
API test &
monitoring
ElasticSearch
Log search tool
Kibana
Dashboard creation
FluentD
Log management
LDAP
Cloud UAM
Oracle
DBMS
CentOS
Linux distribution
Windows OS
Chef
CM tool
Ant
Build tool
Maven
Build tool
Docker
Container tool
Packer
Cloud gold image
creation
Vagrant
Virtual Dev Env
Creation
GITHub
Code repo
QTP
Test tool
HPSV
Service
virtualisation
Selenium
Web browser test
automation
Cucumber
BDD Test dev tool
Atom
IDE
SourceTree
GIT mgt tool
Eclipse
IDE
Gulp
FE build tool
NPM
FE build tool
Grunt
FE build tool
SauceLabs
Virtual browser test
Dashing.io
Dashboard creation
Cloudability
AWS utilisation
viewer
JIRA
Task/bug tracking
Confluence
Wiki
Fisheye
Code viewer
JIRA capture
Capture
screenshots
JIRA Agile
Agile boards plug-in
JIRA Enhancer
JIRA additional
features
JIRA Agile Cards
Ticket printing
Intelligent Reports
JIRA reporting
Gliffy
Drawing plug-in
Crucible
Peer code review
JIRA Suite
Additional features
JIRA Zephyr
Test plug-in
AWS SES
Email send service
AWS IAM
Identity access
AWS
Cloudformation
Infra deploy tool
AWS Route 53
DNS mgt
AWS Cloudwatch
Monitoring
AWS EC2
VM set-up service
AWS Cloudfront
CDN service
AWS S3
Elastic storage
AWS VPC
Virtual private
cloud tool
AWS RDS
Managed RDBMS
AWS tool
£££’s
Open Source
Key
WE LIKE TOOLS REALLY…

27. BTW We built
this….

28. Summary
DevOps can be considered ‘Mature Agile’
• we care about the full SDLC (cradle to grave)
• we want to deliver light touch governance through our tooling – actually we
like the challenge
• we want to drive best practice through automation
• get a kick out of demonstrating the ‘art of the possible’
And we really do get the need for maximum automated testing available as early in
the lifecycle as possible
• And we shout about it. Loud.

29. Thank you and
goodnight!

30. www.QualiTestGroup.com
Thank you!