1.
DevTestOps
Cultural Change Through
Quality Assurance
Module 1 - Talk the Talk

2.
Module Contents
What is DevOps?
What is DevTestOps?
Module Objectives
❏ Learn, comprehend and practice DevOps principles and
fundamentals
❏ Understanding of where testing plays a part
❏ Become a competent and courageous DevOps
Team member.

3.
Module Contents...
VM Basics
https://www.virtualbox.org/
https://www.centos.org/download/
Docker Basics
https://store.docker.com/editions/community/docker-ce-server-centos
Source Code Management
https://git-scm.com/

4.
What is DevOps?
The WIKI definition of DevOps
DevOps is a software engineering practice that
aims at unifying software development (Dev) and
software operation (Ops). The main characteristic
of the DevOps movement is to strongly advocate
automation and monitoring at all steps of
software construction, from integration, testing,
releasing to deployment and infrastructure
management. DevOps aims at shorter
development cycles, increased deployment
frequency, more dependable releases, in close
alignment with business objectives

5.
What is DevOps? Continue...
Examples of Practices?

6.
What is DevOps? Continue...
Examples of Practices?
- Law Practice
- Accounting Practice
- Physiotherapy Practice
- Agile Practice

7.
What is DevOps? Continue...
Examples of Practices?
- DevOps Practice

8.
What is DevOps? Continue...
The Wall of Confusion

9.
What is DevOps? Continue...

10.
What is DevOps? Continue...
6 Principles of DevOps
1. Customer Centric Action
➢ Meeting customer requirements through short
feedback loops
➢ Courage to act and innovate continuously
➢ Pivot When strategy is not working
➢ Invest in products and services for customer
delight

11.
What is DevOps? Continue...
2. Create With the End In Mind
➢ Let go of waterfall process-oriented models
➢ Act as product companies selling to real
customers
➢ All members require an engineering mindset
to envision and realise the product

12.
What is DevOps? Continue...
3. End to End Responsibility
➢ Vertically organised teams
➢ Team is responsible till end of life
➢ Full support including quality and performance

13.
What is DevOps? Continue...
4. Cross Functional
Autonomous Teams
➢ Fully responsible
➢ T-Shaped all rounders
➢ Hotbed for growth

14.
What is DevOps? Continue...
5. Continuous Improvement
➢ Fail Fast
➢ Minimise waste
➢ Optimise for speed costs and ease of delivery
➢ Experimentation is paramount
➢ If it hurts do it more

15.
What is DevOps? Continue...
6. Automate Everything You Can
➢ Adopt CI culture with high cycle rates and
instant feedback
➢ Includes infrastructure and container based
cloud platforms
➢ Synonymous with drive to renew delivery
methods

16.
What is DevOps? Continue...
Key Elements of DevOps Teams
❏ Team Building

17.
What is DevOps? Continue...
Key Elements of DevOps Teams
❏ Team Building
❏ CI and Problem Solving

18.
What is DevOps? Continue...
Key Elements of DevOps Teams
❏ Team Building
❏ CI and Problem Solving
❏ Leadership and
Feedback

19.
What is DevOps? Continue...
Key Elements of DevOps Teams
❏ Team Building
❏ CI and Problem Solving
❏ Leadership and Feedback
❏ Courage and Experimentation

20.
What is DevOps? Continue...
Traditional Organisation Model

21.
What is DevOps? Continue...
DevOps Organisation Model

22.
What is DevOps? Continue...
A Common Microservice example

23.
What is DevOps? Continue...
Solution Architecture for DevOps

24.
What is DevOps? Continue...
Teams Supporting a Monolithic Architecture

25.
What is DevOps? Continue...
DevOps Supporting a Microservice Architecture

26.
What is DevOps? Continue...
Stats from Puppet

27.
What is DevOps? Continue...
Summary
History and Definition
6 Principles of DevOps
Key Elements of DevOps Teams
Organisational Model
Solution Architecture

28.
What is DevOps? Continue...
Questions?
pmateos@planittesting.com
@paulconsults
https://www.linkedin.com/in/paul-mateos-213b2167/

29.
What is DevOps? Continue...
https://blogs.msdn.microsoft.com/uk_faculty_connection/2016/06/23/devops-the-wall-of-confusion-un
derstanding-the-basics-of-devops/
https://www.mulesoft.com/resources/api/microservices-devops-better-together
https://www.devopsagileskills.org/dasa-devops-principles/
http://leanmanufacturingtools.org/77/the-seven-wastes-7-mudas/
https://www.slideshare.net/SherryChang/evolving-team-structure-in-devops
https://www.slideshare.net/DevOpstastic/support-that-devops-change-agents-need
http://www.planitdevops.com
References

30.
What is DevOps? Continue...
https://purplegriffon.com/quizzes/devops-mini
Quiz TIme

31.
DevTestOps
Cultural Change Through
Quality Assurance
Module 1 - Talk the Talk

32.
What is DevTestOps?
DevOps with Quality Assurance as the
focul point
Bringing Planit’s Strengths in Agile,
Automation, Engineering and Quality
Assurance together
Implementation of Testing practices
throughout the DevOps End to End
process

33.
What is DevTestOps? Continue...
Risk Appetites - Severity / Occurrence

34.
What is DevTestOps? Continue...
Risk Appetites - Change of Mentality

35.
What is DevTestOps? Continue...
Strategic Objective
What does the organisation want versus what they need?
● Speed To Market
● Product Quality
● Or Stability of the Production Environment

36.
What is DevTestOps? Continue...
Paths To Communication - 5 Steps
1. Identify - Anyone who works outside of your team
2. Connect - Make connections with each individual
3. Invite - Widen the connection by bringing in new
people
4. Mark - Recording the communication for future
use
5. Widen - the path through activities that engage

37.
What is DevTestOps?
Testing pyramid
● The reality is Test Automation is
expensive and takes a long time to
implement
● Micro service testing is quick to execute
but still costly to develop
● Unit testing requires dedication, discipline
and a FAITH

38.
What is DevTestOps?
The Bug filter
● Bugs move between layers
● A good start but not complete

39.
What is DevTestOps? Continue...
The Bug filter - Unit Sorting
Smaller bugs are caught by smaller filters while larger bugs are caught by larger
filters

40.
What is DevTestOps? Continue...
The Bug filter - Unit Sorting
● Six layers to the DevOps bug filter.
● Top three are for testing that occurs in the development
environment
○ unit testing
○ integration testing
○ and end-to-end testing
● Bottom three are for the information that is captured in production
that can be used to detect bugs and determine product quality
○ Alerting
○ Monitoring
○ and logging

41.
What is DevTestOps? Continue...
Creating Tests At Each Filter
Unit Testing
● TDD (Test Driven Development) is a crucial foundation
● Unit test frameworks for specific code
● Mocking services to isolate integrations
● Powerful when implemented within a CI/CD pipeline and run
against integration environments
● Can be difficult to implement is some circumstances
● GUI interactions
● Databases
● Legacy code
● As DevOps engineers, this is not optional

42.
What is DevTestOps? Continue...
Creating Tests At Each Filter
Integration Testing
● Runs unit tests in a complex context (component)
● Bugs can be introduced from indirect interactions with other
services and applications. (System)
● Manual testing can provide great value
● Microservices is a must have architecture
● Knowledge of tools is a must for all testers
● SOUPUI
● POSTMAN
● Concept regarding NFR’s should be introduced here
● CI/CD tooling becomes essential to build, test, deploy and
then test again
● Product fitness rather than code quality

43.
What is DevTestOps? Continue...
Creating Tests At Each Filter
End-to-End Testing
● Is more efficient through DevOps principles
● Strongly supported by automation
● Scalable environments
● Coded change and deployments
● Coded tests and failure highlighting
● Planit champions this phase
● The fork in the road. Where DevOps can be introduced
● DevOps brings more exploratory and targeted testing ability
to satisfy customer feedback

44.
What is DevTestOps? Continue...
Creating Tests At Each Filter
Monitoring and Alerts Testing
● Issues should be detected by monitors
● Real or Mocked data can be used
● Controlled data easier to distinguish
● Rules based so standard testing practices apply
● Ensure alerts are sent and received successfully
● Ensure they are also fixable
● Ensure they detail the issue clearly

45.
What is DevTestOps? Continue...
Creating Tests At Each Filter
Log Testing
● Records everything that is happening
● At application, Server and Cloud
level
● Errors that generate no logs
highlights issues with logging
● Reveals user behavior and hidden
issues

46.
What is DevTestOps? Continue...
Testing in Production
● A/B Testing - Multiple versions released for
feedback
● Beta Testing - Released to a subset of users
● Feature Toggles - Change configuration in the
application
● Cannery Servers - Adjust deployment if too many
issues
● “TIP” is a hot topic on the interwebs

47.
What is DevTestOps? Continue...
Classical Costs Still Rings True

48.
What is DevTestOps? Continue...
Other Tests within DevOps
● Orchestration Pipeline Testing
● Code Quality Gates
● Infrastructure Configuration Testing
● Bug Bash
● Crowdsourced
● Infrastructure Resilience Testing - chaos
monkey
● Repeatable Performance and Security Tests

49.
What is DevTestOps? Continue...
Summary
Risk Appetite
Strategic Objective
Paths To Communication
Testing Pyramid
Bug Filter
Testing In Production and Other Tests in DevOps

50.
What is DevTestOps? Continue...
Questions?
pmateos@planittesting.com
@paulconsults
https://www.linkedin.com/in/paul-mateos-213b2167/

51.
What is DevTestOps? Continue...
A Practical Guide To Testing in DevOps - Katrina Clokie
http://www.belatrixsf.com/blog/an-introduction-to-testing-in-production/
http://infiniteundo.com/post/158179632683/abandoning-the-pyramid-of-testing-in-favor-of-a
https://www.slideshare.net/dhelper/benefit-from-unit-testing-in-the-real-world
https://test.io/software-testing-guide/devops-best-practices-integrating-qa-devops/
References

52.
VM and Docker Basics
● Setting up a centos VM
● Installing Docker
● Pull Jenkins Docker image and start it up

53.
VM and Docker Basics Continue...
● Oracle Virtualbox installed
○ Developer Tools
○ Networking Tools
● Create new RedHat linux vm
● 50GB HDD, 2048MB RAM
● Networking: NAT with port forwarding
○ Host Port: 8080 Guest Port: 8080

54.
VM and Docker Basics Continue...
● Docker CE installed
Difference between VM and Docker?
- VM’s are complete machines while
docker is a collection of libraries sharing
the host’s kernel
- There are now vSphere Containers

55.
VM and Docker Basics Continue...
● Create Docker Container with Jenkins
Master
● Create Docker Container with Jenkins
Slave
● Use Docker compose to start up both
services