The DevOps movement is here. Companies across many industries are breaking down siloed IT departments and federating them into product development teams. Testing and its practices are at the heart of these changes. Traditionally, IT organizations have been staffed with mostly manual testers and a limited number of automation and performance engineers. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add greater value to the business. DevOps really starts with testing. Join Adam Auerbach as he explains what DevOps is and how it relates to testing. He describes how testing must change from top to bottom and how to access your own environment to identify improvement opportunities. Adam dives into practices like service virtualization, test data management, and continuous testing so you can understand where you are now and identify steps needed to instill a DevOps testing culture in your team and organization.

1. MH
Half-day Tutorials
Monday, April 30th, 2018
8:30 AM
Instill a DevOps Testing Culture in Your
Team and Organization
Presented by:
Adam Auerbach
Lincoln Financial Group
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888---268---8770 ·· 904---278---0524 - info@techwell.com - http://www.stareast.techwell.com/

2. Adam Auerbach
Lincoln Financial Group
Vice president of quality and DevOps engineering at Lincoln Financial Group,
Adam Auerbach is responsible for leading the implementation of continuous
testing and continuous delivery across the organization. Previously Adam was the
senior director of technology for advanced testing and release services at Capital
One Financial Corporation. Adam led Capital One’s enterprise performance and
automated testing departments as well as enterprise release management and
test data management. He provided leadership for the agile transformation of
Capital One’s quality assurance group and led the enterprise adoption of DevOps
and acceptance test-driven development.

3. 4/9/18
1
Testing and DevOps
Building a Modern DevOps Enterprise Testing Organization
Adam Auerbach
VP Quality and DevOps Engineering
Quality and DevOps Engineering @ Lincoln Financial
2
Performance
Testing
Service
Virtualization
Automated
Testing
DevOps
Test Data
Quality
Engineering
Community of
Practice
@ AdamAuerbach@BugMan31

4. 4/9/18
2
3
Agenda
• DevOps and Continuous Testing Overview (60 minutes)
• Exercise 1 – (30 minutes)
• Exercise 2 – (30 minutes)
• Break – (10 minutes)
• Readout and collaboration – (50 minutes)
DevOps Overview

5. 4/9/18
3
5
Traditional QA Organization
LOB
Team
LOB
Team
LOB
Team
Automation
Performance
Security
6
Agile
HARDENINGSPRINT 2SPRINT 1

6. 4/9/18
4
7
What is DevOps?
DevOps is a Philosophy Change
8
DELIVER HIGH QUALITY WORKING SOFTWARE FASTER
DevOps is a philosophy where teams
are accountable for everything
required to get their code
developed, tested and deployed to
production, while shared service
teams provide the automation and
tools to enable them.
“You Build it You Own it”
Agile Pod Prod Support
Arch.
Infrastructure
Shared Services (e.g. Security
Testing, Perf Testing)
(Dev, QA, PO, BSA)
+

7. 4/9/18
5
DevOps is the Next Part of Your Agile Journey
9
Number of Releases
SoftwareQuality
Agile
DevOps
Continuous Delivery
Waterfall
Accelerates software
development with
iterative, incremental
releases
Enables faster feedback
for developers thru
automation and
increased responsibilities
Allows code to flow via an
automated software factory
that deploys constantly
10
What is DevOps?

8. 4/9/18
6
11
Automate Everything
12
Fast Feedback

9. 4/9/18
7
13
Remove Constraints
14
Collaboration Early and Often

10. 4/9/18
8
15
Focus on Flow
16
Agile + DevOps
DevOps
Continuous
Integration
Continuous
Testing
Continuous
Monitoring
Continuous
Delivery
Agile

11. 4/9/18
9
What is Continuous
Testing?
18
Pipelines
PIPELINES

12. 4/9/18
10
19
Real-Time Automation
• Team Focused
• Stress on Stories
• Add a Test
• Run all Tests
• Write Code
• Refactor Until Added Test Passes
BEHAVIOR-DRIVEN DEVELOPMENT (BDD)
TEST-DRIVEN DEVELOPMENT (TDD)
• Developer Focused
• Closer to Unit Level
• Add a Test
• Run all Tests
• Write Code
• Refactor Until Added Test Passes
ACCEPTANCE TEST-DRIVEN DEVELOPMENT (ATDD)
• Team Focused
• Stress on Acceptance Criteria
• Add a Test
• Run all Tests
• Write Code
• Refractor Until Added Test Passes
Testing Approach
Software Testing Ice-cream
Cone Anti-Pattern
watermelon.com
Automated GUI Tests
Integration Tests
Unit
Tests
Automated
GUI Tests
Automated Unit Tests
Manual Session
Based Testing
Automated API Tests
Automated Integration Tests
Automated Component Tests
Manual Tests
Ideal Software
Testing Pyramid
watermelon.com
CURRENT DESTINATION

13. 4/9/18
11
Open Source Tools
22
Environments

14. 4/9/18
12
23
Test Data Management
EFFECTIVE DATA EFFICIENT DATA
PRODUCTION DATA
Building the right data Getting it right the first time
Reduce Security Exception Dependency
1 2
3
INTEGRATED ENVIRONMENT VIRTUALIZED ENVIRONMENT
24
Service Virtualization
VS.

15. 4/9/18
13
25
Dashboards
26
Dashboard Cont’d

16. 4/9/18
14
Class Excersises
28
Exercise #1 - Identify manual tasks and lost time
• Break into 2 groups
• Group 1 –
• Create a list of all tasks performed in your QA departments that is
manual.
• Prioritize the list based on time and frequency of the task
• Group 2 –
• Create a list of all things QA has to wait to be performed in order to
do their work.
• Prioritize the list based on time and frequency of the wait.
• 20 minutes for the teams and 10 minutes for the readouts.

17. 4/9/18
15
29
Exercise #2 – Identify solutions
• Break into 2 groups
• Group 1 –
• For the top 5 items
• Identify how you would automate the task, if possible.
• Identify the challenges
• Group 2 –
• For the top 5 items
• Identify how you would remove the wait time
• Identify the challenges.
• 30 minutes for the groups
30
Exercise #3 – Class readout and discussion
• Each group will provide a readout of their work and we will discuss their
solutions as a class.
• 25 minutes for each group.

18. 4/9/18
16
Thank You
Testing and DevOps
Building a Modern DevOps Enterprise Testing Organization
Adam Auerbach
VP Quality and DevOps Engineering