The DevOps movement is here. Now, companies with mature systems are breaking down siloed IT departments and federating them into product development teams and departments. Testing and its practices are at the heart of these changes. Traditionally, development organizations have been filled with mostly manual testers and a limited number of automation and performance engineers. Adam Auerbach says this has to change. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add more value to the business. Based on his experiences at Capital One, Adam explores what the DevOps movement is all about, its core values, and proven patterns for how testing must evolve. In organizations implementing DevOps, testers and test teams—with the right skills and mindset—can help lead the transformation and not be left behind. You and your team can be the agents that help your organization and its culture change to reap the benefits of the DevOps revolution.
Testing and DevOps: Organizations and Their Culture Must Change
1.
K2
Keynote
10/4/17
10:00
Testing
and
DevOps:
Organizations
and
Their
Culture
Must
Change
Presented
by:
Adam
Auerbach
Lincoln
Financial
Group
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/
2. Adam
Auerbach
Adam
Auerbach
is
the
Vice
President
of
Quality
and
DevOps
Engineering
at
Lincoln
Financial
Group.
Adam
is
responsible
for
leading
the
implementation
of
Continuous
Testing
and
Continuous
Delivery
across
Lincoln.
Prior
to
joining
LFG,
Adam
was
the
senior
director
of
technology
for
advanced
testing
and
release
services
for
Capital
One
Financial
Corporation.
Adam
led
Capital
One—
’s
enterprise
performance
and
automated
testing
departments
as
well
as
enterprise
release
management
and
test
data
management.
While
at
Capital
One,
he
provided
leadership
for
the
agile
transformation
of
their
quality
assurance
group
and
led
the
enterprise
adoption
of
DevOps
and
acceptance
test6 driven
development.
Before
joining
Capital
One,
Adam
was
with
Chase
and
other
financial
and
insurance
companies,
in
various
leadership
positions
focused
on
quality
and
agile
practices.
3. 10/17/17
1
Tes$ng
and
DevOps:
Organiza-ons
and
their
culture
must
change
Adam
Auerbach
VP
Quality
and
DevOps
Engineering
Quality
and
DevOps
Engineering
@
Lincoln
Financial
2
Performance
Tes-ng
Service
Virtualiza$on
Automated
Tes-ng
DevOps
Test
Data
Quality
Engineering
Community
of
Prac-ce
@BugMan31
@
AdamAuerbach
4. 10/17/17
2
Tradi$onal
QA
Organiza$on
3
LOB
Team
LOB
Team
LOB
Team
Automa$on
Performance
Security
Agile
4
HARDENING
SPRINT
2
SPRINT
1
Test
Test
Design
Design
Build
Build
Defect
Fix
Test
5. 10/17/17
3
What
is
DevOps?
5
DevOps
is
a
philosophy
change
6
DELIVER
HIGH
QUALITY
WORKING
SOFTWARE
FASTER
DevOps
is
a
philosophy
where
teams
are
accountable
for
everything
required
to
get
their
code
developed,
tested
and
deployed
to
produc-on,
while
shared
service
teams
provide
the
automa-on
and
tools
to
enable
them.
“You
Build
it
You
Own
it”
Agile
Pod
Prod
Support
Arch.
Infrastructure
Shared
Services
(e.g.
Security
TesAng,
Perf
TesAng)
(Dev,
QA,
PO,
BSA)
6. 10/17/17
4
DevOps
is
the
next
part
of
your
Agile
journey
7
Number
of
Releases
SoJware
Quality
Agile
DevOps
Con$nuous
Delivery
Waterfall
We
are
here
Accelerates
soPware
development
with
itera-ve,
incremental
releases
Enables
faster
feedback
for
developers
thru
automa-on
and
increased
responsibili-es
Allows
code
to
flow
via
an
automated
soPware
factory
that
deploys
constantly
8
What
is
DevOps?
7. 10/17/17
5
9
Automate
Everything
10
Fast
Feedback
8. 10/17/17
6
11
Remove
Constraints
12
Collabora$on
early
and
oVen
10. 10/17/17
8
How
does
Tes$ng
have
to
change?
15
16
Pipelines
Pipelines
11. 10/17/17
9
Real-‐Time
Automa$on
17
ACCEPTANCE
TEST-‐DRIVEN
DEVELOPMENT
(ATDD)
• Team
Focused
• Stress
on
Acceptance
Criteria
• Add
a
Test
• Run
all
Tests
• Write
Code
• Refractor
Un-l
Added
Test
Passes
BEHAVIOR-‐DRIVEN
DEVELOPMENT
(BDD)
• Team
Focused
• Stress
on
Stories
• Add
a
Test
• Run
all
Tests
• Write
Code
• Refactor
Un-l
Added
Test
Passes
TEST-‐DRIVEN
DEVELOPMENT
(TDD)
• Developer
Focused
• Closer
to
Unit
Level
• Add
a
Test
• Run
all
Tests
• Write
Code
• Refactor
Un-l
Added
Test
Passes
Tes$ng
Approach
SoVware
Tes$ng
Ice-‐cream
Cone
An$-‐Pa^ern
watermelon.com
Automated
GUI
Tests
Integra-on
Tests
Unit
Tests
Automated
GUI
Tests
Automated
Unit
Tests
Manual
Session
Based
Tes-ng
Automated
API
Tests
Automated
Integra-on
Tests
Automated
Component
Tests
Manual
Tests
Ideal
SoVware
Tes$ng
Pyramid
watermelon.com
CURRENT
DESTINATION
13. 10/17/17
11
Test
Data
Management
21
EFFECTIVE
DATA
EFFICIENT
DATA
PRODUCTION
DATA
Building
the
right
data
Ge9ng
it
right
the
first
;me
Reduce
Security
Excep;on
Dependency
1
2
3
INTEGRATED
ENVIRONMENT
VIRTUALIZED
ENVIRONMENT
Service
Virtualiza$on
22
VS.
15. 10/17/17
13
How
do
you
start?
25
26
Iden$fy
areas
for
improvement
16. 10/17/17
14
27
Empower
people
to
drive
change
Next
Stop
Agile
Process
Training
Waterfall
Process
Test
Data
Management
Tools
Technical
Processes
28
Build
a
community
Demos
Lunch
&
Learns
Roadshows
Office
Hours
Internal
Conferences
17. 10/17/17
15
Common
challenges
29
Lack
of
Technical
resources
Lack
of
funding
Project
delivery
mindset
3rd
party
dependencies
Ques-ons
on
direc-on
Top
down
and
bo^om
up
support
is
cri$cal
30
Take
inventory
of
your
team
SME
DevOps
Waterfall
Process
Driven
Agile
Innova$ve
Business
Focus
YOUR
TEAM
Programming
18. 10/17/17
16
Iden$fy
your
future
leaders
31
PEOPLE
MANAGERS
TECHNOLOGISTS
Training
32
Use
mul$ple
levers
to
get
more
technical
Sourcing
New
Hires
Resul$ng
in
increased
technical
team
members
19. 10/17/17
17
33
Enterprise
groups
focus
on
enablement
Focus
on:
Technology
&
Support
Shippable
code
Focus
on:
TEAM
MEMBERS
ENTERPRISE
TEAMS
34
Embrace
DevOps
for
Tes$ng
AUTOMATION
Performance
Tes$ng
Test
Data
Agile
Team
20. 10/17/17
18
35
Pilot
like
crazy
Discovery
Pilot
36
Proven
metrics
21. 10/17/17
19
Des$na$on
37
Enterprise
Products
Teams
are
enabled
via
Enterprise
solu$ons
LOB
Team
LOB
Team
LOB
Team
Tes-ng
tools
and
frameworks
Performance
and
monitoring
tools
Test
Data
Management
Tool
Service
Virtualiza-on
Test
Environment
tooling
QUESTIONS
38