With fingers wagging and eyes squinting, they query Why didn’t you find this problem during testing? How many times have you tried to defend yourself with things like We can’t test everything or It’s a corner case? Everyone knows you can’t improve quality with testing alone, so what can you do? Marianne Hollier shares practices and tools that help improve your test effectiveness and overall software quality. Learn how early collaboration across your whole team can remove bottlenecks and surprises. See how capturing and agreeing on interfaces between dependent systems can eliminate common issues that occur when systems are finally integrated for testing and nothing works. Understand how service virtualization and test automation go hand-in-hand to get your testing effort started earlier to achieve higher coverage more quickly. Join Marianne to learn how continuous integration and continuous deployment can get your test environments ready to test immediately after a new build is made—with no wasted time.

1.
W17
DevOps
&
Testing
10/5/16
15:00
A
DevOps
Primer:
Whole
Team
Approaches
for
Better
Software
Quality
Presented
by:
Marianne
Hollier
IBM
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Marianne
Hollier
Marianne
Hollier
is
an
IBM
and
Open
Group
Master
Certified
IT
specialist
in
application
development.
She
has
strong,
practical
expertise
in
measurably
improving
the
software
development
lifecycle
and
driving
the
necessary
cultural
changes
to
make
it
work.
Marianne
is
instrumental
in
architecting,
tailoring,
and
deploying
the
IBM
DevOps
best
practices
as
well
as
appropriate
software
development
tools
on
many
types
of
projectsÛÓfrom
large
to
small,
long
to
fast-­‐
track,
agile
to
traditional.
Marianne
is
passionate
about
all
things
testingÛÓprocess,
tools,
culture,
and
automation.
Her
experience
is
broad-­‐based,
spanning
both
custom
projects
and
standard
software
packages
that
apply
to
pharmaceutical,
refining,
telecommunications,
healthcare,
financial,
automotive,
government,
and
retail
industries.

3. 9/21/16
1
© 2016 IBM Corporation
Marianne Hollier
DevOps Continuous Testing Offering Manager
hollie22@us.ibm.com
A DevOps Primer:
Whole Team Approaches for
Better Software Quality
Agenda
• What is DevOps?
• Testing challenges
• DevOps pipeline
• Quality improvement scenario
• Key DevOps practices
• Get started
2

4. 9/21/16
2
© IBM Corporation 3
3
IBM
DevOps
Enterprise
capability for
continuous
software
delivery
3
Continuous
Customer
Feedback &
Optimization
Collaborative
Development
Continuous Release
and Deployment
Continuous
Monitoring
Continuous Business Planning
Continuous
Testing
Speeding
continuous
innovation
Enabling continuous
delivery of innovations
Providing feedback
for continuous
learning
Current approaches are not helping
Today’s testing challenges still remain
Development VelocityTesting Labor Test Lab Complexity
Outsourcing labor is no longer
a sustainable model as global
wages are increasing
Product and application
complexity are driving
test lab cost increases
Test teams can no longer
keep up with agile
development
Heterogeneous Environments
Public Cloud
Private Cloud
Data Warehouse Mainframe
Enterprise
Service Bus
Directory
Identity
File
systems
Collaboration
Web/
Internet
Routing
Service
Third-party
Services Portals
Content
Providers EJB
Shared
ServicesArchives
Business
Partners
Messaging
Services
Traditional
Testing
Other costs
Testing costs
4

5. 9/21/16
3
Modern testing needs to adapt to the DevOps world
Dev
Dev
Dev
Test
Test
Test
Dev QA
Dev QA
Dev QA
Before DevOps
manual testing
could keep up
In the DevOps
world manual
testing can’t
keep up!
Testing needs to
change with
automation and
collaboration
5
Benefits of test automation hampered by “big bang” integration
6

6. 9/21/16
4
Continuous testing can help achieve software quality earlier
7
• Test earlier
• Find architectural issues sooner
• Reduce costs
• Decrease risk
• Capture
requirements
• Create plans and
tasks
Plan
• Develop/ update
code
• Commit code
• Build the code
• Run unit tests
Code • Deploy to test
environments
• DEV, QA, UAT
• Deploy to
producIon
Deploy
• IntegraIon
• FuncIonal
• Performance
• Security
• Acceptance
Test • Monitor
performance
• OpImize
infrastructure
• NoIfy of issues
Monitor
DevOps Pipeline for building applications
Iterate
8

7. 9/21/16
5
Our Quality Improvement scenario
Plan for
applicaIon
test
Make a
build and
unit test
Automate
integraIon
tests
Deploy build
to test
environment
IdenIfy
high-risk
integraIons
Deploy to
QA
environment
Start stubs
and run
integraIon
tests
Deploy to
producIon
Quality
improvement
Automate
performance
tests
Run system
tests
Report on
quality
objecIves
Create
needed
stubs
Gather test
data
Manage
performance
and noIfy
on issues
Test
applicaIon
performance
Con*nuous Tes*ng
9
Key Roles
10
IdenIfies the appropriate
techniques and tools and
implements the automated
tests
Plans and executes integraIon
and system level tests
The key stakeholder that
prioriIzes requirements by
value
SomeImes referred to as
“scrum master” – drives the
overall development work
Captures non-funcIonal
requirements as well as
interfaces between applicaIon
components and middleware
components
One or more people doing the
actual coding and unit tesIng
of the new features
Works with OperaIons and
Development to automate the
creaIon and deployment of
test and producIon
environments
Monitors and manages the
producIon applicaIons and
environments

8. 9/21/16
6
Quality Improvement: Part 1
11
System Architect, Test Automation Specialist and Developer
12

9. 9/21/16
7
Test Automation Specialist and Infrastructure Engineer
13
Quality Improvement: Part 2
14

10. 9/21/16
8
Test Automation Specialist and Developer
15
Product Owner
16

11. 9/21/16
9
Key DevOps Practices
• Early Collaboration
– Include the whole team as early as possible to reduce surprises
• Continuous Integration
– Deliver code changes in small bites and test as early as possible
• Continuous Testing
– Leverage service virtualization and test automation to test earlier and faster
• Continuous Monitoring
– Incorporate application monitoring early in the lifecycle, not just in production
• Continuous Feedback
– Incorporate end-user interaction as early as possible, as well as feedback from
all team members
17
Get started with Continuous Testing
18
Identify your bottlenecks
• Test efficiency – are you able to run tests and provide feedback quickly?
• Test effectiveness – are you running the fewest number of tests that find
the largest number of problems?
• Bottlenecks may be upstream or downstream from test execution
Start with your biggest pain point
• Delays in test environment provisioning?
• Waiting on dependent systems to start testing?
• Unable run all the needed tests before the next build is available?
• Performance problems found in production?
• Significant problems when all systems are integrated?
• Misunderstood requirements – code and tests don’t align?

12. 9/21/16
10
Rational Test Workbench Starter Editions
Get up and running quickly, at no cost, with Rational Test Workbench.
Rational Test Workbench Web UI Tester Starter Edition
Create and run functional tests for modern web applications.
Rational Integration Tester Starter Edition
Record and test the APIs that your systems use every day.
Rational Performance Tester Starter Edition
Examine how your systems perform under load.
Get the Rational Test Workbench Starter Editions now on IBM developerWorks
19
IBM Resources on DevOps and Continuous Testing
20
Click here to
download the
eBooks!
IBM Continuous Testing
Community

13. 9/21/16
11
21
Questions?
© 2016 IBM Corporation
Thank You
© Copyright IBM Corporation 2016. All rights reserved. The information contained in these materials is
provided for informational purposes only, and is provided AS IS without warranty of any kind, express or
implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these
materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any
warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the
applicable license agreement governing the use of IBM software. References in these materials to IBM
products, programs, or services do not imply that they will be available in all countries in which IBM operates.
Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole
discretion based on market opportunities or other factors, and are not intended to be a commitment to future
product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, and other IBM
products and services are trademarks of the International Business Machines Corporation, in the United States,
other countries or both. Other company, product, or service names may be trademarks or service marks of
others.
www.ibm.com/devops