This document discusses how to fit performance testing into DevOps environments. It recommends adopting best practices like shifting performance testing left to earlier stages, conducting continuous performance validation as part of continuous delivery, automating what can be automated while accelerating other tests, and ensuring collaboration between performance engineers and DevOps teams. The presentation provides an example of how a performance testing tool can integrate into a DevOps toolchain at different stages like build, deploy, test and release. It emphasizes the importance of performance testing for software quality in fast-paced DevOps environments.

1. 2015
How To Fit Performance Testing In Devops
Adopt smart, thorough performance testing
techniques in time-constrained DevOps environments

2. Speakers
Kirk Tuxhorn
Senior Performance Engineer
Neotys
Sylvain Fambon
VP of Marketing
Neotys

3. Neotys: Continuous Performance Testing Solution

4. The context of performance testing in DevOps environments
Performance Testing in DevOps Best Practices
Demo – How to fit a Performance Testing Tool into a DevOps Toolchain
Agenda

5. The context of performance testing in
DevOps environments

6. Optimized software delivery is a competitive advantage
Software delivery is accelerating dramatically to sustain the competitive advantage
Organizations adopt Agile and DevOps as a solution
Software Delivery Pace is Accelerating Dramatically

7. High Performing DevOps Enables Better Software Quality
Source: 2016 State of DevOps Report

8. • Performance testing is time consuming and still highly manual
• Modern applications are complex: risk of failure is high
• Risk assessment is required
• Provisioning testing environments can be challenging
• More test = more data. How do you act on it?
But performance testing in DevOps is challenging

9. • Testing maturity is “a key differentiator”
Yet Performance Testing is Essential to DevOps
“DevOps – It’s all about continuous testing!”
by Marc Hornbeek, Principal Consultant - DevOps at Trace3
(Source: devops.com)
• Performance is a key determinant in the user
experience
• Performance bottlenecks impact
• Business and brand
• Employee’s productivity

10. Best Practices
Running Performance Testing in a Devops
Environment

11. 1. Shift left
2. Continuous performance validation for Continuous Delivery
3. What can be automated vs what can be accelerated
4. Collaboration to support the Devops culture
5. Performance Engineers become key players
Best Practices

12. How automated are your performance testing processes?
• Fully automated
• Mostly automated with Continuous Integration
• Mostly manual
Polling Question #1

13. Why Shift Left?
• Manages the frequency of releases and cost of testing maintenance
• Identify performance issues earlier
How?
• Test APIs, microservices & components
1) Shift Left: Test Early

14. 2) Continuous performance validation for Continuous Delivery
Early Risk Analysis
1
Understand the application
Evaluate the performance risk
Write performance SLAs in user stories
Automation
2 Start Testing Early Components
Faster more frequent performance testing
Validation
3 System Wide Testing
Test against your (business) SLAs
Monitor
4 Detect problems in production
Collaborate

15. 3) What can be automated vs what can be accelerated
Unit component
Component
Production
activity
Limit
Development
Environment
QA
Environment
Soak
Connection
Mobile
Performance
Or
User
experience
Failover Cloud
Preproduction
Environment
Production
Environnement
Robustness of the code Architecture stability End to end experience
Load in
production
Test can (should) be
automated
Test can be accelerated
Complexityofperformancetest
Stage in the SDLC
Single
User

16. Re-Using Functional Testing Assets Enables Acceleration

17. • Use of collaborative tools to distribute test results to the entire Devops team
• Enable real time monitoring during tests
• Customize test data views
4) Collaboration To Support The DevOps Culture

18. Performance engineering is a cultural transformation
Performance Engineers skill set needs to evolve
• Methodology proficiency
• New performance testing practices (More frequent component/API testing,
Monitoring, Feedback loop)
• Integration skills
• Automation skills
• Dev expertise
5 ) Performance Engineers: Supporters of The DevOps Team

19. Who manages performance testing within your Devops organisation?
• Performance engineers in a separate test team
• Decentralized performance engineer from the test team in the Devops team
• Each devops team levarages its own performance testing skills
Polling Question #2

20. Integrating Performance Testing in Your Devops
Tool Chain

21. receive task
integration
test
Typical Integration in the Devops Toolchain
Build
reques
t
Deploy Release
DE
V
QA
UA
T
PROD
Test
Prepare
Build
Compile
Packagin
g
Integration Functional Performance Security

22. Stage 1: Build
Build
reques
t
Prepare
Build
Compile
Packagin
g
New tasks is
created in Jira
Developer codes
Commit
code
CI server detects change
Prepare the new build
Deploy ReleaseTest
1
2
3
4
5
Build automation tool
Source control

23. Build
reques
t
Deploy ReleaseTest
Stage 2: Deploy
DE
V
QA
UA
T
Deployment scripts
Deployment tools
Chef
3 environments where build may
be deployed depending on
where you are in the
development process

24. Build
reques
t
Deploy ReleaseTest
Stage 3: Test
receive task
integration
test
DEV QA UAT
Integration Functional Performance Security
Test Pass or fail
Jira: collaboration

25. Continuous Integration
• Specific Plugin
• API or command line
Other Test Tools
• Reuse functional test assets
• Service Virtualization
Deployment Tools
• Test Infrastructure
• SLA Pass/Fail JUnit
Integration Strategy

26. Stage 4 : Release and Operate
PROD
Build
reques
t
Deploy ReleaseTest

27. Demo – How to fit a Performance Testing
Tool into a DevOps Toolchain

28. Next Steps
Contact Us
Visit Neotys blog www.neotys.com/blog/
Download NeoLoad Free Edition
go.neotys.com/NeoLoad-Free-Edition-Download.html
Questions?
www.neotys.com/solutions/devops-performance-testing-solution