Continuous Testing- A Key Ingredient for Success in Agile & DevOps

Continuous Testing – A key
Ingredient for Success in Agile
& DevOps
By
ChakravarthyS Devarakonda
Asst. VicePresident

Agenda
Unsolicited Distribution is Restricted. Copyright © 2017 - 18, Cigniti Technologies
• AnalystSpeak
• DevOpsinanutshell
• DevOpsvsDevTestOps
• ContinuousTesting&DeliveryleveragingQE
• UnitTesting&SAST
• Automation,Performance&Security
• ServiceVirtualization
• ContinuousIntegration,Deployments&Delivery
• ContinuousAutomation&DeliveryPipeline
• CollaborativePlatform&Howdoesithelp?

ANALYST SPEAK
1

Continuous Testing is important in DevOps
Test is not a step. In a DevOps pipeline, it is a continuous discipline requiring
the need to visualize and assess the test from a plan, through production
and back.
Agile and DevOps practices are disrupting old ways of testing software and
applications to keep up with the increasing demand for quality at speed. Testing
comes earlier in the cycle (shifting left) and is becoming faster, better, and
smarter.

Continuous Testing is important in DevOps
DevOps toolchains are often built from
discrete and sometimes disconnected
tools, making it difficult to understand
where bottlenecks are in the
application delivery pipeline
DevOps initiatives continue to grow
in complexity, driving many IT
organizations to seek tools that
provide greater control and visibility
over the entire DevOps process
Toolchain orchestration promises to simplify the
DevOps toolchain; however, one tool is not able to
support the entire toolchain resulting in additional
integration, ownership and handoff challenges
when using multiple orchestration tools
Orchestrating the DevOps toolchain
can be compelling, but it can introduce
another set of challenges
01
02 03
04

DevOps adoption Is no longer Just For unicorns
Source: Forrester’s Q1 2017 Global DevOps Benchmark Online Survey Source: Forrester Data Global Business Technographics Developer Survey, 2016 &
Forrester Data Global Business Technographics Infrastructure Survey, 2016
Culture and
people come First;
process follows
Release velocity is pivotal There’s no “easy” button

DevOps Adoption is Growing

DevOps Tool Adoption is Growing as well

Adoption and Strategies vary acrossVertical Segments
$
Business Services And construction
Ponder The Leap Into Automation
Media, entertainment, &
Leisure embark On The
Journey To Differentiate
With DevOps
Public Sector And Healthcare cling To Manual Audit
And compliance Procedures
Retail And Wholesale DevOps Adoption Drive
Differentiated Interactions With clients
Compliance-wary companies need not fear automation
Utilities And Telecommunications
Lead The Pack To Drive Business
Transformation
Data must drive strategy, with services
that emphasize connectivityFinancial Services Accelerate To compete
With Fintechs
DevOps offers key benefits in fraud detection, time-to-market,
and scalability
Get ready for the next generation of digital stores
DevOps drives identi cation
to execution to improve
customer experience
Manufacturing Transitions
Beyond Automation To
Show DevOps Momentum
Manufacturers improve
efficiency and insight by
expanding the role of
technology
Technology is
becoming part of the
business

DevTestOps & Continuous Testing is key for DevOps success
DevOps
End to End Automation
Overall Cost Reduction – Automation,
Reduce Cost of Quality
Fail – First, Fail - Fast, Fail- Often – Early
Defects Detection
Collaborative Communication utilizing
Dashboard
Continuous Testing
DevTestOps
Equal Importance for Dev, Operations
and Testing
All actions – Development --> Operations
Emphasis on Automation & Analytics
Continuous Integration + Continuous
Deployment (CI/CD)
Dev --> Ops Process Reduced from
Months to Days/Hours
Continuous Feedback & Improvement
Supported
Maturing Processes and Organizations
Focus

Devops in a nutshell
Build Dev
Automated Unit
Testing & SAST
Build Packing &
Automated
Deployments
Continuous Testing
• Functional Automation
• Non Functional Automation
Production
Deployments
Post Production
Monitoring & Feedback
Amplification
Virtual Server(s)
Physical
server(s)
CI
Server
Hosted Tools
Infrastructure
Automation &
operations

Continuous Testing & Delivery Leveraging QE elements
Client Layer Web Server App Server Component 1 Component 2 Component 3
Target System
Target System
Service
Virtualization
Data
Engineering Dashboard
Application Under Test
CI
Platform
UI
Services
App Component
Continuous Monitoring
Health Check
InfrastructureApplication
{Through scripts} {Through metrics configuration}
Functional Automation
Continuous Testing
Non - Functional Automation
Leveraging a collaborative
dashboard to monitor
application release
activities & health
Enabling continuous testing
by overcoming the
dependency factors involved
Centralised data management through
automated TDM tools & other mechanisms
for on demand testing
Ensuring faster feedback for
issues remediation through
continuous infrastructure &
application monitoring in
production & test environments
Adopting Quality Engineering
approach to provide end to end
automated functional and non
– functional test coverage for
each release
Ensuring quality during rapid and frequent build
deployments by leveraging robust CI tools
integrated with test automation frameworks
Third party system
Efficient Tools
Leverage

DevOps Journey
DevOps Journey Automation Maturity
Partial Agile
No Automation
in place
Functional and Non
Functional
Automation
Adopted CI and
CD tools
Implemented Dual Shift
Approach
Build - Deployment –
Release Automation
Matured Agile
Organization
Transformed to TDD
/ ATDD / BDD
DevOps
Matured Collaborative
Agile ( Dev, QA and Ops)
Cost Elements ROI Factors
» Tools for Automation and
Non Functional Tests
» Process Standardization
» On Demand Test Envts.
» End-to-End Automation
and Test Coverage
» Tools and process for CI
and CD
» Infra Maintenance
» DevOps Process
» Org. Change Mgmt.
(People, Process and Systems)
» Cost of Maintenance
» Technology Adoption
» Unified Automation
Framework
» Best suited tools
» Tool optimization and
consolidation
» Quality Check gates
» Automation Stabilization
» Continuous Testing –
Continuous Feedback
Quality Engineering
approach for early testing
(Functional and Non Functional)
» Build Reliability
» Uninterrupted Service
Delivery

In Detail
KEY QUALITY
ENGINEERING ELEMENTS

Unit Testing & SAST
JUNIT/NUNIT
Static Code Analysis
Build Server2
2
CI Server
Unit Testing
1
Open Source/Commercial Tools
3
Results Analysis &
Vulnerable
identification
Manual Analysis
for false positive
removal
Final Summary
Code Commit
1
I D E
(Technology Agnostic)
Code Repository
» Unit testing is performed using respective open source tools that
integrated with the IDE used for code development
» CI Server picks the build from the code repository/version controlling
tool once the developer commits the build
» CI Server initiates the static code analysis by triggering a request to
the configured open source / commercial tools
» Input Validation
» Output Encoding
» Secure handling of credentials
» Session Management
» Cryptographic practices
» Error handling / Logging
» Data Protection
» Communication Security
» Memory management
Secure Coding Guidelines & Standards

Automation,Performance & Security
CI PlatformBuild
Version
Controlling Tool
Test Automation Performance Testing
Security Testing (DAST)
Test Automation
Tools
Test Management
ToolsTest Automation
Framework
Performance Testing
Tools
Performance Testing
Scenarios
Monitoring
Tools
Trigger Monitoring High Level
Analysis Report
Trigger Automated
Test Execution
DAST Tools
Monitoring
Tools
Results Analysis &
Vulnerabilities
Identification
Manual Analysis
(False Positives)
Automated
Trigger
Final
Summary
Report

TDD blended with DevOps
Function 1- Unit
Test case
Function 2- Unit
Test case
Function 3 - Unit
Test case
Feature broken
into functions
Add a Developer
test
Run Developer
tests at the unit level
Make code changesRun the
Developer testsPass,
Functionality completeDevelopment Ends
Fail
Fail
Pass
Pass,
Functionality
Incomplete
FinalizedFeature
Unit Testing, Integration
Tests and Performance
Tests in reference to
development under
DevOps practices.
DevOps Specific
input to feature
requirements
Feature Spec
file
Automated Configuration
Management frameworks
(Ansible/Chef/Puppet)
CI/CD framework
(Continuous Integration/
Continuous Delivery)
TDD
DevOps
Continuous
Feedback
DevOps Analytics Platform
Continuous
Deployment

ServiceVirtualization
T
Legacy
ERP
Databases
Mainframe
» Automatically Capture realistic
behavior
» Build VS Image, collection
services requests and Service
responses
» Optimize models as desired
Create Virtual
Services
Playback against Virtual Service
Test
(Manual, Automated)
System Under
Test (SUT)
Virtual Service
SOAP Request
SOAP Response
JMS Publish
Exercise SUT
ServiceRequest
ServiceResponse
Service Response
Service Request
Service Response
Service Request
ServiceResponse
ServiceRequest
DevTest
Deploy SV tool
on cloud
Advantages
✓ Accessible from Multiple Geographies (Cloud & On-premise)
✓ Deploy & Bring Up/down to Optimize Resources & Cloud Spend
✓ Easily Customized as the UI or Business Needs Change
✓ Leverage Docker Containers, enabling Continuous Integration of
Code Changes and Parallel Test Instances
✓ Reduce CapEx (Capital Expenditure) by Hosting in AWS (Amazon
Web Services) Cloud
✓ Dynamically Support Infrastructure & Policy as Code, with Fugue…
✓ Configurations
✓ Configurations
✓ Configurations
✓ Configurations
Target Service
DevTest
Typical DevTest SV Usage Scenario

Continuous- Integration vs.Delivery vs.Deployment
C o n t i n u o u s - I n t e g r a t i o n v s . D e l i v e r y v s . D e p l o y m e n t
Source Control
Commit Changes
Build
Run Build And Unit Tests
Staging
Deploy to test Environment Run Integration
Tests, Load Test, & Other Tests
Production
Deploy to Production Environment
V1.1
Automated Automated
Continuous
Deployment
Continuous
Delivery
Automatic Deploy
Approve Deploy
Continuous
Integration

Continuous Automation & Delivery Pipeline
ORCESTRAED CONTINIOUS INTEGRATION PIPE LINE
Test
SCM Repo Adoption Build Package Deploy Host / Stage
Jenkins
Requirements Document & verify
Container as a SCM service using Rancher/Docker
Environment Environment
Resources Resources
Private Container Service
User Directory Container Registry
Ops ToolsDevOps
Pipeline
Enterprise Micro
service Stack
Reporting &
Monitoring
On demand test bed using
containers
Test Automation
Tools
Web & Mobile
browsers
Advance
Analytics
Dashboards
Tableau
DeliveryPipelineContinuous
Automation
Requirements
Automated
Testing
Test Automation
Scripts
CI Enabled
Execution
Results
Automated Performance
DAST
Regression & Risk Based
Continuous Development
Coding
Compile & Build
Code
Quality
SAST
Unit testing
Continuous Deployment
Deploy Apps
Release
Management
On Demand Infra
Collaborative
Dashboard
Continuous
Monitoring
Operations
Enablement

Collaborative DevTest Platform & How does it help?
Build OperateIntegrate Deploy
Application Health Release Readiness Quality Metrics Go – No Go Decision Enabler
Release Progression Build Status Test Status Predict Analysis
LiveDashboard
Test Automation
Tools
Test Management
Tools
Performance
Testing Tools
DAST Tools
Monitoring
Tools

Q & A
Thank You
Connect at:
chakri@cigniti.com
+1 469 955 0841

Continuous Testing- A Key Ingredient for Success in Agile & DevOps

More Related Content

What's hot

Similar to Continuous Testing- A Key Ingredient for Success in Agile & DevOps

More from SmartBear

Recently uploaded

Continuous Testing- A Key Ingredient for Success in Agile & DevOps