Shifting left: Continuous testing for better app quality and security

Shifting left:
Continuous testing for better
app quality & security

© Copyright 2016 NowSecure, Inc. All Rights Reserved. Proprietary information.
Connect
Twitter: @NowSecureMobile | @GuerrillaQA
—
Subscribe to #MobSec5, our weekly mobile security news digest
http://mobsec5.nowsecure.com/
—
Web: nowsecure.com | guerrillaqa.com

Steven Winter
Founder & Chief Strategist, GuerrillaQA
Andrew Hoog
CEO & Co-founder, NowSecure

Contents
● Why deploy more quickly?
● Going fast, achieving quality, & saving money
● What now? Must do’s!
● Continuous testing in practice
● Q & A

Why deploy mobile
apps more quickly?

The business value of more frequent deployments
Happier customers
New features / improvements increase
customer satisfaction & lead to faster
realization of revenue from new features.
Fix defects faster
Identifying flaws earlier & shortening
the feedback loop leads to less
expensive, faster fixes.
Reduce risk
Smaller deployments include fewer
things that can go wrong, & those
failures are easier to fix

Pain experienced as a result of infrequent releases
Dissatisfied customers
App users churn due to their perception
that the developer is not responsive with
improvements & new features.
Slower reaction time
Improvements & fixes take longer to
be released, are more expensive, &
leave customers dissatisfied longer.
High-risk, complex deploys
Monolithic releases include more
dependencies & potential failures resulting
in more expensive & time-consuming fixes.

What does the ideal look like?
Company Deploy Frequency Deploy Lead Time
Amazon 23,000 / day minutes
Google 5,500 / day minutes
Netflix 500 / day minutes
Facebook 1 / day hours
Twitter 3 / week hours
Typical enterprise Once every 9 months Months or quarters
Kim, Gene. "Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win" 2014.

How frequently are others deploying?
https://blog.newrelic.com/2016/02/04/data-culture-survey-results-faster-deployment/

Where are you in your journey? First steps
Automate what
testing you can
Take advantage of
Continuous Integration
Shift security &
performance testing left

You can go fast, achieve
quality, & save money

Earlier testing & remediation prevents technical debt
Requirements /
Architecture
Coding Integration /
Component
Testing
System /
Acceptance
Testing
Production /
Post-Release
Source: National Institute of Standards & Technology
The cost for fixing
vulnerabilities is
30xhigher after an app
has been deployed

The cost of fixing a P1 mobile bug in production
Case Study
Team Hours
Detection & communication 20
Verification 16
Fix 40
Build, test, certify the fix 60
Customer acceptance 40
Post-publish verification 20
Total hours 196
As well as
● Loss of client & app user confidence
● Negative app ratings
● Derailment of feature development & release
$35K
In total costs

Automation pays for itself with repeatability
Manual Testing
Automated Testing
Releases
Time /
Effort
Time Savings

Development / Integration Staging Production
Dev Team
Version
Control
Build & Unit
Tests
Automated
Acceptance
Tests
Release
User
Acceptance
Tests
Check-in
Check-in
Check-in
Trigger
Trigger
Trigger
Trigger
Trigger Approval
Approval
Feedback
Feedback
Feedback
Feedback
Feedback
Feedback
Engineer QA DevOps

Development / Integration Staging Production
Dev Team
Version
Control
Build & Unit
Tests
Automated
Acceptance
Tests
Release
User
Acceptance
Tests
Check-in
Check-in
Check-in
Trigger
Trigger
Trigger
Trigger
Trigger Approval
Approval
Feedback
Feedback
Feedback
Feedback
Feedback
Feedback
Engineer QA DevOps
Shift security & performance
Testing to the left

CI + CT = !!
Continuous Integration
+
Continuous Testing
=
Productivity multiplier
● Sets the stage for “set-it-and-forget-it” deployment
● Deliver higher quality code at lower risk in less time
● “Parallelizes” testing
○ Security, regression, performance, etc.
○ Simultaneously
● Repurpose test scripts
○ Write once
○ Use everywhere

What now?
Must do’s!

1. Agree & commit to improving
Must do’s!

Must do’s!
2. Plan testing & automation scripting up front

Must do’s!
3. Agree on test coverage

Must do’s!
3. Agree on Test Coverage
4. Measure, measure, measure

Must do’s!
3. Agree on Test Coverage
4. Measure, measure, measure
5. Plan for test script maintenance

Continuous testing in practice

Case study: Value realized in just a few hours

Steven’s experience at scale
From 4 months to Nightly

Scaling your automated testing based on maturity
Small Medium Enterprise
● Leverage open-source tools
● Build CI environment
● Create a basic smoke test
● Expand test coverage
● Leverage cloud platform services
● Plug security & performance
testing into CI
● Create smoke tests for each
feature (not the entire app)
● Prioritize by feature’s
success / risk
● Pick the top three & go!

Let’s talk
NowSecure
+1 312.878.1100
@NowSecureMobile
www.nowsecure.com
GuerrillaQA
+1 415.763.TEST
@GuerrillaQA
www.guerrillaqa.com
Subscribe to #MobSec5 - a collection of the week’s mobile
news that matters - http://mobsec5.nowsecure.com/

Shifting left: Continuous testing for better app quality and security

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Shifting left: Continuous testing for better app quality and security

Similar to Shifting left: Continuous testing for better app quality and security (20)

More from NowSecure

More from NowSecure (17)

Recently uploaded

Recently uploaded (20)

Shifting left: Continuous testing for better app quality and security