Deliver Flawless Mobile Apps Faster with CI/CD & CT

Perfecto by Perforce © 2020 Perforce Software, Inc. and NowSecure, Inc
Deliver Flawless Mobile Apps Faster With CI/CD & CT

2 | Advanced Codeless Testing for Web Apps
Today’s Speakers:
Eran Kinsbruner
• Chief Evangelist and author at Perfecto
• Blogger and speaker
• 19+ years in development & testing
• Author of “The Digital Quality Handbook”
and “Continuous Testing for DevOps
Professionals”
• @ek121268
Brian Reed • Chief Mobility Officer at NowSecure
• Leading the Mobile DevSecOps Charge at NowSecure
• Helping Fortune 2000 and Gov agencies deliver high quality and
secure mobile apps faster
• Advisor, Speaker, & Writer
• @reed_on_the_run

perfecto.io & NowSecure.com3 | Perfecto by Perforce © 2020 Perforce Software, Inc. and NowSecure, Inc
Today’s Agenda
1
2
The CI/CT/CD trifecta & DevOps
How to fit automated security and functional testing inside the DevOps process
https://www.perfecto.io/resources/state-test-automation
3 Common pitfalls in mobile app security and how to overcome them
5 Q&A
4 Fundamentals of continuous testing (CT) strategy for CI/CD/CT pipelines

Mobile Apps Drive Global Economy

The Mobile App Conundrum
100%
of mobile devs want to
build great apps.
100%
of mobile companies
want happy customers.
85%
of mobile apps
have security bugs.
70%
of mobile apps leak
personal data and
violate GDPR/CCPA.
9%
of organizations
automate over 75% of
their test cases.
14%
of organizations can
release software daily.

OPTIMIZED
DELIVERY PIPELINE Innovation
Throughput
Quality of
Output
Time Cost
Today — DevOps Process Probably Looks Like This
Process
Impact
Organizational
Impact
2-3 WEEKS 1-3 WEEKS
❌ Unstable
❌ Labor-intensive
❌ Cluttered
❌ Slow
• Manual testing.
• Unreliable and flaky
executions.
• Long time to analyze results
and fix issues.
• Slows time to release.
• Increases risk and reduces
flexibility during the cycle.
• Reduces innovation time
versus bug fixes time.
• QA and security often not
part of the daily cycle.
• Testing holds back
innovation.
”End of
Cycle”
Testing

Continuous Integration — Ability to merge all developer
code and automatically build apps throughout the day.
Introduction to CI/CD
Dev Functional
Test
Acceptance
Test
Security
Test
Deploy
Continuous Deployment — Ability to automatically
deploy new app functionality throughout the day.
Build
Continuous Integration Continuous Delivery

What Is Continuous Testing?
Continuous testing is the process of executing automated
high-value tests as a part of the software delivery pipeline
in order to obtain feedback on business risks associated
with a software release upon every code change.

CT Enables Efficient CI/CD
Source: Dan Ashby

Source: DORA Report
Key Benefits of a Mature DevOps Program

Leverage Test Automation to Optimize the Pipeline
vs
OPTIMIZED
PIPELINE Innovation
Throughout
Quality of
Output
Time Cost

Contributing Factors: Common Challenges in Test Automation
Automation Skillset
Merging Tests Into Pipeline
Escaped Defects Due to Noise
Advanced Automation Scenarios
Challenge to Setup Test Environment Designed for
Testability
Time Spent to Analyze Reports
Testing Is Done Separately
Test Maintenance & Digital Platforms Coverage Lack of Time to Automate
It all boils down to people, processes, and/or technology.

How to Fit Automated Functional and Security Testing
into Mobile DevOps Pipeline

T H E D E V O P S M A N I F E S T O
 Continuous testing over testing at the end.
 Embracing all testing activities over only automated functional testing.
 Testing what gives value over testing everything.
 Testing across the team over testing in siloed testing departments.
 Product coverage over code coverage.
The Agile Testing Manifesto

Working Together
D E V Q A S E C

The Software Delivery Lifecycle
Commit
Code
Build
Binary
Deploy
Staging
Test Binary
Requirements & Design
Common Goals
• Build high-quality software.
• Bring together security, QA,
& dev.
• Improve test coverage.
• Build testing into the pipeline.
• Enable faster release cycles with
scalability.
• Improve productivity and
efficiency.

The Software Delivery Lifecycle — Functional Testing
High Mobile Quality
& UX Requires
• Balance between real and virtual
device testing.
• Testing against real user conditions.
• Leveraging a cloud-based solution to
continuously maintain your lab.
• Automation of the key business
transactions.
• Fast feedback driven by smart
reporting and analysis.
Commit
Code
Build Binary
Deploy
Staging
Test Binary
Real Device Cloud-Based Testing
and Actionable Feedback
Unit & Smoke Testing on
Virtual/Real Platforms

The Software Delivery Lifecycle — Security Testing
Security Testing
• Tests in dev, build and prod.
• Leverages automation and direct
toolchain integrations.
• Binary testing provides most complete
code and risk coverage.
• Tune testing frequency and depth to
mobile app risk level.
• Dev remediation instructions for speed.
• High accuracy for low false positives.
Commit
Code
Build Binary
Deploy
Test Binary
Auto Monitor in Production
Staging
SCA Repo Security Scans
Auto Security Tests Every Build
Auto Generates Issue Tickets
Static Source
Security Scans

Common Pitfalls in Mobile App Security
& How to Overcome Them

Mobile App Security Risks Are Real & Pervasive

OWASP Mobile Top 10 — Areas Of Common Failure
M1 - Improper Platform Usage Misuse of features like Touch ID, permissions, keychain. 4% Fail
M2 - Insecure Data Storage Data leakage, client-side injection, weak server-side controls. 50% Fail
M3 - Insecure Communication Poor handshake, SSL/TLS/cert issues, transfer in clear text. 48% Fail
M4 - Insecure Authentication Improper identity management, weak session management. 5% Fail
M5 - Insufficient Cryptography Lack of crypto, improper crypto use. 8% Fail
M6 - Insecure Authorization Improper local authentication, forced browsing. 2% Fail
M7 - Client Code Quality Code mistakes e.g. buffer overflows, format string vulns. 32% Fail
M8 - Code Tampering Binary patching, method hooking/swizzling, memory mods. 11% Fail
M9 - Reverse Engineering Exposure to attacker reversing tools. 32% Fail
M10 - Extraneous Functionality Dev/QA inadvertent disabling security, hidden backdoors. 47% Fail

Inside the Mobile Attack Surface
Code Functionality
Data at Rest Data in Motion
Data Center
& App Backend
• GPS spoofing
• Buffer overflow
• allowBackup Flag
• allowDebug Flag
• Code obfuscation
• Configuration manipulation
• Escalated privileges
• URL schemes
• GPS leaking
• Integrity/tampering/repacking
• Side channel attacks
• App signing key unprotected
• JSON-RPC
• Automatic reference counting
• Dynamic runtime injection
• Unintended permissions
• UI overlay/pin stealing
• Intent hijacking
• Zip directory traversal
• Clipboard data
• World readable files
• Data caching
• Data stored in application directory
• Decryption of keychain
• Data stored in log files
• Data cached in memory/RAM
• Data stored in SD card
• OS data caching
• Passwords & data accessible
• No/weak encryption
• TEE/Secure enclave processor
• Side channel leak
• SQLite database
• Emulator variance
• Wi-Fi (no/weak encryption)
• Rogue access point
• Packet sniffing
• Man-in-the-middle
• Session hijacking
• DNS poisoning
• TLS Downgrade
• Fake TLS certificate
• Improper TLS validation
• HTTP Proxies
• VPNs
• Weak/no local authentication
• App transport security
• Transmitted to insecure server
• Zip files in transit
• Cookie “httpOnly” flag
• Cookie “secure” flag
• Android rooting/iOS jailbreak
• User-initiated code
• Confused deputy attack
• Media/file format parsers
• Insecure 3rd party libraries
• World writable files
• World writable executables
WEB + SAST VENDORS
APPS
FRAMEWORKS
NATIVE LIBRARIES
KERNEL
HAL
HARDWARE
TEST
APP
API Backends
Network &
Cloud Services

Data Center
& App Backend
Network &
Cloud Services
NowSecure Mobile AppSec Testing Checklist
APPS
FRAMEWORKS
NATIVE LIBRARIES
KERNEL
HAL
HARDWARE
TEST
APP
✓ Man in the middle: cert validation
✓ Man in the middle: hostname veri.
✓ Man in the middle: HTTP connections
✓ SSL downgrade
✓ Unprotected TLS traffic
✓ Cookie flags
✓ Certificate validity
✓ …
✓ App files and log files
✓ Keychain
✓ SD Card
✓ World writable files
✓ World readable files
✓ RAM
✓ Unencrypted credential storage
✓ SQLite databases
✓ Secure enclave processor
✓ …
✓ Development flags
✓ Automatic reference counting
✓ Stack smashing
✓ Bad authentication/authorization
✓ Root access
✓ Path traversal
✓ SQL injection
✓ Vulnerable third party libraries
✓ Heartbleed
✓ Bad cryptography
✓ App transport security
✓ Obfuscation
✓ …
Code Functionality Data in MotionData at Rest
Automated Mobile App Security Testing on Real Devices
Analyzes the binary post-compilation
to discover vulnerabilities including
those in third-party libraries.
Static Testing [SAST]
Inspects the binary at runtime collecting
telemetry from the “inside out” to find
vulnerabilities with near zero false
positives.
Interactive Testing [IAST]
Attacks the binary, device, network, and
APIs at runtime from the “outside in” to find
vulnerabilities with near zero false
positives.
Dynamic Testing [DAST]
TEST
APP

perfecto.io & nowsecure.com24 | Perfecto by Perforce © 2020 Perforce Software, Inc. and NowSecure, Inc
NowSecure Platform
One Portal for Your Mobile App Security & Privacy Testing Needs
• Web Interface
• Apple App Store
• Google Play
• CI/CD Plugins
• Enterprise App Store
• MDM/EMM Integration
• Restful API
• Interactive Binary Analysis
• Dynamic Binary Analysis
• Static Binary Analysis
• CVSS Security Score
• Compliance Checks
• Findings Descriptions
• Remediation Instructions
1
NowSecure Automated
Analysis Engine
Device Pool
Upload/Download Binary Fully Automated Testing
• Web Interface
• Report PDF
• Restful API & JSON
• Issue Tracking Tools
• Vulnerability Management
Dashboards
3
Flexible Output Options
2

Commit
Code
Build Binary
Deploy
Staging
Test Binary
NowSecure Powers Your Secure Toolchain
Auto Test Every Build
Auto Generate Issue Tickets

The Path Towards Continuous Testing

1 2 3 4 5
Stable automation Daily cycle Increase coverage Reach 95% Continuous testing
W H A T Y O U ’ L L G E T
W H A T Y O U ’ L L N E E D
• 99.9% availability lab
• Evidence collection
• Process integration
• CI
• Defect tracking
• Vuln tracking
• Execution control
• Dashboarding
• Create scripts
• Maintain scripts
• Understand what’s
wrong
• Skillset matched tool
• False negative
detection in reports
• Accurate findings for
low False positives
• Threat-modelling to
tune testing to risk
• Test on real devices
• Run tests daily
• Run tests on each build
• Get results in minutes
• Fast feedback loops
• Valuable coverage
• Meaningful daily
feedback
• Advance validations
• Visual automation
• Basic orchestration
• Up-to-date lab
• Scaled lab
• Analysis grouping
• Role-based routing
• Automate all that
possible & reasonable
• Advanced
orchestration
• Elasticity
• Sharding
• Environment
control
• Scaled reporting
• Nightly 
Continuously
• Cloud execution
• Very high scale
D E V O P S F R I E N D L Y Z O N E
The Path to Continuous Testing

Source: John Ferguson Smart
Continuous Testing, Feedback, Visibility, and Business Value —
A Full Team Objective. How and When Does Security Fit?

Pipeline Example: What Good Looks Like
Relevant Unit Testing
High-Value Smoke Testing:
Functional, API, Integration, Component and Security Testing
Regression on Real Devices:
Mixed Functional, Non-Functional, Performance and Security Testing

Perfecto Continuous Testing Platform
Cloud-Based Test
Environment
Smart Automation
Test Creation
& Execution
Authoring
Debugging Validations
Maintenance
Orchestration
Scheduling Test Environment Control
Self-Healing
Elastic
Artifacts Collection
Browsers &
Desktop
Mobile
Devices
Mobile
Simulators / Emulators
Smart Reporting
& Analytics
Smart
Analytics
Heatmaps
Root Cause
Analysis
Cross-Platform
Analysis
Continuous
Integration
Analysis

Perfecto’s Smart Continuous Testing Solution
Smart Execution
Fast and parallel test execution with
multi-team orchestration abilities and
management.
Smart Analytics
"Single pane of glass" provides
visibility and scales to support
millions of test results.
Smart Creation
Automation creation that matches your
team’s skillset (Appium, Espresso,
XCUITest, Quantum BDD).
Smart Lab
Always on and stable. Always up
to date. Supports all mobile OS
and platforms.
The Perfecto
human factor
increases your
chances to succeed.
v
Black Belt
Testing Experts
Training
Dedicated
Success Manager
24/7 VIP
Support

The 4 Key Pillars of Continuous Testing in DevOps
Automation for effective use of Time, Tools, & Resources
This is where value is being
realized, quality is improved
Unified Functionality + Security Approach serves DevOps with greater effectiveness.
Automated test analysis
• Fast feedback
• Root cause analysis
• Risk based coverage
Test creation and
maintenance
• Authoring tool
• Validations
• Accuracy
• Coverage

Eran Kinsbruner
Perfecto
Brian Reed
NowSecure

Deliver Flawless Mobile Apps Faster with CI/CD & CT

More Related Content

What's hot

Similar to Deliver Flawless Mobile Apps Faster with CI/CD & CT

More from Perfecto by Perforce

Recently uploaded

Deliver Flawless Mobile Apps Faster with CI/CD & CT