Presentation at DevSecCon Boston 2018

1. BOSTON 10-11 SEPT 2018
ADVANCED MOBILE SECURITY AUTOMATION IN
YOUR SDLC WITH BDD TESTS
DAVIDE CIOCCIA

2. BOSTON 10-11 SEPT 2018
#whoami
• Davide Cioccia
Security Engineer @ ING
Co-Founder @ IoTsec.eu
• Mobile Security Lead
• OWASP MSTG contributor
• IoT, IIoT breaker
• @davide107
• davidecioccia.com

3. BOSTON 10-11 SEPT 2018
Agenda
• Background
• BDD introduction
• Mobile BDD security tests
• Workshop

4. BOSTON 10-11 SEPT 2018
Background

5. BOSTON 10-11 SEPT 2018
Why mobile security matters?

6. BOSTON 10-11 SEPT 2018
The mistake
WRITE_EXTERNAL_STORAGE

7. BOSTON 10-11 SEPT 2018
Why do developers make mistakes?
• We need it ASAP
• Agile environment requires multiple release per day/week
• Security requirements are not followed
• Threat model not in place
• Rely on SAST tools for vulnerability detection
• Security testing is not embedded in the CI/CD
• Security is too late in the SDLC

8. BOSTON 10-11 SEPT 2018
Security teams challenges
• Provide security at the DevOps speed
• Have developers understand security
• Detect vulnerabilities in early stage
• Have Pentesters focus on “serious” stuff

9. BOSTON 10-11 SEPT 2018
Mobile security challenges
• Different Operating Systems
• Client side testing
• Different apps (native, hybrid, web)
• Different languages (poor SAST tool support)
• Different security controls

10. BOSTON 10-11 SEPT 2018
Improve the testing
• Security must be an accelerator and not a step back
• Extend the SDLC with security integration tests

11. BOSTON 10-11 SEPT 2018
BDD Security Tests

12. BOSTON 10-11 SEPT 2018
Testing in Agile
BDD
ATDD
TDD

13. BOSTON 10-11 SEPT 2018
What do we solve with BDD

14. BOSTON 10-11 SEPT 2018
Why BDD in security
• BDD offers more precise guidance on organizing the conversation between developers,
testers and security experts
• Notations originating in the BDD approach, in particular the given-when-then canvas,
are closer to everyday language and have a shallower learning curve
• Tools targeting a BDD approach generally afford the automatic generation of technical
and end user documentation from BDD “specifications"

15. BOSTON 10-11 SEPT 2018
BDD steps

16. BOSTON 10-11 SEPT 2018
Cucumber as engine for BDD
Features + Step definitions
Java
C#
Python
Ruby
JavaScript
…more
Gherkin

17. BOSTON 10-11 SEPT 2018
BDD test example
Business facing
Technology facing

18. BOSTON 10-11 SEPT 2018
Mobile BDD security

19. BOSTON 10-11 SEPT 2018
OWASP MSTG to BDD
Test scripts
.features
steps

20. BOSTON 10-11 SEPT 2018
Process
Test scripts APK
.features
steps.rb
Emulator / Device

21. BOSTON 10-11 SEPT 2018
Full process in CI/CD
Security Requirements
Threat modelling
(abuse case generation)
Implement BDD standard
security tests
Implement BDD application
specific security tests
Test against acceptance
environment
MSTG Test casesMASVS Checklist
Manual PT
Identify what can be
automated
Requirements Design Code Build Test Release Deploy Operate

22. BOSTON 10-11 SEPT 2018
Attack surface
22
Application layer OS and architecture layer Network layer
AUTHENTICATION
ACCESS CONTROLS
SESSION MANAGEMENT
ENCRYPTION
OBFUSCATION
INPUT & ERROR VALIDATION
DATA PROTECTION
MORE
PERMISSION MODEL
SERVICES
LIBRARIES
RESIDUAL DATA
MORE
CERTIFICATE PINNING
ENCRYPTION
MITM
URL WHITELISTING
WEB SERVER ASSESSMENT
NETWORK SCAN
MORE
INTERACTION WITH OS

23. BOSTON 10-11 SEPT 2018
What are we going to do
• Automate MASVS using Calabash, Gherkin and Ruby
• Identify what we can automate from the MSTG
• Extend UI/UX testing framework to create security integration tests
• Write BDD tests

24. BOSTON 10-11 SEPT 2018
Why
• MASVS is becoming the standard de facto for security testing
• MSTG is the technical sister (thanks Sven Sneiler and Bernard Muller)
• All the checks are currently performed manually from pentesters, security
engineers, developers,
• ..or integrating SAST tools in the pipeline. But SAST is not too smart!
• With BDD security is pushed left in the SDLC

25. BOSTON 10-11 SEPT 2018
Benefits
• Increase security maturity of the teams
• Perform security integration tests on every build improves the code
• Simplify pentesterts life
• Decrease TTR (Time To Release) and enhance security
• Translate threats in tests
• Have a ready-to-use documentation

26. BOSTON 10-11 SEPT 2018
WORKSHOP

27. BOSTON 10-11 SEPT 2018
Setup
• Dockerfile
• Calabash
• Android SDK
• Android tools
• JDK
• Genymotion for Personal use (FREE)
• Emulate any Android device
• IDE / Text Editor of your choice
• Recommended: Sublime with Gherkin syntax plugin

28. BOSTON 10-11 SEPT 2018
Outcome
• .features
• A Feature File is an entry point to the Cucumber tests. This is a file
where you will describe your tests in Descriptive language (Like
English).

29. BOSTON 10-11 SEPT 2018
Steps
• security_steps.rb
• Implementation of the Gherkin syntax
• Ruby function with parameters in input
• We are going to use the android tools and to perform analysis on the
device

30. BOSTON 10-11 SEPT 2018
.features
Feature: Logs must not contain sensitive information
@first_scenario
Scenario: As a user I insert my sensitive information and I
check that they are not reflected in the logfiles
Given I clean "all" the application log

31. BOSTON 10-11 SEPT 2018
security_steps.rb
Given /^I clean "(.*)" the application log$/ do |log|
%x(adb logcat -b #{log} -c)
end

32. BOSTON 10-11 SEPT 2018
Clone the GitHub repository
https://github.com/ing-bank/bdd-mobile-security-automation-framework

33. BOSTON 10-11 SEPT 2018
Let’s do some work

34. BOSTON 10-11 SEPT 2018
Thank you