SlideShare a Scribd company logo
1 of 115
Download to read offline
Christian Schneider | @cschneider4711
Security DevOps
Free pentester’s time to focus on high-hanging fruits
`whoami`
» Software Developer, Whitehat Hacker & Trainer
» Freelancer since 1997
» Focus on JavaEE & Web Security
» Speaker at Conferences
» @cschneider4711
www.

mail@
Christian-Schneider.net
Security – Status Quo
Static Application Security Testing (SAST)
Dynamic Application Security Testing (DAST)
Fun Part
The Ratio Problem
Dev Ops
Sec
100 10 1: :
The Frequency Problem
more Rollouts than Pentests
Security DevOps ?
Integrate Security Tools into Build-Jobs / CI-Chain
on every
commit or nightly
Why Security DevOps?
» Keep up with rollout pace in agile projects
» Automate certain checks as best as possible within
the build-chain
» Early feedback to Devs
» Solution that scales across the company
» Does not remove the manual pentest requirement!
» Aims to free pentesters’ time to hunt more
high-hanging bugs
Different levels of "Security DevOps" integration…
» Security DevOps Maturity Model (SDOMM)
» Can be seen as some automation tips within
OpenSAMM’s security practices
» Verification: Security Testing
» Verification: Code Reviews
» Allows to define a RoadMap for projects
implementing Security DevOps
four axes each with four
belts as incremental steps
implicit master
… what levels will we cover?
Four different axes
4 3 2 1 1 2 3 4
4
3
2
1
1
2
3
4
Dynamic Depth
Intensity
Static DepthConsolidation
4 3 2 1 1 2 3 4
4
3
2
1
1
2
3
4
Dynamic Depth
Intensity
Static DepthConsolidation
Four different axes
Let’s explore these axes …
» … by showing how to implement this
with OpenSource solutions used in the 

Security & Development domains.
Axis of "Dynamic Depth"
How deep are dynamic scans executed
within a Security DevOps CI chain?

i.e. "where" are dynamic 

security tests applied?
Axis "Dynamic Depth": Level 1
Scanning of public attack surface (pre-auth):
» Spidering of UI layer
» No requirement to authenticate scanner with target
» Easy integration of scanner(s) in nightly build as
post-step
» "Throw tool at it (in CI-chain) and see what it
generates…"
Zed Attack Proxy (ZAP)
"OWASP ZAP" features:
»passive scanning (Proxy / Spider)
»active scanning (Proxy / Spider)
»manual payload delivery (Intercepting Proxy)
»Spider (classic & AJAX)
»Fuzzing
»Plugins
»…
Request/Response History (Proxy)
Active Scanning…
OWASP ZAP
ZAP in SecDevOps?
"OWASP ZAP" features
relevant for Security DevOps integration:
»Headless operation mode / daemon
»REST-API
»Highly scriptable
»CLI
UI- and Service-Tests in CI
… enhanced with OWASP ZAP
passive & active
scanning
place report (html,xml) 

into workspace
HTML report of ZAP scan
ZAP + Jenkins = SecDevOps?
"OWASP ZAP" (spider & scanner) + Jenkins plugin “ZAProxy"
» Allows us to "Spider & Scan" as step in build job 

via Jenkins plugin "ZAProxy"
» Point plugin config to URL of integration system to test
» Plugin saves HTML-report in project’s job for inspection
» Best as separate Jenkins job to run during nightly build
(duration)
– Use different ZAP proxy ports for different builds to allow

parallel execution of different project build jobs
Jenkins Plugin "ZAProxy": ZAP Startup
Jenkins Plugin "ZAProxy": ZAP Scan
Arachni in SecDevOps?
"Arachni Scanner" features relevant for Security
DevOps integration:
» Passive & active scanning (Proxy / Spider)
» Uses internally a headless browser-cluster 

(for apps with lots of JS)
» Automation?
– direct CLI + RPC-API (with CLI)
» Web-UI (helpful when permanently 

running as server)
Arachni + Jenkins = SecDevOps?
"Arachni Scanner" + Jenkins CLI step in build
» Start in build job as CLI step and point to URL
of system under test
» Generate HTML report and place into
workspace for inspection
» Better execute within nightly build job
(due to duration)
» Alternatively: 

Start remote (also via CLI) to run async
HTML report of Arachni scan
(Summary) Bildquelle: Arachni Scanner
HTML report of Arachni scan 

(Details) Bildquelle: Arachni Scanner
BDD-Security in SecDevOps?
BDD-based framework for functional and technical security tests:
» Technical security tests 

(i.e. check against XSS, SQL-Injection, XXE, etc.)
– uses ZAP as scanning engine (among others)
» Functional security tests 

(i.e. check UserA can’t access data from UserB)
» Tightly integrates with Selenium based app navigation workflows
» Uses JBehave for G/W/T stories & reporting
» Can run within CI (Jenkins, etc.) due to JBehave
or as JUnit tests
BDD-Security Story: Scan for XSS
Bildquelle: BDD-Security
HTML report of BDD-Security
Bildquelle: BDD-Security
Gauntlt in SecDevOps?
BDD-based framework for executing many security tools/
scanners:
» Integrates scanners like Arachni, ZAP, sqlmap, etc.
– Easy to integrate "your custom scanner tool" with
Gauntlt as well
» Allows to call different scan polices via BDD-stories (G/W/T)
» Integration with Jenkins (or other build servers) by either
– Linking Gauntlt’s HTML report to build, or by
– modifying how Gauntlt calls Cucumber to produce
JUnit output
Other dynamic scanners exist also…
Some have different focus
» Nikto
» w3af
» IronWASP
» Nessus
» Mittn (BDD-based)
» Gauntlt (BDD-based)
» Minion (integration framework)
» OWTF (integration framework)
» good commercial ones
» etc.
Axis "Dynamic Depth": Level 2
Scanning of authenticated parts (= "post-auth")
via UI layer
» Properly maintaining sessions
» Logout-detection & automatic re-login
» Different users / roles
» Spider & scan post-auth

Handling of hardening measures of application under test
» CSRF-Tokens, etc.
Guide ZAP into Post-Auth in CI
Use ZAP manually (1x) to configure "Context": Auth, RegExps for 

Logged-In/Out Indicators, Users etc.
– save as "ZAP Session-File" (could be in code repo)
– use that "Session-File" from code repo as starting point of scan 

(loaded as ZAP session during build job). 

Note: Current version of ZAP has a bugfix pending for loading creds from session file
One can set these auth values and/or additional data via ZAP’s REST-API
during each build before scan starts (from Jenkins/Maven/…)
– use that to define current active session etc. during spider & scan
Also Scripts in JavaScript or Zest can be registered in ZAP context 

to programmatically give authentication to ZAP
Login config example within ZAP
ZAProxy Jenkins Plugin: ZAP session use
Guide Arachni into Post-Auth
Give authentication infos to Arachni
(Auth, Logged-In Indicators, Users)
» Use Arachni "autologin" plugin to specify via command line
– Login URL, formfield names, credentials, logged-in
indicator, excludes
» Alternatively write custom ruby script for "login_script" plugin
– Individual custom login logic possible
– Logged-In indicators (RegExp) to know when to re-login


./arachni 

--plugin=autologin:

url=https://example.com/login.action,

parameters='j_username=foo&j_password=bar',

check='Logout' 

--scope-exclude-pattern=logout.action 

https://example.com/
Login config example
within Arachni (used in CI)
Or individual ruby script if more custom login logic required…
Eventually also --session-check-url & --session-check-pattern
>> Easier to achieve Post-Auth Scan with Arachni (compared to others)

>> Can also be ordered via CLI remote/async during build (nightly) or scheduled (WebUI)
Arachni Web-UI:
Scan scheduling
Bildquelle: Arachni Scanner
Guide BDD-Security into Post-Auth
Use Selenium to navigate through the login process
» Based on excellent integration of BDD-Security with Selenium
» Separate app navigation code (Selenium) from Security
testing code
» Use Selenium class (that handles login) within BDD stories
» Perform further spidering & active scanning (through ZAP)
post-auth
public class ShopApplicationScanHelper
extends WebApplication implements ILogin {
// ... integrates with BDD-Security via parent class & interface ...
}
public class ShopApplicationScanHelper
extends WebApplication implements ILogin {
@Override
public void openLoginPage() {
}
@Override
public void login(Credentials credentials) {
}
@Override
public boolean isLoggedIn(String role) {
}
public class ShopApplicationScanHelper
extends WebApplication implements ILogin {
@Override
public void openLoginPage() {
driver.get(Config.getInstance().getBaseUrl() + "customer/login");
verifyTextPresent("Login");
}
@Override
public void login(Credentials credentials) {
UserPassCredentials creds = new UserPassCredentials(credentials);
driver.findElement(By.id("username")).clear();
driver.findElement(By.id("username")).sendKeys(creds.getUsername());
driver.findElement(By.id("password")).clear();
driver.findElement(By.id("password")).sendKeys(creds.getPassword());
driver.findElement(By.name("_action_login")).click();
}
@Override
public boolean isLoggedIn(String role) {
if (driver.getPageSource().contains("My Account")) {
return true;
} else {
return false;
}
What about CSRF-Tokens?
ZAP can handle CSRF-Tokens
» Can be individualised via configuration
BDD-Security uses (among others) ZAP for scanning
Arachni can also handle CSRF-Tokens 

Apply targeted delta scanning:
• Developer teams maintain "scope delta files" in SCM
• like "what parts of the application have changed during this sprint" 

(= URL patterns)
• could’ve been recorded in proxy-based tools (like ZAP) by Devs
• DAST Scanner uses it to scan only this during onCommit or nightly CI run
• Feed this delta information as argument to scope definitions

Arachni: --scope-include-pattern / --scope-exclude-pattern

ZAP: Include/Excludes in Context (via Session-File or REST-API)

BDD-Security: maintain as "table file" of RegExps as scope
If scanning takes too long? (1/2)
If scanning takes too long? (2/2)
Train spiders about the structure of application to reduce scan
redundancy and avoid scanning static resources
• ZAP: use RegExp "Exclude from scanner" rules to prevent
redundant scans in data-driven applications (i.e. scan only one
instance of it)
• Arachni: --scope-redundant-path-pattern / --scope-auto-redundant
as well as --scope-exclude-pattern="logout.|.js|.css|.jpg|.png"
Don’t forget to scan complete application at least on nightly or weekly run
• Only apply full scan, if simple smoke-tests are green
Axis "Dynamic Depth": Level 3
Separate scanning of different application layers / backends
• Scan internal WebServices (e.g. SOAP / REST) = directly scan backends
• Detect and scan parameter positions within XML, JSON, …
• Scan from "within" the different application’s layers
• IAST with distributed agents & instrumentation aims into that direction
• At least one simple step in that direction:
• Use the proxy also between your backend service calls
Backend scans with ZAP
How to achieve this with ZAP?
• ZAP operates as proxy server: place it between backend calls
• ZAP can inject payloads in observed XML tags/attributes & JSON fields
• Capture service call traffic in integration test during CI while either
A. frontend UI tests execute service backend calls indirectly, or
B. executing service tests that directly access the service endpoint
• Automatically scan as new requests are seen: "ATTACK Mode"
Also keep an eye on an alpha-level SOAP-Scanner ZAP addon
Backend scans with Arachni
How to achieve this with Arachni?
• Arachni can also operate as proxy: place it between backend
calls
• Use passive proxy plugin to "train" Arachni of the XML / JSON
requests
• New addition in v1.1 to extract XML / JSON input vectors
from it
• Use that collected input vector data to feed the active scan for 

the observed requests
Axis "Dynamic Depth": Level 4
Targeted scanning of individual forms / wizards (UI)
and service layers
» More individualised workflow coverage
(not just simple spidering)
» Business-logic compliant usage patterns & inputs
– "fill shopping cart followed by checkout process"
– "access backend WebServices in special order to test
workflow", etc.
» Custom coded security tests tailored to the application
ZAP with special workflows (1/3)
Many ways exist…
The simplest one could be: 

Re-use existing UI tests (Selenium, …)
» Proxy this traffic through ZAP in
“ATTACK-Mode" (in security test phase of build)
» Optionally use ZAP Attack-Policies to specify/limit
certain attack types
ZAP with special workflows (2/3)
A more customised handling of individual workflows can
be achieved:
Re-use & enhance existing "UI test code" at the desired

workflow steps with calls to ZAP’s (REST)-API ordering attacks
» Basically it’s like Unit-Test code that uses Selenium along 

with ZAP-Calls at the proper positions in application workflow
» Type of "ordered attacks" can again be defined via policies
» Start ZAP as Daemon from Jenkins via plugin
public class ShopApplicationTest { // = regular JUnit unit test
@Before

public void setup() {
}
@Test

public void testShippingAddressStep() {


}
@Test

public void testBillingAddressStep() {
}
}
public class ShopApplicationTest { // = regular JUnit unit test
@Before

public void setup() {
// 1. start new proxy session in running ZAP (via REST-API call)
// 2. create Selenium driver (proxying through running ZAP)
}
@Test

public void testShippingAddressStep() {


}
@Test

public void testBillingAddressStep() {
}
}
public class ShopApplicationTest { // = regular JUnit unit test
@Before

public void setup() {
// 1. start new proxy session in running ZAP (via REST-API call)
// 2. create Selenium driver (proxying through running ZAP)
}
@Test

public void testShippingAddressStep() {
// 1. use Selenium to fill shopping cart
// 2. use Selenium to proceed to checkout
// 3. use Selenium to provide reasonable shipping address data


}
@Test

public void testBillingAddressStep() {
}
}
public class ShopApplicationTest { // = regular JUnit unit test
@Before

public void setup() {
// 1. start new proxy session in running ZAP (via REST-API call)
// 2. create Selenium driver (proxying through running ZAP)
}
@Test

public void testShippingAddressStep() {
// 1. use Selenium to fill shopping cart
// 2. use Selenium to proceed to checkout
// 3. use Selenium to provide reasonable shipping address data
// 4. set attack policy (types & strength) in running ZAP (API)
/* 5. call ZAP (API) to actively scan the last seen URL
(optionally define parameter excludes via API 

or ZAP "input vector scripts" if custom input format) */
}
@Test

public void testBillingAddressStep() {
}
}
See https://github.com/continuumsecurity/zap-webdriver 

for a great working example of Selenium ZAP integration
public class ShopApplicationTest { // = regular JUnit unit test
@Before

public void setup() {
// 1. start new proxy session in running ZAP (via REST-API call)
// 2. create Selenium driver (proxying through running ZAP)
}
@Test

public void testShippingAddressStep() {
// 1. use Selenium to fill shopping cart
// 2. use Selenium to proceed to checkout
// 3. use Selenium to provide reasonable shipping address data
// 4. set attack policy (types & strength) in running ZAP (API)
/* 5. call ZAP (API) to actively scan the last seen URL
(optionally define parameter excludes via API 

or ZAP "input vector scripts" if custom input format) */
}
@Test

public void testBillingAddressStep() {
// same idea as above ... just continue with the pattern
}
}
ZAP with special workflows (3/3)
Alternatively "train" ZAP about the workflow by
recording Zest scripts
»Keep an eye on "Sequence Scanning"
alpha-level ZAP AddOn
–Still alpha-level (as of June 2015), but
interesting approach
BDD with special workflows
Use Selenium to further drive BDD-Security initiated checks:
» Selenium-based test code navigates application workflows
» This code is integrated with BDD (via Java interfaces),
so that:
– BDD-Security stories can use that code to navigate 

and generate traffic
– This generated traffic will be scanned by ZAP via BDD
public class ShopApplicationScanHelper
extends WebApplication implements ILogin {
// ... already covered login handling code
// in BDD-Security example ...
public boolean fillShoppingCartAndCheckout() {
// custom Selenium (recorded) code 

// to drive certain use cases (like shopping) ...
}
}
Use this navigation method inside BDD-Security scenario:
...

Given a new scanning session
And the page flow described in the method: 

fillShoppingCartAndCheckout is run through the proxy
...
If no Selenium test code exists?
Simply give developer teams access to ZAP to (at least)
pre-seed the scanner:
» Developer teams use browser to navigate app workflows while proxying
– Thereby seed the ZAP session(s) with navigation nodes/workflows
– Save the ZAP session(s) and check-in into SCM (Git, SVN, …)
» Point the Jenkins ZAP plugin to the saved ZAP session(s) as starting
point
» Devs can add to this list of URLs for ZAP with each new UI
BTW: ZAP is also available as Docker image…
Sometimes it’s useful to have programmed unit-tests directly 

calling a business API
• Test directly at the API-level (i.e. Java code level)
• Utilise fuzzing values from popular sources like "fuzzdb" 

and/or generate on-the-fly with fuzzer "radamsa"
• Check the API method’s response 

(like regular unit test, but seeking for signs 

of successful attack tests)
Security-UnitTests for APIs (1/2)
Security-UnitTests for APIs (2/2)
Leverage the benefit of direct API-level testing 

(i.e. testing "from within")
• Place "probes" into the application to watch out for successful
attack tests
• Check some OWASP AppSensor detection points for
potential attack triggers (RASP-like)
• Error logs are good candidates to check also
• Mock objects (Mockito, etc.) or weave aspects (AOP)
collecting data (i.e. method calls and arguments, etc.)
Axis of "Static Depth"
How deep is static code analysis performed 

within a Security DevOps CI chain?

i.e. "where" are static 

security tests applied?
Axis "Static Depth": Level 1
Assurance that no third-party code with known
vulnerabilities is used
» Check application’s server-side dependencies
– For Java applications: JAR files, etc.
» Check application’s client-side dependencies
– JavaScript frameworks, etc.

Useful even for projects not currently under development
» New vulnerabilities could become public: 

Build regularly to check
Java libs: OWASP Dependency-Check
Scans all dependencies (even transitive ones) 

against CVE list
» Available as Maven plugin and Ant task
» CLI version also available
» Jenkins plugin for nice reporting 

(and build breaking thresholds)
» Not false-positive free
– regularly needs some time to triage findings
Reporting in Jenkins
Findings (CVE-Details)
JavaScript libs: retire.js
Scans application’s JavaScript files against list
of known vulnerable ones
» Available also as Maven Plugin …
» … and as CLI 

(point it to your folder containing .js files)
Axis "Static Depth": Level 2
Scan important parts of source code
for vulnerability patterns
» At least important parts of applications’ codebases are
scanned
– like in-house reused code & custom developed
frameworks
– for big multi-project Maven projects: 

at least frontend and backend projects
– or code of changed modules within a sprint
Java: FindSecurityBugs
Plugin for FindBugs with enhanced checks for security issues
in Java code
» Runs within FindBugs, so that it can execute in Maven and/or Sonar
» Generates XML-file with potential findings
» Jenkins plugin for representing results
(also possible to display in Sonar)
– Integrates with Jenkins "Static Code Analysis" as part of FindBugs
– Delta reporting of new (and fixed) findings & trends
FindBugs in Jenkins: Trends, etc.
in Sonar: Vulnerability & Code-Pointer
JavaScript: ScanJS
Scans JavaScript code for vulnerabilities
» Most importantly DOM-based XSS (Cross-Site Scripting)
» Flags also "Usage of sensitive API" issues
» Executable as CLI from within Jenkins build steps
» Optional Web-UI (running as local server)
ScanJS Findings as HTML: DOM-XSS
Ruby on Rails apps: Brakeman
Scans RoR code for vulnerabilities
» CLI based
» Nicely integrates with Jenkins
– Rendering scan results and trend analysis
» Allows to compare scan results (delta detection)
– "What has been fixed and what was newly introduced?"
Other static scanners exist also…
Some have different focus
» PrimeJS (like ScanJS for JavaScript DOM-XSS etc.)
» PMD (for Java code regarding API safety issues)
» good commercial ones
» etc.
Axis "Static Depth": Level 3
Scan the complete applications’ source code 

for vulnerability patterns
» Just like "Level 2", but now for the complete codebase

(no excludes)
– i.e. all projects of a multi-project Maven project
– best via parent or corporate pom.xml
For FindSecurityBugs this means:
» pre-compile even all JSPs (if you have any) 

to scan them
Axis "Static Depth": Level 4
Scan code of (important) third-party dependencies 

used in application
» If the component is open-source:
– include it in regular scanning activities
» If the component is closed-source:
– consider using code scanners that operate on binaries
• FindBugs (hence its FindSecurityBugs plugin) can also 

scan Java binaries, since it operates on Bytecode
Axis of "Intensity"
How intense are the majority of the
executed attacks within a Security
DevOps CI chain?

i.e. "what" is being checked for?
Axis "Intensity": Level 1
Dynamic checks: Only passive scanning
» Simply proxy testcase generated traffic through passive operating
scanners
» Quick-win for using existing testcase code (i.e. Service- or UI-tests)
» Can execute within existing tests that execute on every commit
– No requirement for a separate nightly CI job
(i.e. doesn’t slow down build)


Static checks: Just scan the code along with other code metrics
» Use the default rules of the scanners and integrate them into CI
Axis "Intensity": Level 2
Use lightweight active scanning options for dynamic checks
» ZAP: active scanning of observed URLs (proxied or spidered)
– perform an "active scan": For Jenkins ZAProxy Plugin
provide a ZAP policy with desired active scan intensity
(or via ZAP’s REST-API during build for other setups)
» Arachni: enable "active scan" during spidering
» BDD-Security stories can call ZAP with policy files defining
attack strength
Better to execute in separate (nightly) build job due it’s to duration
Think of resetting databases (+ more environment) prior or after
test execution
Axis "Intensity": Level 3
Use heavyweight scanning options on important parts of application
Dynamic checks: create "riskier" custom attack profiles
ZAP:
– set "Threshold" to "Low" / "Strength" to "High" (or selectively "Insane")
– enable more scanning rules via Policy Files (could be maintained &
used via SCM)
– use ZAP’s "Advanced SQLInjection Scanner" extension (or directly
call sqlmap)
Arachni:
– Enable the more aggressive "audit flags" like --audit-forms, …
Static checks: FindSecurityBugs: set "Threshold"

to "Low" / "Effort" to "Max"
Axis "Intensity": Level 4
Use customised rule sets for dynamic checks
ZAP 

(which is very extensible via custom scripts, recored and
customised Zest, or plugins):
» Custom coded scan scripts as active & passive scan rules for
company-individual checks
» Security regression tests (in Zest) against previous found
vulns (also logic flaws)
» Custom input vector scripts: e.g. define certain injection
points in custom encoded params
» Proxy scripts: for example for generic request/response
modifications
Scripting possibilities in ZAP
… and for static code scans?
Use customised rule sets for static checks
FindSecurityBugs
» Custom coded rule files (in Java) to check for wrong usages of 

server-side inhouse frameworks (or alike)
ScanJS
» Custom rule files (in JSON) to check for wrong usages of
client-side inhouse frameworks (or alike)
Axis of "Consolidation"
How complete is the process of handling
findings within a Security DevOps CI chain?

i.e. "how" are the results used?
Axis "Consolidation": Level 1
Generate human-readable (HTML) reports from tools and link
them in Jenkins
» All relevant mentioned static and dynamic scanners generate
HTML reports
» Collect and publish them in Jenkins build: via Jenkins 

"HTML Publisher Plugin"

Jenkins "HTML Publisher Plugin":
Configuration of HTML reports to link
Jenkins "HTML
Publisher Plugin":
Result in build
HTML report of ZAP scan
Axis "Consolidation": Level 1 (cont.)
Use simple criteria to "break the build" on heavy findings 

(ok, at least "unstable")

» Most scanners have capabilities to automatically flag the build
» Dependency-Check (Severity Threshold),
» BDD-Security (with the JBehave-stories),
» FindSecurityBugs (via Sonar when rated as blocker),
» Arachni (via Gauntlt execution with BDD-like stories),
» etc.

» For others: at least do a simple log parse from 

Jenkins "Log Parser Plugin" to flag the build as 

unstable and/or broken
OWASP Dependency-Check: 

Thresholds to break build
FindSecurityBugs: 

IDE support & Collaboration
FindBugs (with FindSecurityBugs) has good Jenkins & IDE integration
Additionally consider placing the FindBugs generated XML report file
within Jenkins
» Developers can load the XML report file into FindBugs IDE plugins 

(Eclipse, IntelliJ, NetBeans, …)
» Developers & Security Analysts can collaborate on bugs
via the IDE plugins
– See https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial 

for more details
• Consider installing it as your private "FindBugs"-Server 

on-premise to keep results only on your servers…
FindSecurityBugs Eclipse Perspective
with Collaboration View + Remediation
Axis "Consolidation": Level 2
Custom logic to make build unstable and/or broken
depending on
» Type of vulnerability (CWE or WASC or …)
» Severity ranking (high risk)
» Confidence level (firm vs. tentative)


Provide useful remediation info to developers
Respect suppression mechanisms to rule out 

false positives
Flagging builds from reports
How (from within a CI job)?
» Most scanners also emit XML reports that can be parsed
– Often a simple XPath count is just fine
» Alternatively fetch the results by accessing the scanner’s API
» Be sure to only break build with findings of 

high severity and high confidence !!!
– Less is more (when it comes to automation)…
» When new check introduce many findings: 

Baseline-Approach: Only break on new findings
Providing useful info to Devs
Lookup CWE-ID (provided by most scanners) in vulndb:
» https://github.com/vulndb/data

(currently 65+ vulns described)
» Useful if the messages from scanners are too generic 

or don’t help much
» Maps vulnerability to useful remediation messages
» Possibility to extend that repository locally with 

company-individual texts (mapping to the technology 

stack) —> that’s where it pays off…
Suppression of false positives
Custom finding list (from XML reports): 

filter out according to list from code repo
BDD-Security: Use false positive tables in story files 

(i.e. text files checked in SCM)
FindSecurityBugs: Use exclusion lists in config (XML filter files) 

& annotation @SuppressWarnings on false positive code lines
Dependency-Check: XML file of suppressions 

(checked-in along with the project)
retire.js: "ignore file" parameters on CLI

(cloud be loaded in CI from code repo)
Axis "Consolidation": Level 3
Consolidation goals:
» Consolidate & de-duplicate findings from different 

scanner reports (with better false positive handling)
» Push consolidated findings into established bug-tracker 

(known to devs)
» Delta analysis & trends over consolidated data sets
ThreadFix as result consolidator
Use a local ThreadFix server, which imports native scanner outputs
• does the heavy lifting of consolidation & de-duplication
• pushes findings toward bug-tracker and IDE (via plugins)
• process can be customised using it’s own REST-API
• ThreadFix imports findings of ZAP, Arachni, FindBugs, Brakeman, etc.
Report generated by ThreadFix
Bildquelle: ThreadFix
Trend Chart generated by ThreadFix
Bildquelle: ThreadFix
Axis "Consolidation": Level 4
Measure the concrete code coverage of your
security testing activities
» Find untested "white spots"
» Derive where static checks and code reviews
should focus more to compensate
Code coverage analysis
Use "OWASP Code Pulse", which instruments your Java app
» collects coverage data during dynamic security testing scans
» generates reports ("code treemaps") of coverage
Code Treemap of dynamic scan coverage
Bildquelle: OWASP Code Pulse
Thank you very much!
Links
OWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
ZAP Selenium Demo https://github.com/continuumsecurity/zap-webdriver
ZAP Jenkins Plugin https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+Plugin
BDD-Security http://www.continuumsecurity.net/bdd-intro.html
Arachni http://www.arachni-scanner.com
OWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_Check
OWASP Dependency Track https://www.owasp.org/index.php/OWASP_Dependency_Track_Project
FindSecurityBugs http://h3xstream.github.io/find-sec-bugs/
FindSecurityBugs-Cloud https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial
retire.js http://bekk.github.io/retire.js/
ScanJS https://github.com/mozilla/scanjs
Jenkins Log Parser Plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin
ThreadFix http://www.threadfix.org
OWASP Code Pulse https://www.owasp.org/index.php/OWASP_Code_Pulse_Project
Seccubus https://www.seccubus.com
vulndb https://github.com/vulndb/data
fuzzdb https://code.google.com/p/fuzzdb/

radamsa https://code.google.com/p/ouspg/wiki/Radamsa

Interested in more web security stuff?
Visit my Blog: www.Christian-Schneider.net
@cschneider4711
Bildquelle: dreamstime.com

More Related Content

What's hot

DevSecOps reference architectures 2018
DevSecOps reference architectures 2018DevSecOps reference architectures 2018
DevSecOps reference architectures 2018Sonatype
 
LENIN PART 2: Totalitarianism
LENIN PART 2: TotalitarianismLENIN PART 2: Totalitarianism
LENIN PART 2: TotalitarianismElizabeth Lugones
 
Continuous and Visible Security Testing with BDD-Security
Continuous and Visible Security Testing with BDD-SecurityContinuous and Visible Security Testing with BDD-Security
Continuous and Visible Security Testing with BDD-SecurityStephen de Vries
 
Chapter 5 - The Selection Structure
Chapter 5 - The Selection StructureChapter 5 - The Selection Structure
Chapter 5 - The Selection Structuremshellman
 
DevSecOps in Baby Steps
DevSecOps in Baby StepsDevSecOps in Baby Steps
DevSecOps in Baby StepsPriyanka Aash
 
Trunk Based Development in the Enterprise - Its Relevance and Economics
Trunk Based Development in the Enterprise - Its Relevance and EconomicsTrunk Based Development in the Enterprise - Its Relevance and Economics
Trunk Based Development in the Enterprise - Its Relevance and EconomicsPerforce
 
The Gulf War
The Gulf WarThe Gulf War
The Gulf WarBen Dover
 
Causes of the Cold War - The Arms Race.pptx
 Causes of the Cold War - The Arms Race.pptx Causes of the Cold War - The Arms Race.pptx
Causes of the Cold War - The Arms Race.pptxmrmarr
 
The Cold War 1945 1990
The Cold War 1945 1990The Cold War 1945 1990
The Cold War 1945 1990mrbruns
 
Testing process
Testing processTesting process
Testing processTerry Cho
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDDDror Helper
 
Code obfuscation
Code obfuscationCode obfuscation
Code obfuscationbijondesai
 
Events of the cold war
Events of the cold warEvents of the cold war
Events of the cold warG_Tweedy
 
Software Testing 101
Software Testing 101Software Testing 101
Software Testing 101QA Hannah
 
Testing strategies in Software Engineering
Testing strategies in Software EngineeringTesting strategies in Software Engineering
Testing strategies in Software EngineeringMuhammadTalha436
 

What's hot (20)

DevSecOps reference architectures 2018
DevSecOps reference architectures 2018DevSecOps reference architectures 2018
DevSecOps reference architectures 2018
 
LENIN PART 2: Totalitarianism
LENIN PART 2: TotalitarianismLENIN PART 2: Totalitarianism
LENIN PART 2: Totalitarianism
 
Continuous and Visible Security Testing with BDD-Security
Continuous and Visible Security Testing with BDD-SecurityContinuous and Visible Security Testing with BDD-Security
Continuous and Visible Security Testing with BDD-Security
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
 
Unipolarity
UnipolarityUnipolarity
Unipolarity
 
Chapter 5 - The Selection Structure
Chapter 5 - The Selection StructureChapter 5 - The Selection Structure
Chapter 5 - The Selection Structure
 
DevSecOps in Baby Steps
DevSecOps in Baby StepsDevSecOps in Baby Steps
DevSecOps in Baby Steps
 
Trunk Based Development in the Enterprise - Its Relevance and Economics
Trunk Based Development in the Enterprise - Its Relevance and EconomicsTrunk Based Development in the Enterprise - Its Relevance and Economics
Trunk Based Development in the Enterprise - Its Relevance and Economics
 
The Gulf War
The Gulf WarThe Gulf War
The Gulf War
 
Causes of the Cold War - The Arms Race.pptx
 Causes of the Cold War - The Arms Race.pptx Causes of the Cold War - The Arms Race.pptx
Causes of the Cold War - The Arms Race.pptx
 
The Cold War 1945 1990
The Cold War 1945 1990The Cold War 1945 1990
The Cold War 1945 1990
 
Testing process
Testing processTesting process
Testing process
 
Code coverage
Code coverageCode coverage
Code coverage
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDD
 
Code obfuscation
Code obfuscationCode obfuscation
Code obfuscation
 
Events of the cold war
Events of the cold warEvents of the cold war
Events of the cold war
 
Software Testing 101
Software Testing 101Software Testing 101
Software Testing 101
 
Testing strategies in Software Engineering
Testing strategies in Software EngineeringTesting strategies in Software Engineering
Testing strategies in Software Engineering
 
Marshall plan (2)
Marshall plan (2)Marshall plan (2)
Marshall plan (2)
 
Wwi propaganda
Wwi propagandaWwi propaganda
Wwi propaganda
 

Similar to Christian Schneider on Security DevOps

Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...Christian Schneider
 
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...Christian Schneider
 
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015Christian Schneider
 
OWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA TestersOWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA TestersJavan Rasokat
 
OWASP_Top_Ten_Proactive_Controls_v32.pptx
OWASP_Top_Ten_Proactive_Controls_v32.pptxOWASP_Top_Ten_Proactive_Controls_v32.pptx
OWASP_Top_Ten_Proactive_Controls_v32.pptxnmk42194
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxcgt38842
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxjohnpragasam1
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxazida3
 
we45 DEFCON Workshop - Building AppSec Automation with Python
we45 DEFCON Workshop - Building AppSec Automation with Pythonwe45 DEFCON Workshop - Building AppSec Automation with Python
we45 DEFCON Workshop - Building AppSec Automation with PythonAbhay Bhargav
 
Maxim Salnikov - Service Worker: taking the best from the past experience for...
Maxim Salnikov - Service Worker: taking the best from the past experience for...Maxim Salnikov - Service Worker: taking the best from the past experience for...
Maxim Salnikov - Service Worker: taking the best from the past experience for...Codemotion
 
Securing Microservices using Play and Akka HTTP
Securing Microservices using Play and Akka HTTPSecuring Microservices using Play and Akka HTTP
Securing Microservices using Play and Akka HTTPRafal Gancarz
 
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
[NDC 2019] Functions 2.0: Enterprise-Grade ServerlessKatyShimizu
 
[NDC 2019] Enterprise-Grade Serverless
[NDC 2019] Enterprise-Grade Serverless[NDC 2019] Enterprise-Grade Serverless
[NDC 2019] Enterprise-Grade ServerlessKatyShimizu
 
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivSelf Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivAmazon Web Services
 
Java 6 [Mustang] - Features and Enchantments
Java 6 [Mustang] - Features and Enchantments Java 6 [Mustang] - Features and Enchantments
Java 6 [Mustang] - Features and Enchantments Pavel Kaminsky
 
Infrastructure as Code: Manage your Architecture with Git
Infrastructure as Code: Manage your Architecture with GitInfrastructure as Code: Manage your Architecture with Git
Infrastructure as Code: Manage your Architecture with GitDanilo Poccia
 
Modern Web 2019 從零開始加入自動化資安測試
Modern Web 2019 從零開始加入自動化資安測試Modern Web 2019 從零開始加入自動化資安測試
Modern Web 2019 從零開始加入自動化資安測試Secview
 
Cloud infrastructure as code
Cloud infrastructure as codeCloud infrastructure as code
Cloud infrastructure as codeTomasz Cholewa
 
Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Yan Cui
 
Securing Infrastructure as a Code - DevFest 2022 Presentation
Securing Infrastructure as a Code - DevFest 2022 PresentationSecuring Infrastructure as a Code - DevFest 2022 Presentation
Securing Infrastructure as a Code - DevFest 2022 PresentationObika Gellineau
 

Similar to Christian Schneider on Security DevOps (20)

Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
 
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
 
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
 
OWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA TestersOWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA Testers
 
OWASP_Top_Ten_Proactive_Controls_v32.pptx
OWASP_Top_Ten_Proactive_Controls_v32.pptxOWASP_Top_Ten_Proactive_Controls_v32.pptx
OWASP_Top_Ten_Proactive_Controls_v32.pptx
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptx
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptx
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptxOWASP_Top_Ten_Proactive_Controls_v2.pptx
OWASP_Top_Ten_Proactive_Controls_v2.pptx
 
we45 DEFCON Workshop - Building AppSec Automation with Python
we45 DEFCON Workshop - Building AppSec Automation with Pythonwe45 DEFCON Workshop - Building AppSec Automation with Python
we45 DEFCON Workshop - Building AppSec Automation with Python
 
Maxim Salnikov - Service Worker: taking the best from the past experience for...
Maxim Salnikov - Service Worker: taking the best from the past experience for...Maxim Salnikov - Service Worker: taking the best from the past experience for...
Maxim Salnikov - Service Worker: taking the best from the past experience for...
 
Securing Microservices using Play and Akka HTTP
Securing Microservices using Play and Akka HTTPSecuring Microservices using Play and Akka HTTP
Securing Microservices using Play and Akka HTTP
 
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
[NDC 2019] Functions 2.0: Enterprise-Grade Serverless
 
[NDC 2019] Enterprise-Grade Serverless
[NDC 2019] Enterprise-Grade Serverless[NDC 2019] Enterprise-Grade Serverless
[NDC 2019] Enterprise-Grade Serverless
 
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivSelf Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
 
Java 6 [Mustang] - Features and Enchantments
Java 6 [Mustang] - Features and Enchantments Java 6 [Mustang] - Features and Enchantments
Java 6 [Mustang] - Features and Enchantments
 
Infrastructure as Code: Manage your Architecture with Git
Infrastructure as Code: Manage your Architecture with GitInfrastructure as Code: Manage your Architecture with Git
Infrastructure as Code: Manage your Architecture with Git
 
Modern Web 2019 從零開始加入自動化資安測試
Modern Web 2019 從零開始加入自動化資安測試Modern Web 2019 從零開始加入自動化資安測試
Modern Web 2019 從零開始加入自動化資安測試
 
Cloud infrastructure as code
Cloud infrastructure as codeCloud infrastructure as code
Cloud infrastructure as code
 
Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)
 
Securing Infrastructure as a Code - DevFest 2022 Presentation
Securing Infrastructure as a Code - DevFest 2022 PresentationSecuring Infrastructure as a Code - DevFest 2022 Presentation
Securing Infrastructure as a Code - DevFest 2022 Presentation
 

Recently uploaded

Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...OnePlan Solutions
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfFerryKemperman
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentationvaddepallysandeep122
 
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in NoidaBuds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in Noidabntitsolutionsrishis
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
How to Track Employee Performance A Comprehensive Guide.pdf
How to Track Employee Performance A Comprehensive Guide.pdfHow to Track Employee Performance A Comprehensive Guide.pdf
How to Track Employee Performance A Comprehensive Guide.pdfLivetecs LLC
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commercemanigoyal112
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based projectAnoyGreter
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 

Recently uploaded (20)

Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdf
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentation
 
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in NoidaBuds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
How to Track Employee Performance A Comprehensive Guide.pdf
How to Track Employee Performance A Comprehensive Guide.pdfHow to Track Employee Performance A Comprehensive Guide.pdf
How to Track Employee Performance A Comprehensive Guide.pdf
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commerce
 
2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based project
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 

Christian Schneider on Security DevOps

  • 1. Christian Schneider | @cschneider4711 Security DevOps Free pentester’s time to focus on high-hanging fruits
  • 2. `whoami` » Software Developer, Whitehat Hacker & Trainer » Freelancer since 1997 » Focus on JavaEE & Web Security » Speaker at Conferences » @cschneider4711 www.
 mail@ Christian-Schneider.net
  • 5. Dynamic Application Security Testing (DAST) Fun Part
  • 6. The Ratio Problem Dev Ops Sec 100 10 1: :
  • 7. The Frequency Problem more Rollouts than Pentests
  • 9. Integrate Security Tools into Build-Jobs / CI-Chain on every commit or nightly
  • 10. Why Security DevOps? » Keep up with rollout pace in agile projects » Automate certain checks as best as possible within the build-chain » Early feedback to Devs » Solution that scales across the company » Does not remove the manual pentest requirement! » Aims to free pentesters’ time to hunt more high-hanging bugs
  • 11. Different levels of "Security DevOps" integration… » Security DevOps Maturity Model (SDOMM) » Can be seen as some automation tips within OpenSAMM’s security practices » Verification: Security Testing » Verification: Code Reviews » Allows to define a RoadMap for projects implementing Security DevOps
  • 12. four axes each with four belts as incremental steps implicit master … what levels will we cover?
  • 13. Four different axes 4 3 2 1 1 2 3 4 4 3 2 1 1 2 3 4 Dynamic Depth Intensity Static DepthConsolidation
  • 14. 4 3 2 1 1 2 3 4 4 3 2 1 1 2 3 4 Dynamic Depth Intensity Static DepthConsolidation Four different axes
  • 15. Let’s explore these axes … » … by showing how to implement this with OpenSource solutions used in the 
 Security & Development domains.
  • 16. Axis of "Dynamic Depth" How deep are dynamic scans executed within a Security DevOps CI chain?
 i.e. "where" are dynamic 
 security tests applied?
  • 17. Axis "Dynamic Depth": Level 1 Scanning of public attack surface (pre-auth): » Spidering of UI layer » No requirement to authenticate scanner with target » Easy integration of scanner(s) in nightly build as post-step » "Throw tool at it (in CI-chain) and see what it generates…"
  • 18. Zed Attack Proxy (ZAP) "OWASP ZAP" features: »passive scanning (Proxy / Spider) »active scanning (Proxy / Spider) »manual payload delivery (Intercepting Proxy) »Spider (classic & AJAX) »Fuzzing »Plugins »…
  • 22. ZAP in SecDevOps? "OWASP ZAP" features relevant for Security DevOps integration: »Headless operation mode / daemon »REST-API »Highly scriptable »CLI
  • 24. … enhanced with OWASP ZAP passive & active scanning place report (html,xml) 
 into workspace
  • 25. HTML report of ZAP scan
  • 26. ZAP + Jenkins = SecDevOps? "OWASP ZAP" (spider & scanner) + Jenkins plugin “ZAProxy" » Allows us to "Spider & Scan" as step in build job 
 via Jenkins plugin "ZAProxy" » Point plugin config to URL of integration system to test » Plugin saves HTML-report in project’s job for inspection » Best as separate Jenkins job to run during nightly build (duration) – Use different ZAP proxy ports for different builds to allow
 parallel execution of different project build jobs
  • 29. Arachni in SecDevOps? "Arachni Scanner" features relevant for Security DevOps integration: » Passive & active scanning (Proxy / Spider) » Uses internally a headless browser-cluster 
 (for apps with lots of JS) » Automation? – direct CLI + RPC-API (with CLI) » Web-UI (helpful when permanently 
 running as server)
  • 30. Arachni + Jenkins = SecDevOps? "Arachni Scanner" + Jenkins CLI step in build » Start in build job as CLI step and point to URL of system under test » Generate HTML report and place into workspace for inspection » Better execute within nightly build job (due to duration) » Alternatively: 
 Start remote (also via CLI) to run async
  • 31. HTML report of Arachni scan (Summary) Bildquelle: Arachni Scanner
  • 32. HTML report of Arachni scan 
 (Details) Bildquelle: Arachni Scanner
  • 33. BDD-Security in SecDevOps? BDD-based framework for functional and technical security tests: » Technical security tests 
 (i.e. check against XSS, SQL-Injection, XXE, etc.) – uses ZAP as scanning engine (among others) » Functional security tests 
 (i.e. check UserA can’t access data from UserB) » Tightly integrates with Selenium based app navigation workflows » Uses JBehave for G/W/T stories & reporting » Can run within CI (Jenkins, etc.) due to JBehave or as JUnit tests
  • 34. BDD-Security Story: Scan for XSS Bildquelle: BDD-Security
  • 35. HTML report of BDD-Security Bildquelle: BDD-Security
  • 36. Gauntlt in SecDevOps? BDD-based framework for executing many security tools/ scanners: » Integrates scanners like Arachni, ZAP, sqlmap, etc. – Easy to integrate "your custom scanner tool" with Gauntlt as well » Allows to call different scan polices via BDD-stories (G/W/T) » Integration with Jenkins (or other build servers) by either – Linking Gauntlt’s HTML report to build, or by – modifying how Gauntlt calls Cucumber to produce JUnit output
  • 37. Other dynamic scanners exist also… Some have different focus » Nikto » w3af » IronWASP » Nessus » Mittn (BDD-based) » Gauntlt (BDD-based) » Minion (integration framework) » OWTF (integration framework) » good commercial ones » etc.
  • 38. Axis "Dynamic Depth": Level 2 Scanning of authenticated parts (= "post-auth") via UI layer » Properly maintaining sessions » Logout-detection & automatic re-login » Different users / roles » Spider & scan post-auth
 Handling of hardening measures of application under test » CSRF-Tokens, etc.
  • 39. Guide ZAP into Post-Auth in CI Use ZAP manually (1x) to configure "Context": Auth, RegExps for 
 Logged-In/Out Indicators, Users etc. – save as "ZAP Session-File" (could be in code repo) – use that "Session-File" from code repo as starting point of scan 
 (loaded as ZAP session during build job). 
 Note: Current version of ZAP has a bugfix pending for loading creds from session file One can set these auth values and/or additional data via ZAP’s REST-API during each build before scan starts (from Jenkins/Maven/…) – use that to define current active session etc. during spider & scan Also Scripts in JavaScript or Zest can be registered in ZAP context 
 to programmatically give authentication to ZAP
  • 40. Login config example within ZAP
  • 41. ZAProxy Jenkins Plugin: ZAP session use
  • 42. Guide Arachni into Post-Auth Give authentication infos to Arachni (Auth, Logged-In Indicators, Users) » Use Arachni "autologin" plugin to specify via command line – Login URL, formfield names, credentials, logged-in indicator, excludes » Alternatively write custom ruby script for "login_script" plugin – Individual custom login logic possible – Logged-In indicators (RegExp) to know when to re-login 

  • 43. ./arachni 
 --plugin=autologin:
 url=https://example.com/login.action,
 parameters='j_username=foo&j_password=bar',
 check='Logout' 
 --scope-exclude-pattern=logout.action 
 https://example.com/ Login config example within Arachni (used in CI) Or individual ruby script if more custom login logic required… Eventually also --session-check-url & --session-check-pattern >> Easier to achieve Post-Auth Scan with Arachni (compared to others)
 >> Can also be ordered via CLI remote/async during build (nightly) or scheduled (WebUI)
  • 45. Guide BDD-Security into Post-Auth Use Selenium to navigate through the login process » Based on excellent integration of BDD-Security with Selenium » Separate app navigation code (Selenium) from Security testing code » Use Selenium class (that handles login) within BDD stories » Perform further spidering & active scanning (through ZAP) post-auth
  • 46. public class ShopApplicationScanHelper extends WebApplication implements ILogin { // ... integrates with BDD-Security via parent class & interface ... }
  • 47. public class ShopApplicationScanHelper extends WebApplication implements ILogin { @Override public void openLoginPage() { } @Override public void login(Credentials credentials) { } @Override public boolean isLoggedIn(String role) { }
  • 48. public class ShopApplicationScanHelper extends WebApplication implements ILogin { @Override public void openLoginPage() { driver.get(Config.getInstance().getBaseUrl() + "customer/login"); verifyTextPresent("Login"); } @Override public void login(Credentials credentials) { UserPassCredentials creds = new UserPassCredentials(credentials); driver.findElement(By.id("username")).clear(); driver.findElement(By.id("username")).sendKeys(creds.getUsername()); driver.findElement(By.id("password")).clear(); driver.findElement(By.id("password")).sendKeys(creds.getPassword()); driver.findElement(By.name("_action_login")).click(); } @Override public boolean isLoggedIn(String role) { if (driver.getPageSource().contains("My Account")) { return true; } else { return false; }
  • 49. What about CSRF-Tokens? ZAP can handle CSRF-Tokens » Can be individualised via configuration BDD-Security uses (among others) ZAP for scanning Arachni can also handle CSRF-Tokens 

  • 50. Apply targeted delta scanning: • Developer teams maintain "scope delta files" in SCM • like "what parts of the application have changed during this sprint" 
 (= URL patterns) • could’ve been recorded in proxy-based tools (like ZAP) by Devs • DAST Scanner uses it to scan only this during onCommit or nightly CI run • Feed this delta information as argument to scope definitions
 Arachni: --scope-include-pattern / --scope-exclude-pattern
 ZAP: Include/Excludes in Context (via Session-File or REST-API)
 BDD-Security: maintain as "table file" of RegExps as scope If scanning takes too long? (1/2)
  • 51. If scanning takes too long? (2/2) Train spiders about the structure of application to reduce scan redundancy and avoid scanning static resources • ZAP: use RegExp "Exclude from scanner" rules to prevent redundant scans in data-driven applications (i.e. scan only one instance of it) • Arachni: --scope-redundant-path-pattern / --scope-auto-redundant as well as --scope-exclude-pattern="logout.|.js|.css|.jpg|.png" Don’t forget to scan complete application at least on nightly or weekly run • Only apply full scan, if simple smoke-tests are green
  • 52. Axis "Dynamic Depth": Level 3 Separate scanning of different application layers / backends • Scan internal WebServices (e.g. SOAP / REST) = directly scan backends • Detect and scan parameter positions within XML, JSON, … • Scan from "within" the different application’s layers • IAST with distributed agents & instrumentation aims into that direction • At least one simple step in that direction: • Use the proxy also between your backend service calls
  • 53. Backend scans with ZAP How to achieve this with ZAP? • ZAP operates as proxy server: place it between backend calls • ZAP can inject payloads in observed XML tags/attributes & JSON fields • Capture service call traffic in integration test during CI while either A. frontend UI tests execute service backend calls indirectly, or B. executing service tests that directly access the service endpoint • Automatically scan as new requests are seen: "ATTACK Mode" Also keep an eye on an alpha-level SOAP-Scanner ZAP addon
  • 54. Backend scans with Arachni How to achieve this with Arachni? • Arachni can also operate as proxy: place it between backend calls • Use passive proxy plugin to "train" Arachni of the XML / JSON requests • New addition in v1.1 to extract XML / JSON input vectors from it • Use that collected input vector data to feed the active scan for 
 the observed requests
  • 55. Axis "Dynamic Depth": Level 4 Targeted scanning of individual forms / wizards (UI) and service layers » More individualised workflow coverage (not just simple spidering) » Business-logic compliant usage patterns & inputs – "fill shopping cart followed by checkout process" – "access backend WebServices in special order to test workflow", etc. » Custom coded security tests tailored to the application
  • 56. ZAP with special workflows (1/3) Many ways exist… The simplest one could be: 
 Re-use existing UI tests (Selenium, …) » Proxy this traffic through ZAP in “ATTACK-Mode" (in security test phase of build) » Optionally use ZAP Attack-Policies to specify/limit certain attack types
  • 57. ZAP with special workflows (2/3) A more customised handling of individual workflows can be achieved: Re-use & enhance existing "UI test code" at the desired
 workflow steps with calls to ZAP’s (REST)-API ordering attacks » Basically it’s like Unit-Test code that uses Selenium along 
 with ZAP-Calls at the proper positions in application workflow » Type of "ordered attacks" can again be defined via policies » Start ZAP as Daemon from Jenkins via plugin
  • 58. public class ShopApplicationTest { // = regular JUnit unit test @Before
 public void setup() { } @Test
 public void testShippingAddressStep() { 
 } @Test
 public void testBillingAddressStep() { } }
  • 59. public class ShopApplicationTest { // = regular JUnit unit test @Before
 public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) } @Test
 public void testShippingAddressStep() { 
 } @Test
 public void testBillingAddressStep() { } }
  • 60. public class ShopApplicationTest { // = regular JUnit unit test @Before
 public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) } @Test
 public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data 
 } @Test
 public void testBillingAddressStep() { } }
  • 61. public class ShopApplicationTest { // = regular JUnit unit test @Before
 public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) } @Test
 public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data // 4. set attack policy (types & strength) in running ZAP (API) /* 5. call ZAP (API) to actively scan the last seen URL (optionally define parameter excludes via API 
 or ZAP "input vector scripts" if custom input format) */ } @Test
 public void testBillingAddressStep() { } }
  • 62. See https://github.com/continuumsecurity/zap-webdriver 
 for a great working example of Selenium ZAP integration public class ShopApplicationTest { // = regular JUnit unit test @Before
 public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) } @Test
 public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data // 4. set attack policy (types & strength) in running ZAP (API) /* 5. call ZAP (API) to actively scan the last seen URL (optionally define parameter excludes via API 
 or ZAP "input vector scripts" if custom input format) */ } @Test
 public void testBillingAddressStep() { // same idea as above ... just continue with the pattern } }
  • 63. ZAP with special workflows (3/3) Alternatively "train" ZAP about the workflow by recording Zest scripts »Keep an eye on "Sequence Scanning" alpha-level ZAP AddOn –Still alpha-level (as of June 2015), but interesting approach
  • 64. BDD with special workflows Use Selenium to further drive BDD-Security initiated checks: » Selenium-based test code navigates application workflows » This code is integrated with BDD (via Java interfaces), so that: – BDD-Security stories can use that code to navigate 
 and generate traffic – This generated traffic will be scanned by ZAP via BDD
  • 65. public class ShopApplicationScanHelper extends WebApplication implements ILogin { // ... already covered login handling code // in BDD-Security example ... public boolean fillShoppingCartAndCheckout() { // custom Selenium (recorded) code 
 // to drive certain use cases (like shopping) ... } }
  • 66. Use this navigation method inside BDD-Security scenario: ...
 Given a new scanning session And the page flow described in the method: 
 fillShoppingCartAndCheckout is run through the proxy ...
  • 67. If no Selenium test code exists? Simply give developer teams access to ZAP to (at least) pre-seed the scanner: » Developer teams use browser to navigate app workflows while proxying – Thereby seed the ZAP session(s) with navigation nodes/workflows – Save the ZAP session(s) and check-in into SCM (Git, SVN, …) » Point the Jenkins ZAP plugin to the saved ZAP session(s) as starting point » Devs can add to this list of URLs for ZAP with each new UI BTW: ZAP is also available as Docker image…
  • 68. Sometimes it’s useful to have programmed unit-tests directly 
 calling a business API • Test directly at the API-level (i.e. Java code level) • Utilise fuzzing values from popular sources like "fuzzdb" 
 and/or generate on-the-fly with fuzzer "radamsa" • Check the API method’s response 
 (like regular unit test, but seeking for signs 
 of successful attack tests) Security-UnitTests for APIs (1/2)
  • 69. Security-UnitTests for APIs (2/2) Leverage the benefit of direct API-level testing 
 (i.e. testing "from within") • Place "probes" into the application to watch out for successful attack tests • Check some OWASP AppSensor detection points for potential attack triggers (RASP-like) • Error logs are good candidates to check also • Mock objects (Mockito, etc.) or weave aspects (AOP) collecting data (i.e. method calls and arguments, etc.)
  • 70. Axis of "Static Depth" How deep is static code analysis performed 
 within a Security DevOps CI chain?
 i.e. "where" are static 
 security tests applied?
  • 71. Axis "Static Depth": Level 1 Assurance that no third-party code with known vulnerabilities is used » Check application’s server-side dependencies – For Java applications: JAR files, etc. » Check application’s client-side dependencies – JavaScript frameworks, etc.
 Useful even for projects not currently under development » New vulnerabilities could become public: 
 Build regularly to check
  • 72. Java libs: OWASP Dependency-Check Scans all dependencies (even transitive ones) 
 against CVE list » Available as Maven plugin and Ant task » CLI version also available » Jenkins plugin for nice reporting 
 (and build breaking thresholds) » Not false-positive free – regularly needs some time to triage findings
  • 74.
  • 76. JavaScript libs: retire.js Scans application’s JavaScript files against list of known vulnerable ones » Available also as Maven Plugin … » … and as CLI 
 (point it to your folder containing .js files)
  • 77. Axis "Static Depth": Level 2 Scan important parts of source code for vulnerability patterns » At least important parts of applications’ codebases are scanned – like in-house reused code & custom developed frameworks – for big multi-project Maven projects: 
 at least frontend and backend projects – or code of changed modules within a sprint
  • 78. Java: FindSecurityBugs Plugin for FindBugs with enhanced checks for security issues in Java code » Runs within FindBugs, so that it can execute in Maven and/or Sonar » Generates XML-file with potential findings » Jenkins plugin for representing results (also possible to display in Sonar) – Integrates with Jenkins "Static Code Analysis" as part of FindBugs – Delta reporting of new (and fixed) findings & trends
  • 79. FindBugs in Jenkins: Trends, etc.
  • 80. in Sonar: Vulnerability & Code-Pointer
  • 81. JavaScript: ScanJS Scans JavaScript code for vulnerabilities » Most importantly DOM-based XSS (Cross-Site Scripting) » Flags also "Usage of sensitive API" issues » Executable as CLI from within Jenkins build steps » Optional Web-UI (running as local server)
  • 82. ScanJS Findings as HTML: DOM-XSS
  • 83. Ruby on Rails apps: Brakeman Scans RoR code for vulnerabilities » CLI based » Nicely integrates with Jenkins – Rendering scan results and trend analysis » Allows to compare scan results (delta detection) – "What has been fixed and what was newly introduced?"
  • 84. Other static scanners exist also… Some have different focus » PrimeJS (like ScanJS for JavaScript DOM-XSS etc.) » PMD (for Java code regarding API safety issues) » good commercial ones » etc.
  • 85. Axis "Static Depth": Level 3 Scan the complete applications’ source code 
 for vulnerability patterns » Just like "Level 2", but now for the complete codebase
 (no excludes) – i.e. all projects of a multi-project Maven project – best via parent or corporate pom.xml For FindSecurityBugs this means: » pre-compile even all JSPs (if you have any) 
 to scan them
  • 86. Axis "Static Depth": Level 4 Scan code of (important) third-party dependencies 
 used in application » If the component is open-source: – include it in regular scanning activities » If the component is closed-source: – consider using code scanners that operate on binaries • FindBugs (hence its FindSecurityBugs plugin) can also 
 scan Java binaries, since it operates on Bytecode
  • 87. Axis of "Intensity" How intense are the majority of the executed attacks within a Security DevOps CI chain?
 i.e. "what" is being checked for?
  • 88. Axis "Intensity": Level 1 Dynamic checks: Only passive scanning » Simply proxy testcase generated traffic through passive operating scanners » Quick-win for using existing testcase code (i.e. Service- or UI-tests) » Can execute within existing tests that execute on every commit – No requirement for a separate nightly CI job (i.e. doesn’t slow down build) 
 Static checks: Just scan the code along with other code metrics » Use the default rules of the scanners and integrate them into CI
  • 89. Axis "Intensity": Level 2 Use lightweight active scanning options for dynamic checks » ZAP: active scanning of observed URLs (proxied or spidered) – perform an "active scan": For Jenkins ZAProxy Plugin provide a ZAP policy with desired active scan intensity (or via ZAP’s REST-API during build for other setups) » Arachni: enable "active scan" during spidering » BDD-Security stories can call ZAP with policy files defining attack strength Better to execute in separate (nightly) build job due it’s to duration Think of resetting databases (+ more environment) prior or after test execution
  • 90. Axis "Intensity": Level 3 Use heavyweight scanning options on important parts of application Dynamic checks: create "riskier" custom attack profiles ZAP: – set "Threshold" to "Low" / "Strength" to "High" (or selectively "Insane") – enable more scanning rules via Policy Files (could be maintained & used via SCM) – use ZAP’s "Advanced SQLInjection Scanner" extension (or directly call sqlmap) Arachni: – Enable the more aggressive "audit flags" like --audit-forms, … Static checks: FindSecurityBugs: set "Threshold"
 to "Low" / "Effort" to "Max"
  • 91. Axis "Intensity": Level 4 Use customised rule sets for dynamic checks ZAP 
 (which is very extensible via custom scripts, recored and customised Zest, or plugins): » Custom coded scan scripts as active & passive scan rules for company-individual checks » Security regression tests (in Zest) against previous found vulns (also logic flaws) » Custom input vector scripts: e.g. define certain injection points in custom encoded params » Proxy scripts: for example for generic request/response modifications
  • 93. … and for static code scans? Use customised rule sets for static checks FindSecurityBugs » Custom coded rule files (in Java) to check for wrong usages of 
 server-side inhouse frameworks (or alike) ScanJS » Custom rule files (in JSON) to check for wrong usages of client-side inhouse frameworks (or alike)
  • 94. Axis of "Consolidation" How complete is the process of handling findings within a Security DevOps CI chain?
 i.e. "how" are the results used?
  • 95. Axis "Consolidation": Level 1 Generate human-readable (HTML) reports from tools and link them in Jenkins » All relevant mentioned static and dynamic scanners generate HTML reports » Collect and publish them in Jenkins build: via Jenkins 
 "HTML Publisher Plugin"

  • 96. Jenkins "HTML Publisher Plugin": Configuration of HTML reports to link
  • 98. HTML report of ZAP scan
  • 99. Axis "Consolidation": Level 1 (cont.) Use simple criteria to "break the build" on heavy findings 
 (ok, at least "unstable")
 » Most scanners have capabilities to automatically flag the build » Dependency-Check (Severity Threshold), » BDD-Security (with the JBehave-stories), » FindSecurityBugs (via Sonar when rated as blocker), » Arachni (via Gauntlt execution with BDD-like stories), » etc.
 » For others: at least do a simple log parse from 
 Jenkins "Log Parser Plugin" to flag the build as 
 unstable and/or broken
  • 101. FindSecurityBugs: 
 IDE support & Collaboration FindBugs (with FindSecurityBugs) has good Jenkins & IDE integration Additionally consider placing the FindBugs generated XML report file within Jenkins » Developers can load the XML report file into FindBugs IDE plugins 
 (Eclipse, IntelliJ, NetBeans, …) » Developers & Security Analysts can collaborate on bugs via the IDE plugins – See https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial 
 for more details • Consider installing it as your private "FindBugs"-Server 
 on-premise to keep results only on your servers…
  • 102. FindSecurityBugs Eclipse Perspective with Collaboration View + Remediation
  • 103. Axis "Consolidation": Level 2 Custom logic to make build unstable and/or broken depending on » Type of vulnerability (CWE or WASC or …) » Severity ranking (high risk) » Confidence level (firm vs. tentative) 
 Provide useful remediation info to developers Respect suppression mechanisms to rule out 
 false positives
  • 104. Flagging builds from reports How (from within a CI job)? » Most scanners also emit XML reports that can be parsed – Often a simple XPath count is just fine » Alternatively fetch the results by accessing the scanner’s API » Be sure to only break build with findings of 
 high severity and high confidence !!! – Less is more (when it comes to automation)… » When new check introduce many findings: 
 Baseline-Approach: Only break on new findings
  • 105. Providing useful info to Devs Lookup CWE-ID (provided by most scanners) in vulndb: » https://github.com/vulndb/data
 (currently 65+ vulns described) » Useful if the messages from scanners are too generic 
 or don’t help much » Maps vulnerability to useful remediation messages » Possibility to extend that repository locally with 
 company-individual texts (mapping to the technology 
 stack) —> that’s where it pays off…
  • 106. Suppression of false positives Custom finding list (from XML reports): 
 filter out according to list from code repo BDD-Security: Use false positive tables in story files 
 (i.e. text files checked in SCM) FindSecurityBugs: Use exclusion lists in config (XML filter files) 
 & annotation @SuppressWarnings on false positive code lines Dependency-Check: XML file of suppressions 
 (checked-in along with the project) retire.js: "ignore file" parameters on CLI
 (cloud be loaded in CI from code repo)
  • 107. Axis "Consolidation": Level 3 Consolidation goals: » Consolidate & de-duplicate findings from different 
 scanner reports (with better false positive handling) » Push consolidated findings into established bug-tracker 
 (known to devs) » Delta analysis & trends over consolidated data sets
  • 108. ThreadFix as result consolidator Use a local ThreadFix server, which imports native scanner outputs • does the heavy lifting of consolidation & de-duplication • pushes findings toward bug-tracker and IDE (via plugins) • process can be customised using it’s own REST-API • ThreadFix imports findings of ZAP, Arachni, FindBugs, Brakeman, etc.
  • 109. Report generated by ThreadFix Bildquelle: ThreadFix
  • 110. Trend Chart generated by ThreadFix Bildquelle: ThreadFix
  • 111. Axis "Consolidation": Level 4 Measure the concrete code coverage of your security testing activities » Find untested "white spots" » Derive where static checks and code reviews should focus more to compensate
  • 112. Code coverage analysis Use "OWASP Code Pulse", which instruments your Java app » collects coverage data during dynamic security testing scans » generates reports ("code treemaps") of coverage
  • 113. Code Treemap of dynamic scan coverage Bildquelle: OWASP Code Pulse
  • 114. Thank you very much!
  • 115. Links OWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project ZAP Selenium Demo https://github.com/continuumsecurity/zap-webdriver ZAP Jenkins Plugin https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+Plugin BDD-Security http://www.continuumsecurity.net/bdd-intro.html Arachni http://www.arachni-scanner.com OWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_Check OWASP Dependency Track https://www.owasp.org/index.php/OWASP_Dependency_Track_Project FindSecurityBugs http://h3xstream.github.io/find-sec-bugs/ FindSecurityBugs-Cloud https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial retire.js http://bekk.github.io/retire.js/ ScanJS https://github.com/mozilla/scanjs Jenkins Log Parser Plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin ThreadFix http://www.threadfix.org OWASP Code Pulse https://www.owasp.org/index.php/OWASP_Code_Pulse_Project Seccubus https://www.seccubus.com vulndb https://github.com/vulndb/data fuzzdb https://code.google.com/p/fuzzdb/
 radamsa https://code.google.com/p/ouspg/wiki/Radamsa
 Interested in more web security stuff? Visit my Blog: www.Christian-Schneider.net @cschneider4711 Bildquelle: dreamstime.com