Secure DevOps: A Puma's Tail

Secure DevOps: A Puma’s Tail
SANS Secure DevOps Summit
Tuesday, October 10th 2017
Eric Johnson (@emjohn20)

Puma Security
• Principal Security
Engineer
• Modern static code
analysis
• DevSecOps automation
• Secure Development
Lifecycle
SANS Institute
• Certified Instructor
DEV541: Secure Coding in Java
DEV534: Secure DevOps
• Course Author
DEV531: Mobile App Security
Essentials
DEV540: Secure DevOps &
Cloud Application Security
DEV544: Secure Coding in .NET
Eric Johnson, CISSP, AWS CD, GSSP, GWAPT
©2018 – Puma Security, LLC

Roadmap
• The DevOps Problem
• Security Unit / Integration Testing
• Static Analysis Options
• Pre-Commit
• Continuous Integration
• Conclusion

Case Study | Travel Industry Breaches
• Airline Company
• 5,000 employees
• 50 .NET C# applications
• 20 software engineers
• 0 application security engineers
• 1 deployment / week

• Continuous Integration via Jenkins
• Continuous Delivery via Jenkins pipeline plugin
Case Study | State of DevOps

• External vendor performing
annual code assessments
• Internal security team receives
1,000 page PDF report
• Internal security team performs
dynamic pen testing, fuzzing,
etc.
Case Study | State of Security

• Security was not invited to the DevOps party
• Internal security team does not have
development background
• Frequent deployments invalidate assessment
results
• Missing a huge opportunity for app sec in the
pipeline
Case Study | The Problem

• Published
October 2016
• Release
frequency up 30x
• Silos still exist
between Sec and
DevOps
HPE | AppSec & DevOps Survey
20%
38%
25%
17%
Security in DevOps
SecDevOps Gated Reviews Network Defenses Nothing

What is SecDevOps
SecDevOps / DevSecOps / DevOpsSec is about
breaking down walls between security and:
• Development
• Operations
• Business

SecDevOps Security Controls
PRE-COMMIT COMMIT ACCEPTANCE DEPLOYMENT
THREAT MODELING
IDE SAST
CODE REVIEWS
SECURITY UNIT TESTS
CI SAST
HIGH RISK CODE
ALERTS
SECURITY ACCEPTANCE
TESTS
CI DAST
AUTOMATED SECURITY
ATTACKS
AUTOMATE
DEPLOYMENT
SECURITY
MONITORING
SECURITY SMOKE
TESTING
SECURITY STORIES
• Security controls in a Continuous Integration
(CI) / Continuous Delivery (CD) pipeline:
SUPPLY CHAIN SCANS MANUAL PEN TESTING VIRTUAL PATCHING

SecDevOps Quick Wins
PRE-COMMIT COMMIT ACCEPTANCE DEPLOYMENT
THREAT MODELING
IDE SAST
CODE REVIEWS
CI SAST
HIGH RISK CODE
ALERTS
CI DAST
AUTOMATED SECURITY
ATTACKS
AUTOMATE
DEPLOYMENT
SECURITY
MONITORING
SECURITY SMOKE
TESTING
• Narrowing the scope and identifying some
quick wins for our case study:
SUPPLY CHAIN SCANS MANUAL PEN TESTING VIRTUAL PATCHING
SECURITY UNIT TESTS
SECURITY ACCEPTANCE
TESTS
SECURITY STORIES

Built on top of standard unit and integration tests to
enforce security requirements:
Security Unit / Integration Testing
SECURITY STORIES
UNIT / ACCEPTANCE
TESTING
PRE-COMMIT
COMMIT
• Create abuse cases and evil user
stories
• Focus on high risk code and business
logic flaws
• Fast execution in the IDE / CI pipeline

• Engineers often stay on the "happy path"
• Prove the code works under normal usage
• Example: positive validation testing for a
normal user's name
The Happy Path
[Fact]
public void NameValidationPositiveTest()
{
bool isValid = Validator.IsNameValid("Bobby Tables");
Assert.Equal(isValid, true);
}
1
2
3
4
5
6

• Security works with engineers to write test cases
• Prove the code is secure under abnormal usage
• Example: Negative validation w/ evil injection
characters
Evil Security Stories
[Fact]
public void ValidateNameNegativeTest()
{
string EVIL = "&<>"()|;!=~*/{}#";
foreach (char s in EVIL.ToArray())
Assert.Equal(Validator.IsNameValid(s.ToString()), false);
}
1
2
3
4
5
6
7

Security identifies high risk code performing the
following functionality:
• Authentication
• Access control
• Output encoding
• Input validation
• High risk business logic
• Data entitlement checks
• Handling confidential data
• Cryptography
High Risk Code Examples

• Automated and configurable security unit
testing framework written by @sethlaw
• Payload lists for XSS and Injection
SQLi, NoSQL, LDAP, XML, XPath, OS vulnerabilities
• Supports Java Spring, ASP.NET MVC, and
Django
• https://github.com/sethlaw/sputr
Security Payload Unit Test Repository (SPUTR)

Limited opportunity for static analysis in CI & CD
pipelines:
SecDevOps Static Analysis
IDE SAST
CI SAST
PRE-COMMIT
COMMIT
• Speed matters (< 10 minutes)
• High accuracy rules
• Low false positive rates

Free / Open Source .NET Options
• CAT.NET
• FxCop
• Visual Studio Code Analysis
• Web Config Security Analyzer
• Custom Roslyn Analyzers

• Purposely vulnerable eCommerce application
• Contains over 50 different vulnerabilities
• Across two different versions:
• Web Forms
• .NET MVC 5
• Contributors:
• Louis Gardina
• Eric Johnson
The Target

CAT.NET v1.1 Security Benchmark
• Widget Town scan results:
• 2 XSS, 1 Unvalidated Redirect issues
• CAT.NET is a very limited security scanner

FxCop / Code Analysis Security Benchmark
• Rule target results from the “Microsoft
Security Rules” rule set
• Widget Town scan results:
• 2 SQL Injection instances, 1 is a false positive

• Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0

Introducing Roslyn
• Open-source C# and VB compilers with code
analysis APIs
• Capable of producing warnings in source code
as you type:

Roslyn Diagnostic Warnings
• Roslyn diagnostics are also reported during
MSBuild compilation:

Session recorded at OWASP AppSec USA 2016:
• Continuous Integration: Live Code Analysis
using Visual Studio and the Roslyn API
https://youtube.com/watch?v=Y8JKVjY-7T0
• Demonstration analyzers from the
presentation:
https://github.com/ejohn20/puma-scan-demo
Building Security Analyzers 101

• Open source security source code analyzer built using
Roslyn
• 50+ application security-specific rules
• Version 1.0.6 is available via NuGet & VS Marketplace
• Install guide, rule docs, source code:
https://www.pumascan.com
https://github.com/pumasecurity
@puma_scan
Introducing the Puma Scan

• SQLi via an EF Command
Creating a Puma Analyzer
[…]
public void Delete(string id)
{
using (WidgetTownDBEntities context = new WidgetTownDBEntities())
{
string q = string.Format("DELETE FROM Contests WHERE Id = {0}", id);
context.Database.ExecuteSqlCommand(q);
}
}
36
37
38
39
40
41
42
43
44
45

• How can we search for this vulnerability?
• Which interface?
ISyntaxAnalyzer
Implementing the Interface
public interface ISyntaxAnalyzer
{
SyntaxKind SinkKind { get; }
ConcurrentStack<VulnerableSyntaxNode> VulnerableSyntaxNodes { get; }
void GetSinks(SyntaxNodeAnalysisContext context);
}
10
11
12
13
14
15
16
17
19

• Syntax Kind Enum
SimpleAssignmentExpression, MemberAccessExpression,
InvocationExpression, ObjectCreationExpression
TrueKeyword, LiteralTextExpression, PlusToken
Choosing the Syntax Kind
[…]
using (WidgetTownDBEntities context = new WidgetTownDBEntities())
{
string q = string.Format("DELETE FROM Contests WHERE Id = {0}", id);
context.Database.ExecuteSqlCommand(q);
}
23
24
25
26
27
28
29

• Configure the analyzer to only inspect InvocationExpression
types
Choosing the Syntax Kind
[…]
public SyntaxKind SinkKind
{
get { return SyntaxKind.InvocationExpression; }
}
[…]
23
24
25
26
27
28
29

• Tread lightly at first – weed out nodes cheaply
Finding the Sink | Cheap Check
[…]
public void GetSinks(SyntaxNodeAnalysisContext context)
{
var syntax = context.Node as InvocationExpressionSyntax;
if (!syntax.ToString().Contains("ExecuteSqlCommand"))
return;
[…]
36
37
38
39
40
41
42
43

• …then a little more aggressive
Finding the Sink | Symbol Check
[…]
{
[…]
var symbol = context.SemanticModel
.GetSymbolInfo(syntax)
.Symbol as IMethodSymbol;
if(!symbol.IsMethod("System.Data.Entity.Database"
,"ExecuteSqlCommand"))
return;
[…]
}
43
44
45
52
53
54
55
56
57
58
62

• Symbol Types
Method symbol
Property symbol
Field symbol
Many more
Finding the Sink | Symbol Type
[…]
var symbol = context.SemanticModel
.GetSymbolInfo(syntax)
.Symbol as IMethodSymbol;
[…]
43
44
45
46
47

• Add the sink to the vulnerable collection
Vulnerable Sink Collection
[…]
{
[…]
if (VulnerableSyntaxNodes.All(p =>
p.Sink.GetLocation() != syntax?.GetLocation()))
{
VulnerableSyntaxNodes.Push(new VulnerableSyntaxNode(syntax));
}
}
55
56
57
58
59
60
61
62
63
64
65
66

• Tell Rosyln what diagnostic to report for this
Supported Diagnostic
[SupportedDiagnostic(
DiagnosticId.SEC0108,
DiagnosticSeverity.Warning,
DiagnosticCategory.Security)]
public class EfExecuteSqlCommandInjectionAnalyzer : ISyntaxAnalyzer
{
[…]
}
10
11
12
13
14
15
16
50
51
52

• Add informative diagnostic description
• Diagnostic Analyzers reporting method plays nicely
with localized strings and resource files.
• Uses default Resouces.resx file
Supported Diagnostic Descriptors

• Suite is a logical grouping of analyzer
• Suite base classes handle the registering of analyzers
and reporting of the results
• Examples of groupings
SQL Injection
Cross site scripting
Path tampering
Open redirects
Analyzer Suites

• Add analyzer to the array of analyzers
Adding an Analyzer to the Suite
[DiagnosticAnalyzer(LanguageNames.CSharp, LanguageNames.VisualBasic)]
public class SqlInjectionDiagnosticSuite : BaseSyntaxDiagnosticSuite
{
public SqlInjectionDiagnosticSuite()
{
Analyzers = new ISyntaxAnalyzer[]{
new EfExecuteSqlCommandInjectionAnalyzer(),
new LinqSqlInjectionAnalyzer(),
new SqlCommandInjectionAnalyzer()
}.ToImmutableArray();
}
}
10
11
12
13
14
15
16
17
18
19
21
22
23
24

• Widget Town Puma scan results:
56 valid issues, 10 false positives
Puma Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 19 3
SQL Injection 2 3
Misconfiguration 16 0
Path Tampering 3 0
Unvalidated Redirect 2 4
Cross-Site Request Forgery 8 0
Poor Password Management 3 0
Certificate Validation Disabled 1 0

Requirements for static scanning in the IDE.
q Display vulnerabilities inside the IDE
q Provide documentation on how to fix the issue
q Allow engineers to suppress false positives
IDE Static Analysis Checklist

q Display vulnerabilities inside the IDE

q Provide documentation on how to fix the issue

q Allow engineers to suppress false positives

Requirements for static scanning in the CI pipeline:
q Pipeline executes accurate static analysis rules
q Process and capture results in pipeline
q Configure build thresholds to mark builds as
unhealthy or failed
q Notify security when issues are suppressed by
engineers
CI Static Analysis Checklist

q Pipeline executes accurate static analysis rules

q Process and capture results in pipeline

q Configure build thresholds to mark builds as
unhealthy or failed

• Welcoming contributions!
• Gather feedback and address edge cases
• Continue to build out additional rule categories:
Cleartext secrets, insecure XML processing, etc.
• Further refine results using data flow analysis to
eliminate false positives
• Identify rules that can apply suggested code fixes
Puma Scan | Future Enhancements

• Scott Sauber - Puma Security Engineer
• James Kashevos – CDD Security Engineer
• Josh Brown-White - Microsoft
• Tom Meschter – Microsoft
• Manish Vasani – Microsoft
• Gitter Rosyln Channel
Acknowledgements

Questions?
Contact Us:
eric.johnson@pumascan.com
@emjohn20
eric.mead@pumascan.com
@ericmmead

Secure DevOps: A Puma's Tail

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Secure DevOps: A Puma's Tail

Similar to Secure DevOps: A Puma's Tail (20)

Recently uploaded

Recently uploaded (20)

Secure DevOps: A Puma's Tail