Continuous Integration: Live Static Analysis with Puma Scan

Continuous Integration
Live Static Analysis with Roslyn
Eric Johnson
Twitter: @emjohn20
Senior Security Consultant
Cypress Data Defense

Eric Johnson, CISSP, GSSP, GWAPT
• Cypress Data Defense
• Senior Security Consultant
• Static code analysis
• Web & mobile app
dynamic assessments
• SDL consulting
• Tools development
– SHIM
– Puma Scan .NET
• SANS Institute
• Certified Instructor
– DEV541: Secure Coding in
Java
– DEV534: Secure DevOps
• Course Author
– DEV531: Mobile App
Security Essentials
– DEV544: Secure Coding in
.NET

Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

Free / Open Source .NET Options
• CAT.NET
• FxCop
• Visual Studio Code Analysis
• Web Config Security Analyzer

Widget Town Target App
• Purposely vulnerable eCommerce application
• Contains over 50 different vulnerabilities
• Across two different versions:
– Web Forms
– .NET MVC
• Contributors:
– Louis Gardina
– Eric Johnson

Microsoft CAT.NET v1.1
• Microsoft Code Analysis Tool (CAT)
• Promising start but fizzled quickly
• Version 1.1 published
– April 2009
• Version 2.0 beta never published
– November 2009
• https://www.microsoft.com/en-
us/download/details.aspx?id=19968

CAT.NET v1.1 Security Benchmark
• Widget Town scan results:
– 2 XSS, 1 Unvalidated Redirect issues
• CAT.NET is a very limited security scanner

FxCop
• GUI and command line binary static analysis of
dotNET code
• Rules primarily target design, naming,
performance, interoperability, globalization,
usage
• Basic security rules exist
– SQL Injection, XSS

Visual Studio Code Analysis
• FxCop wrapper baked into Visual Studio
• Security rules covered by the “Microsoft
Security Rules” rule set
• Custom rules can be created using the
BaseFxCopRule
• https://msdn.microsoft.com/en-
us/library/3z0aeatx(v=vs.140).aspx

Code Analysis Security Benchmark
• Rule target results from the “Microsoft
Security Rules” rule set
• Widget Town scan results:
– 2 SQL Injection instances, 1 is a false positive

• Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0

Introducing Roslyn
• Open-source C# and Visual Basic compilers
with code analysis APIs
• Capable of producing warnings in code as you
type:

Getting Started
• Prerequisites:
– Visual Studio 2015
– Visual Studio 2015 Extensibility Tools
– .NET Compiler Platform ("Roslyn") SDK
• Described in detail in this MSDN Magazine
article by Alex Turner:
– https://msdn.microsoft.com/en-
us/magazine/dn879356.aspx

Creating a Code Analyzer Project
• File > New Project
• Templates > Visual C#
> Extensibility
• Select Analyzer with
Code Fix (NuGet +
VSIX) template

Roslyn Syntax Visualizer
• Included in the .NET Compiler
Platform SDK
• Facilitates inspection of a syntax
tree for any C# or VB code file open
inside Visual Studio
• Each node displays a properties grid
for the item selected in the tree
including:
– Semantics, symbols, types, values, etc.

Code Analyzer 101
• Roslyn exposes the following API’s to simplify
code analysis:
– DiagnosticAnalyzer
– DiagnosticDescriptor
– AnalysisContext
– SyntaxKinds

• Decorate the custom analyzer with the
DiagnosticAnalyzer attribute
• Inherit from the DiagnosticAnalyzer base class
Diagnostic Analyzer Class
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
//Insert awesome analyzer logic here
}
1
2
3
4
5

• Define the diagnostic’s id, title, message, severity,
and description
Diagnostic Descriptor Class
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10

• Add the diagnostic descriptor to the rule’s list of
supported diagnostics
Diagnostic Descriptor List
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10

• Determines when Roslyn calls back to your
analyzer code
• http://bit.ly/2dStJru
Analysis Context Events
Context Registration Options
RegisterCodeBlockAction RegisterSymbolAction
RegisterCompilationAction RegisterSyntaxNodeAction
RegisterCompilationStartAction RegisterSyntaxTreeAction
RegisterSemanticModelAction

• Determines the syntax nodes or symbol the
analyzers are inspecting
• Hundreds of options are available, some
commonly used items:
Symbol / Syntax Kind Options
Syntax Kinds Symbol Kinds
MethodDeclaration Event
ObjectCreationExpression Field
InvocationExpression Method
SimpleAssignmentExpression Parameter

• Believe it or not, this is all you need to build a
real analyzer
• WARNING: Intense Roslyn code flagging
ASP.NET Identity for weak password length
coming next!
Password Length Analyzer Example

• Override the Initialize method
• Register the SyntaxNodeAction event listener
• Target the ObjectCreateExpression nodes
Initializing an Analysis Context
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
[…]
public override void Initialize(AnalysisContext context)
{
context.RegisterSyntaxNodeAction(AnalyzeSyntaxNode
,SyntaxKind.ObjectCreationExpression);
}
}
1
2
3
4
5
6
7
8
9
10

• Retrieve the incoming object creation node
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

• Check the object type’s name
[…]
context)
{
return;
.Symbol as ISymbol;
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

• Verify the symbol is in the Identity namespace
[…]
context)
{
return;
.Symbol as ISymbol;
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

• Retrieve the initializer expressions
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

• Find and read the expression’s constant value
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

• Minimum length requirement check
[…]
//Warn if length < 12 chars
if(minLength < 12)
{
var diagnostic = Diagnostic.Create(Rule,
statement.GetLocation());
context.ReportDiagnostic(diagnostic);
}
}
}
}
36
37
38
39
40
41
42
43
44
45
46

• Report the diagnostic to the compiler
[…]
//Warn if length < 12 chars
if(minLength < 12)
{
var diagnostic = Diagnostic.Create(Rule,
statement.GetLocation());
context.ReportDiagnostic(diagnostic);
}
}
}
}
36
37
38
39
40
41
42
43
44
45
46

• Proof that 34 lines of code can create a static
analysis rule flagging poor password
management policies

Non-Code Files
• What about non-code files?
• Security issues commonly exist in non-code
files:
– Configuration files (.config, .json)
– View markup files (.cshtml, .html, .aspx, .ascx)
– External references (.js, .css)
– Non-compiled languages (SQL, node, python, rails)

Additional Files
• Additional files were designed to feed
configuration data to code analyzers
– Password complexity rules, authentication
timeout values, etc.

Additional Files Analyzer
• But, we need to analyze and create diagnostic
warnings in non-code files
– .config, .json, .cshtml, .aspx, .ascx, etc.
• Not officially supported as of Visual Studio
2015 Update 3
• Open git issue
– https://github.com/dotnet/roslyn/issues/11097

Additional Files Analyzer Roadblocks
• Additional files are not automatically loaded
into the analysis context
• Creating a diagnostic with an additional file
location causes the error to disappear

Additional File Item Names
• Each project file targeted for analysis must set
its additional file item names property group
to all content files:
<PropertyGroup>
[…]
<AdditionalFileItemNames>
$(AdditionalFileItemNames);Content
</AdditionalFileItemNames>
</PropertyGroup>

Additional File Diagnostic
• Do not include the source location in
additional file diagnostics
• Workaround: leverage the message arguments
parameter to display path and line info in the
error list:
string messageFormat = "Debug compilation is enabled.
{0}({1}): {2}”;
context.ReportDiagnostic(Diagnostic.Create(Rule,
Location.None, path, lineNumber, line));

Additional File Analyzer Diagnostics
• Diagnostics reported on web.config
vulnerabilities in the error list:

Additional Files Analyzer Limitations
• Additional files are not automatically loaded
after installing the NuGet package
– Open ticket to correct this in the NuGet installer
• Manual edits required to project files when
using the extension (.vsix) installer
• Error list double click navigation is not
supported
• No spellcheck (squiggles) in non-code files

Demo Code Repo
• Sample analyzers from this talk are available in
git:
– https://github.com/ejohn20/puma-scan-demo

Introducing the Puma Scan
• Open source Visual Studio Roslyn security source
code analyzer extension
• Over 40 application security-specific rules
• Version 1.0 is available via NuGet & Visual Studio
Marketplace
• Install, rule docs, source code:
– https://www.pumascan.com
– https://github.com/pumasecurity
– @puma_scan

• Widget Town Puma scan results:
– 54 valid issues, 10 false positives
Puma Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 19 3
SQL Injection 2 3
Misconfiguration 16 0
Path Tampering 3 0
Unvalidated Redirect 2 4
Cross-Site Request Forgery 8 0
Poor Password Management 3 0
Certificate Validation Disabled 1 0

Future Enhancements
• Welcoming contributors!
• Gather feedback and address edge cases
• Continue to build out additional rule categories
– Crypto, cleartext secrets, XML processing, etc.
• Further refine results using data flow analysis to
eliminate false positives
• Identify rules that can apply suggested code fixes

Acknowledgements
• Eric Mead – Cypress Data Defense
• Tom Meschter – Microsoft
• Manish Vasani – Microsoft
• Gitter Rosyln Channel

Thank you for attending!
Email: eric.johnson@cypressdefense.com
Twitter: @emjohn20

Continuous Integration: Live Static Analysis with Puma Scan

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Continuous Integration: Live Static Analysis with Puma Scan

Similar to Continuous Integration: Live Static Analysis with Puma Scan (20)

Recently uploaded

Recently uploaded (20)

Continuous Integration: Live Static Analysis with Puma Scan