Puma Scan is a software security Visual Studio analyzer extension providing real time, continuous source code analysis as development teams write code. Vulnerabilities are immediately displayed in the development environment as spell check and compiler warnings, preventing security bugs from entering your applications.
4. Free / Open Source .NET Options
• CAT.NET
• FxCop
• Visual Studio Code Analysis
• Web Config Security Analyzer
5. Widget Town Target App
• Purposely vulnerable eCommerce application
• Contains over 50 different vulnerabilities
• Across two different versions:
– Web Forms
– .NET MVC
• Contributors:
– Louis Gardina
– Eric Johnson
6. Microsoft CAT.NET v1.1
• Microsoft Code Analysis Tool (CAT)
• Promising start but fizzled quickly
• Version 1.1 published
– April 2009
• Version 2.0 beta never published
– November 2009
• https://www.microsoft.com/en-
us/download/details.aspx?id=19968
7. CAT.NET v1.1 Security Benchmark
• Widget Town scan results:
– 2 XSS, 1 Unvalidated Redirect issues
• CAT.NET is a very limited security scanner
9. Visual Studio Code Analysis
• FxCop wrapper baked into Visual Studio
• Security rules covered by the “Microsoft
Security Rules” rule set
• Custom rules can be created using the
BaseFxCopRule
• https://msdn.microsoft.com/en-
us/library/3z0aeatx(v=vs.140).aspx
10. Code Analysis Security Benchmark
• Rule target results from the “Microsoft
Security Rules” rule set
• Widget Town scan results:
– 2 SQL Injection instances, 1 is a false positive
11. • Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0
12. • Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0
14. Introducing Roslyn
• Open-source C# and Visual Basic compilers
with code analysis APIs
• Capable of producing warnings in code as you
type:
15. Getting Started
• Prerequisites:
– Visual Studio 2015
– Visual Studio 2015 Extensibility Tools
– .NET Compiler Platform ("Roslyn") SDK
• Described in detail in this MSDN Magazine
article by Alex Turner:
– https://msdn.microsoft.com/en-
us/magazine/dn879356.aspx
16. Creating a Code Analyzer Project
• File > New Project
• Templates > Visual C#
> Extensibility
• Select Analyzer with
Code Fix (NuGet +
VSIX) template
17. Roslyn Syntax Visualizer
• Included in the .NET Compiler
Platform SDK
• Facilitates inspection of a syntax
tree for any C# or VB code file open
inside Visual Studio
• Each node displays a properties grid
for the item selected in the tree
including:
– Semantics, symbols, types, values, etc.
19. Code Analyzer 101
• Roslyn exposes the following API’s to simplify
code analysis:
–DiagnosticAnalyzer
–DiagnosticDescriptor
–AnalysisContext
–SyntaxKinds
20. • Decorate the custom analyzer with the
DiagnosticAnalyzer attribute
• Inherit from the DiagnosticAnalyzer base class
Diagnostic Analyzer Class
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
//Insert awesome analyzer logic here
}
1
2
3
4
5
21. • Define the diagnostic’s id, title, message, severity,
and description
Diagnostic Descriptor Class
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10
22. • Add the diagnostic descriptor to the rule’s list of
supported diagnostics
Diagnostic Descriptor List
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10
23. • Determines when Roslyn calls back to your
analyzer code
• http://bit.ly/2dStJru
Analysis Context Events
Context Registration Options
RegisterCodeBlockAction RegisterSymbolAction
RegisterCompilationAction RegisterSyntaxNodeAction
RegisterCompilationStartAction RegisterSyntaxTreeAction
RegisterSemanticModelAction
24. • Determines the syntax nodes or symbol the
analyzers are inspecting
• Hundreds of options are available, some
commonly used items:
Symbol / Syntax Kind Options
Syntax Kinds Symbol Kinds
MethodDeclaration Event
ObjectCreationExpression Field
InvocationExpression Method
SimpleAssignmentExpression Parameter
25. • Believe it or not, this is all you need to build a
real analyzer
• WARNING: Intense Roslyn code flagging
ASP.NET Identity for weak password length
coming next!
Password Length Analyzer Example
26. • Override the Initialize method
• Register the SyntaxNodeAction event listener
• Target the ObjectCreateExpression nodes
Initializing an Analysis Context
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
[…]
public override void Initialize(AnalysisContext context)
{
context.RegisterSyntaxNodeAction(AnalyzeSyntaxNode
,SyntaxKind.ObjectCreationExpression);
}
}
1
2
3
4
5
6
7
8
9
10
27. • Retrieve the incoming object creation node
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
28. • Check the object type’s name
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
29. • Verify the symbol is in the Identity namespace
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
30. • Retrieve the initializer expressions
Identity Password Length Analyzer
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
31. • Find and read the expression’s constant value
Identity Password Length Analyzer
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
37. Additional Files
• Additional files were designed to feed
configuration data to code analyzers
– Password complexity rules, authentication
timeout values, etc.
38. Additional Files Analyzer
• But, we need to analyze and create diagnostic
warnings in non-code files
– .config, .json, .cshtml, .aspx, .ascx, etc.
• Not officially supported as of Visual Studio
2015 Update 3
• Open git issue
– https://github.com/dotnet/roslyn/issues/11097
39. Additional Files Analyzer Roadblocks
• Additional files are not automatically loaded
into the analysis context
• Creating a diagnostic with an additional file
location causes the error to disappear
40. Additional File Item Names
• Each project file targeted for analysis must set
its additional file item names property group
to all content files:
<PropertyGroup>
[…]
<AdditionalFileItemNames>
$(AdditionalFileItemNames);Content
</AdditionalFileItemNames>
</PropertyGroup>
41. Additional File Diagnostic
• Do not include the source location in
additional file diagnostics
• Workaround: leverage the message arguments
parameter to display path and line info in the
error list:
string messageFormat = "Debug compilation is enabled.
{0}({1}): {2}”;
context.ReportDiagnostic(Diagnostic.Create(Rule,
Location.None, path, lineNumber, line));
42. Additional File Analyzer Diagnostics
• Diagnostics reported on web.config
vulnerabilities in the error list:
43. Additional Files Analyzer Limitations
• Additional files are not automatically loaded
after installing the NuGet package
– Open ticket to correct this in the NuGet installer
• Manual edits required to project files when
using the extension (.vsix) installer
• Error list double click navigation is not
supported
• No spellcheck (squiggles) in non-code files
44. Demo Code Repo
• Sample analyzers from this talk are available in
git:
– https://github.com/ejohn20/puma-scan-demo
46. Introducing the Puma Scan
• Open source Visual Studio Roslyn security source
code analyzer extension
• Over 40 application security-specific rules
• Version 1.0 is available via NuGet & Visual Studio
Marketplace
• Install, rule docs, source code:
– https://www.pumascan.com
– https://github.com/pumasecurity
– @puma_scan
48. Future Enhancements
• Welcoming contributors!
• Gather feedback and address edge cases
• Continue to build out additional rule categories
– Crypto, cleartext secrets, XML processing, etc.
• Further refine results using data flow analysis to
eliminate false positives
• Identify rules that can apply suggested code fixes
49. Acknowledgements
• Eric Mead – Cypress Data Defense
• Tom Meschter – Microsoft
• Manish Vasani – Microsoft
• Gitter Rosyln Channel
50. Thank you for attending!
Email: eric.johnson@pumascan.com
Twitter: @emjohn20