This document discusses using the Roslyn compiler API to build .NET static code analyzers. It begins with an overview of existing free and open source .NET static analysis tools. It then covers the basics of the Roslyn API and how to create a code analyzer that checks for weak password lengths in ASP.NET Identity. It also discusses challenges with analyzing non-code files and demonstrates a tool called Puma Scan that contains over 40 security rules for .NET applications. The document encourages contributions to help expand analysis capabilities and rule coverage.

1. Continuous Integration
Live Static Analysis with Roslyn
Eric Johnson
Twitter: @emjohn20
Senior Security Consultant
Cypress Data Defense

2. Eric Johnson, CISSP, GSSP, GWAPT
• Cypress Data Defense
• Senior Security Consultant
• Static code analysis
• Web & mobile app
dynamic assessments
• SDL consulting
• Tools development
– SHIM
– Puma Scan .NET
• SANS Institute
• Certified Instructor
– DEV541: Secure Coding in
Java
– DEV534: Secure DevOps
• Course Author
– DEV531: Mobile App
Security Essentials
– DEV544: Secure Coding in
.NET

3. Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

4. Free / Open Source .NET Options
• CAT.NET
• FxCop
• Visual Studio Code Analysis
• Web Config Security Analyzer

5. Widget Town Target App
• Purposely vulnerable eCommerce application
• Contains over 50 different vulnerabilities
• Across two different versions:
– Web Forms
– .NET MVC
• Contributors:
– Louis Gardina
– Eric Johnson

6. Microsoft CAT.NET v1.1
• Microsoft Code Analysis Tool (CAT)
• Promising start but fizzled quickly
• Version 1.1 published
– April 2009
• Version 2.0 beta never published
– November 2009
• https://www.microsoft.com/en-
us/download/details.aspx?id=19968

7. CAT.NET v1.1 Security Benchmark
• Widget Town scan results:
– 2 XSS, 1 Unvalidated Redirect issues
• CAT.NET is a very limited security scanner

8. FxCop
• GUI and command line binary static analysis of
dotNET code
• Rules primarily target design, naming,
performance, interoperability, globalization,
usage
• Basic security rules exist
– SQL Injection, XSS

9. Visual Studio Code Analysis
• FxCop wrapper baked into Visual Studio
• Security rules covered by the “Microsoft
Security Rules” rule set
• Custom rules can be created using the
BaseFxCopRule
• https://msdn.microsoft.com/en-
us/library/3z0aeatx(v=vs.140).aspx

10. Code Analysis Security Benchmark
• Rule target results from the “Microsoft
Security Rules” rule set
• Widget Town scan results:
– 2 SQL Injection instances, 1 is a false positive

11. • Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0

12. • Widget Town combined CAT.NET and VS Code
analysis scan results:
Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 2 0
SQL Injection 1 1
Unvalidated Redirect 1 0

13. Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

14. Introducing Roslyn
• Open-source C# and Visual Basic compilers
with code analysis APIs
• Capable of producing warnings in code as you
type:

15. Getting Started
• Prerequisites:
– Visual Studio 2015
– Visual Studio 2015 Extensibility Tools
– .NET Compiler Platform ("Roslyn") SDK
• Described in detail in this MSDN Magazine
article by Alex Turner:
– https://msdn.microsoft.com/en-
us/magazine/dn879356.aspx

16. Creating a Code Analyzer Project
• File > New Project
• Templates > Visual C#
> Extensibility
• Select Analyzer with
Code Fix (NuGet +
VSIX) template

17. Roslyn Syntax Visualizer
• Included in the .NET Compiler
Platform SDK
• Facilitates inspection of a syntax
tree for any C# or VB code file open
inside Visual Studio
• Each node displays a properties grid
for the item selected in the tree
including:
– Semantics, symbols, types, values, etc.

18. Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

19. Code Analyzer 101
• Roslyn exposes the following API’s to simplify
code analysis:
– DiagnosticAnalyzer
– DiagnosticDescriptor
– AnalysisContext
– SyntaxKinds

20. • Decorate the custom analyzer with the
DiagnosticAnalyzer attribute
• Inherit from the DiagnosticAnalyzer base class
Diagnostic Analyzer Class
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
//Insert awesome analyzer logic here
}
1
2
3
4
5

21. • Define the diagnostic’s id, title, message, severity,
and description
Diagnostic Descriptor Class
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10

22. • Add the diagnostic descriptor to the rule’s list of
supported diagnostics
Diagnostic Descriptor List
[…]
private static DiagnosticDescriptor Rule =
new DiagnosticDescriptor(Id, Title, MessageFormat,
Category, DiagnosticSeverity.Warning,
isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor>
SupportedDiagnostics {
get { return ImmutableArray.Create(Rule); }
}
1
2
3
4
5
6
7
8
9
10

23. • Determines when Roslyn calls back to your
analyzer code
• http://bit.ly/2dStJru
Analysis Context Events
Context Registration Options
RegisterCodeBlockAction RegisterSymbolAction
RegisterCompilationAction RegisterSyntaxNodeAction
RegisterCompilationStartAction RegisterSyntaxTreeAction
RegisterSemanticModelAction

24. • Determines the syntax nodes or symbol the
analyzers are inspecting
• Hundreds of options are available, some
commonly used items:
Symbol / Syntax Kind Options
Syntax Kinds Symbol Kinds
MethodDeclaration Event
ObjectCreationExpression Field
InvocationExpression Method
SimpleAssignmentExpression Parameter

25. • Believe it or not, this is all you need to build a
real analyzer
• WARNING: Intense Roslyn code flagging
ASP.NET Identity for weak password length
coming next!
Password Length Analyzer Example

26. • Override the Initialize method
• Register the SyntaxNodeAction event listener
• Target the ObjectCreateExpression nodes
Initializing an Analysis Context
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class MyAwesomeAnalyzer : DiagnosticAnalyzer
{
[…]
public override void Initialize(AnalysisContext context)
{
context.RegisterSyntaxNodeAction(AnalyzeSyntaxNode
,SyntaxKind.ObjectCreationExpression);
}
}
1
2
3
4
5
6
7
8
9
10

27. • Retrieve the incoming object creation node
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

28. • Check the object type’s name
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

29. • Verify the symbol is in the Identity namespace
Identity Password Length Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext
context)
{
var statement=context.Node as ObjectCreationExpressionSyntax;
if(string.Compare(statement?.Type.ToString()
, "PasswordValidator", StringComparison.Ordinal) != 0)
return;
var symbol = context.SemanticModel.GetSymbolInfo(statement)
.Symbol as ISymbol;
if (string.Compare(symbol?.ContainingNamespace.ToString()
,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0)
return;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

30. • Retrieve the initializer expressions
Identity Password Length Analyzer
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

31. • Find and read the expression’s constant value
Identity Password Length Analyzer
[…]
var initializer = statement.Initializer as
InitializerExpressionSyntax;
if (initializer?.Expressions.Count == 0)
return;
int minLength = 0;
foreach (AssignmentExpressionSyntax expression in
initializer.Expressions)
{
var value = context.SemanticModel.GetConstantValue
(expression.Right);
if (value.HasValue &&
expression.Left.ToString().Equals("RequiredLength"))
minLength = (int)value.Value;
}
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

32. • Minimum length requirement check
Identity Password Length Analyzer
[…]
//Warn if length < 12 chars
if(minLength < 12)
{
var diagnostic = Diagnostic.Create(Rule,
statement.GetLocation());
context.ReportDiagnostic(diagnostic);
}
}
}
}
36
37
38
39
40
41
42
43
44
45
46

33. • Report the diagnostic to the compiler
Identity Password Length Analyzer
[…]
//Warn if length < 12 chars
if(minLength < 12)
{
var diagnostic = Diagnostic.Create(Rule,
statement.GetLocation());
context.ReportDiagnostic(diagnostic);
}
}
}
}
36
37
38
39
40
41
42
43
44
45
46

34. • Proof that 34 lines of code can create a static
analysis rule flagging poor password
management policies
Identity Password Length Analyzer

35. Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

36. Non-Code Files
• What about non-code files?
• Security issues commonly exist in non-code
files:
– Configuration files (.config, .json)
– View markup files (.cshtml, .html, .aspx, .ascx)
– External references (.js, .css)
– Non-compiled languages (SQL, node, python, rails)

37. Additional Files
• Additional files were designed to feed
configuration data to code analyzers
– Password complexity rules, authentication
timeout values, etc.

38. Additional Files Analyzer
• But, we need to analyze and create diagnostic
warnings in non-code files
– .config, .json, .cshtml, .aspx, .ascx, etc.
• Not officially supported as of Visual Studio
2015 Update 3
• Open git issue
– https://github.com/dotnet/roslyn/issues/11097

39. Additional Files Analyzer Roadblocks
• Additional files are not automatically loaded
into the analysis context
• Creating a diagnostic with an additional file
location causes the error to disappear

40. Additional File Item Names
• Each project file targeted for analysis must set
its additional file item names property group
to all content files:
<PropertyGroup>
[…]
<AdditionalFileItemNames>
$(AdditionalFileItemNames);Content
</AdditionalFileItemNames>
</PropertyGroup>

41. Additional File Diagnostic
• Do not include the source location in
additional file diagnostics
• Workaround: leverage the message arguments
parameter to display path and line info in the
error list:
string messageFormat = "Debug compilation is enabled.
{0}({1}): {2}”;
context.ReportDiagnostic(Diagnostic.Create(Rule,
Location.None, path, lineNumber, line));

42. Additional File Analyzer Diagnostics
• Diagnostics reported on web.config
vulnerabilities in the error list:

43. Additional Files Analyzer Limitations
• Additional files are not automatically loaded
after installing the NuGet package
– Open ticket to correct this in the NuGet installer
• Manual edits required to project files when
using the extension (.vsix) installer
• Error list double click navigation is not
supported
• No spellcheck (squiggles) in non-code files

44. Demo Code Repo
• Sample analyzers from this talk are available in
git:
– https://github.com/ejohn20/puma-scan-demo

45. Roadmap
• .NET Static Analysis Options
• The Roslyn API
• Code Analyzer
• Additional Files Analyzer
• Puma Scan
• Future Enhancements

46. Introducing the Puma Scan
• Open source Visual Studio Roslyn security source
code analyzer extension
• Over 40 application security-specific rules
• Version 1.0 is available via NuGet & Visual Studio
Marketplace
• Install, rule docs, source code:
– https://www.pumascan.com
– https://github.com/pumasecurity
– @puma_scan

47. • Widget Town Puma scan results:
– 54 valid issues, 10 false positives
Puma Scan Result Summary
Category Valid False Positive
Cross-Site Scripting 19 3
SQL Injection 2 3
Misconfiguration 16 0
Path Tampering 3 0
Unvalidated Redirect 2 4
Cross-Site Request Forgery 8 0
Poor Password Management 3 0
Certificate Validation Disabled 1 0

48. Future Enhancements
• Welcoming contributors!
• Gather feedback and address edge cases
• Continue to build out additional rule categories
– Crypto, cleartext secrets, XML processing, etc.
• Further refine results using data flow analysis to
eliminate false positives
• Identify rules that can apply suggested code fixes

49. Acknowledgements
• Eric Mead – Cypress Data Defense
• Tom Meschter – Microsoft
• Manish Vasani – Microsoft
• Gitter Rosyln Channel

50. Thank you for attending!
Email: eric.johnson@cypressdefense.com
Twitter: @emjohn20