Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Continuous	Integration
Live	Static	Analysis	with	Roslyn
Eric	Johnson	
Twitter:	@emjohn20
Senior	Security	Consultant
Cypres...
Eric	Johnson,	CISSP,	GSSP,	GWAPT
• Cypress	Data	Defense
• Senior	Security	Consultant
• Static	code	analysis
• Web	&	mobile...
Roadmap
• .NET	Static	Analysis	Options
• The	Roslyn	API
• Code	Analyzer
• Additional	Files	Analyzer
• Puma	Scan
• Future	E...
Free	/	Open	Source	.NET	Options
• CAT.NET
• FxCop
• Visual	Studio	Code	Analysis
• Web	Config Security	Analyzer
Widget	Town	Target	App
• Purposely	vulnerable	eCommerce application
• Contains	over	50	different	vulnerabilities
• Across	...
Microsoft	CAT.NET v1.1
• Microsoft	Code	Analysis	Tool	(CAT)
• Promising	start	but	fizzled	quickly
• Version	1.1	published
...
CAT.NET v1.1	Security	Benchmark
• Widget	Town	scan	results:
– 2	XSS,	1	Unvalidated	Redirect	issues
• CAT.NET is	a	very	lim...
FxCop
• GUI	and	command	line	binary	static	analysis	of	
dotNET code
• Rules	primarily target	design,	naming,	
performance,...
Visual	Studio	Code	Analysis
• FxCop wrapper	baked	into	Visual	Studio
• Security	rules	covered	by	the	“Microsoft	
Security	...
Code	Analysis	Security	Benchmark
• Rule	target	results	from	the	“Microsoft	
Security	Rules”	rule	set
• Widget	Town	scan	re...
• Widget	Town	combined	CAT.NET and	VS	Code	
analysis	scan	results:
Scan	Result	Summary
Category Valid False	Positive
Cross...
• Widget	Town	combined	CAT.NET and	VS	Code	
analysis	scan	results:
Scan	Result	Summary
Category Valid False	Positive
Cross...
Roadmap
• .NET	Static	Analysis	Options
• The	Roslyn	API
• Code	Analyzer
• Additional	Files	Analyzer
• Puma	Scan
• Future	E...
Introducing	Roslyn	
• Open-source	C#	and	Visual	Basic	compilers	
with	code	analysis	APIs
• Capable	of	producing	warnings	i...
Getting	Started
• Prerequisites:
– Visual	Studio	2015
– Visual	Studio	2015	Extensibility	Tools
– .NET	Compiler	Platform	("...
Creating	a	Code	Analyzer	Project
• File	>	New	Project
• Templates	>	Visual	C#	
>	Extensibility
• Select	Analyzer	with	
Cod...
Roslyn	Syntax	Visualizer
• Included	in	the	.NET	Compiler	
Platform	SDK
• Facilitates	inspection	of	a	syntax	
tree	for	any	...
Roadmap
• .NET	Static	Analysis	Options
• The	Roslyn	API
• Code	Analyzer
• Additional	Files	Analyzer
• Puma	Scan
• Future	E...
Code	Analyzer	101
• Roslyn	exposes	the	following	API’s	to	simplify	
code	analysis:
– DiagnosticAnalyzer
– DiagnosticDescri...
• Decorate	the	custom	analyzer	with	the	
DiagnosticAnalyzer attribute
• Inherit	from	the	DiagnosticAnalyzer base	class
Dia...
• Define	the	diagnostic’s	id,	title,	message,	severity,	
and	description
Diagnostic	Descriptor	Class
[…]
private static Di...
• Add	the	diagnostic	descriptor	to	the	rule’s	list	of	
supported	diagnostics
Diagnostic	Descriptor	List
[…]
private static...
• Determines	when	Roslyn	calls	back	to	your	
analyzer	code
• http://bit.ly/2dStJru
Analysis	Context	Events
Context	Registr...
• Determines	the	syntax	nodes	or	symbol the	
analyzers	are	inspecting
• Hundreds	of	options	are	available,	some	
commonly	...
• Believe	it	or	not,	this	is	all	you	need	to	build	a	
real	analyzer
• WARNING:	Intense	Roslyn	code	flagging	
ASP.NET	Ident...
• Override	the	Initialize	method	
• Register	the	SyntaxNodeAction event	listener
• Target	the	ObjectCreateExpression nodes...
• Retrieve	the	incoming	object	creation	node
Identity	Password	Length	Analyzer
[…]
private static void AnalyzeSyntaxNode(S...
• Check	the	object	type’s	name
Identity	Password	Length	Analyzer
[…]
private static void AnalyzeSyntaxNode(SyntaxNodeAnaly...
• Verify	the	symbol	is	in	the	Identity	namespace
Identity	Password	Length	Analyzer
[…]
private static void AnalyzeSyntaxNo...
• Retrieve	the	initializer	expressions
Identity	Password	Length	Analyzer
[…]
var initializer = statement.Initializer as
In...
• Find	and	read	the	expression’s	constant	value
Identity	Password	Length	Analyzer
[…]
var initializer = statement.Initiali...
• Minimum	length	requirement	check
Identity	Password	Length	Analyzer
[…]
//Warn if length < 12 chars
if(minLength < 12)
{
...
• Report	the	diagnostic	to	the	compiler
Identity	Password	Length	Analyzer
[…]
//Warn if length < 12 chars
if(minLength < 1...
• Proof	that	34	lines	of	code	can	create	a	static	
analysis	rule	flagging	poor	password	
management	policies
Identity	Pass...
Roadmap
• .NET	Static	Analysis	Options
• The	Roslyn	API
• Code	Analyzer
• Additional	Files	Analyzer
• Puma	Scan
• Future	E...
Non-Code	Files
• What	about	non-code	files?
• Security	issues	commonly	exist	in	non-code	
files:
– Configuration	files	(.c...
Additional	Files
• Additional	files	were	designed	to	feed	
configuration	data	to	code	analyzers
– Password	complexity	rule...
Additional	Files	Analyzer
• But,	we	need	to	analyze	and	create	diagnostic	
warnings	in	non-code	files
– .config,	.json,	.c...
Additional	Files	Analyzer	Roadblocks
• Additional	files	are	not	automatically	loaded	
into	the	analysis	context
• Creating...
Additional	File	Item	Names
• Each	project	file	targeted	for	analysis	must	set	
its	additional	file	item	names	property	gro...
Additional	File	Diagnostic
• Do	not	include	the	source	location	in	
additional	file	diagnostics
• Workaround:	leverage	the...
Additional	File	Analyzer	Diagnostics
• Diagnostics	reported	on	web.config
vulnerabilities	in	the	error	list:
Additional	Files	Analyzer	Limitations
• Additional	files	are	not	automatically	loaded	
after	installing	the	NuGet package
...
Demo	Code	Repo
• Sample	analyzers	from	this	talk	are	available	in	
git:
– https://github.com/ejohn20/puma-scan-demo
Roadmap
• .NET	Static	Analysis	Options
• The	Roslyn	API
• Code	Analyzer
• Additional	Files	Analyzer
• Puma	Scan
• Future	E...
Introducing	the	Puma	Scan
• Open	source	Visual	Studio	Roslyn	security	source	
code	analyzer	extension
• Over	40	applicatio...
• Widget	Town	Puma	scan	results:
– 54	valid	issues,	10	false	positives
Puma	Scan	Result	Summary
Category Valid False	Posit...
Future	Enhancements
• Welcoming	contributors!
• Gather	feedback	and	address	edge	cases
• Continue	to	build	out	additional	...
Acknowledgements
• Eric	Mead	– Cypress	Data	Defense
• Tom	Meschter – Microsoft
• Manish	Vasani – Microsoft
• Gitter Rosyln...
Thank	you	for	attending!
Email:	eric.johnson@cypressdefense.com
Twitter:	@emjohn20
Upcoming SlideShare
Loading in …5
×

Continuous Integration: Live Static Analysis with Puma Scan

960 views

Published on

Slides from OWASP AppSec USA 2016.

For over 10 years, Visual Studio has provided basic source code analysis through FxCop and StyleCop. While these code analyzers focus mainly on design conformance, code consistency, and best practices, there is very little support for enforcing secure coding techniques. To address this gap, Microsoft started a project back in 2011 called CAT.NET to help identify secure coding bugs such as XSS, SQL Injection, and XPath Injection. Unfortunately, CAT.NET failed and never made it past the first version. Aside from purchasing expensive commercial static analysis tools, .NET development teams have been left without an easy way to integrate secure code scanning rules into Visual Studio. At least until now...

With the release of Visual Studio 2015, the open-source .NET Compiler Platform (aka “Roslyn”) exposes a set of code analysis APIs capable of querying the source code, identifying a security issue, and reporting it as code is written! In this talk, we will explore the code analysis APIs and show you how to create a live static analysis rule. Come prepared to see demonstrations of Visual Studio static analysis rules in action, and walk away with a static analysis rule pack to run against your organization’s .NET applications.

Published in: Technology
  • Be the first to comment

Continuous Integration: Live Static Analysis with Puma Scan

  1. 1. Continuous Integration Live Static Analysis with Roslyn Eric Johnson Twitter: @emjohn20 Senior Security Consultant Cypress Data Defense
  2. 2. Eric Johnson, CISSP, GSSP, GWAPT • Cypress Data Defense • Senior Security Consultant • Static code analysis • Web & mobile app dynamic assessments • SDL consulting • Tools development – SHIM – Puma Scan .NET • SANS Institute • Certified Instructor – DEV541: Secure Coding in Java – DEV534: Secure DevOps • Course Author – DEV531: Mobile App Security Essentials – DEV544: Secure Coding in .NET
  3. 3. Roadmap • .NET Static Analysis Options • The Roslyn API • Code Analyzer • Additional Files Analyzer • Puma Scan • Future Enhancements
  4. 4. Free / Open Source .NET Options • CAT.NET • FxCop • Visual Studio Code Analysis • Web Config Security Analyzer
  5. 5. Widget Town Target App • Purposely vulnerable eCommerce application • Contains over 50 different vulnerabilities • Across two different versions: – Web Forms – .NET MVC • Contributors: – Louis Gardina – Eric Johnson
  6. 6. Microsoft CAT.NET v1.1 • Microsoft Code Analysis Tool (CAT) • Promising start but fizzled quickly • Version 1.1 published – April 2009 • Version 2.0 beta never published – November 2009 • https://www.microsoft.com/en- us/download/details.aspx?id=19968
  7. 7. CAT.NET v1.1 Security Benchmark • Widget Town scan results: – 2 XSS, 1 Unvalidated Redirect issues • CAT.NET is a very limited security scanner
  8. 8. FxCop • GUI and command line binary static analysis of dotNET code • Rules primarily target design, naming, performance, interoperability, globalization, usage • Basic security rules exist – SQL Injection, XSS
  9. 9. Visual Studio Code Analysis • FxCop wrapper baked into Visual Studio • Security rules covered by the “Microsoft Security Rules” rule set • Custom rules can be created using the BaseFxCopRule • https://msdn.microsoft.com/en- us/library/3z0aeatx(v=vs.140).aspx
  10. 10. Code Analysis Security Benchmark • Rule target results from the “Microsoft Security Rules” rule set • Widget Town scan results: – 2 SQL Injection instances, 1 is a false positive
  11. 11. • Widget Town combined CAT.NET and VS Code analysis scan results: Scan Result Summary Category Valid False Positive Cross-Site Scripting 2 0 SQL Injection 1 1 Unvalidated Redirect 1 0
  12. 12. • Widget Town combined CAT.NET and VS Code analysis scan results: Scan Result Summary Category Valid False Positive Cross-Site Scripting 2 0 SQL Injection 1 1 Unvalidated Redirect 1 0
  13. 13. Roadmap • .NET Static Analysis Options • The Roslyn API • Code Analyzer • Additional Files Analyzer • Puma Scan • Future Enhancements
  14. 14. Introducing Roslyn • Open-source C# and Visual Basic compilers with code analysis APIs • Capable of producing warnings in code as you type:
  15. 15. Getting Started • Prerequisites: – Visual Studio 2015 – Visual Studio 2015 Extensibility Tools – .NET Compiler Platform ("Roslyn") SDK • Described in detail in this MSDN Magazine article by Alex Turner: – https://msdn.microsoft.com/en- us/magazine/dn879356.aspx
  16. 16. Creating a Code Analyzer Project • File > New Project • Templates > Visual C# > Extensibility • Select Analyzer with Code Fix (NuGet + VSIX) template
  17. 17. Roslyn Syntax Visualizer • Included in the .NET Compiler Platform SDK • Facilitates inspection of a syntax tree for any C# or VB code file open inside Visual Studio • Each node displays a properties grid for the item selected in the tree including: – Semantics, symbols, types, values, etc.
  18. 18. Roadmap • .NET Static Analysis Options • The Roslyn API • Code Analyzer • Additional Files Analyzer • Puma Scan • Future Enhancements
  19. 19. Code Analyzer 101 • Roslyn exposes the following API’s to simplify code analysis: – DiagnosticAnalyzer – DiagnosticDescriptor – AnalysisContext – SyntaxKinds
  20. 20. • Decorate the custom analyzer with the DiagnosticAnalyzer attribute • Inherit from the DiagnosticAnalyzer base class Diagnostic Analyzer Class [DiagnosticAnalyzer(LanguageNames.CSharp)] public class MyAwesomeAnalyzer : DiagnosticAnalyzer { //Insert awesome analyzer logic here } 1 2 3 4 5
  21. 21. • Define the diagnostic’s id, title, message, severity, and description Diagnostic Descriptor Class […] private static DiagnosticDescriptor Rule = new DiagnosticDescriptor(Id, Title, MessageFormat, Category, DiagnosticSeverity.Warning, isEnabledByDefault: true, description: Description); public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics { get { return ImmutableArray.Create(Rule); } } 1 2 3 4 5 6 7 8 9 10
  22. 22. • Add the diagnostic descriptor to the rule’s list of supported diagnostics Diagnostic Descriptor List […] private static DiagnosticDescriptor Rule = new DiagnosticDescriptor(Id, Title, MessageFormat, Category, DiagnosticSeverity.Warning, isEnabledByDefault: true, description: Description); public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics { get { return ImmutableArray.Create(Rule); } } 1 2 3 4 5 6 7 8 9 10
  23. 23. • Determines when Roslyn calls back to your analyzer code • http://bit.ly/2dStJru Analysis Context Events Context Registration Options RegisterCodeBlockAction RegisterSymbolAction RegisterCompilationAction RegisterSyntaxNodeAction RegisterCompilationStartAction RegisterSyntaxTreeAction RegisterSemanticModelAction
  24. 24. • Determines the syntax nodes or symbol the analyzers are inspecting • Hundreds of options are available, some commonly used items: Symbol / Syntax Kind Options Syntax Kinds Symbol Kinds MethodDeclaration Event ObjectCreationExpression Field InvocationExpression Method SimpleAssignmentExpression Parameter
  25. 25. • Believe it or not, this is all you need to build a real analyzer • WARNING: Intense Roslyn code flagging ASP.NET Identity for weak password length coming next! Password Length Analyzer Example
  26. 26. • Override the Initialize method • Register the SyntaxNodeAction event listener • Target the ObjectCreateExpression nodes Initializing an Analysis Context [DiagnosticAnalyzer(LanguageNames.CSharp)] public class MyAwesomeAnalyzer : DiagnosticAnalyzer { […] public override void Initialize(AnalysisContext context) { context.RegisterSyntaxNodeAction(AnalyzeSyntaxNode ,SyntaxKind.ObjectCreationExpression); } } 1 2 3 4 5 6 7 8 9 10
  27. 27. • Retrieve the incoming object creation node Identity Password Length Analyzer […] private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext context) { var statement=context.Node as ObjectCreationExpressionSyntax; if(string.Compare(statement?.Type.ToString() , "PasswordValidator", StringComparison.Ordinal) != 0) return; var symbol = context.SemanticModel.GetSymbolInfo(statement) .Symbol as ISymbol; if (string.Compare(symbol?.ContainingNamespace.ToString() ,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0) return; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  28. 28. • Check the object type’s name Identity Password Length Analyzer […] private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext context) { var statement=context.Node as ObjectCreationExpressionSyntax; if(string.Compare(statement?.Type.ToString() , "PasswordValidator", StringComparison.Ordinal) != 0) return; var symbol = context.SemanticModel.GetSymbolInfo(statement) .Symbol as ISymbol; if (string.Compare(symbol?.ContainingNamespace.ToString() ,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0) return; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  29. 29. • Verify the symbol is in the Identity namespace Identity Password Length Analyzer […] private static void AnalyzeSyntaxNode(SyntaxNodeAnalysisContext context) { var statement=context.Node as ObjectCreationExpressionSyntax; if(string.Compare(statement?.Type.ToString() , "PasswordValidator", StringComparison.Ordinal) != 0) return; var symbol = context.SemanticModel.GetSymbolInfo(statement) .Symbol as ISymbol; if (string.Compare(symbol?.ContainingNamespace.ToString() ,"Microsoft.AspNet.Identity", StringComparison.Ordinal) != 0) return; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  30. 30. • Retrieve the initializer expressions Identity Password Length Analyzer […] var initializer = statement.Initializer as InitializerExpressionSyntax; if (initializer?.Expressions.Count == 0) return; int minLength = 0; foreach (AssignmentExpressionSyntax expression in initializer.Expressions) { var value = context.SemanticModel.GetConstantValue (expression.Right); if (value.HasValue && expression.Left.ToString().Equals("RequiredLength")) minLength = (int)value.Value; } 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
  31. 31. • Find and read the expression’s constant value Identity Password Length Analyzer […] var initializer = statement.Initializer as InitializerExpressionSyntax; if (initializer?.Expressions.Count == 0) return; int minLength = 0; foreach (AssignmentExpressionSyntax expression in initializer.Expressions) { var value = context.SemanticModel.GetConstantValue (expression.Right); if (value.HasValue && expression.Left.ToString().Equals("RequiredLength")) minLength = (int)value.Value; } 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
  32. 32. • Minimum length requirement check Identity Password Length Analyzer […] //Warn if length < 12 chars if(minLength < 12) { var diagnostic = Diagnostic.Create(Rule, statement.GetLocation()); context.ReportDiagnostic(diagnostic); } } } } 36 37 38 39 40 41 42 43 44 45 46
  33. 33. • Report the diagnostic to the compiler Identity Password Length Analyzer […] //Warn if length < 12 chars if(minLength < 12) { var diagnostic = Diagnostic.Create(Rule, statement.GetLocation()); context.ReportDiagnostic(diagnostic); } } } } 36 37 38 39 40 41 42 43 44 45 46
  34. 34. • Proof that 34 lines of code can create a static analysis rule flagging poor password management policies Identity Password Length Analyzer
  35. 35. Roadmap • .NET Static Analysis Options • The Roslyn API • Code Analyzer • Additional Files Analyzer • Puma Scan • Future Enhancements
  36. 36. Non-Code Files • What about non-code files? • Security issues commonly exist in non-code files: – Configuration files (.config, .json) – View markup files (.cshtml, .html, .aspx, .ascx) – External references (.js, .css) – Non-compiled languages (SQL, node, python, rails)
  37. 37. Additional Files • Additional files were designed to feed configuration data to code analyzers – Password complexity rules, authentication timeout values, etc.
  38. 38. Additional Files Analyzer • But, we need to analyze and create diagnostic warnings in non-code files – .config, .json, .cshtml, .aspx, .ascx, etc. • Not officially supported as of Visual Studio 2015 Update 3 • Open git issue – https://github.com/dotnet/roslyn/issues/11097
  39. 39. Additional Files Analyzer Roadblocks • Additional files are not automatically loaded into the analysis context • Creating a diagnostic with an additional file location causes the error to disappear
  40. 40. Additional File Item Names • Each project file targeted for analysis must set its additional file item names property group to all content files: <PropertyGroup> […] <AdditionalFileItemNames> $(AdditionalFileItemNames);Content </AdditionalFileItemNames> </PropertyGroup>
  41. 41. Additional File Diagnostic • Do not include the source location in additional file diagnostics • Workaround: leverage the message arguments parameter to display path and line info in the error list: string messageFormat = "Debug compilation is enabled. {0}({1}): {2}”; context.ReportDiagnostic(Diagnostic.Create(Rule, Location.None, path, lineNumber, line));
  42. 42. Additional File Analyzer Diagnostics • Diagnostics reported on web.config vulnerabilities in the error list:
  43. 43. Additional Files Analyzer Limitations • Additional files are not automatically loaded after installing the NuGet package – Open ticket to correct this in the NuGet installer • Manual edits required to project files when using the extension (.vsix) installer • Error list double click navigation is not supported • No spellcheck (squiggles) in non-code files
  44. 44. Demo Code Repo • Sample analyzers from this talk are available in git: – https://github.com/ejohn20/puma-scan-demo
  45. 45. Roadmap • .NET Static Analysis Options • The Roslyn API • Code Analyzer • Additional Files Analyzer • Puma Scan • Future Enhancements
  46. 46. Introducing the Puma Scan • Open source Visual Studio Roslyn security source code analyzer extension • Over 40 application security-specific rules • Version 1.0 is available via NuGet & Visual Studio Marketplace • Install, rule docs, source code: – https://www.pumascan.com – https://github.com/pumasecurity – @puma_scan
  47. 47. • Widget Town Puma scan results: – 54 valid issues, 10 false positives Puma Scan Result Summary Category Valid False Positive Cross-Site Scripting 19 3 SQL Injection 2 3 Misconfiguration 16 0 Path Tampering 3 0 Unvalidated Redirect 2 4 Cross-Site Request Forgery 8 0 Poor Password Management 3 0 Certificate Validation Disabled 1 0
  48. 48. Future Enhancements • Welcoming contributors! • Gather feedback and address edge cases • Continue to build out additional rule categories – Crypto, cleartext secrets, XML processing, etc. • Further refine results using data flow analysis to eliminate false positives • Identify rules that can apply suggested code fixes
  49. 49. Acknowledgements • Eric Mead – Cypress Data Defense • Tom Meschter – Microsoft • Manish Vasani – Microsoft • Gitter Rosyln Channel
  50. 50. Thank you for attending! Email: eric.johnson@cypressdefense.com Twitter: @emjohn20

×