Static analysis, development complexity and code quality; How the C# analyzer is designed and operates; Diagnostic; Functional check on actual projects (SelfTester).

1. 1/41St. Petersburg. May 31–June 1 2019
Software quality assurance days
The 25th international conference
concerning SW
sqadays.com
Sergey Khrenov
PVS-Studio, Tula, Russia
Specifics of static analyzer development and
testing

2. 2/41
Название темы. Может быть длинное, даже не одна строка, а две или три
Static analysis, development complexity
and code quality

3. 3/41
• Does not replace, but
supplements code review
• Finding errors in code without
executing it
• Allows to control the code quality
in large projects
Static analysis …

4. 4/41
Typical error (project Mono)

5. 5/41
Typical error (project Mono)

6. 6/41
V3012 The '?:' operator, regardless of its conditional expression, always
returns one and the same value: Color.FromArgb (150, 179, 225).
ProfessionalColorTable.cs 258
Typical error (project Mono)

7. 7/41
Issues
• Scientific content
• Different programming languages
• Cross-platform
• Standards support (CWE, MISRA, SEI CERT…)
• Classical testing methods are not enough

8. 8/41
How to achieve quality?
• Joint code reviews (it works!)
• Unit-tests
• UI-tests
• Functional tests
• Load testing
• Static analysis
• Check of actual projects pool (SelfTester)

9. 9/41
How the C# analyzer is designed and operates

10. 10/41
MSBuild, Roslyn and each and all

11. 11/41
MSBuild, Roslyn and each and all

12. 12/41
int x = 1;
Syntax tree

13. 13/41
Semantic model
 Getting information about the object
 Getting information about the object type
 Getting constant values
x = 1;
x
Semantic Model
System.Int32 x = 1;

14. 14/41
Accessing the project
void GetProjects(string solutionPath, string projectPath)
{
MSBuildWorkspace workspace = MSBuildWorkspace.Create();
Solution currSolution = workspace.OpenSolutionAsync(solutionPath)
.Result;
IEnumerable<Project> projects = currSolution.Projects;
Project currProject = workspace.OpenProjectAsync(projectPath)
.Result;
}

15. 15/41
Accessing the project (all projects in solution)
void GetProjects(string solutionPath, string projectPath)
{
MSBuildWorkspace workspace = MSBuildWorkspace.Create();
Solution currSolution = workspace.OpenSolutionAsync(solutionPath)
.Result;
IEnumerable<Project> projects = currSolution.Projects;
Project currProject = workspace.OpenProjectAsync(projectPath)
.Result;
}

16. 16/41
Accessing the project (specified project)
void GetProjects(string solutionPath, string projectPath)
{
MSBuildWorkspace workspace = MSBuildWorkspace.Create();
Solution currSolution = workspace.OpenSolutionAsync(solutionPath)
.Result;
IEnumerable<Project> projects = currSolution.Projects;
Project currProject = workspace.OpenProjectAsync(projectPath)
.Result;
}

17. 17/41
Project analysis
void ProjectAnalysis(Project project)
{
Compilation compilation = project.GetCompilationAsync().Result;
foreach (var file in project.Documents)
{
SyntaxTree tree = file.GetSyntaxTreeAsync().Result;
SemanticModel model = compilation.GetSemanticModel(tree);
Visit(tree.GetRoot());
}
}

18. 18/41
Project analysis (traversing all files)
void ProjectAnalysis(Project project)
{
Compilation compilation = project.GetCompilationAsync().Result;
foreach (var file in project.Documents)
{
SyntaxTree tree = file.GetSyntaxTreeAsync().Result;
SemanticModel model = compilation.GetSemanticModel(tree);
Visit(tree.GetRoot());
}
}

19. 19/41
Project analysis (getting and traversing the tree)
void ProjectAnalysis(Project project)
{
Compilation compilation = project.GetCompilationAsync().Result;
foreach (var file in project.Documents)
{
SyntaxTree tree = file.GetSyntaxTreeAsync().Result;
SemanticModel model = compilation.GetSemanticModel(tree);
Visit(tree.GetRoot());
}
}

20. 20/41
Overridden node-traversing methods
public override void VisitIfStatement(IfStatementSyntax node)
{
base.VisitIfStatement(node);
}
public override void VisitForStatement(ForStatementSyntax node)
{
base.VisitForStatement(node);
}
….

21. 21/41
Diagnostic

22. 22/41
Diagnostic V3006: missing throw
public void DoSomething(int index)
{
if (index < 0)
new ArgumentOutOfRangeException(); // <= V3006
else
....
}
// The correct version of the code:
throw new ArgumentOutOfRangeException();

23. 23/41
1. Follow traversing nodes of the type ObjectCreationExpressionSyntax
(creating an object using the operator new);
2. Check that the type of the created object is System.Exception or a
derived one (use a semantic model);
3. Check that the created object is not used in any way;
4. Issue a warning.
Diagnostics V3006: missing throw

24. 24/41
public class V3006CSharpRule : IVisitObjectCreationExpressionRule
{
....
public void VisitObjectCreationExpression(
SemanticModelAdapter model,
VisitInfo visitInfo,
ObjectCreationExpressionSyntax node,
AnalysisResults results)
{
....
}
}
Diagnostic V3006: missing throw

25. 25/41
Diagnostics development
1. Creating positive and negative tests
2. Development of a prototype that meets the test
requirements
3. Improvement of diagnostics and tests based on the
results after checking a pool of actual projects
(SelfTester)
4. Exception handling, minimization of false positives
5. Re-run on actual projects, saving changes

26. 26/41
Some statistics

27. 27/41
Positive tests

28. 28/41
Negative tests

29. 29/41
Negative and positive tests

30. 30/41
Functional check on actual projects
(SelfTester)

31. 31/41
• The tool for batch verification of actual
projects
• SelfTester for С/С++ and C# uses local project
pool
• SelfTester for Java downloads projects of the
certain version from the repository on GitHub
PVS-Studio SelfTester

32. 32/41
Requirements to SelfTester
• Сross-platform
• Parallelism
• Available GUI
• Flexible settings
• Presentation of the result in a convenient
form
• Comparison with the result of the previous
run
• Ability to quickly fixate or ignore differences

33. 33/41
SelfTester tasks
• Any change of the analyzer’s core requires tests’ restart
• The core is a set of common internal mechanisms of the
analyzer
• Creating a new diagnostic requires tests’ restart
• Including a new project in the test pool requires a
restart of the tests
• The main task is to detect defects in the analyzer’s
behavior
• Often defects represent expected behavior

34. 34/41
Report of found bugs (*.plog)

35. 35/41
SelfTester: operational algorithm

36. 36/41
Log comparison result

37. 37/41
PVS-Studio SelfTester (С#)

38. 38/41
Going into production
• Improvements based on users
feedback (including internal
feedback)
• Improvements related to changes
in the core behavior (refinement
of mechanisms, support of new
language standards, etc.)

39. 39/41
Five features of a good static analyzer
according to PVS-Studio
• Fast code processing
• Minimum false positives
• Developed means of integration
• Simplicity of introduction into large projects
• Support

40. 40/41
Questions

41. 41/41
Sergey Khrenov
C# developer, PVS-Studio
khrenov@viva64.com
www.viva64.com
Contacts