BenchmarkDotNet is a powerful .NET library for benchmarking code. It allows developers to easily write benchmarking code, run benchmarks across different runtimes and environments, and view results in various formats like markdown, CSV, HTML and plain text. Key features include support for multiple .NET implementations, automatic warmup and overhead evaluation, parameterization of benchmarks, and diagnostic tools to analyze benchmark performance.

1. BENCHMARKDOTNET -
POWERFUL .NET LIBRARY FOR
BENCHMARKING
Larry Nung

2. AGENDA
Introduction
Getting started
Jobs
Columns
Diagnosers
Exporters
Params
Setup
Baseline
Reference
Q & A 2

3. INTRODUCTION
3

4. INTRODUCTION
 A powerful .NET library for benchmarking
 Standard benchmarking routine: generating an isolated
project per each benchmark method; auto-selection of
iteration amount; warmup; overhead evaluation; statistics
calculation; and so on.
 Supported runtimes: Full .NET Framework, .NET Core (RTM),
Mono
 Supported languages: C#, F#, and Visual Basic
 Supported OS: Windows, Linux, MacOS
 Easy way to compare different environments
(x86 vs x64, LegacyJit vs RyuJit, and so on; see: Jobs)
 Reports: markdown, csv, html, plain text, png plots.
 Advanced features: Baseline, Params
 Powerful diagnostics based on ETW events
(see BenchmarkDotNet.Diagnostics.Windows) 4

5. GETTING STARTED
5

6. INSTALLATION
 Install-Package BenchmarkDotNet
6

7. INSTALLATION
7

8. INSTALLATION
8

9. INSTALLATION
9

10. INSTALLATION
10

11. WRITE CODE TO BENCHMARK
using BenchmarkDotNet.Attributes;
…
public class ProgramBenchmarker {
protected Program m_Program { get; set; } = new
Program();
[Benchmark] public void Test() {
m_Program.Test();
}
}
…
11

12. RUN THE BENCHMARK
using BenchmarkDotNet.Running;
…
public class Program {
static void Main(string[] args) {
var summary =
BenchmarkRunner.Run<ProgramBenchmarker>();
}
public void Test() {… }
}
…
}
12

13. VIEW RESULTS
13

14. ANALYZE RESULTS
14

15. JOBS
15

16. JOBS
 Describes how to run your benchmark
 Avaliable Jobs
 DryJob
 ClrJob
 CoreJob
 MonoJob
 LegacyJitX86Job
 LegacyJitX64
 RyuJitX64Job
 SimpleJob
 LongRunJob
 MediumRunJob
 ShortRunJob
 VeryLongRunJob 16

17. JOBS
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Attributes.Jobs;
...
[ShortRunJob]
public class ProgramBenchmarker {
...
}
17

18. JOBS
18

19. COLUMNS
19

20. COLUMNS
 A column in the summary table
 Available Columns
 NamespaceColumn
 MedianColumn
 MinColumn
 MaxColumn
 RankColumn
 e.t.c
20

21. COLUMNS
using BenchmarkDotNet.Attributes.Columns;
…
[NamespaceColumn]
[MedianColumn]
[MinColumn]
[MaxColumn]
[RankColumn]
[RankColumn(NumeralSystem.Roman)]
[OrderProvider(SummaryOrderPolicy.FastestToSlowest)]
public class ProgramBenchmarker {
…
} 21

22. COLUMNS
22

23. COLUMNS
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs;
…
[Config(typeof(Config))]
public class ProgramBenchmarker {
private class Config : ManualConfig {
public Config() {
Add(new TagColumn("HashCode", item =>
item.GetHashCode().ToString()));
}
}
…
}
23

24. COLUMNS
24

25. COLUMNS
using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Reports;
using BenchmarkDotNet.Running;
…
public class HashCodeColumn : IColumn {
public string ColumnName { get; } = "HashCode";
public HashCodeColumn() { }
public bool IsDefault(Summary summary, Benchmark benchmark) =>
false;
public string GetValue(Summary summary, Benchmark benchmark) =>
benchmark.Target.Method.Name.GetHashCode().ToString();
public bool IsAvailable(Summary summary) => true;
public bool AlwaysShow => true;
public ColumnCategory Category => ColumnCategory.Custom;
public string Id { get; } = "1";
public int PriorityInCategory { get; } = 0;
public override string ToString() => ColumnName;
}
25

26. COLUMNS
…
[Config(typeof(Config))]
public class ProgramBenchmarker {
private class Config : ManualConfig {
public Config() {
Add(new HashCodeColumn()));
}
}
…
} 26

27. DIAGNOSERS
27

28. DIAGNOSERS
 Can attach to your benchmark and get some useful
info
 Available Diagnosers
 MemoryDiagnoser
28

29. DIAGNOSERS
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Diagnosers;
...
[Config(typeof(Config))]
public class ProgramBenchmarker {
private class Config : ManualConfig {
public Config() {
Add(MemoryDiagnoser.Default);
}
}
...
} 29

30. DIAGNOSERS
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Diagnosers;
...
[MemoryDiagnoser]
public class ProgramBenchmarker {
...
}
30

31. DIAGNOSERS
31

32. EXPORTERS
32

33. EXPORTERS
 Allows you to export results of your benchmark in
different formats.By default, files with results will be
located
in .BenchmarkDotNet.Artifactsresults directory.
 Available Exporters
 HtmlExporter
 CsvExporter
 MarkdownExporter
 AsciiDocExporter
 CsvMeasurementsExporter
 PlainExporter
 JsonExporter
33

34. CONFIG
using BenchmarkDotNet.Attributes.Exporters;
…
[AsciiDocExporter]
[CsvMeasurementsExporter]
[PlainExporter]
[JsonExporter]
public class ProgramBenchmarker {
…
}
34

35. CONFIG
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Exporters;
using BenchmarkDotNet.Exporters.Csv;
using BenchmarkDotNet.Exporters.Json;
…
[Config(typeof(Config))]
public class ProgramBenchmarker {
private class Config : ManualConfig {
public Config() {
Add(AsciiDocExporter.Default);
Add(CsvMeasurementsExporter.Default);
Add(PlainExporter.Default);
Add(JsonExporter.Default);
}
}
…
}
35

36. HTMLEXPORTER
36

37. MARKDOWNEXPORTER
37

38. CSVEXPORTER
38

39. ASCIIDOCEXPORTER
39

40. CSVMEASUREMENTSEXPORTER
40

41. PLAINEXPORTER
41

42. JSONEXPORTER
42

43. PARAMS
43

44. PARAMS
using BenchmarkDotNet.Attributes;
...
public class ProgramBenchmarker {
[Params(100, 200)]
public int Parameter { get; set; }
protected Program m_Program { get; set; } = new
Program();
[Benchmark] public void Test() {
m_Program.Test();
}
}
44

45. PARAMS
45

46. SETUP
46

47. SETUP
using BenchmarkDotNet.Attributes;
...
public class ProgramBenchmarker {
protected Program m_Program { get; set; }
[Setup]
public void Setup() {
m_Program = new Program();
}
[Benchmark]
public void Test() {
m_Program.Test();
}
} 47

48. BASELINE
48

49. BASELINE
using System.Threading;
using BenchmarkDotNet.Attributes;
...
public class ProgramBenchmarker {
protected Program m_Program { get; set; } = new Program();
[Benchmark(Baseline = true)]
public void Test1() {
m_Program.Test();
Thread.Sleep(10);
}
[Benchmark]
public void Test2() {
m_Program.Test();
Thread.Sleep(20);
}
} 49

50. BASELINE
50

51. REFERENCE
51

52. REFERENCE
 NuGet Gallery | BenchmarkDotNet 0.10.3
 https://www.nuget.org/packages/BenchmarkDotNet
 dotnet/BenchmarkDotNet: Powerful .NET library for
benchmarking
 https://github.com/dotnet/BenchmarkDotNet
 Home - BenchmarkDotNet Documentation
 http://benchmarkdotnet.org/
52

53. Q&A
53

54. QUESTION & ANSWER
54