Kostiantyn Yelisavenko "Mastering Macro Benchmarking in .NET"

1. Mastering Macro
Benchmarking in
.NET

2. Konstantin
Yelisavenko
Lead .NET Developer
Work in Luxoft since 2016
kyelisavenko@luxoft.com

3. Micro vs Macro Benchmarking

4. Micro vs Macro Benchmarking
Micro
✘ Repeatable measurement of specific
sections of code
✘ Abstracted from virtual machine
warmup, garbage collections
and other side effects

5. Micro vs Macro Benchmarking
Micro
✘ Repeatable measurement of specific
sections of code
✘ Abstracted from virtual machine
warmup, garbage collections
and other side effects
Macro
✘ Repeatable measurement of whole
application or it’s part performance
from user point of view
✘ Abstracted from performance
overhead caused by monitoring
tools

7. Donald Knuth

8. “
Premature optimization
is the root of all evil

9. When we need optimization?

10. When we need optimization?
Obvious reasons :

11. When we need optimization?
Obvious reasons :
✘ Slow performance

12. When we need optimization?
Obvious reasons :
✘ Slow performance
✘ Memory consumption increasing

13. When we need optimization?
Obvious reasons :
✘ Slow performance
✘ Memory consumption increasing
Unobvious reasons :

14. When we need optimization?
Obvious reasons :
✘ Slow performance
✘ Memory consumption increasing
Unobvious reasons :
✘ GC.Collect()

15. private void memoryCheckTimer_Tick(object sender, EventArgs e)
{
var timer = sender as DispatcherTimer;
if (timer != null)
{
Process eProcess = Process.GetCurrentProcess();
GC.Collect();
eProcess.Refresh();
var ci = new Microsoft.VisualBasic.Devices.ComputerInfo();
ulong mem = ulong.Parse(CI.AvailablePhysicalMemory.ToString()) / (1024 * 1024);
If (mem <= ulong.Parse(ConfigurationManager.AppSettings.Get("MemoryLowThreshold")))
{
ShowWarningMessage("Low Memory Warning",
string.Format(System.Configuration.ConfigurationManager.AppSettings.Get("MemoryLowMessage"),
Environment.NewLine));
timer.Stop();
timer.Tick -= memoryCheckTimer_Tick;
}
}
}

16. private void memoryCheckTimer_Tick(object sender, EventArgs e)
{
var timer = sender as DispatcherTimer;
if (timer != null)
{
Process eProcess = Process.GetCurrentProcess();
GC.Collect();
eProcess.Refresh();
var ci = new Microsoft.VisualBasic.Devices.ComputerInfo();
ulong mem = ulong.Parse(CI.AvailablePhysicalMemory.ToString()) / (1024 * 1024);
If (mem <= ulong.Parse(ConfigurationManager.AppSettings.Get("MemoryLowThreshold")))
{
ShowWarningMessage("Low Memory Warning",
string.Format(System.Configuration.ConfigurationManager.AppSettings.Get("MemoryLowMessage"),
Environment.NewLine));
timer.Stop();
timer.Tick -= memoryCheckTimer_Tick;
}
}
}

17. private void memoryCheckTimer_Tick(object sender, EventArgs e)
{
var timer = sender as DispatcherTimer;
if (timer != null)
{
Process eProcess = Process.GetCurrentProcess();
GC.Collect();
eProcess.Refresh();
var ci = new Microsoft.VisualBasic.Devices.ComputerInfo();
ulong mem = ulong.Parse(CI.AvailablePhysicalMemory.ToString()) / (1024 * 1024);
If (mem <= ulong.Parse(ConfigurationManager.AppSettings.Get("MemoryLowThreshold")))
{
ShowWarningMessage("Low Memory Warning",
string.Format(System.Configuration.ConfigurationManager.AppSettings.Get("MemoryLowMessage"),
Environment.NewLine));
timer.Stop();
timer.Tick -= memoryCheckTimer_Tick;
}
}
}

18. When we need optimization?
Obvious reasons :
✘ Slow performance
✘ Memory consumption increasing
Unobvious reasons :
✘ GC.Collect()
✘ Subscriptions management violated

19. When we need optimization?
Obvious reasons :
✘ Slow performance
✘ Memory consumption increasing
Unobvious reasons :
✘ GC.Collect()
✘ Subscriptions management violated
✘ UI freeze (“Not Responding”)

21. Vilfredo Pareto

22. According to Pareto Principle

23. According to Pareto Principle
20%of your codebase

24. According to Pareto Principle
20%of your codebase
do 80% of all job

25. 20%
You should concentrate on those

26. What to consider before optimization?

27. What to consider before optimization?
Operation system
Windows, Linux, MacOS

28. What to consider before optimization?
Operation system
Windows, Linux, MacOS
Runtime type
CLR, Core CLR, Mono

29. What to consider before optimization?
Operation system
Windows, Linux, MacOS
Runtime type
CLR, Core CLR, Mono
Configuration
X86 or x64

30. What to consider before optimization?
Operation system
Windows, Linux, MacOS
Runtime type
CLR, Core CLR, Mono
Garbage collector type
GC, Boehm, Sgen
Configuration
X86 or x64

31. Also pay attention to…

32. Also pay attention to…
✘ How much time do you have for this task?

33. Also pay attention to…
✘ How much time do you have for this task?
✘ What target scenarios should be optimized?

34. Also pay attention to…
✘ How much time do you have for this task?
✘ What target scenarios should be optimized?
✘ How should you measure?

35. What to read before?

36. What to read before?
✘ “Writing High-Performance .NET Code” by Ben Watson
✘ “Under the Hood of .NET Memory Management” by Chris Farrell and
Nick Harrison
✘ “Detecting and Solving Memory Problems in .NET” by Alexey Totin
✘ Jetbrains official documentation
✘ Pluralsight course - “.NET Performance Optimization and Profiling
with JetBrains dotTrace” by Xavier Morera

37. Profiling stages

38. Profiling stages
collection visualization analysis

39. Snapshot strategies

40. Snapshot strategies
Performance
✘ Sampling (call time measurement using call stacks)

41. Snapshot strategies
Performance
✘ Sampling (call time measurement using call stacks)
✘ Tracing (number of calls using profiling API)

42. Snapshot strategies
Performance
✘ Sampling (call time measurement using call stacks)
✘ Tracing (number of calls using profiling API)
✘ Line-by-line (each statement execution time inside
function)

43. Snapshot strategies
Performance
✘ Sampling (call time measurement using call stacks)
✘ Tracing (number of calls using profiling API)
✘ Line-by-line (each statement execution time inside
function)
✘ Timeline (each thread activity visualization)

44. Snapshot strategies
Memory
✘ Image of managed heap created

45. Snapshot strategies
Memory
✘ Image of managed heap created
✘ Simulation of GC traversing through reference tree

46. What tools are available?

47. What tools are available?
Macro
✘ Visual Studio Profiling Tools
✘ “dotTrace” and “dotMemory”
from Jetbrains
✘ “ANTS Memory Profiler” from
Redgate
✘ “YourKit”
✘ Old good “Xperf” and
“PerfMon”

48. What tools are available?
Macro
✘ Visual Studio Profiling Tools
✘ “dotTrace” and “dotMemory”
from Jetbrains
✘ “ANTS Memory Profiler” from
Redgate
✘ “YourKit”
✘ Old good “Xperf” and
“PerfMon”
Micro
✘ “BenchmarkDotNet”
✘ ???

49. dotTrace features

50. dotTrace features
✘ Performance profiling with various levels of details

51. dotTrace features
✘ Performance profiling with various levels of details
✘ Hotspots and bottlenecks detection

52. dotTrace features
✘ Performance profiling with various levels of details
✘ Hotspots and bottlenecks detection
✘ Threads Timeline

53. dotTrace features
✘ Performance profiling with various levels of details
✘ Hotspots and bottlenecks detection
✘ Threads Timeline
✘ Performance gain forecasting

54. dotTrace features
✘ Performance profiling with various levels of details
✘ Hotspots and bottlenecks detection
✘ Threads Timeline
✘ Performance gain forecasting
✘ Remote profiling

55. dotTrace features
✘ Performance profiling with various levels of details
✘ Hotspots and bottlenecks detection
✘ Threads Timeline
✘ Performance gain forecasting
✘ Remote profiling
✘ Unit tests profiling

56. dotMemory features

57. dotMemory features
✘ Shows all allocated objects consumed be the app

58. dotMemory features
✘ Shows all allocated objects consumed be the app
✘ Shows current state of Finalization Queue

59. dotMemory features
✘ Shows all allocated objects consumed be the app
✘ Shows current state of Finalization Queue
✘ Shows how effectively collections are utilized

60. dotMemory features
✘ Shows all allocated objects consumed be the app
✘ Shows current state of Finalization Queue
✘ Shows how effectively collections are utilized
✘ Shows memory leaks

61. Out of profiling scope
✘ UI client side
✘ Database interactions
✘ Network and infrastructure communications
✘ Native Apps (iOS, Android)

62. Demo

63. Conclusion

64. Conclusion
✘ You can get answers to…
○ Why this s**t is working slowly?
○ Why this s**t is so hungry?

65. Conclusion
✘ You can get answers to…
○ Why this soft is working slowly?
○ Why this soft is so hungry?

66. Conclusion
✘ You can get answers to…
○ Why this soft is working slowly?
○ Why this soft is so hungry?
✘ More descriptive Technical debt items

67. Conclusion
✘ You can get answers to…
○ Why this soft is working slowly?
○ Why this soft is so hungry?
✘ More descriptive Technical debt items
✘ Ability to forecast performance gains

68. Conclusion
✘ You can get answers to…
○ Why this soft is working slowly?
○ Why this soft is so hungry?
✘ More descriptive Technical debt items
✘ Ability to forecast performance gains
✘ Can understand how dependent your project from 3-rd party
components

69. Conclusion
✘ You can get answers to…
○ Why this soft is working slowly?
○ Why this soft is so hungry?
✘ More descriptive Technical debt items
✘ Ability to forecast performance gains
✘ Can understand how dependent your project from 3-rd party
components
✘ Can create performance trend

70. thanks!
Any questions?