Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

YouTube videos are no longer supported on SlideShare

View original on YouTube

PERFORMANCE AND
BENCHMARKING
David Wengier
@ch00k
premature optimization is
the root of all evil
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 29

Performance and Benchmarking

0

Share

Download to read offline

A brief discussion of performance optimization and when to do it, of how to benchmark and use BenchmarkDotNet, and some common gotchas in the .NET framework that you may or may not be aware of.

Presented at Melbourne Alt.Net on 29th Aug 2017
Code used to generate benchmarks: https://github.com/davidwengier/Benchmark
Recording of the session: https://www.youtube.com/watch?v=S8zhGaT6vv8

More Related Content

You Might Also Like

Related Books

Free with a 30 day trial from Scribd

See all

Performance and Benchmarking

  1. 1. PERFORMANCE AND BENCHMARKING David Wengier @ch00k
  2. 2. premature optimization is the root of all evil
  3. 3. premature optimization is the root of all evil - Donald Knuth
  4. 4. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%” - Donald Knuth
  5. 5. “Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%” - Donald Knuth
  6. 6. “Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%” - Donald Knuth “Structured Programming with Go To Statements”,
  7. 7. WHEN TO OPTIMIZE? • Optimization is fine when we know we need to • Look at the algorithm before the code • Micro-benchmarking is fine if the code is on the hot path • When experience tells you some things are objectively better • When the better performing code has other benefits
  8. 8. HOW TO OPTIMIZE? • Know the context • Question assumptions • Release builds only • Measure • Measure • Measure
  9. 9. HOW TO OPTIMIZE? public static void Profile(Action func) { DateTime start = DateTime.Now; for (int i = 0; i < 100000; i++) { func(); } Console.WriteLine("Time Elapsed {0} ms", (DateTime.Now - start).TotalMilliseconds); }
  10. 10. HOW TO OPTIMIZE? public static void Profile(Action func) { Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High; Thread.CurrentThread.Priority = ThreadPriority.Highest; func(); Stopwatch watch = new Stopwatch(); GC.Collect(); GC.WaitForPendingFinalizers(); GC.Collect(); watch.Start(); for (int i = 0; i < 100000; i++) { func(); } watch.Stop(); Console.WriteLine("Time Elapsed {0} ms", watch.Elapsed.TotalMilliseconds); }
  11. 11. BENCHMARKDOTNET (not Benchmark.Net) public class Program { public static void Main(string[] args) { BenchmarkRunner.Run<Program>(); } private string input = "1000"; [Benchmark] public int IntParse() { return int.Parse(input); } [Benchmark] public int ConvertToInt32() { return Convert.ToInt32(input); } }
  12. 12. HOW IT WORKS • Separate process, separate run for each diagnostic. • Pilot: The best operation count will be chosen. • IdleWarmup, IdleTarget: Overhead will be evaluated. • MainWarmup: Warmup of the main method. • MainTarget: Main measurements. • Result = MainTarget – AverageOverhead http://benchmarkdotnet.org/HowItWorks.htm
  13. 13. LETS MICRO BENCHMARK! • NO! • Its too slow.
  14. 14. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  15. 15. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  16. 16. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  17. 17. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  18. 18. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  19. 19. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION
  20. 20. STRING.JOIN VS LINQ VS STRINGBUILDER VS CONCATENATION • Combining 3 x 20 character strings: • StringBuilder: 126ns, 236 b • Concatnation: 267ns, 213 b • Combining 3 x 2,000,000 character strings: • StringBuilder: 122.3 ms, 220 mb • Concatenation: 230.9 ms, 343 mb
  21. 21. DICTIONARY<> VS IDICTIONARY<>
  22. 22. DICTIONARY<> VS IDICTIONARY<> 0 bytes allocated 32 bytes allocated
  23. 23. LOOPS VS LINQ
  24. 24. LOOPS VS LINQ 0 bytes 20 bytes 32 bytes
  25. 25. EXCEPTIONS VS NOT EXCEPTIONS
  26. 26. EXCEPTIONS VS NOT EXCEPTIONS 408 bytes 0 bytes
  27. 27. HYBRIDDICTIONARY public class HybridDictionary: IDictionary { // These numbers have been carefully tested to be optimal. // Please don't change them without doing thorough performance // testing. private const int CutoverPoint = 9;
  28. 28. • Should you always use Dictionary<K,V> instead of IDictionary<K,V>? No, it depends. • Should you always use structs instead of classes? No, it depends. • Should you always use for instead of foreach? No, it depends. • Should you always use StringBuilder instead of concatenation? No, it depends. • Should you always use traditional loops instead of Linq? No, it depends. • Should you always avoid throwing exceptions? No, it depends. • Should you always specify initial capacity on Lists and StringBuilders? Yes. Be nice. SUMMARY

Editor's Notes

  • Heard this?
    Who said it?
  • Donald Knuth. You may notice the formatting is odd.. Well, that’s because its part of a bigger quote
  • Most people don’t know this. 97% of the time its evil, the rest its not. But why?
    Notice the quotes.
  • Consider maintenance and debugging. Readability. Etc.
    Bonus points, what paper was it from?
  • Ironically, a quote that lots of developers spout as being gospel comes from a paper arguing that programming with go to statements is perfectly fine. A debate for another time.
  • So when do we optimize? When we should.
    Split the string? What about streams? StringSegments? Don’t focus on the loop
    Measure the hot path
    Pragmatism says draw on your experience
    Other benefits, like reusability or readability.
  • To avoid a micro-benchmark, know the context of your data. How much? How often?
    Questions assumptions: “the compiler will fix this”. Will it?
    Release builds only!
    Measure
  • DateTime is as precise as stopwatch, but not as granular
    Doesn’t account for JITing
  • High affinity, so nothing else on the machine matters (hopefully)
    JIT first
    Get GC out of the way, because GC pauses are bad for benchmarking
    Then go
  • Or use BenchmarkDotNet
  • So I’ve told you how to benchmark, and that you should basically never do it, so lets do it.
  • Except its too slow. BenchmarkDotNet is too awesome for live demos. I have some canned ones.
  • Concat wins
    Then stringbuilder
    StringBuilder again
    Now StringBuilder with capacity, and that trend continues forever
    If we add more, we can see stringbuilder and string join are both a lot fast
    As we keep going, the difference gets bigger
    Looks like a slam dunk right? Always use a string builder with capacity?
    BUT- don’t forget context!!! Look what happens as the strings get longer. Now it doesn’t matter so much, as long as you don’t concatenate
  • Concat wins
    Then stringbuilder
    StringBuilder again
    Now StringBuilder with capacity, and that trend continues forever
    If we add more, we can see stringbuilder and string join are both a lot fast
    As we keep going, the difference gets bigger
    Looks like a slam dunk right? Always use a string builder with capacity?
    BUT- don’t forget context!!! Look what happens as the strings get longer. Now it doesn’t matter so much, as long as you don’t concatenate
  • Concat wins
    Then stringbuilder
    StringBuilder again
    Now StringBuilder with capacity, and that trend continues forever
    If we add more, we can see stringbuilder and string join are both a lot fast
    As we keep going, the difference gets bigger
    Looks like a slam dunk right? Always use a string builder with capacity?
    BUT- don’t forget context!!! Look what happens as the strings get longer. Now it doesn’t matter so much, as long as you don’t concatenate
  • Concat wins
    Then stringbuilder
    StringBuilder again
    Now StringBuilder with capacity, and that trend continues forever
    If we add more, we can see stringbuilder and string join are both a lot fast
    As we keep going, the difference gets bigger
    Looks like a slam dunk right? Always use a string builder with capacity?
    BUT- don’t forget context!!! Look what happens as the strings get longer. Now it doesn’t matter so much, as long as you don’t concatenate
  • Back to context, again production code.. For 3 small strings, concatenation is slower, but not measurably so. But it uses less memory. The overhead of StringBuilder might make it a winner. For large strings, not so.

    Stopwatch granularity is around 300ns, so the difference in the first one is effectively unmeasurable in production .NET code.
  • The overhead of a class vs struct, virtual calls, etc. makes the difference.
    So if you use Idictionary as your parameters are you’re supposed to do, on a hot path that might be a negative.
    Question assumptions.. How many of you think “coding to interfaces” is _always_ better?
  • I’m an old man so I like loops, but Linq has an overhead.
  • Overhead in method invocations and allocations. Even the Where vs FirstOrDefault examples, because there are more iterators being allocated.
  • 50% chance of having an invalid number. The important thing is that both benchmarks will use exactly the same input.
  • We all know exceptions are slow, but they’re really slow. They have to collect debug information, context, stack trace etc. Don’t use for control flow.
  • Measure! Take a leaf out of microsofts book. Know your input data. Know your context. Use the correct type.
  • ×