You don’t want to prematurely optimise, but sometimes you want to optimise, the question is - where to start? Profiling and Benchmarking can help you figure out what your application is doing and where performance problems could arise - allowing you to find (and fix!) them before your customers do.
If you aren’t already benchmarking your code this talk will offer some starting points. We’ll look at how to accurately benchmark in .NET and things to avoid. Along the way we’ll also discover some surprising code optimisations!
Performance and how to measure it - ProgSCon London 2016Matt Warren
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Performance is a feature! - London .NET User GroupMatt Warren
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Starting with the premise that "Performance is a Feature", Matt Warren will show you how to measure, what to measure and how to get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
The presentation covers:
Why we should care about performance
Pitfalls to avoid when measuring performance
How the .NET Garbage Collector can hurt performance
Real-world performance lessons from open-source code
The webinar recording can be found here: http://www.postsharp.net/blog/post/webinar-recording-performance-is-a-feature
Have you ever stopped to think about all the things that have to take place when you execute a .NET program? As the quote from Neal Ford says "Understand one level below your usual abstraction", this talk will look at why this is important and how can it help you if we apply it to the .NET framework. We will delve into the internals of the recently open-sourced .NET Core Runtime, looking at what happens, when it happens and why. Using freely available diagnostic tools such as PerfView, libraries including ClrMD and even the source code itself! Along the way we'll examine the Execution Engine, Type Loader, Just-in-Time (JIT) Compiler and the CLR Hosting API, to see how all these components play a part in making a 'Hello World' app possible.
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
Is your profiler speaking the same language as you? -- Docklands JUGSimon Maple
Profilers are absolute beasts. And profilers might prove useful to pinpoint the performance issues in your Java applications.
By using profilers, developers are fortunate to find the root cause of an issue at hand. However, it requires effort to actually comprehend the data collected by the profiler. Due to the inherent complexity of the data, one has to understand how this data is collected. And thus understand how the profiler actually works.
During this talk we will go through the classic profiler features. What is a hotspot? What is the difference between sampling and instrumentation from the profiler point of view? What are the problems with either of those methods? What is the time budget of the application? And more!
I will also showcase a new kid on the block among the profiling tools: XRebel. This tool provides insight into application behaviour and permits the developers to discover application level issues.
Performance and how to measure it - ProgSCon London 2016Matt Warren
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Performance is a feature! - London .NET User GroupMatt Warren
Starting with the premise that "Performance is a Feature", this session will look at how to measure, what to measure and how get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
Starting with the premise that "Performance is a Feature", Matt Warren will show you how to measure, what to measure and how to get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
The presentation covers:
Why we should care about performance
Pitfalls to avoid when measuring performance
How the .NET Garbage Collector can hurt performance
Real-world performance lessons from open-source code
The webinar recording can be found here: http://www.postsharp.net/blog/post/webinar-recording-performance-is-a-feature
Have you ever stopped to think about all the things that have to take place when you execute a .NET program? As the quote from Neal Ford says "Understand one level below your usual abstraction", this talk will look at why this is important and how can it help you if we apply it to the .NET framework. We will delve into the internals of the recently open-sourced .NET Core Runtime, looking at what happens, when it happens and why. Using freely available diagnostic tools such as PerfView, libraries including ClrMD and even the source code itself! Along the way we'll examine the Execution Engine, Type Loader, Just-in-Time (JIT) Compiler and the CLR Hosting API, to see how all these components play a part in making a 'Hello World' app possible.
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
Is your profiler speaking the same language as you? -- Docklands JUGSimon Maple
Profilers are absolute beasts. And profilers might prove useful to pinpoint the performance issues in your Java applications.
By using profilers, developers are fortunate to find the root cause of an issue at hand. However, it requires effort to actually comprehend the data collected by the profiler. Due to the inherent complexity of the data, one has to understand how this data is collected. And thus understand how the profiler actually works.
During this talk we will go through the classic profiler features. What is a hotspot? What is the difference between sampling and instrumentation from the profiler point of view? What are the problems with either of those methods? What is the time budget of the application? And more!
I will also showcase a new kid on the block among the profiling tools: XRebel. This tool provides insight into application behaviour and permits the developers to discover application level issues.
Verification of Concurrent and Distributed SystemsMykola Novik
Building correct concurrent and distributed systems is hard and very challenging task also high complexity of such software increases the probability of human error in design and architecture. On practice standard verification techniques in industry are necessary but not sufficient. In my talk we will discuss formal specification and verification language that helps engineers design, specify, reason about and verify complex, real-life algorithms and software systems.
A short and fast journey through some of the profiling options available in the Ruby 2.x world, including a look at flamegraphs and new ways of tracking memory usage in the MRI.
Down to Stack Traces, up from Heap DumpsAndrei Pangin
Глубже стек-трейсов, шире хип-дампов
Stack trace и heap dump - не просто инструменты отладки; это потайные дверцы к самым недрам виртуальной Java машины. Доклад будет посвящён малоизвестным особенностям JDK, так или иначе связанным с обоходом хипа и стеками потоков.
Мы разберём:
- как снимать дампы в продакшне без побочных эффектов;
- как работают утилиты jmap и jstack изнутри, и в чём хитрость forced режима;
- почему все профилировщики врут, и как с этим бороться;
- познакомимся с новым Stack-Walking API в Java 9;
- научимся сканировать Heap средствами JVMTI;
- узнаем о недокументированных функциях Хотспота и других интересных штуках.
.NET Fest 2019. Николай Балакин. Микрооптимизации в мире .NETNETFest
Что делать, если все, что можно уже закэшировано, а код всё ещё тормозит? В этом докладе мы обсудим, как работают некоторые низкоуровневые механизмы .NET и как мы с их помощью можем выиграть драгоценные секунды, когда счет идет на отдельные такты процессора.
Optimizing Parallel Reduction in CUDA : NOTESSubhajit Sahu
Highlighted notes on Optimizing Parallel Reduction in CUDA
While doing research work under Prof. Dip Banerjee, Prof. Kishore Kothapalli.
Interesting optimizations, i should try these soon as PageRank is basically lots of sums.
Adam Sitnik "State of the .NET Performance"Yulia Tsisyk
MSK DOT NET #5
2016-12-07
In this talk Adam will describe how latest changes in.NET are affecting performance.
Adam wants to go through:
C# 7: ref locals and ref returns, ValueTuples.
.NET Core: Spans, Buffers, ValueTasks
And how all of these things help build zero-copy streams aka Channels/Pipelines which are going to be a game changer in the next year.
Verification of Concurrent and Distributed SystemsMykola Novik
Building correct concurrent and distributed systems is hard and very challenging task also high complexity of such software increases the probability of human error in design and architecture. On practice standard verification techniques in industry are necessary but not sufficient. In my talk we will discuss formal specification and verification language that helps engineers design, specify, reason about and verify complex, real-life algorithms and software systems.
A short and fast journey through some of the profiling options available in the Ruby 2.x world, including a look at flamegraphs and new ways of tracking memory usage in the MRI.
Down to Stack Traces, up from Heap DumpsAndrei Pangin
Глубже стек-трейсов, шире хип-дампов
Stack trace и heap dump - не просто инструменты отладки; это потайные дверцы к самым недрам виртуальной Java машины. Доклад будет посвящён малоизвестным особенностям JDK, так или иначе связанным с обоходом хипа и стеками потоков.
Мы разберём:
- как снимать дампы в продакшне без побочных эффектов;
- как работают утилиты jmap и jstack изнутри, и в чём хитрость forced режима;
- почему все профилировщики врут, и как с этим бороться;
- познакомимся с новым Stack-Walking API в Java 9;
- научимся сканировать Heap средствами JVMTI;
- узнаем о недокументированных функциях Хотспота и других интересных штуках.
.NET Fest 2019. Николай Балакин. Микрооптимизации в мире .NETNETFest
Что делать, если все, что можно уже закэшировано, а код всё ещё тормозит? В этом докладе мы обсудим, как работают некоторые низкоуровневые механизмы .NET и как мы с их помощью можем выиграть драгоценные секунды, когда счет идет на отдельные такты процессора.
Optimizing Parallel Reduction in CUDA : NOTESSubhajit Sahu
Highlighted notes on Optimizing Parallel Reduction in CUDA
While doing research work under Prof. Dip Banerjee, Prof. Kishore Kothapalli.
Interesting optimizations, i should try these soon as PageRank is basically lots of sums.
Adam Sitnik "State of the .NET Performance"Yulia Tsisyk
MSK DOT NET #5
2016-12-07
In this talk Adam will describe how latest changes in.NET are affecting performance.
Adam wants to go through:
C# 7: ref locals and ref returns, ValueTuples.
.NET Core: Spans, Buffers, ValueTasks
And how all of these things help build zero-copy streams aka Channels/Pipelines which are going to be a game changer in the next year.
An Apex Approach to performance assessment
LimitsProfiler is a package that allows you to natively profile your Apex code (including a Visualforce UI). You can find it at github.com/apexlarson/LimitsProfiler
An exploration into RxJava on Android for the experienced, yet uninitiated software engineer. This presentation explores Declarative vs Imperative programming paradigms and expands the discussion into Functional Reactive Programming. It explains the benefits of the observer contract, high-order functions, and schedulers available in RxJava. It also explains the purpose of the Android integration libraries: RxAndroid, RxLifecycle, and RxBindings.
5 must have patterns for your microservice - techoramaAli Kheyrollahi
"Netflix is actually a log generating application that just happens to stream movies"
Building a service/Microservice is itself easy. Scaling it on the cloud is not that hard either but operating, maintaining and iterating a production large scale service is not just about linearisation. As Cockcroft points out, telemetry and monitoring is the most important aspect of building Microservices
We discuss 5 patterns that any serious Microservice should have:
- Canary (an endpoint reporting health of underlying dependencies)
- IO monitor (measuring all calls from Microservice to external dependencies)
- A circuit breaker
- An ActivityId-Propagator
- An exception and short timeout retry policy
QA Fest 2019. Антон Молдован. Load testing which you always wantedQAFest
Десь рік тому ми почали працювати над новою версією наших продуктів. Саме тоді ми почали випробовувати різні технології, архітектури, підходи, а головне це — міряти performance, бо без цього в highload проектах взагалі не вижити.
При проектуванні “любої” системи нам потрібно знати її ліміти:
- скільки паралельних запитів може обробити мікросервіс за допустиму latency?
- як багато запитів може витримати база даних, яку ми використовуємо?
- як довго потрібно чекати на Push повідомлення?
- як довго триває розподілена транзакція та між якими сервісами відбувається найбільша затримка?
І таких питань у нас було безліч. В процесі тестування ми використовували різний tooling: JMeter, ab, Gatling, але всі вони надавали дуже лімітовані можливості. Нам не вдавалося нормально покрити push flow (WebSockets/SSE), різні бази даних, було складно імітувати різний workloads (update/read).
На цій зустрічі я розповім про наш досвід застосування load testing:
- що використовуємо для тестування баз даних, мікросервісів;
- як готуємо Pull/Push тести та як адаптуємо тести під різні протоколи (HTTP/WebSockets/SSE);
- які виникають проблеми з замірами latency.
Моя доповідь дуже практична, тому після неї ви зможете з легкістю почати застосовувати load testing у себе на проекті.
Monitoring Complex Systems: Keeping Your Head on Straight in a Hard WorldBrian Troutwine
This talk will provide motivation for the extensive instrumentation of complex computer systems and make the argument that such systems. This talk will provide practical starting points in Erlang projects and maintain a perspective on the human organization around the computer system. Brian will focus on getting started with instrumentation in a systematic way and follow up with the challenge of interpreting and acting on metrics emitted from a production system in a way which does not overwhelm operators’ ability to effectively control or prioritize faults in the system. He’ll use historical examples and case studies from my work to keep the talk anchored in the practical.
Talk objectives:
Brian hopes to convince the audience of two things:
* that monitoring and instrumentation is an essential component of any long-lived system and
* that it's not so hard to get started, after all.
He’ll keep a clear-eyed view of what works and is difficult in practice so that the audience can make a reasoned decision after the talk.
Target audience:
This talk would appeal to engineers with long-running production employments, operations folks and Erlangers in general.
Instrumenting application code is like flossing your teeth. Developers know they ought to be doing it more often. Code instrumentation is an important practice for establishing baseline performance metrics and identifying bottlenecks. Getting the right metrics is core to understanding how much concurrency your application can handle, determining what latency is normal for the application, and indicating when performance is deviating from those norms.
While most developers acknowledge the value of instrumentation, few actually implement it. If Bytecode injection sounds as scary as a root canal, take heart, effective instrumentation doesn't have to be complicated. I've written an open-source instrumentation framework to encourage developers to get the metrics they need to pilot their application safely. We'll examine some strategies for code instrumentation, run some load tests, and make sense of the numbers.
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
Monitoring is the key to successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open source stacks easily accessible to any developer.
We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast growing, high performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there.
Similar to Where the wild things are - Benchmarking and Micro-Optimisations (20)
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
3. Premature Optimization
“We should forget about small efficiencies, say
about 97% of the time: premature
optimization is the root of all evil.Yet we
should not pass up our opportunities in that
critical 3%.“
- Donald Knuth
4. ProfilingTools
• ANTS Performance Profiler - Redgate
• dotTrace & dotMemory - Jet Brains
• PerfView - Microsoft (free)
• Visual Studio Profiling Tools (Ultimate, Premium or Professional)
• MiniProfiler - Stack Overflow (free)
10. static void Profile(int iterations, Action action)
{
action(); // warm up
GC.Collect(); // clean up
var watch = new Stopwatch();
watch.Start();
for (int i = 0; i < iterations; i++)
{
action();
}
watch.Stop();
Console.WriteLine("Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
Benchmarking small code samples in C#, can this implementation be improved?
http://stackoverflow.com/q/1047218/4500
11. private static T Result;
static void Profile<T>(int iterations, Func<T> func)
{
func(); // warm up
GC.Collect(); // clean up
var watch = new Stopwatch();
watch.Start();
for (int i = 0; i < iterations; i++)
{
Result = func();
}
watch.Stop();
Console.WriteLine("Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
Benchmarking small code samples in C#, can this implementation be improved?
http://stackoverflow.com/q/1047218/4500
12. BenchmarkDotNet project
Andrey Akinshin (the ‘Boss’)
@andrey_akinshin
http://aakinshin.net/en/blog/
Matt Warren (me)
Adam Sitnik (.NET Core guru)
@SitnikAdam
http://adamsitnik.com/
15. Benchmarking library that is:
•Accurate
•Easy-to-use
•Helpful
Stopwatch under the hood http://aakinshin.net/en/blog/dotnet/stopwatch/
LegacyJIT-x86 and first method call http://aakinshin.net/en/blog/dotnet/legacyjitx86-and-first-method-call/
Goals of BenchmarkDotNet
17. What BenchmarkDotNet doesn’t do
•Multi-threaded benchmarks
•Integrate with C.I builds
•Unit test runner integration
•Anything else?
http://github.com/dotnet/BenchmarkDotNet/issues/
18. “Other Benchmarking tools are available”
• NBench
• https://github.com/petabridge/NBench
• Microsoft Xunit performance
• http://github.com/Microsoft/xunit-performance/
• Lambda Micro Benchmarking (“Clash of the Lambdas”)
• https://github.com/biboudis/LambdaMicrobenchmarking
• Etimo.Benchmarks
• http://etimo.se/blog/etimo-benchmarks-lightweight-net-benchmark-tool/
• MeasureIt
• https://blogs.msdn.microsoft.com/vancem/2009/02/06/measureit-update-tool-for-
doing-microbenchmarks-for-net/
19. How it works
An invocation of the target method is an operation.
A bunch of operations is an iteration.
Iteration types:
• Pilot:The best operation count will be chosen.
• IdleWarmup, IdleTarget: BenchmarkDotNet overhead will be evaluated.
• MainWarmup:Warmup of the main method.
• MainTarget: Main measurements.
• Result = MainTarget – AverageOverhead
http://benchmarkdotnet.org/HowItWorks.htm
20. What happens under the covers?
Image credit Albert Rodríguez @UncleFirefox
22. Scale of benchmarks
•millisecond - ms
• One thousandth of one second, single webapp request
•microsecond - us or µs
• One millionth of one second, several in-memory operations
•nanosecond - ns
• One billionth of one second, single operations
23. Who ‘times’ the timers?
[Benchmark]
public long StopwatchLatency()
{
return Stopwatch.GetTimestamp();
}
[Benchmark]
public long StopwatchGranularity()
{
// Loop until Stopwatch.GetTimestamp()
// gives us a different value
long lastTimestamp =
Stopwatch.GetTimestamp();
while (Stopwatch.GetTimestamp() ==
lastTimestamp)
{
}
return lastTimestamp;
}
[Benchmark]
public long DateTimeLatency()
{
return DateTime.Now.Ticks;
}
[Benchmark]
public long DateTimeGranularity()
{
// Loop until DateTime.Now
// gives us a different value
long lastTimestamp = DateTime.Now.Ticks;
while (DateTime.Now.Ticks == lastTimestamp)
{
}
return lastTimestamp;
}
24. BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4800MQ CPU 2.70GHz, ProcessorCount=8
Frequency=2630673 Hz, Resolution=380.1309 ns, Timer=TSC
[Host] : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1590.0
Job-FIDMNL : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1590.0
Method | Mean | StdDev | Allocated |
--------------------- |---------------- |------------ |---------- |
StopwatchLatency | ?? ns | ?? ns | ?? B |
StopwatchGranularity | ?? ns | ?? ns | ?? B |
DateTimeLatency | ?? ns | ?? ns | ?? B |
DateTimeGranularity | ?? ns | ?? ns | ?? B |
Who ‘times’ the timers?
25. BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4800MQ CPU 2.70GHz, ProcessorCount=8
Frequency=2630673 Hz, Resolution=380.1309 ns, Timer=TSC
[Host] : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1590.0
Job-FIDMNL : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1590.0
Method | Mean | StdDev | Allocated |
--------------------- |---------------- |------------ |---------- |
StopwatchLatency | 12.9960 ns | 0.1609 ns | 0 B |
StopwatchGranularity | 374.3049 ns | 2.4388 ns | 0 B |
DateTimeLatency | 682.2320 ns | 8.9341 ns | 32 B |
DateTimeGranularity | 996,025.6492 ns | 413.9175 ns | 47.34 kB |
Who ‘times’ the timers?
26. Loop-the-Loop
”Avoid foreach loop on everything except raw arrays?”
[Benchmark(Baseline = true)]
public int ForLoopArray()
{
var counter = 0;
for (int i = 0; i < anArray.Length; i++)
counter += anArray[i];
return counter;
}
[Benchmark]
public int ForEachArray()
{
var counter = 0;
foreach (var i in anArray)
counter += i;
return counter;
}
[Benchmark]
public int ForLoopList()
{
var counter = 0;
for (int i = 0; i < aList.Count; i++)
counter += aList[i];
return counter;
}
[Benchmark]
public int ForEachList()
{
var counter = 0;
foreach (var i in aList)
counter += i;
return counter;
}
38. Search - Linear v Binary
private static int LinearSearch(
Data[] set, int key)
{
for (int i = 0; i < set.Length; i++)
{
var c = set[i].Key - key;
if (c == 0)
{
return i;
}
if (c > 0)
{
return ~i;
}
}
return ~set.Length;
}
private static int BinarySearch(
Data[] set, int key)
{
int i = 0;
int up = set.Length - 1;
while (i <= up)
{
int mid = (up - i) / 2 + i;
int c = set[mid].Key - key;
if (c == 0)
{
return mid;
}
if (c < 0)
i = mid + 1;
else
up = mid - 1;
}
return ~i;
}
39. Search - Linear v Binary
private readonly Data[][] dataSet;
private Data[] currentSet;
private int currentMid;
private int currentMax;
[Params(1, 2, 3, 4, 5, 7, 10, 12, 15)]
public int Size
{
set
{
currentSet = dataSet[value];
currentMax = value - 1;
currentMid = value / 2;
}
}
42. readonly fields
public struct Int256
{
private readonly long bits0, bits1,
bits2, bits3;
public Int256(long bits0, long bits1,
long bits2, long bits3)
{
this.bits0 = bits0; this.bits1 = bits1;
this.bits2 = bits2; this.bits3 = bits3;
}
public long Bits0 { get { return bits0; } }
public long Bits1 { get { return bits1; } }
public long Bits2 { get { return bits2; } }
public long Bits3 { get { return bits3; } }
}
private readonly Int256 readOnlyField =
new Int256(1L, 5L, 10L, 100L);
private Int256 field =
new Int256(1L, 5L, 10L, 100L);
[LegacyJitX86Job, LegacyJitX64Job, RyuJitX64Job]
public class Program
{
[Benchmark]
public long GetValue()
{
return field.Bits0 + field.Bits1 +
field.Bits2 + field.Bits3;
}
[Benchmark]
public long GetReadOnlyValue()
{
return readOnlyField.Bits0 +
readOnlyField.Bits1 +
readOnlyField.Bits2 +
readOnlyField.Bits3;
}
}
This is what we aim for
This is why we wanted to build BenchmarkDotNet
Aim is to make a framework that can accurately measure milli, micro and nano benchmarks.But in reality the main use-cases are probably milli/micro benchmarks, so these must work above all else!!
(Even lower down!!!)
picoseconds: 1…1000 ps
One trillionth of one second, pipelining
Avoid foreach loop on everything except raw arrays