.NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund
•Download as PPTX, PDF•
0 likes•227 views
.NET Core Summer event, 2019 in Brno, CZ - 2019/7/9 Talk: .NET Core Networking stack and performance by Karel Zikmund https://www.wug.cz/brno/akce/1152--NET-Core-Summer-Event
Recommended
More Related Content
What's hot
What's hot (20)
Similar to .NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund
Similar to .NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund (20)
More from Karel Zikmund
More from Karel Zikmund (20)
Recently uploaded
Recently uploaded (20)
.NET Core Summer event 2019 in Brno, CZ - .NET Core Networking stack and performance -- Karel Zikmund
- 1. .NET Core Networking stack and Performance .NET Core Summer event 2019 – Brno, CZ Karel Zikmund ( @ziki_cz)
- 2. Agenda • Networking stack architecture evolution • .NET Framework, UWP and .NET Core • Networking stack in .NET Core • Direction and plans • Status & perf results • General BCL performance thoughts & observations
- 3. .NET Framework UWP .NET Core Networking – Architecture Evolution HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5
- 4. .NET Framework UWP .NET Core Windows Linux / Mac Sockets, SslStream, … Sockets SslStream, Dns, … Windows Networking – Architecture Evolution HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 HttpWebRequest + ServicePoint HttpClient + HttpClientHandler WinRT APIs win9net.dll HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0
- 5. Networking – Architecture Evolution .NET Framework HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 .NET Core Windows Linux / Mac Sockets, SslStream, … HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0
- 6. Networking – Architecture Evolution .NET Framework HttpWebRequest + ServicePoint Sockets SslStream, Dns, … Windows HttpClient + HttpClientHandler 4.5 .NET Core Windows Linux / Mac Sockets, SslStream, … HttpClient WinHttpHandler WinHttp.dll CurlHandler libcurl OpenSSL HttpWebRequest + ServicePoint 2.0 .NET Core 2.1 HttpWebRequest + ServicePoint HttpClient ManagedHandler Windows Linux / Mac OpenSSL Sockets SslStream, Dns, …
- 7. Networking – Technical Roadmap https://github.com/dotnet/designs/issues/9 1. Foundation – rock solid • Sockets, SSL, DNS 2. Web stack (client) – high perf & consistency • HttpClient, ClientWebSocket 3. Emerging technologies • HTTP/2 … coming in .NET Core 3.0 as opt-in, mainly for gRPC scenarios • RIO, QUIC 4. Maintenance components • (Http/Ftp/File)WebRequest + ServicePoint, Mail, HttpListener
- 8. Networking – Investments https://github.com/dotnet/designs/issues/9 1. Foundation – rock solid • Sockets, SSL, DNS 2. Web stack (client) – high perf & consistency • HttpClient, ClientWebSocket 3. Emerging technologies • HTTP/2 … coming in .NET Core 3.0 as opt-in, mainly for gRPC scenarios • RIO, QUIC 4. Maintenance components • (Http/Ftp/File)WebRequest + ServicePoint, Mail, HttpListener
- 9. Networking – Investments short-term SocketsHttpHandler – introduced in .NET Core 2.1 (1 year) Next steps: • Documentation (API / conceptual) • Production diagnostics • Testing – multi-machine, load / perf / stress automation • High-perf scenarios – new APIs? • Sync APIs on HttpClient/SocketsHttpHandler • Backlog (15% of all CoreFX) • More perf focus … of course
- 10. Networking – Focus on Perf Scenarios / workloads: • Micro-benchmarks vs. benchmarks vs. real-world scenarios (feedback) Metrics: • RPS (Response per second) • Throughput • Latency • Connection density Important properties: • Percentiles (95% / 99%) • Scale up • Resources (CPU) utilization (90%-95% ideal)
- 11. Networking – Perf Test Environment • Repeatability – isolated environment (reduce noise) • 2 machines: • 4-core • 16 GB RAM • 2x NIC: 1x 1 Gbps + 1x 10 Gbps • 8 servers: • 12-core • 64 GB RAM • 2x NIC: 1x 1 Gbps + 1x 40 Gbps A B External network 10 Gbps 1 Gbps 1 Gbps External network S.1 S.8… 40 Gbps 1 Gbps 1 Gbps 1 Gbps
- 12. Networking – Sockets Perf Results • Micro-benchmark only (disclaimer: Netty/Go impl may be inefficient) • Linux 2 CPUs: • Gap for small payloads (driver problem) • SSL: • 22% gap (to be investigated) GB/s 256 B 4 KB 64 KB 1 MB .NET Core 0.09 0.77 1.09 1.10 Netty 0.11 0.48 0.66 0.67 Go 0.12 0.82 1.10 1.11 SSL - GB/s 256 B 4 KB 64 KB 1 MB .NET Core 0.04 0.31 0.71 0.87 Netty 0.03 0.12 0.15 0.15 Go 0.06 0.56 0.98 1.12
- 13. Networking – SocketsHttpHandler • Introduced in .NET Core 2.1 … May 2018 • Windows: Parity with Go • Linux: 15% gap against Windows (hard problem)
- 14. Networking – SSL Perf • Indicators: • Sockets micro-benchmarks (22% overhead) • TechEmpower benchmark • Customer reports • Attempt for rewrite by community (@drawaes) • Some end-to-end wins, but no micro-benchmark wins • Next steps: • Measure & analyze micro-benchmarks & end-to-end scenarios
- 15. Industry Benchmarks • TechEmpower benchmark • More end-to-end, with DB, etc. • Useful for overall platform performance comparison • Round 17
- 16. Industry Benchmarks - History • Round 15 (from end of 2017 … 1.5 years old) • ASP.NET Core at #5 entry (jump from #14 in Round 14)
- 17. Industry Benchmarks Platform performance shows how fast your app could be, one day … but it is not everything: • Productivity • Tooling • Developer availability (in-house/to hire) • Documentation • Community • etc.
- 18. Performance – Lessons Learned • Plan for performance during design • Understand scenario, set goals • Prototype and measure early • Optimize what’s important – measure • Avoid micro-optimizations • Understand the big picture • Don’t guess root cause – measure • Minimize repro – it’s worth it!
- 19. BCL Performance • Fine-tuned over 15 years • Opportunities are often trade-offs (memory vs. speed, etc.) • How to identify scenarios which matter • Customer feedback • OSS helps: • More eyes on code • More reports • Motivated contributors • Perf improvements in .NET Core (.NET blog post by Stephen Toub) • More Span/friends usage, arrays, strings, parsing, RegEx, Collections, Networking, IO
- 20. BCL Performance – What not to take? • Specialized collections • BCL designed for usability and decent perf for 95% customers • Code complexity (maintainability) vs. perf wins • APIs for specialized operations (e.g. to save duplicate lookup) • Creates complexity • May leak implementation into API surface
- 21. Wrap Up • Proactive investments into .NET Networking stack • Consistency across platforms • Great performance for all workloads • Ongoing scenario/feedback-based improvements in BCL perf • Performance in general is: • Important • But not the only important thing • Tricky to get right in the right place
Editor's Notes
- Why Networking? Networking is even more important in many modern workloads – microservices, Cloud I work on team making some of these cool investments in Networking and BCL These are NOT the only investments in BCL and .NET/.NET Core – just those I have close relation to at my daily job
- Client stack HttpWebRequest since .NET 1.0 Exceptions on errors (404) 4.5 added HttpClient, driven by WCF Pipeline architecture (extensibility) No http2 support Original plan to later re-wire HttpClient directly on fundamentals – huge compat!!!
- UWP: WinRT APIs designed by Windows Networking team at he same time as 4.5 – almost 1:1 mapping win9net.dll is client library - http2 support .NET Core: http2 support – server library HttpWebRequest for compatibility in .NET Core 2.0 (.NET Standard 2.0), but “obsoleted” libcurl with NSS, not just OpenSSL Different behaviors on different OS’s or even Linux distros – inconsistencies in behavior
- Key values: Consistency and Perf wins
- Foundation – performance (& reliability) Usable for both server and client scenarios Web stack – consistency & performance (& reliability) Important for middleware scenarios (not just 1 server) Emerging technologies – new protocols, capabilities and performance motivated RIO = Registered I/O (Win8+) QUIC = Quick UDP Internet Protocol (TCP/TSL replacement) … Latency improvements (esp. for streaming) Maintenance components – minimal investments – mostly for reliability and the most necessary standards support (preserve customers’ investments)
- Foundation – performance (& reliability) Usable for both server and client scenarios Web stack – consistency & performance (& reliability) Important for middleware scenarios (not just 1 server) Emerging technologies – new protocols, capabilities and performance motivated RIO = Registered I/O (Win8+) QUIC = Quick UDP Internet Protocol (TCP/TSL replacement) … Latency improvements (esp. for streaming) Maintenance components – minimal investments – mostly for reliability and the most necessary standards support (preserve customers’ investments)
- RPS (Response per second) – e.g. web servers Throughput – e.g. streaming video/music Latency – e.g. real-time trading Connection density – e.g. messaging services, IoT/devices
- Repeatability Non-networking micro-benchmarks – time (wallclock), memory … classic CLR perf lab disables many OS features Cloud (Azure) attempt Full control of environment needed
- Example how we measure perf and compare, not statement about platform comparison Column is payload size (1B -> 1MB, each column is 16x bigger than previous one) Note: RPS as metric in second table better tells the story of scaling based on payload size 22% gap on SSL against Go Note: .NET Core 1.1 was at 0.47 GB/s at 1MB – 2x improvement in 2.0/2.1
- Sockets micro-benchmarks – 22% gap TechEmpower benchmark - http vs. https larger diff than other platforms Historical reports on some .NET Framework scenarios: https vs. http
- Warning: Not everyone builds hyper-scale service Even if you think you are, don’t forget that most scaling apps are rewritten every 2-3 years You don’t have to be perfect on day 1, evolve over time
- Design performance E.g. for micro-services architecture, model the workloads, use fake data / CPU/network utilization Optimize what’s important: 90-10 rule (95-5) … what contributes to app performance Story: Arguing over d.s. List vs. array when the data is <100 items and the service has 3 hops between servers Guess root cause mistakes: Conclusions based on 1 dump (or 2 in better case)
- Thousands of APIs, hard to pick the right ones Problem: Using telemetry from MS/partners, partner teams reports, working closely with customers and community OSS: We often ask about the scenario, to understand the big picture Examples: (quite often 30% and more)