.NET Memory Primer (Martin Kulov)

•

1 like•527 views

In this session we will look over the various ways .NET is collecting memory, tips how to help GC perform better and tools that will save your day. This is a must attend session for those who still do not know how to troubleshoot memory issues. For the rest it is a nice refresh and new look of features in .NET 4.5. As usual there will be lots of demos.

Technology

Premium community conference on Microsoft technologies itcampro@ itcamp14#
.NET Memory Primer
Martin Kulov
martin@kulov.net

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Huge thanks to our sponsors & partners!

Premium community conference on Microsoft technologies itcampro@ itcamp14#
"Out of CPU, memory and disk, memory is typically
the most important for overall system performance."
Mark Russinovich
“All you worry about in a .NET application is the
memory.”
John Robbins

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• x86
–2 ^ 32 bits = 4GB /0x FFFF FFFF/
• x64
–2 ^ 64 bits = 16 EB /0x FFFF FFFF' FFFF FFFF/
Virtual Memory Limits

Premium community conference on Microsoft technologies itcampro@ itcamp14#
x86 Memory Mapping
* PFN - Page Frame Number database

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Max 4GB
–Windows Client, Windows Srv 2008 Standard
• Max 128GB
–Windows Srv 2003 SP1 Datacenter (PAE)
x86 Physical Memory Limits

Premium community conference on Microsoft technologies itcampro@ itcamp14#
x64 Memory Mapping

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Max 4TB - Windows Srv 2012 Standard
• Limited per SKU
x64 Physical Memory Limits

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Canonical Form Addresses
48-bit implementation 56-bit implementation 64-bit implementation

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Virtual Address Space

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Code
• Data
• Stacks
• Heaps
User Mode Memory

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Created for Each Thread
• Default to 1MB
• Hold Method Data /stack frame/
–Parameters
–Local variables
–Return address
• First In, First Out
Stacks

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Stack Layout

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• ChildEBP RetAddr Caller,Callee
• 08e4e1a4 751b149d _WaitForSingleObjectEx@12+0x98, calling _ZwWaitForSingleObject@12
• 08e4e1e8 718b53c2 ?LeaveRuntimeNoThrow@Thread@@SGJI@Z+0xd7, calling __EH_epilog3
• 08e4e210 755b1194 _WaitForSingleObjectExImplementation@12+0x75, calling
_WaitForSingleObjectEx@12
• 08e4e228 718b54d7 ?LoadImage@PEImage@@SGPAV1@PAUHINSTANCE__@@@Z+0x1af
• ...
• 08e4f5f4 71a10647 ?intermediateThreadProc@Thread@@CGKPAX@Z+0x49
• 08e4f784 71a10635 ?intermediateThreadProc@Thread@@CGKPAX@Z+0x37, calling
__alloca_probe_16
• 08e4f798 755b336a @BaseThreadInitThunk@12+0xe
• 08e4f7a4 77639f72 ___RtlUserThreadStart@8+0x70
• 08e4f7e4 77639f45 __RtlUserThreadStart@8+0x1b, calling ___RtlUserThreadStart@8
Call Stack Example

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Hold Dynamically Allocated Data
• 1 Heap Per Logical Processor
• Heap Has Segments
–Process Heap
–Code Heap /JITed code/
–Small Object Heap /SOH/
–Large Object Heap /LOH/
Heaps

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Stack
–Value Types /Int32, Bool, Struct, etc…/
–Pointers to Reference Types
• Heap
–Reference Types /Object, String, Array, etc…/
–Free Areas
Allocating .NET Memory

Premium community conference on Microsoft technologies itcampro@ itcamp14#
DEMO: ALLOCATING MEMORY

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Stack References
• Static References /Fields, ThreadStatic/
• CPU Registers
• Interop References /COM, API calls/
• Finalization Queue References
Object Roots /GC Roots/

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• a.k.a. Generational Garbage Collector /GC/
• Three Generations /SOH/
–Gen0 – short lived
–Gen1 – medium lived
–Gen2 – long lived
Nondeterministic Finalization

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Before GC #1
Gen1 Gen0
Before GC #500
Gen2
Gen2
Gen2 Gen1 Gen0
Gen0
Before GC #0
Before GC #2
Gen2 Gen1 Gen0
Before GC #100
Gen2
Gen2 Gen1 Gen0

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Gen0 is Full
• Induced GC /System.GC.Collect()/
• System Pressure
Collection - When

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Rule of Thumb – Ratio 1:10:100
• .NET CLR Memory% time in GC
• .NET CLR Memory# Induced GC
• .NET CLR Memory# Gen X collections
Collection - Cost

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Size > 85KB
• Memory is marked as free during Gen2
• Avoid Temporary Large Objects
• Reuse Objects in LOH If Possible
• Many LOH Segments
• Fragmentation Problems
Large Object Heap

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Suspend Managed Threads
• Collect Garbage
• Resume Managed Threads
• Two Phases of GC
–Mark
–Compact
Collection - How

Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Workstation GC – Non Concurrent
• Server GC – Non Concurrent
• Workstation GC – Concurrent
–Background GC /New in .NET 4/
• Server GC – Background /New in .NET 4.5/
GC Types

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Workstation GC

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Server GC

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Concurrent GC – Before and After

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Testing Concurrent Server GC

Premium community conference on Microsoft technologies itcampro@ itcamp14#
Q & A

What's hot

Mongodb meetup

Eytan Daniyalzade

Introduction to mongo db

Lawrence Mwai

Parallel computing with GPars

Pablo Molnar

The cardinality of monitoring data we are collecting today continues to rise, in no small part due to the ephemeral nature of containers and compute platforms like Kubernetes. Querying a flat dataset comprised of an increasing number of metrics requires searching through millions and in some cases billions of metrics to select a subset to display or alert on. The ability to use wildcards or regex within the tag name and values of these metrics and traces are becoming less of a nice-to-have feature and more useful for the growing popularity of ad-hoc exploratory queries. In this talk we will look at how Prometheus introduced the concept of a reverse index existing side-by-side with a traditional column based TSDB in a single process. We will then walk through the evolution of M3’s metric index, starting with ElasticSearch and evolving over the years to the current M3DB reverse index. We will give an in depth overview of the alternate designs and dive deep into the architecture of the current distributed index and the optimizations we’ve made in order to fulfill wildcards and regex queries across billions of metrics.

FOSDEM 2020: Querying over millions and billions of metrics with M3DB's index

Rob Skillington

FleetDB

Diego Pacheco

Ceph Month 2021: RADOS Update

Ceph Community

Ai meetup Neural machine translation updated

2040.io

Bsdtw17: mariusz zaborski: case studies of sandboxing base system with capsicum

Scott Tsai

DConf 2016: Keynote by Walter Bright

Andrei Alexandrescu

Ceph is a mature open source software-defined storage solution that was created over a decade ago. During that time new faster storage technologies have emerged including NVMe and Persistent memory. The crimson project aim is to create a better Ceph OSD that is more well suited to those faster devices. The crimson OSD is built on the Seastar C++ framework and can leverage these devices by minimizing latency, cpu overhead, and cross-core communication. This talk will discuss the project design, our current status, and our future plans.

Crimson: Ceph for the Age of NVMe and Persistent Memory

ScyllaDB

Performance

Christophe Marchal

In a Nutshell: Rancher

Jeffrey Sica

In this talk we'll showcase the new support for LXD containers in OpenNebula. The talk will describe the basic functionality of the new drivers and will provide some hints on the integration internals. LXD support will be released in OpenNebula 5.8 and it will let you manage LXD containers in your cloud using the same interfaces as with VMs, leveraging all the OpenNebula ecosystem and functionality including: Marketplace, multi-tenancy or service composition with OpenNebula Flow.

OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...

OpenNebula Project

Ceph is an open source distributed file system addressing file, block, and object storage use cases. Next generation storage devices require a change in strategy, so the community has been developing crimson-osd, an eventual replacement for ceph-osd intended to minimize cpu overhead and improve throughput and latency. Seastore is a new backing store for crimson-osd targeted at emerging storage technologies including persistent memory and ZNS devices.

Seastore: Next Generation Backing Store for Ceph

ScyllaDB

Logs are one of the most important sources to monitor and reveal some significant events of interest. In this presentation, we introduced an implementation of log streams processing architecture based on Apache Flink. With fluentd, different kinds of emitted logs are collected and sent to Kafka. After having processed by Flink, we try to build a dash board utilizing elasticsearch and kibana for visualization.

Log Event Stream Processing In Flink Way

George T. C. Lai

mypipe: Buffering and consuming MySQL changes via Kafka

Hisham Mardam-Bey

This presentation is for Go developers and operators of Go applications who are interested in reducing costs and latency, or debugging problems such as memory leaks, infinite loops, performance regressions, etc. of such applications. We'll start with a brief description of the unique aspects of the Go runtime, and then take a look at the builtin profilers as well as Go's execution tracer. Additionally we'll look at the interoperability with popular observability tools such as Linux perf and bpftrace. After this presentation you should have a good idea of the various tools you can use, and which ones might be the most useful to you in a production environment.

Continuous Go Profiling & Observability

ScyllaDB

Clique square storage

INRIA-OAK

Declarative Infrastructure Tools

Yulia Shcherbachova

What's hot (19)

Mongodb meetup

Introduction to mongo db

Parallel computing with GPars

FOSDEM 2020: Querying over millions and billions of metrics with M3DB's index

FleetDB

Ceph Month 2021: RADOS Update

Ai meetup Neural machine translation updated

Bsdtw17: mariusz zaborski: case studies of sandboxing base system with capsicum

DConf 2016: Keynote by Walter Bright

Crimson: Ceph for the Age of NVMe and Persistent Memory

Performance

In a Nutshell: Rancher

OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...

Seastore: Next Generation Backing Store for Ceph

Log Event Stream Processing In Flink Way

mypipe: Buffering and consuming MySQL changes via Kafka

Continuous Go Profiling & Observability

Clique square storage

Declarative Infrastructure Tools

Similar to .NET Memory Primer (Martin Kulov)

ITCamp 2011 - Catalin Zima - Common pitfalls in Windows Phone 7 game development

ITCamp

14 scaleabilty wics

ashish61_scs

ITCamp 2011 - Cristian Lefter - SQL Server code-name Denali

ITCamp

Cryptography - You're doing it wrong! (Attila Balazs)

ITCamp

Machine Learning at the Limit John Canny, UC Berkeley How fast can machine learning and graph algorithms be? In "roofline" design, every kernel is driven toward the limits imposed by CPU, memory, network etc. This can lead to dramatic improvements: BIDMach is a toolkit for machine learning that uses rooflined design and GPUs to achieve two- to three-orders of magnitude improvements over other toolkits on single machines. These speedups are larger than have been reported for *cluster* systems (e.g. Spark/MLLib, Powergraph) running on hundreds of nodes, and BIDMach with a GPU outperforms these systems for most common machine learning tasks. For algorithms (e.g. graph algorithms) which do require cluster computing, we have developed a rooflined network primitive called "Kylix". We can show that Kylix approaches the rooline limits for sparse Allreduce, and empirically holds the record for distributed Pagerank. Beyond rooflining, we believe there are great opportunities from deep algorithm/hardware codesign. Gibbs Sampling (GS) is a very general tool for inference, but is typically much slower than alternatives. SAME (State Augmentation for Marginal Estimation) is a variation of GS which was developed for marginal parameter estimation. We show that it has high parallelism, and a fast GPU implementation. Using SAME, we developed a GS implementation of Latent Dirichlet Allocation whose running time is 100x faster than other samplers, and within 3x of the fastest symbolic methods. We are extending this approach to general graphical models, an area where there is currently a void of (practically) fast tools. It seems at least plausible that a general-purpose solution based on these techniques can closely approach the performance of custom algorithms. Bio John Canny is a professor in computer science at UC Berkeley. He is an ACM dissertation award winner and a Packard Fellow. He is currently a Data Science Senior Fellow in Berkeley's new Institute for Data Science and holds a INRIA (France) International Chair. Since 2002, he has been developing and deploying large-scale behavioral modeling systems. He designed and protyped production systems for Overstock.com, Yahoo, Ebay, Quantcast and Microsoft. He currently works on several applications of data mining for human learning (MOOCs and early language learning), health and well-being, and applications in the sciences.

SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...

Chester Chen

Title: Sista: Improving Cog’s JIT performance Speaker: Clément Béra Thu, August 21, 9:45am – 10:30am Video Part1 https://www.youtube.com/watch?v=X4E_FoLysJg Video Part2 https://www.youtube.com/watch?v=gZOk3qojoVE Description Abstract: Although recent improvements of the Cog VM performance made it one of the fastest available Smalltalk virtual machine, the overhead compared to optimized C code remains important. Efficient industrial object oriented virtual machine, such as Javascript V8's engine for Google Chrome and Oracle Java Hotspot can reach on many benchs the performance of optimized C code thanks to adaptive optimizations performed their JIT compilers. The VM becomes then cleverer, and after executing numerous times the same portion of codes, it stops the code execution, looks at what it is doing and recompiles critical portion of codes in code faster to run based on the current environment and previous executions. Bio: Clément Béra and Eliot Miranda has been working together on Cog's JIT performance for the last year. Clément Béra is a young engineer and has been working in the Pharo team for the past two years. Eliot Miranda is a Smalltalk VM expert who, among others, has implemented Cog's JIT and the Spur Memory Manager for Cog.

Sista: Improving Cog’s JIT performance

ESUG

So you've learned what elasticity means and why it is important to consider scalability in your cloud application architecture. But how can you easily manage your code in order to implement all the theories around scalability? During this session, I will talk and demo the most common patterns used when designing a cloud application: the valet key pattern, the sharding pattern, the materialized view pattern, the event sourcing pattern and the CQRS pattern.

Patterns for Scalability in Windows Azure Applications (Alex Mang)

ITCamp

Linux Perf Tools

Raj Pandey

Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...

Daosheng Mu

Important amounts of information are circulating in big companies in order to conduct current tactical or strategic operations. Information systems implemented have a life cycle that provides functionality for a long time. However, from time to time current systems run outdated technology, so they must be replaced with others that provide more flexibility and functionality. The challenge is to ensure the transfer of data, information and functionality from an old system to another. The subject of this presentation is to expose a success story in migrating data and information within the company OMV/Petrom, which involved the transfer of information content of the eRoom document management system to SharePoint 2010. We detail the design stages of migration, triggering the process, its performance and the effective closure of the old platform. Each stage has a certain specific technical and managerial course also incidents that were successfully overcome in a project conducted over a year.

Migrating from eRoom to SharePoint, A Success Story (Valy Greavu)

ITCamp

ITCamp 2013 - Martin Kulov - Demystifying Visual Studio 2012 Performance Tools

ITCamp

ITCamp 2013 - Alessandro Pilotti - Git crash course for Visual Studio devs

ITCamp

amrapali builders @@ hacking challenges.pdf

amrapalibuildersreviews

There’s a special Microsoft product that has come a long way since its first OS/2 version. Actually this year, the 12th version of the product marks 25 years from the start, 25 years of Microsoft SQL Server. Over the years, we have seen many amazing and innovative technologies such as SQL CLR, Service Broker, Resource Governor or recently the cloud-based version of SQL Server. And each time we concluded that we have seen it all, another version came to surprise us. SQL Server 2014 is no exception and I guarantee that some of its features will know your socks off! I will say just “In-Memory OLTP” and “Updatable Clustered Columnstore Indexes” and it should be enough. But there’s more than that! Come to this session to find out for yourself!

SQL Server 2014 for Developers (Cristian Lefter)

ITCamp

ITCamp 2013 - Tobiasz Koprowski - 2AM A Disaster Just Began

ITCamp

2AM. We sleeping well. And our mobile ringing and ringing. Message: DISASTER! In this session (on slides) we are NOT talk about potential disaster (such BCM); we talk about: What happened NOW? Which tasks should have been finished BEFORE. Is virtual or physical SQL matter? We talk about systems, databases, peoples, encryption, passwords, certificates and users. In this session (on few demos) I'll show which part of our SQL Server Environment are critical and how to be prepared to disaster. In some documents I'll show You how to be BEST prepared.

KoprowskiT_it_camp2013 - 2amADisasterJustBegan

Tobias Koprowski

You played around with containers? You feel you can handle the adrenaline rush of publishing your containers in production? Well hold on there because there are some aspects you need to consider before you start rushing to production. How you will handle auto-scalling? What about updates / upgrades? Downtime of your app? Version 1 and Version 2? CI/CD? Etc. This session is about deploying your services on containers using the Azure Kubernetes managed offering. You will learn about what problems you might encounter and how to handle them during your deployment journey, and we will cover the main features of Kubernetes and how they can be of use to you

ITCamp 2019 - Florin Loghiade - Azure Kubernetes in Production - Field notes...

ITCamp

ITCamp 2013 - Lorant Domokos - Chasing the one codebase, multiple platforms d...

ITCamp

causos da linha de frente - #rsonrails 2011

bzanchet

In this video from Switzerland HPC Conference, Martin Hilgeman from Dell presents: HPC Workload Efficiency and the Challenges for System Builders. "With all the advances in massively parallel and multi-core computing with CPUs and accelerators it is often overlooked whether the computational work is being done in an efficient manner. This efficiency is largely being determined at the application level and therefore puts the responsibility of sustaining a certain performance trajectory into the hands of the user. It is observed that the adoption rate of new hardware capabilities is decreasing and lead to a feeling of diminishing returns. This presentation shows the well-known laws of parallel performance from the perspective of a system builder. It also covers through the use of real case studies, examples of how to program for energy efficient parallel application performance." Watch the video: http://wp.me/p3RLHQ-gIS Learn more: http://dell.com and http://www.hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Trends in Systems and How to Get Efficient Performance

inside-BigData.com

Similar to .NET Memory Primer (Martin Kulov) (20)

ITCamp 2011 - Catalin Zima - Common pitfalls in Windows Phone 7 game development

14 scaleabilty wics

ITCamp 2011 - Cristian Lefter - SQL Server code-name Denali

Cryptography - You're doing it wrong! (Attila Balazs)

SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...

Sista: Improving Cog’s JIT performance

Patterns for Scalability in Windows Azure Applications (Alex Mang)

Linux Perf Tools

Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...

Migrating from eRoom to SharePoint, A Success Story (Valy Greavu)

ITCamp 2013 - Martin Kulov - Demystifying Visual Studio 2012 Performance Tools

ITCamp 2013 - Alessandro Pilotti - Git crash course for Visual Studio devs

amrapali builders @@ hacking challenges.pdf

SQL Server 2014 for Developers (Cristian Lefter)

ITCamp 2013 - Tobiasz Koprowski - 2AM A Disaster Just Began

KoprowskiT_it_camp2013 - 2amADisasterJustBegan

ITCamp 2019 - Florin Loghiade - Azure Kubernetes in Production - Field notes...

ITCamp 2013 - Lorant Domokos - Chasing the one codebase, multiple platforms d...

causos da linha de frente - #rsonrails 2011

Trends in Systems and How to Get Efficient Performance

More from ITCamp

ITCamp 2019 - Stacey M. Jenkins - Protecting your company's data - By psychol...

ITCamp

ITCamp 2019 - Silviu Niculita - Supercharge your AI efforts with the use of A...

ITCamp

Understanding skills is key to managing any organisation. Skills are not necessarily related to your job, your qualifications or your studies, they are related to what you can do and the responsibilities you have (or should have) within your organisation. Through a systematic and structured approach to understanding, analysing and classifying skills, the business can become more effective, staff has a better understanding of their roles and responsibilities, there is increased job satisfaction, and clear career and training progression plans can be defined.

ITCamp 2019 - Peter Leeson - Managing Skills

ITCamp

ITCamp 2019 - Mihai Tataran - Governing your Cloud Resources

ITCamp

Color. It has the power to evoke emotions and empower the effectiveness of a product, but it also has the ability to ruin otherwise meticulously crafted user experiences. It often rules from the shadows, disguised as a purely aesthetic element and a mean of beautification. Let’s see how to overtake control and strategically use color in digital product development. Product teams often fail to remember that color has an enormous impact on our response to visual stimulation during human-computer interaction. The most immediate and direct psychological impact on experiences is of course - color. With its complexity and various levels of subconscious effects, it triggers an emotional response. Color doesn’t live in a vacuum, and we need to start considering it in the context of use. There are many aspects that we need to take into account: target audience and their potential visual impairments, cultural background and individual difference, previous experiences and memories, the physical environment of use and compliance with the brand. In this talk, we will immerse into approaches and best practices that product teams should take for strategic use of color in their product design process. After a basic introduction to color theory and psychology (to make sure everyone is up to speed), we will elaborate in detail how even subtle differences in color schemes have a significant impact on interface perception and product success. We will show a series of interface examples we tested on various users and do some live testing on site as well.

ITCamp 2019 - Ivana Milicic - Color - The Shadow Ruler of UX

ITCamp

Clean Architecture as a term is around for a while. However, the path to implement it is not always clear nor easy to follow. When projects fail for reasons that are primary technical, the reason is often uncontrolled complexity. The complexity goes out of hand when the code lacks structure, when it lacks Clean Architecture. In this session, I will show how to achieve consistency by implementing Clean Architecture through structure, rather than relying on discipline only. We will look at some basic building blocks of an application infrastructure which will enforce the way dependencies are created, how dependency injection is used or how separation of the data access concerns is enforced.

ITCamp 2019 - Florin Coros - Implementing Clean Architecture

ITCamp

ITCamp 2019 - Florin Flestea - How 3rd Level support experience influenced m...

ITCamp

Let's face it, our world will be taken over by robots, or at least our jobs as the scary ML & AI speculations seem to say. But until that day arrives, I want to take you on a hypothetical journey of designing and creating a fully automated restaurant of the future, where a fine tuned and efficiently orchestrated group of RoboChefs will cook your desired meal perfectly each time. And all of this is possible thanks to Actions, Timers, Monitors, Orchestrators, Sub-Orchestrators and more, all concepts from Azure Durable Functions, the real focus of this session, an extension to Functions that adds state, and which are part of Azure's Serverless Compute technologies.

ITCamp 2019 - Emil Craciun - RoboRestaurant of the future powered by serverle...

ITCamp

ITCamp 2019 - Eldert Grootenboer - Cloud Architecture Recipes for The Enterprise

ITCamp

Blockchain is one of the main legal tech trends today and, like any new technology, comes with strings attached. Issues like enforceability of smart contracts, performance risks, data privacy and compliance with various regulations in different jurisdictions are main legal concerns. The session will focus on the main legal risks by means of case studies and offer a hands-on approach for risk management in case of blockchain and architectures of distributed ledgers.

ITCamp 2019 - Cristiana Fernbach - Blockchain Legal Trends

ITCamp

ML.NET is an open source, machine learning framework built in .NET and runs on Windows, Linux and macOS. It allows developers to integrate custom machine learning into their applications without any prior expertise in developing or tuning machine learning models. Enhance your .NET apps with sentiment analysis, price prediction, fraud detection and more using custom models built with ML.NET In this Session, Andy will show not only the core of ML.NET but best practices around Azure Data Lake and data in general when using .NET

ITCamp 2019 - Andy Cross - Machine Learning with ML.NET and Azure Data Lake

ITCamp

Andy Cross, Director of Elastacloud, Microsoft Regional Director, Azure MVP and all round good guy, gives a session on how to successfully build or transform a business using AI technologies. Over the last years, Elastacloud have delivered analytics projects to a variety of customers. The greatest challenges around AI are both technical and organisational. The existing landscape of process and strategy doesn't solve these challenges in combination, and the gap between causes friction and the failure of AI projects. When modelling the outcome of actions that were informed by AI, possibly enacted by AI, the standard risk modelling approaches need to be transformed to include a factor that can change over time to represent the effectiveness of the AI solutions. Given that we should accept errors as part of the AI solution, and that errors are reinforcing of better future decisions, we need to project risk as a decreasing vector over time.

ITCamp 2019 - Andy Cross - Business Outcomes from AI

ITCamp

ITCamp 2019 - Andrea Saltarello - Modernise your app. The Cloud Story

ITCamp

Thanks to the recently released v4 of the Bot Framework SDK, creating your first bot is a breeze; still, implementing a production viable one is no easy task since several aspects must be taken into account such as user authentication, integration within existing apps, multi language support, technical considerations (e.g.: Azure Functions vs. MVC Core, Blob Storage vs. CosmosDB) and, last but not least, operational costs. Moreover, you might want to reuse your bot’s Azure hosted, Cognitive Services-backed code to address Amazon’s Alexa users to avoid the need to implement (and evolve) it twice. Eager to learn how to do that for real? Don’t miss this code-based talk then.

ITCamp 2019 - Andrea Saltarello - Implementing bots and Alexa skills using Az...

ITCamp

ITCamp 2019 - Alex Mang - I'm Confused Should I Orchestrate my Containers on ...

ITCamp

You may have heard me talk about the capabilities of Azure Logic Apps and Azure Functions before, but now I'm taking it up a few notches! And this is mostly because a lot of things have changed over the past few months in terms of serverless and cloud-native applications. Join me at this session during which you will get to do a deep dive with me on the ins and outs of Azure Functions when it comes to developer real applications, not just 'Hello, World's and the brand-new, top-notch Azure Service Fabric Mesh offering. I will finger point each bad practice and the things you should avoid, but at the end of the day we'll have created a highly scalable, production-ready application. So, how far and how fast can we actually go... now?

ITCamp 2019 - Alex Mang - How Far Can Serverless Actually Go Now

ITCamp

Marcus Vitruvius Pollio, commonly known as Vitruvius, was a Roman author, architect, civil engineer and military engineer during the 1st century BC. He is known for his multi-volume work entitled “De architectura” and his discussion of perfect proportion in architecture and the human body, which led, among others to the famous drawing by Leonardo da Vinci called the “Vitruvian Man”. Within the principles of “Vitruvian Quality”, we seek to find those perfect proportions and how to align all components of the business architecture in order to make them fit the human needs of the impacted stakeholders.

ITCamp 2019 - Peter Leeson - Vitruvian Quality

ITCamp

This session might look like a joke, and it partially is. On one hand it is a parody about how the most recent trends in industry can significantly increase the cost associated with launching an application (design, development, hosting & operations, etc). However, it is also a live demo of how you can incrementally evolve your application to take advantage of all the cool technologies out there without needing the actual a million dollars.

ITCamp 2018 - Ciprian Sorlea - Million Dollars Hello World Application

ITCamp

JavaScript is no longer meant just for front-end or for scripting kiddies to play with. And it's no longer just a language, it's become an entire ecosystem, a lifestyle. However, it has its downsides. And TypeScript is here to fill in some of the gaps. In this session we will look at how to use TypeScript along with some other technologies to build large scale distributed applications that are Enterprise ready yet Developer friendly.

ITCamp 2018 - Ciprian Sorlea - Enterprise Architectures with TypeScript And F...

ITCamp

What does it take to connect a Google Home to a .NET container running in the cloud? Surprisingly, not much! In this talk, we will use Dialogflow to setup a Google Home device to talk to a .NET container managed by Kubernetes Engine. We will take a look at some of the Google Cloud services such as Machine Learning APIs, BigQuery, Stackdriver diagnostics and see how they can elevate our Google Home to the next level. If you’re curious about what Google has to offer for your .NET apps, this talk is for you!

ITCamp 2018 - Mete Atamel Ian Talarico - Google Home meets .NET containers on...

ITCamp

More from ITCamp (20)

ITCamp 2019 - Stacey M. Jenkins - Protecting your company's data - By psychol...

ITCamp 2019 - Silviu Niculita - Supercharge your AI efforts with the use of A...

ITCamp 2019 - Peter Leeson - Managing Skills

ITCamp 2019 - Mihai Tataran - Governing your Cloud Resources

ITCamp 2019 - Ivana Milicic - Color - The Shadow Ruler of UX

ITCamp 2019 - Florin Coros - Implementing Clean Architecture

ITCamp 2019 - Florin Flestea - How 3rd Level support experience influenced m...

ITCamp 2019 - Emil Craciun - RoboRestaurant of the future powered by serverle...

ITCamp 2019 - Eldert Grootenboer - Cloud Architecture Recipes for The Enterprise

ITCamp 2019 - Cristiana Fernbach - Blockchain Legal Trends

ITCamp 2019 - Andy Cross - Machine Learning with ML.NET and Azure Data Lake

ITCamp 2019 - Andy Cross - Business Outcomes from AI

ITCamp 2019 - Andrea Saltarello - Modernise your app. The Cloud Story

ITCamp 2019 - Andrea Saltarello - Implementing bots and Alexa skills using Az...

ITCamp 2019 - Alex Mang - I'm Confused Should I Orchestrate my Containers on ...

ITCamp 2019 - Alex Mang - How Far Can Serverless Actually Go Now

ITCamp 2019 - Peter Leeson - Vitruvian Quality

ITCamp 2018 - Ciprian Sorlea - Million Dollars Hello World Application

ITCamp 2018 - Ciprian Sorlea - Enterprise Architectures with TypeScript And F...

ITCamp 2018 - Mete Atamel Ian Talarico - Google Home meets .NET containers on...

Recently uploaded

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

CNv6 Instructor Chapter 6 Quality of Service

giselly40

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Civil Lines Women Seeking Men

Delhi Call girls

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Recently uploaded (20)

Data Cloud, More than a CDP by Matt Robison

Handwritten Text Recognition for manuscripts and early printed texts

CNv6 Instructor Chapter 6 Quality of Service

Finology Group – Insurtech Innovation Award 2024

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

What Are The Drone Anti-jamming Systems Technology?

GenAI Risks & Security Meetup 01052024.pdf

Axa Assurance Maroc - Insurer Innovation Award 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

08448380779 Call Girls In Civil Lines Women Seeking Men

🐬 The future of MySQL is Postgres 🐘

How to Troubleshoot Apps for the Modern Connected Worker

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Boost PC performance: How more available memory can improve productivity

.NET Memory Primer (Martin Kulov)

1. Premium community conference on Microsoft technologies itcampro@ itcamp14# .NET Memory Primer Martin Kulov martin@kulov.net

2. Premium community conference on Microsoft technologies itcampro@ itcamp14# Huge thanks to our sponsors & partners!

3. Premium community conference on Microsoft technologies itcampro@ itcamp14# "Out of CPU, memory and disk, memory is typically the most important for overall system performance." Mark Russinovich “All you worry about in a .NET application is the memory.” John Robbins

4. Premium community conference on Microsoft technologies itcampro@ itcamp14# • x86 –2 ^ 32 bits = 4GB /0x FFFF FFFF/ • x64 –2 ^ 64 bits = 16 EB /0x FFFF FFFF' FFFF FFFF/ Virtual Memory Limits

5. Premium community conference on Microsoft technologies itcampro@ itcamp14# x86 Memory Mapping * PFN - Page Frame Number database

6. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Max 4GB –Windows Client, Windows Srv 2008 Standard • Max 128GB –Windows Srv 2003 SP1 Datacenter (PAE) x86 Physical Memory Limits

7. Premium community conference on Microsoft technologies itcampro@ itcamp14# x64 Memory Mapping

8. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Max 4TB - Windows Srv 2012 Standard • Limited per SKU x64 Physical Memory Limits

9. Premium community conference on Microsoft technologies itcampro@ itcamp14# Canonical Form Addresses 48-bit implementation 56-bit implementation 64-bit implementation

10. Premium community conference on Microsoft technologies itcampro@ itcamp14# Virtual Address Space

11. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Code • Data • Stacks • Heaps User Mode Memory

12. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Created for Each Thread • Default to 1MB • Hold Method Data /stack frame/ –Parameters –Local variables –Return address • First In, First Out Stacks

13. Premium community conference on Microsoft technologies itcampro@ itcamp14# Stack Layout

14. Premium community conference on Microsoft technologies itcampro@ itcamp14# • ChildEBP RetAddr Caller,Callee • 08e4e1a4 751b149d _WaitForSingleObjectEx@12+0x98, calling _ZwWaitForSingleObject@12 • 08e4e1e8 718b53c2 ?LeaveRuntimeNoThrow@Thread@@SGJI@Z+0xd7, calling __EH_epilog3 • 08e4e210 755b1194 _WaitForSingleObjectExImplementation@12+0x75, calling _WaitForSingleObjectEx@12 • 08e4e228 718b54d7 ?LoadImage@PEImage@@SGPAV1@PAUHINSTANCE__@@@Z+0x1af • ... • 08e4f5f4 71a10647 ?intermediateThreadProc@Thread@@CGKPAX@Z+0x49 • 08e4f784 71a10635 ?intermediateThreadProc@Thread@@CGKPAX@Z+0x37, calling __alloca_probe_16 • 08e4f798 755b336a @BaseThreadInitThunk@12+0xe • 08e4f7a4 77639f72 ___RtlUserThreadStart@8+0x70 • 08e4f7e4 77639f45 __RtlUserThreadStart@8+0x1b, calling ___RtlUserThreadStart@8 Call Stack Example

15. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Hold Dynamically Allocated Data • 1 Heap Per Logical Processor • Heap Has Segments –Process Heap –Code Heap /JITed code/ –Small Object Heap /SOH/ –Large Object Heap /LOH/ Heaps

16. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Stack –Value Types /Int32, Bool, Struct, etc…/ –Pointers to Reference Types • Heap –Reference Types /Object, String, Array, etc…/ –Free Areas Allocating .NET Memory

17. Premium community conference on Microsoft technologies itcampro@ itcamp14# DEMO: ALLOCATING MEMORY

18. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Stack References • Static References /Fields, ThreadStatic/ • CPU Registers • Interop References /COM, API calls/ • Finalization Queue References Object Roots /GC Roots/

19. Premium community conference on Microsoft technologies itcampro@ itcamp14# • a.k.a. Generational Garbage Collector /GC/ • Three Generations /SOH/ –Gen0 – short lived –Gen1 – medium lived –Gen2 – long lived Nondeterministic Finalization

20. Premium community conference on Microsoft technologies itcampro@ itcamp14# Before GC #1 Gen1 Gen0 Before GC #500 Gen2 Gen2 Gen2 Gen1 Gen0 Gen0 Before GC #0 Before GC #2 Gen2 Gen1 Gen0 Before GC #100 Gen2 Gen2 Gen1 Gen0

21. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Gen0 is Full • Induced GC /System.GC.Collect()/ • System Pressure Collection - When

22. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Rule of Thumb – Ratio 1:10:100 • .NET CLR Memory% time in GC • .NET CLR Memory# Induced GC • .NET CLR Memory# Gen X collections Collection - Cost

23. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Size > 85KB • Memory is marked as free during Gen2 • Avoid Temporary Large Objects • Reuse Objects in LOH If Possible • Many LOH Segments • Fragmentation Problems Large Object Heap

24. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Suspend Managed Threads • Collect Garbage • Resume Managed Threads • Two Phases of GC –Mark –Compact Collection - How

25. Premium community conference on Microsoft technologies itcampro@ itcamp14# • Workstation GC – Non Concurrent • Server GC – Non Concurrent • Workstation GC – Concurrent –Background GC /New in .NET 4/ • Server GC – Background /New in .NET 4.5/ GC Types

26. Premium community conference on Microsoft technologies itcampro@ itcamp14# Workstation GC

27. Premium community conference on Microsoft technologies itcampro@ itcamp14# Server GC

28. Premium community conference on Microsoft technologies itcampro@ itcamp14# Concurrent GC – Before and After

29. Premium community conference on Microsoft technologies itcampro@ itcamp14# Testing Concurrent Server GC

30. Premium community conference on Microsoft technologies itcampro@ itcamp14# Q & A

.NET Memory Primer (Martin Kulov)

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to .NET Memory Primer (Martin Kulov)

Similar to .NET Memory Primer (Martin Kulov) (20)

More from ITCamp

More from ITCamp (20)

Recently uploaded

Recently uploaded (20)

.NET Memory Primer (Martin Kulov)