Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Staying friendly with the gc

13,853 views

Published on


.Net Garbage Collector is an important factor in improving development efficiency by eliminating the need to manually manage memory.
But sometimes, it's overhead may cause very long pauses in application execution. In this talk, I will show why this happens and how we can deal with it.

Published in: Technology
  • Be the first to comment

Staying friendly with the gc

  1. 1. STAYING FRIENDLY WITH THE GC Oren Eini oren@ravendb.net
  2. 2. CONTEXT • RavenDB • Document database • Written in .NET • Managed code • High performance requirements
  3. 3. THE GARBAGE COLLECTOR • It cleans up after you • “No more memory leaks” • Avoid manual memory management • Allow you to focus on the real problems you need to solve… • Can be faster than manual memory management
  4. 4. GARBAGE COLLECTION • Can be expensive • Will pause the entire process • Can cause pathologic behavior • Will only find out about such issues very late in the game (production)
  5. 5. HOW IT WORKS? • Generations • Bump allocator • Mark & Sweep • Allocations are cheap • Memory is retained long term or quickly discarded • Freeing memory is cheap, scan small sections, free in bulk
  6. 6. THE HEAP • Allocated on Gen 0 • If survived GC, moved to Gen 1 • Collections happens on generations • Gen 0 – often • Gen 1 – sometimes • Gen 2 - rare Large Object Heap Gen 0 • Per core Gen 1Gen 2
  7. 7. BAD CASES? • Many objects • Cost per object, not per size • 100 x byte[10] > 10 x byte[100] • Retained for a while, then discarded • Pinned memory (I/O, interop) • Memory fragmentation • LOH fragmentation JSON documents DataSet Xml Document Strings > 80Kb
  8. 8. CONSEQUENCES • High performance • High concurrency • Complex objects • Retentions for caching, indexing, etc • Direct I/O requires pinning • We saw 90%+ CPU time, all in GC
  9. 9. SUPPORT CALL ROULETTE • Got pretty bad • Hard to retro-fix • Without manual memory management, how do you control it? GC Memory Frament- ation
  10. 10. DO YOU NEED THIS? • Requirements: • High performance (tens of thousands of request / second) • Throughput (dealing with 100,000s – 1,000,000s documents / second) • Size 1 (dealing with documents that are multi KB range) • Size 2 (dealing with databases that are tens to hundreds of GB typically, TBs as well) • Latency matters (KPI measurement)
  11. 11. SOLUTION • Don’t allocate • No allocations, no GC • Buffering, pooling, re-use objects • ArrayPool<T>, MemoryPool<T> • Span<T>, Memory<T> • Stackalloc • struct vs. class • Understand the GC • Work with it • Long lived allocations & object reuse • Very short term allocations • Nothing in between
  12. 12. USING THE STACK Unmanaged Heap Managed Heap Stack • Free allocation & free • Limited • Stack Overflow Exception • Process die • struct, enum, ValueType • stackalloc • Unsafe – sometimes • Beware of boxing
  13. 13. ANALYZE THIS?
  14. 14. BETTER…
  15. 15. NO ALLOCATIONS
  16. 16. BENEFITS • Other: • Code is simpler for the compiler • More chances for optimizations • Auto vectorization, instruction level parallelism • Might allow inlining • Can be used in hot spots
  17. 17. UNMANAGED HEAP • You control allocations / free • Can happen when you want it to • Adjust to your own behavior • Still benefit from pooling • Avoid excessive alloc / free (system calls!) • Marshal.AllocHGlobal • Marhsal.FreeHGlobal
  18. 18. REVERSE STRING… Heap allocation Heap allocation Heap allocation
  19. 19. DOING IT MANUALLY Heap allocation Heap allocation
  20. 20. UNSAFE CODE… Heap allocation Allow to access the memory directly
  21. 21. MODERN C#
  22. 22. USING UNMANAGED MEMORY • Span<T> / Memory<T> • T[] - array • T* - unmanaged memory • But how to manage? • Arena Allocator • Context • GC is: • _allocated = 0; Segment – 64 KB Segment – 128 KB Segment – 256KB Allocated
  23. 23. RESULTS… Unmanaged Managed
  24. 24. BEWARE • Memory leak • Use after free • Buffer overlap • Double free • Buffer overrun • With great power…
  25. 25. THE CONTEXT • A unit of work in the application • Processing of a single request • Handling a single document • Short lived • Frequent • Scope of actions • Hang memory management • Reset arena • Pool objects • Pool contexts • Explicit in API, explicit in design
  26. 26. TWO FACED GC
  27. 27. QUESTIONS?

×