More Related Content
What's hot
What's hot (20)
Similar to Staying friendly with the gc
Similar to Staying friendly with the gc (20)
More from Oren Eini
More from Oren Eini (7)
Recently uploaded
Recently uploaded (20)
Staying friendly with the gc
- 1. STAYING FRIENDLY WITH THE GC Oren Eini oren@ravendb.net
- 2. CONTEXT • RavenDB • Document database • Written in .NET • Managed code • High performance requirements
- 3. THE GARBAGE COLLECTOR • It cleans up after you • “No more memory leaks” • Avoid manual memory management • Allow you to focus on the real problems you need to solve… • Can be faster than manual memory management
- 4. GARBAGE COLLECTION • Can be expensive • Will pause the entire process • Can cause pathologic behavior • Will only find out about such issues very late in the game (production)
- 5. HOW IT WORKS? • Generations • Bump allocator • Mark & Sweep • Allocations are cheap • Memory is retained long term or quickly discarded • Freeing memory is cheap, scan small sections, free in bulk
- 6. THE HEAP • Allocated on Gen 0 • If survived GC, moved to Gen 1 • Collections happens on generations • Gen 0 – often • Gen 1 – sometimes • Gen 2 - rare Large Object Heap Gen 0 • Per core Gen 1Gen 2
- 7. BAD CASES? • Many objects • Cost per object, not per size • 100 x byte[10] > 10 x byte[100] • Retained for a while, then discarded • Pinned memory (I/O, interop) • Memory fragmentation • LOH fragmentation JSON documents DataSet Xml Document Strings > 80Kb
- 8. CONSEQUENCES • High performance • High concurrency • Complex objects • Retentions for caching, indexing, etc • Direct I/O requires pinning • We saw 90%+ CPU time, all in GC
- 9. SUPPORT CALL ROULETTE • Got pretty bad • Hard to retro-fix • Without manual memory management, how do you control it? GC Memory Frament- ation
- 10. DO YOU NEED THIS? • Requirements: • High performance (tens of thousands of request / second) • Throughput (dealing with 100,000s – 1,000,000s documents / second) • Size 1 (dealing with documents that are multi KB range) • Size 2 (dealing with databases that are tens to hundreds of GB typically, TBs as well) • Latency matters (KPI measurement)
- 11. SOLUTION • Don’t allocate • No allocations, no GC • Buffering, pooling, re-use objects • ArrayPool<T>, MemoryPool<T> • Span<T>, Memory<T> • Stackalloc • struct vs. class • Understand the GC • Work with it • Long lived allocations & object reuse • Very short term allocations • Nothing in between
- 12. USING THE STACK Unmanaged Heap Managed Heap Stack • Free allocation & free • Limited • Stack Overflow Exception • Process die • struct, enum, ValueType • stackalloc • Unsafe – sometimes • Beware of boxing
- 13. ANALYZE THIS?
- 14. BETTER…
- 15. NO ALLOCATIONS
- 16. BENEFITS • Other: • Code is simpler for the compiler • More chances for optimizations • Auto vectorization, instruction level parallelism • Might allow inlining • Can be used in hot spots
- 17. UNMANAGED HEAP • You control allocations / free • Can happen when you want it to • Adjust to your own behavior • Still benefit from pooling • Avoid excessive alloc / free (system calls!) • Marshal.AllocHGlobal • Marhsal.FreeHGlobal
- 18. REVERSE STRING… Heap allocation Heap allocation Heap allocation
- 19. DOING IT MANUALLY Heap allocation Heap allocation
- 20. UNSAFE CODE… Heap allocation Allow to access the memory directly
- 21. MODERN C#
- 22. USING UNMANAGED MEMORY • Span<T> / Memory<T> • T[] - array • T* - unmanaged memory • But how to manage? • Arena Allocator • Context • GC is: • _allocated = 0; Segment – 64 KB Segment – 128 KB Segment – 256KB Allocated
- 24. BEWARE • Memory leak • Use after free • Buffer overlap • Double free • Buffer overrun • With great power…
- 25. THE CONTEXT • A unit of work in the application • Processing of a single request • Handling a single document • Short lived • Frequent • Scope of actions • Hang memory management • Reset arena • Pool objects • Pool contexts • Explicit in API, explicit in design
- 26. TWO FACED GC
- 27. QUESTIONS?
Editor's Notes
- Important: “fixing” managed variable has some overhead as well!