Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Taming Metaspace: a look at the machinery, and a proposal for a better one | FOSDEM 2020

72 views

Published on

My talk at FOSDEM 2020 in the Java Devroom.

When examining memory footprint of a JVM process, the delta between Java heap usage and actual working set size can be surprisingly large. The JVM uses off-heap memory for a lot of things: thread stacks, compiler arenas, code heap, byte buffers, GC control... however, one of the largest consumers of off-heap memory can be class metadata. Class metadata are stored in Metaspace, which includes the Compressed Class Space.

The talk explores what Metaspace actually is and what is stored there; describes the architecture of the Metaspace allocator and the Compressed Class Space; how it interacts with the GC.

It also describes a new Metaspace prototype which a much improved elasticity and reduced footprint developed at SAP (see also upcoming JEP "Elastic Metaspace": https://bugs.openjdk.java.net/browse/JDK-8221173).

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Taming Metaspace: a look at the machinery, and a proposal for a better one | FOSDEM 2020

  1. 1. PUBLIC Thomas Stüfe, SAP Feb 01, 2020 Taming Metaspace
  2. 2. © 2019 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they should not be relied upon in making purchasing decisions. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies. See www.sap.com/copyright for additional trademark information and notices. www.sap.com/contactsap Follow us
  3. 3. 3PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ (Taken from https://spring.io/blog/2019/03/11/memory-footprint-of-the-jvm Note: Metaspace is mislabeled as Compressed Class Space) Metaspace can consume a lot of memory. Can we improve that? Yes, within reason. Why do we care?
  4. 4. Basics
  5. 5. 6PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Metaspace contains class metadata – Klass, Constant Pool, Method, Annotations, Bytecode, JIT Counters etc.  Used to live in Java Heap (Permanent Generation) pre JDK 8  JDK 8: PermGen Removal -> Metaspace is born – Inspired by JRockit VM – JEP 122: “JEP 122: Remove the Permanent Generation”  SAP involvement: – JDK 11: partial rework (chunk coalescation, JDK-8198423) – Analysis tools: jcmd VM.metaspace, VM.classloaders – many smaller fixes/cleanups – JDK 15 (?): rewrite Metaspace
  6. 6. 7PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Metadata are usually allocated when classes are loaded  All metadata a loader accumulated is freed in bulk after the loader has been collected – Exception: Metadata may be deallocated earlier (class redefinition, load errors etc) but that’s uncommon. – Deallocated space still belongs to the loader. Metadata lifecycle
  7. 7. 8PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Bulk delete allows arena style allocation! – No need to track individual allocations – Simple pointer bump allocation possible: cheap and allows tight packing  We know the size distribution of typical allocations  Case against malloc in particular: – CompressedClassSpace – Platform specific limitations (e.g. sbrk hits java heap) Why write a customized allocator?
  8. 8. 9PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Compressed Class Space is a (small) part of Metaspace  Optimization for 64bit platforms  Only on 64bit, if –XX:+UseCompressedClassPointers (on by default) Compressed Class Space
  9. 9. 10PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Compressed Class Space Java object Klass vtable itable … Mark word 64bit Klass* Heap Metaspace
  10. 10. 11PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Compressed Class Space Java object Klass vtable itable … Mark word 32bit index base Compressed Class Space Heap Shared classes Klass* = (base + 32 bit index) [<< 3]
  11. 11. 12PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Metaspace has two parts „Compressed“ class space D1 „non class“ Metaspace Commit HWM K 1 …D2 D2 K 2 Sizes per class: • ~1K Klass (500+ … 500K) • ~6K non-class (~2K … xxK) Klass structures Everything else -XX:CompressedClassSpaceSize
  12. 12. Current implementation
  13. 13. 16PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Current implementation (1) (much simplified) Loader top • Loader owns a chunk of memory. • Allocates from it via pointer bump. • Remember: we do not need to track individual allocations for freeing.
  14. 14. 17PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Current implementation (2) (much simplified) Loader • If chunk is used up, Loader aquires a new one from the metaspace allocator. • Retired chunks are kept in list • Leftover space is kept for later reuse Current chunk „Retired chunks“ leftover space
  15. 15. 18PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Current implementation (3) (much simplified) Used Reserved Loader A Loader B Loader C C A B A A A C B B A A B C AB B Committed A A A A A A A B B B B B C C C Chunks are carved from metaspace memory as they are allocated. GC Threshold
  16. 16. 19PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Current implementation (4) (much simplified) Loader A Loader B Loader C A A A A A A A B B B B B C C C When a loader dies, its chunks are marked as free… Used Reserved C A B A A A C B B A A B C AB B Committed GC Threshold
  17. 17. 20PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Current implementation (5) (very much simplified) Loader A Loader C A A A A A A A C C C …and added to global freelists, sorted by size. Chunk Freelists Used Reserved C A A A A C A A C A Committed GC Threshold
  18. 18. 21PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Freelists can get huge. – We have seen used:free ratios of 1:3 and worse – =>Metaspace is not really elastic.  Intra-chunk waste – At some point loader typically stops loading classes; remaining chunk space is wasted – Worse with many tiny loaders (reflection delegator classes, lambda anonymous classes)  Code bloat – Expensive to maintain. – Code base grew over time and has gotten overly complicated. Problems with the current implementation
  19. 19. 22PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Huge freelists: Committed vs used space, after class unloading ~270M 
  20. 20. 23PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ jcmd 27265 VM.metaspace 27265: … Waste (percentages refer to total committed size 373,48 MB): Committed unused: 280,00 KB ( <1%) Waste in chunks in use: 2,45 KB ( <1%) Free in chunks in use: 6,34 MB ( 2%) Overhead in chunks in use: 186,75 KB ( <1%) In free chunks: 269,56 MB ( 72%) Deallocated from chunks in use: 998,98 KB ( <1%) (1763 blocks) -total-: 277,33 MB ( 74%) Huge Freelists (jcmd VM.metaspace output)
  21. 21. Reimplementation
  22. 22. 26PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Uncommit chunks in freelists ● Makes Metaspace more elastic  Delay committing chunks until they are actually used ● Loader commits as needed (like a thread stack) ● Removes the penalty of handing out large chunks to class loaders ● Mitigates intrachunk waste problem (at least for large chunks) Basic idea
  23. 23. 27PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  (Linux): we decommit with mmap(MAP_NORESERVE) && mprotect(PROT_NONE) – Higher commit/uncommit fragmentation results in higher number of VMAs – Kernel keeps vma structures in list and rb tree – Too many of them may affect vma lookup – And we may hit process limits  So: keep an eye on commit granularity Concern: keep number of virtual memory areas low
  24. 24. 28PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Commit granules Before: commit HWM Now: Granule size Adjustable, defaults to 64k
  25. 25. 29PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ  Odd chunk geometry – Difficult to merge and split – High fragmentation – Complex code  Chunk headers are a problem Current chunk allocation scheme unsuited for uncommitting chunks
  26. 26. 30PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ • Power 2 based buddy allocation scheme • Chunks sized from 1K … 4M in pow2 steps • Dead simple to split and merge. • Low external defragmentation -> Leads to larger free contiguous areas. • Standard algorithm widely known Pow 2 based buddy allocator for chunks
  27. 27. 32PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Buddy allocator: Deallocation  Mark chunk as free  If buddy chunk is free and unsplit: remove from freelist and merge with chunk – Repeat until root chunk sized reached or until buddy is not free  Return result chunk to free list 64K 256K128K64K 64K 256K64K 64K 64K 512K 64K 256K64K 64K 64K
  28. 28. 33PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Chunk headers needed to go page page page page ChunkHeaderPool page page page page Header Before Now
  29. 29. 34PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Granules and chunks Chunks Granules 256K 64K 32 K 1 6 K 1 6 K 128K  A larger chunk can span multiple granules (1:n)  Multiple small chunks can cover a single granule (n:1) 1:n n:1
  30. 30. 35PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Granules and chunks Chunks Granules 256K 64K 32 K 1 6 K 1 6 K 128K  Free chunks spanning 1+ granules can be uncommitted  A chunk spanning >1 granules can be committed on demand
  31. 31. 36PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ • Got rid of humongous chunks :) • Got rid of occupancy map • Better deallocation management • Chunks can now often grow in-place • Saves overhead and reduces intrachunk waste • Code is cleaner and more maintainable; better separation of concerns and testability. What else changed
  32. 32. 37PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Result: Committed vs used, Stock JDK14 ~270M 
  33. 33. 38PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Result: Committed vs used, Patched JDK14
  34. 34. 39PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Result: committed Metaspace, Stock vs Patched VM
  35. 35. 40PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Result: RSS, Stock vs Patched VM
  36. 36. 41PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ Bottomline: Results  Metaspace is now elastic. Freelist problem solved (almost).  Modest decrease in consumption even without mass class unloading ● Wildfly standalone after startup: 61m->54m, -7m, (11%) ● Eclipse CDT, hotspot project after C++ indexing: 138m->129m, -9m (12%) ● jruby helloworld.rb (invokedynamic, compile=FORCE): 41m->38m, -3m, (1.2%) ● → But, still more ideas and room for improvement.  Better overall code quality ● Better separation of concerns, better testability, more generic coding
  37. 37. 42PUBLIC© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ How do we go from here?  Patch is stable. Needs more tests and may need smaller fixes but it works.  Patch lives in jdk/sandbox repository, branch "stuefe-new-metaspace-branch“ – http://hg.openjdk.java.net/jdk/sandbox/  JEP exists in Draft state (“Elastic Metaspace”: https://openjdk.java.net/jeps/8221173 )  JDK15? – Very difficult to bring such a large patch upstream  A good candidate for backporting! – Would make a lot of sense in 11/8 – Large patch but Metaspace is quite isolated. Should not be too much of a hassle.
  38. 38. Contact information: Thomas Stüfe @tstuefe thomas.stuefe@sap.com stuefe.de Thank you.
  39. 39. Q/A

×