Heapoff memory wtf


Published on

What is Heap-off memory and how use cache heap off with Apache DirectMemory

Published in: Technology, Education

Heapoff memory wtf

  1. 1. Heap Off Memory WTF ??Reducing Heap memory stress
  2. 2. Abstract* Java memory fundamental* heap off memory principles* heap-off cache with Apache DirectMemory
  3. 3. /meOlivier Lamy* Open Source Architect @Talend Apache Team* Apache Member : Maven, Archiva, Tomcat,DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com
  4. 4. Java Memory Fundamental
  5. 5. Java Memory Fundamental The cool part* Automatic memory allocation (do you rememberwhen you used malloc)* Garbage collector (GC) (no more free)
  6. 6. Java Memory Fundamental The bad part* stop-the-world mode* proportionnal to memorys size* unpredictable unresponsive application :complicated with tight SLA
  7. 7. Heap Off MemoryNot in the Heap process Limited GC stop « Unlimited storage »
  8. 8. Heap Off Memory How to use that ? Native libraries with JNA https://github.com/twall/jna ByteBuffer @since 1.4 with nio ByteBuffer.allocate( size ) (not off heap) ByteBuffer.allocateDirect( size )You can use sun.misc.Unsafe but can cause issue and not portable
  9. 9. Heap Off Memory How to use that ?You can manipulate only Buffer (ByteBuffer byte[] etc..) ! So you must serialize/deserialize your datasMaximum size with -XX:MaxDirectMemorySize=
  10. 10. Our use case : Cache RAM : 10-60 ns Network : 10000-30000 nsSSD Disk : 70000-120000 nsDisk : 3000000-10000000 ns Cache must use memory !
  11. 11. On-Heap Cache Objects stored by reference (no de/serialisation)GC storm effect when refreshing/removing objects!
  12. 12. Off-Heap Cache de/serialisation overhead (hopefully some very performant libraries exist for that)Cache objects payload no more affecting GC
  13. 13. Solutions :* Terracotta BigMemory (off-heap storage ontop of ehcache)* Infinispan (by Jboss)* Huge collections* Apache DirectMemory
  14. 14. Apache Direct Memory Goals :Apache Direct Memory is a multi layered cache implementation featuring off-heap memorystorage to enable caching of java objects without degrading jvm performance.
  15. 15. Apache Direct MemoryJoined Apache Incubator end 201112 developpers ATMUnder development : Memory allocation service just rewrite APIs are subject to be changed and bugs to be found !
  16. 16. Design & principlesByteBuffer.allocateDirect is the foundation of the cacheByteBuffers are allocated in big chunk and splitted for internal use
  17. 17. Design & principlesBuild on layers CachingService : serialize object (pluggable) MemoryManagerService: Compute ByteBuffer access ByteBufferAllocatorService: Eventually deals with ByteBuffer
  18. 18. ByteBuffers AllocationMerging Strategy No memory wasted Suffers from fragmentation Need synchronization at de/allocationFixed Size buffers allocation Memory wasted if size not correctly configured No fragmentation
  19. 19. Use case (1)Multi layers cacheMost used objects are cached on heap, the rest off-heap (maybe overflow to disk)Sounds like ehcache with BigMemoryHard coded class to use : net.sf.ehcache.store.offheap.OffHeapStoreSo same package class name in Apache DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940) Demo
  20. 20. Use case (2)Cache Server« à la » memcache tru http(s)But with a REST Api and written in JavaClient API availableDemo
  21. 21. > PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1
  22. 22. Next StepsJSR 107 ?BenchmarksComponents Integration (Cassandra, Tomcat etc..)Dynamic cache size modificationManagement/Monitoring
  23. 23. Linkshttp://incubator.apache.org/directmemory/https://issues.apache.org/jira/browse/DIRECTMEMORYDemo project : https://bitbucket.org/olamy/heap-off-memory- wtfSlides : http://www.slideshare.net/olamy
  24. 24. Thanks &&Questions