Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

My Old Friend Malloc

2,708 views

Published on

Today higher heaps and bigger RAM amounts are typical for standard Java server applications. We often then get to the limit of performance, predictability or GC pause times and using Off-Heap technologies we can lower that pains.

Published in: Technology, Education

My Old Friend Malloc

  1. 1. www.hazelcast.com
  2. 2. WHO AM I Christoph Engelbert(@noctarius2k) 8+ years of professionalJavadevelopment Specialized to performance, GC, traffic topics Apache DirectMemoryPMC Previous companies incl. Ubisoftand HRS OfficialInhabitantof Hazelcastia CastMapRMapReduce for Hazelcast3 Co-Author JavaOff-heap JEP Proposal www.hazelcast.com
  3. 3. TOPICS JavaMemoryLayout Definition of Off-Heap? DirectMemoryAccess? Whydo we need Off-Heap Memory? Options for Off-Heap Storage Advantages and Disadvantages of (Off- )Heap Available Frameworks (Notso) SecretFuture Smalldemonstration www.hazelcast.com
  4. 4. JAVA MEMORY LAYOUT A FAST WALK-THROUGH www.hazelcast.com
  5. 5. JAVA MEMORY LAYOUT (1/5) www.hazelcast.com
  6. 6. JAVA MEMORY LAYOUT (2/5) YOUNG GENERATION - EDEN SPACE Onlyused for Object-Allocation TLAB¹ allocation available Affected byMinor and Major GC Objects moved to Survivor on Minor GC ¹TLAB:ThreadLocalAllocationBuffer www.hazelcast.com
  7. 7. JAVA MEMORY LAYOUT (3/5) YOUNG GENERATION - SURVIVOR SPACES Always one From-Space and one To-Space From-Space contains livingobjects on lastGC Minor GC switches the Survivor Spaces Alive Objects are moved to To-Space Affected byMinor and Major GC Longlivingobjects eventuallymoved to Tenured www.hazelcast.com
  8. 8. JAVA MEMORY LAYOUT (4/5) TENURED SPACE Contains onlylonglivingobjects Affected onlybyMajor GC More objects means more GC spend time Concurrentobjectinspectation available LongStop-The-World pauses possible Notoptimalfor bigdatasets www.hazelcast.com
  9. 9. JAVA MEMORY LAYOUT (5/5) PERMGEN / META SPACE Class bytecodes Class metadata Runtime information Code Compilation Cache etc. www.hazelcast.com
  10. 10. G1 (GARBAGE FIRST COLLECTOR) GenerationalGC Heap splitted into same sized regions Everyregion is either Eden, Survivor or Tenured space www.hazelcast.com
  11. 11. DEFINITION OF OFF-HEAP? www.hazelcast.com
  12. 12. Off-Heap is a(continuously) self allocated, managed and freed directmemoryregion. Itis notunder controlof the JavaGarbageCollector and needs custom written allocation and cleanup of data-regions. Off-Heap can notbe used for allocation of Javaobjects. DEFINITION OF OFF-HEAP? www.hazelcast.com
  13. 13. I'M DUKE SKYWALKER I'M HERE TO RESCUE YOU! www.hazelcast.com
  14. 14. DIRECT MEMORY ACCESS? GIVE YOURSELF TO THE DARK SIDE www.hazelcast.com
  15. 15. DIRECT MEMORY ACCESS? ARE YOU SERIOUS? Fastmemoryarea Notaffectingthe GC Officiallyavailable since Java1.4 Serialization overhead when storingobjects Limited by-XX:MaxDirectMemorySize www.hazelcast.com
  16. 16. WHY DO WE NEED OFF-HEAP MEMORY? www.hazelcast.com
  17. 17. WHY DO WE NEED OFF-HEAP MEMORY? Storingof huge datasets Zero-Copywrites to channels Lower pressure on GC /less GC pauses Storage Space onlylimited byRAM Compactdatarepresentation possible IPC SharedMemorywith Memory-Mapped-Files www.hazelcast.com
  18. 18. OPTIONS FOR OFF-HEAP STORAGE www.hazelcast.com
  19. 19. OPTIONS FOR OFF-HEAP STORAGE (1/3) JNI (Java Native Interface) int*buffer=(int*)malloc(1024* sizeof(int)); if(buffer==NULL)throwOutOfMemoryException(); for(inti=0;i<1024;i++) buffer[sizeof(int)*i]=i; free(buffer); www.hazelcast.com
  20. 20. OPTIONS FOR OFF-HEAP STORAGE (2/3) (Direct)ByteBuffer ByteBufferbuffer=ByteBuffer.allocateDirect(1024*4); for(inti=0;i<1024;i++) buffer.putInt(i); //bufferisautomaticallyfreedbyGC www.hazelcast.com
  21. 21. OPTIONS FOR OFF-HEAP STORAGE (3/3) sun.misc.Unsafe Unsafeunsafe=trickToRetrieveUnsafe(); longaddress=unsafe.allocateMemory(1024*4); for(inti=0;i<1024;i++) unsafe.putInt(address+4*i,i); unsafe.freeMemory(address); www.hazelcast.com
  22. 22. ADVANTAGES AND DISADVANTAGES OF (OFF-)HEAP www.hazelcast.com
  23. 23. ADVANTAGES ON-HEAP No messingwith malloc Automatic Garbage Collection No need for Serialization www.hazelcast.com
  24. 24. DISADVANTAGES ON-HEAP Slowdown on high allocation rate Bigoverhead on smallobjects (header) Heavilydependingon GC Combination www.hazelcast.com
  25. 25. ADVANTAGES OFF-HEAP No limitation*in size Compactdatalayout Zero-Copysocketread/write possible Frameworks available to manage memory www.hazelcast.com
  26. 26. DISADVANTAGES OFF-HEAP Allocation is up to you Deallocation is up to you Datalayoutis up to you Serialization maybe required JNI requires native library www.hazelcast.com
  27. 27. AVAILABLE FRAMEWORKS www.hazelcast.com
  28. 28. AVAILABLE FRAMEWORKS (1/5) Apache DirectMemory TerracottaBigMemory HazelcastElasticMemory MapDB.org etc. www.hazelcast.com
  29. 29. AVAILABLE FRAMEWORKS (2/5) Apache DirectMemory CacheService<String,String>cacheService=newDirectMemory<...>() .setNumberOfBuffers(10).newCacheService(); cacheService.put("MyKey","SomeValue"); Stringvalue=cacheService.get("MyKey"); www.hazelcast.com
  30. 30. AVAILABLE FRAMEWORKS (3/5) Terracotta BigMemory <ehcachexml:noNamespaceSchemaLocation="..." name="MyCacheDefinition"> <cachename="MyOffheapCache"maxBytesLocalOffHeap="2G"/> </ehcache> CacheManagercacheManager=newCacheManager(); CachedataStore=cacheManager.get("MyOffheapCache"); Elementelement=newElement("MyKey","SomeValue"); dataStore.put(element); Stringvalue=(String)dataStore.get("MyKey").getObjectValue(); www.hazelcast.com
  31. 31. AVAILABLE FRAMEWORKS (4/5) Hazelcast Offheap <hazelcastxml:noNamespaceSchemaLocation="..."> <mapname="MyMap"> <storage-type>OFFHEAP</storage-type> </map> </hazelcast> HazelcastInstancehz=Hazelcast.newInstance(); //ReturnsaIMapextendsConcurrentMap Map<String,String>map=hz.getMap("MyMap"); map.put("MyKey","SomeValue"); Stringvalue=map.get("MyKey"); www.hazelcast.com
  32. 32. AVAILABLE FRAMEWORKS (5/5) MapDB.org DBdb=DBMaker.newDirectMemoryDB().sizeLimit(2).make(); //ReturnsaHTreeMapextendsConcurrentMap Map<String,String>map=db.createHashMap("cache").make(); map.put("MyKey","SomeValue"); Stringvalue=map.get("MyKey"); www.hazelcast.com
  33. 33. FURTHER INFORMATION AVAILABLE The Must-Read for Off-Heap http://bit.ly/must-read-off-heap PacketObjectDescription from IBM http://bit.ly/packet-objects CompactDatastructures (Martin Thompson) http://bit.ly/compact-datastructures C++ Like Performance for Serialization http://bit.ly/serialization-performance Tricks with DirectMemoryAccess in Java http://bit.ly/direct-memory-tricks www.hazelcast.com
  34. 34. THE OFF-HEAP JEP PROPOSAL MAY THE FORCE BE WITH YOU. THE FORCE IS STRONG WITH THIS ONE. www.hazelcast.com
  35. 35. OFF-HEAP JEP PROPOSAL ByteBuffer like interface Fully64-bitsizes and offsets Compare-And-Swap operations Volatile and ordered operations Optionalboundingchecks Supports Memorymapping SupportForeign Function Interface (FFI) JEP www.hazelcast.com
  36. 36. OFF-HEAP JEP PROPOSAL - CODE EXAMPLE DISCLAIMER: PROVISIONAL API importjavax.direct.*; BytesFactoryfactory=createBytesFactory(); Bytesbytes=factory.boundsChecking(false).deallocationChecks(false) .freeOnGC(false).create(ByteOrder.LITTLE_ENDIAN,1024*4); for(inti=0;i<1024;i++) bytes.putVolatileInt(i); bytes.release(); www.hazelcast.com
  37. 37. OFF-HEAP JEP PROPOSAL ProposalDiscussion Group http://bit.ly/offheap-ml ProposalText http://bit.ly/proposal-text FFI JEP 191 http://bit.ly/ffi-jep www.hazelcast.com
  38. 38. @noctarius2k @hazelcast http://www.sourceprojects.com http://github.com/noctarius THANK YOU! ANY QUESTIONS? Images:www.clipartist.info,GnomeNebulaTheme,KDEtheme,www.grabsteine-klnt.de www.hazelcast.com

×