www.hazelcast.com
WHO AM I
Christoph Engelbert(@noctarius2k)
8+ years of professionalJavadevelopment
Specialized to performance, GC, traffic...
TOPICS
JavaMemoryLayout
Definition of Off-Heap?
DirectMemoryAccess?
Whydo we need Off-Heap Memory?
Options for Off-Heap St...
JAVA MEMORY LAYOUT
A FAST WALK-THROUGH
www.hazelcast.com
JAVA MEMORY LAYOUT (1/5)
www.hazelcast.com
JAVA MEMORY LAYOUT (2/5)
YOUNG GENERATION - EDEN SPACE
Onlyused for Object-Allocation
TLAB¹ allocation available
Affected ...
JAVA MEMORY LAYOUT (3/5)
YOUNG GENERATION - SURVIVOR SPACES
Always one From-Space and one To-Space
From-Space contains liv...
JAVA MEMORY LAYOUT (4/5)
TENURED SPACE
Contains onlylonglivingobjects
Affected onlybyMajor GC
More objects means more GC s...
JAVA MEMORY LAYOUT (5/5)
PERMGEN / META SPACE
Class bytecodes
Class metadata
Runtime information
Code Compilation Cache
et...
G1 (GARBAGE FIRST COLLECTOR)
GenerationalGC
Heap splitted into same sized regions
Everyregion is either Eden, Survivor or ...
DEFINITION OF
OFF-HEAP?
www.hazelcast.com
Off-Heap is a(continuously) self allocated,
managed and freed directmemoryregion.
Itis notunder controlof the JavaGarbageC...
I'M DUKE SKYWALKER
I'M HERE TO RESCUE YOU!
www.hazelcast.com
DIRECT MEMORY
ACCESS?
GIVE YOURSELF TO THE DARK SIDE
www.hazelcast.com
DIRECT MEMORY ACCESS?
ARE YOU SERIOUS?
Fastmemoryarea
Notaffectingthe GC
Officiallyavailable since Java1.4
Serialization o...
WHY DO WE NEED
OFF-HEAP MEMORY?
www.hazelcast.com
WHY DO WE NEED OFF-HEAP MEMORY?
Storingof huge datasets
Zero-Copywrites to channels
Lower pressure on GC /less GC pauses
S...
OPTIONS FOR
OFF-HEAP STORAGE
www.hazelcast.com
OPTIONS FOR OFF-HEAP STORAGE (1/3)
JNI (Java Native Interface)
int*buffer=(int*)malloc(1024* sizeof(int));
if(buffer==NULL...
OPTIONS FOR OFF-HEAP STORAGE (2/3)
(Direct)ByteBuffer
ByteBufferbuffer=ByteBuffer.allocateDirect(1024*4);
for(inti=0;i<102...
OPTIONS FOR OFF-HEAP STORAGE (3/3)
sun.misc.Unsafe
Unsafeunsafe=trickToRetrieveUnsafe();
longaddress=unsafe.allocateMemory...
ADVANTAGES
AND
DISADVANTAGES
OF (OFF-)HEAP
www.hazelcast.com
ADVANTAGES ON-HEAP
No messingwith malloc
Automatic Garbage Collection
No need for Serialization
www.hazelcast.com
DISADVANTAGES ON-HEAP
Slowdown on high allocation rate
Bigoverhead on smallobjects (header)
Heavilydependingon GC Combinat...
ADVANTAGES OFF-HEAP
No limitation*in size
Compactdatalayout
Zero-Copysocketread/write possible
Frameworks available to man...
DISADVANTAGES OFF-HEAP
Allocation is up to you
Deallocation is up to you
Datalayoutis up to you
Serialization maybe requir...
AVAILABLE
FRAMEWORKS
www.hazelcast.com
AVAILABLE FRAMEWORKS (1/5)
Apache DirectMemory
TerracottaBigMemory
HazelcastElasticMemory
MapDB.org
etc.
www.hazelcast.com
AVAILABLE FRAMEWORKS (2/5)
Apache DirectMemory
CacheService<String,String>cacheService=newDirectMemory<...>()
.setNumberOf...
AVAILABLE FRAMEWORKS (3/5)
Terracotta BigMemory
<ehcachexml:noNamespaceSchemaLocation="..."
name="MyCacheDefinition">
<cac...
AVAILABLE FRAMEWORKS (4/5)
Hazelcast Offheap
<hazelcastxml:noNamespaceSchemaLocation="...">
<mapname="MyMap">
<storage-typ...
AVAILABLE FRAMEWORKS (5/5)
MapDB.org
DBdb=DBMaker.newDirectMemoryDB().sizeLimit(2).make();
//ReturnsaHTreeMapextendsConcur...
FURTHER INFORMATION AVAILABLE
The Must-Read for Off-Heap
http://bit.ly/must-read-off-heap
PacketObjectDescription from IBM...
THE
OFF-HEAP JEP PROPOSAL
MAY THE FORCE BE WITH YOU.
THE FORCE IS STRONG WITH THIS ONE.
www.hazelcast.com
OFF-HEAP JEP PROPOSAL
ByteBuffer like interface
Fully64-bitsizes and offsets
Compare-And-Swap operations
Volatile and orde...
OFF-HEAP JEP PROPOSAL - CODE EXAMPLE
DISCLAIMER: PROVISIONAL API
importjavax.direct.*;
BytesFactoryfactory=createBytesFact...
OFF-HEAP JEP PROPOSAL
ProposalDiscussion Group
http://bit.ly/offheap-ml
ProposalText
http://bit.ly/proposal-text
FFI JEP 1...
@noctarius2k
@hazelcast
http://www.sourceprojects.com
http://github.com/noctarius
THANK YOU!
ANY QUESTIONS?
Images:www.cli...
Upcoming SlideShare
Loading in …5
×

My Old Friend Malloc

2,228 views
2,070 views

Published on

Today higher heaps and bigger RAM amounts are typical for standard Java server applications. We often then get to the limit of performance, predictability or GC pause times and using Off-Heap technologies we can lower that pains.

Published in: Technology, Education
1 Comment
10 Likes
Statistics
Notes
No Downloads
Views
Total views
2,228
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
1
Likes
10
Embeds 0
No embeds

No notes for slide

My Old Friend Malloc

  1. 1. www.hazelcast.com
  2. 2. WHO AM I Christoph Engelbert(@noctarius2k) 8+ years of professionalJavadevelopment Specialized to performance, GC, traffic topics Apache DirectMemoryPMC Previous companies incl. Ubisoftand HRS OfficialInhabitantof Hazelcastia CastMapRMapReduce for Hazelcast3 Co-Author JavaOff-heap JEP Proposal www.hazelcast.com
  3. 3. TOPICS JavaMemoryLayout Definition of Off-Heap? DirectMemoryAccess? Whydo we need Off-Heap Memory? Options for Off-Heap Storage Advantages and Disadvantages of (Off- )Heap Available Frameworks (Notso) SecretFuture Smalldemonstration www.hazelcast.com
  4. 4. JAVA MEMORY LAYOUT A FAST WALK-THROUGH www.hazelcast.com
  5. 5. JAVA MEMORY LAYOUT (1/5) www.hazelcast.com
  6. 6. JAVA MEMORY LAYOUT (2/5) YOUNG GENERATION - EDEN SPACE Onlyused for Object-Allocation TLAB¹ allocation available Affected byMinor and Major GC Objects moved to Survivor on Minor GC ¹TLAB:ThreadLocalAllocationBuffer www.hazelcast.com
  7. 7. JAVA MEMORY LAYOUT (3/5) YOUNG GENERATION - SURVIVOR SPACES Always one From-Space and one To-Space From-Space contains livingobjects on lastGC Minor GC switches the Survivor Spaces Alive Objects are moved to To-Space Affected byMinor and Major GC Longlivingobjects eventuallymoved to Tenured www.hazelcast.com
  8. 8. JAVA MEMORY LAYOUT (4/5) TENURED SPACE Contains onlylonglivingobjects Affected onlybyMajor GC More objects means more GC spend time Concurrentobjectinspectation available LongStop-The-World pauses possible Notoptimalfor bigdatasets www.hazelcast.com
  9. 9. JAVA MEMORY LAYOUT (5/5) PERMGEN / META SPACE Class bytecodes Class metadata Runtime information Code Compilation Cache etc. www.hazelcast.com
  10. 10. G1 (GARBAGE FIRST COLLECTOR) GenerationalGC Heap splitted into same sized regions Everyregion is either Eden, Survivor or Tenured space www.hazelcast.com
  11. 11. DEFINITION OF OFF-HEAP? www.hazelcast.com
  12. 12. Off-Heap is a(continuously) self allocated, managed and freed directmemoryregion. Itis notunder controlof the JavaGarbageCollector and needs custom written allocation and cleanup of data-regions. Off-Heap can notbe used for allocation of Javaobjects. DEFINITION OF OFF-HEAP? www.hazelcast.com
  13. 13. I'M DUKE SKYWALKER I'M HERE TO RESCUE YOU! www.hazelcast.com
  14. 14. DIRECT MEMORY ACCESS? GIVE YOURSELF TO THE DARK SIDE www.hazelcast.com
  15. 15. DIRECT MEMORY ACCESS? ARE YOU SERIOUS? Fastmemoryarea Notaffectingthe GC Officiallyavailable since Java1.4 Serialization overhead when storingobjects Limited by-XX:MaxDirectMemorySize www.hazelcast.com
  16. 16. WHY DO WE NEED OFF-HEAP MEMORY? www.hazelcast.com
  17. 17. WHY DO WE NEED OFF-HEAP MEMORY? Storingof huge datasets Zero-Copywrites to channels Lower pressure on GC /less GC pauses Storage Space onlylimited byRAM Compactdatarepresentation possible IPC SharedMemorywith Memory-Mapped-Files www.hazelcast.com
  18. 18. OPTIONS FOR OFF-HEAP STORAGE www.hazelcast.com
  19. 19. OPTIONS FOR OFF-HEAP STORAGE (1/3) JNI (Java Native Interface) int*buffer=(int*)malloc(1024* sizeof(int)); if(buffer==NULL)throwOutOfMemoryException(); for(inti=0;i<1024;i++) buffer[sizeof(int)*i]=i; free(buffer); www.hazelcast.com
  20. 20. OPTIONS FOR OFF-HEAP STORAGE (2/3) (Direct)ByteBuffer ByteBufferbuffer=ByteBuffer.allocateDirect(1024*4); for(inti=0;i<1024;i++) buffer.putInt(i); //bufferisautomaticallyfreedbyGC www.hazelcast.com
  21. 21. OPTIONS FOR OFF-HEAP STORAGE (3/3) sun.misc.Unsafe Unsafeunsafe=trickToRetrieveUnsafe(); longaddress=unsafe.allocateMemory(1024*4); for(inti=0;i<1024;i++) unsafe.putInt(address+4*i,i); unsafe.freeMemory(address); www.hazelcast.com
  22. 22. ADVANTAGES AND DISADVANTAGES OF (OFF-)HEAP www.hazelcast.com
  23. 23. ADVANTAGES ON-HEAP No messingwith malloc Automatic Garbage Collection No need for Serialization www.hazelcast.com
  24. 24. DISADVANTAGES ON-HEAP Slowdown on high allocation rate Bigoverhead on smallobjects (header) Heavilydependingon GC Combination www.hazelcast.com
  25. 25. ADVANTAGES OFF-HEAP No limitation*in size Compactdatalayout Zero-Copysocketread/write possible Frameworks available to manage memory www.hazelcast.com
  26. 26. DISADVANTAGES OFF-HEAP Allocation is up to you Deallocation is up to you Datalayoutis up to you Serialization maybe required JNI requires native library www.hazelcast.com
  27. 27. AVAILABLE FRAMEWORKS www.hazelcast.com
  28. 28. AVAILABLE FRAMEWORKS (1/5) Apache DirectMemory TerracottaBigMemory HazelcastElasticMemory MapDB.org etc. www.hazelcast.com
  29. 29. AVAILABLE FRAMEWORKS (2/5) Apache DirectMemory CacheService<String,String>cacheService=newDirectMemory<...>() .setNumberOfBuffers(10).newCacheService(); cacheService.put("MyKey","SomeValue"); Stringvalue=cacheService.get("MyKey"); www.hazelcast.com
  30. 30. AVAILABLE FRAMEWORKS (3/5) Terracotta BigMemory <ehcachexml:noNamespaceSchemaLocation="..." name="MyCacheDefinition"> <cachename="MyOffheapCache"maxBytesLocalOffHeap="2G"/> </ehcache> CacheManagercacheManager=newCacheManager(); CachedataStore=cacheManager.get("MyOffheapCache"); Elementelement=newElement("MyKey","SomeValue"); dataStore.put(element); Stringvalue=(String)dataStore.get("MyKey").getObjectValue(); www.hazelcast.com
  31. 31. AVAILABLE FRAMEWORKS (4/5) Hazelcast Offheap <hazelcastxml:noNamespaceSchemaLocation="..."> <mapname="MyMap"> <storage-type>OFFHEAP</storage-type> </map> </hazelcast> HazelcastInstancehz=Hazelcast.newInstance(); //ReturnsaIMapextendsConcurrentMap Map<String,String>map=hz.getMap("MyMap"); map.put("MyKey","SomeValue"); Stringvalue=map.get("MyKey"); www.hazelcast.com
  32. 32. AVAILABLE FRAMEWORKS (5/5) MapDB.org DBdb=DBMaker.newDirectMemoryDB().sizeLimit(2).make(); //ReturnsaHTreeMapextendsConcurrentMap Map<String,String>map=db.createHashMap("cache").make(); map.put("MyKey","SomeValue"); Stringvalue=map.get("MyKey"); www.hazelcast.com
  33. 33. FURTHER INFORMATION AVAILABLE The Must-Read for Off-Heap http://bit.ly/must-read-off-heap PacketObjectDescription from IBM http://bit.ly/packet-objects CompactDatastructures (Martin Thompson) http://bit.ly/compact-datastructures C++ Like Performance for Serialization http://bit.ly/serialization-performance Tricks with DirectMemoryAccess in Java http://bit.ly/direct-memory-tricks www.hazelcast.com
  34. 34. THE OFF-HEAP JEP PROPOSAL MAY THE FORCE BE WITH YOU. THE FORCE IS STRONG WITH THIS ONE. www.hazelcast.com
  35. 35. OFF-HEAP JEP PROPOSAL ByteBuffer like interface Fully64-bitsizes and offsets Compare-And-Swap operations Volatile and ordered operations Optionalboundingchecks Supports Memorymapping SupportForeign Function Interface (FFI) JEP www.hazelcast.com
  36. 36. OFF-HEAP JEP PROPOSAL - CODE EXAMPLE DISCLAIMER: PROVISIONAL API importjavax.direct.*; BytesFactoryfactory=createBytesFactory(); Bytesbytes=factory.boundsChecking(false).deallocationChecks(false) .freeOnGC(false).create(ByteOrder.LITTLE_ENDIAN,1024*4); for(inti=0;i<1024;i++) bytes.putVolatileInt(i); bytes.release(); www.hazelcast.com
  37. 37. OFF-HEAP JEP PROPOSAL ProposalDiscussion Group http://bit.ly/offheap-ml ProposalText http://bit.ly/proposal-text FFI JEP 191 http://bit.ly/ffi-jep www.hazelcast.com
  38. 38. @noctarius2k @hazelcast http://www.sourceprojects.com http://github.com/noctarius THANK YOU! ANY QUESTIONS? Images:www.clipartist.info,GnomeNebulaTheme,KDEtheme,www.grabsteine-klnt.de www.hazelcast.com

×