Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
On-heap cache vs Off-heap cache
Radek Grębski
@RadekGrebski
https://github.com/rgrebski
PRESENTATION PLAN
1. Heap vs off-heap
2. Heap memory
3. Off-heap memory
3.1 Memory mapped file
3.2 Unsafe and ByteBuffers
...
1. HEAP VS OFF-HEAP
vs
2. HEAP MEMORY
JVM memory
JDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace
(-XX:MaxMetaspaceSize)
Native memory
JVM Pro...
Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (...
3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes /
unallocated
byte[]
byte[]
3.1. MEMORY MAPPED FILE
Off-heap memoryJVM Process
OS memory
JVM Process
/tmp/myFile.dat
byte[]
byte[] byte[]
byte[]
3.2. UNSAFE AND BYTEBUFFERS
How to allocate memory using standard Java classes?
java.nio.ByteBuffer:
HeapByteBuffer (on-he...
3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer.allocate ( <2GB )
JvmUtils.verifyJvmArgumentsPresent("-Xmx2g");
ByteBuffer byteBuff...
1. HEAP VS OFF-HEAP
vs
3.2. UNSAFE AND BYTEBUFFERS
Unallocating ByteBuffer memory
public static void callCleaner(ByteBuffer byteBuffer){
//Direct...
3.2. UNSAFE AND BYTEBUFFERS
Allocating memory using sun.misc.Unsafe
sun.misc.Unsafe::getUnsafe():
public static Unsafe get...
3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file using Java API
File mappedFile = new File("/tmp/mappedFile.tmp");
mappedFil...
3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file content
3.2. UNSAFE AND BYTEBUFFERS
Big endian vs Little endian
0xCAFEBABE
Address 00 01 02 03
Big endian CA FE BA BE
Address 00 0...
3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer endianess
@Test
public void testEndianess() throws NoSuchMethodException, IllegalAc...
3.3. OFF-HEAP MEMORY ADVANTAGES
Off-heap storage
Pros and cons
No GC!
Manual GC!
Memory leaks
Persistence
IPC
Latency
Dura...
4. CACHE
HashMap
Collections.synchronizedMap(..)
ConcurrentHashMap
4.1. CHRONICLE
http://chronicle.software
Chronicle Map Chronicle Set Chronicle Queue
Chronicle Logger Java Thread Affinity
2. HEAP MEMORY
JVM memory
JDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace
(-XX:MaxMetaspaceSize)
Native memory
JVM Pro...
4.1. CHRONICLE MAP
Creating off-heap map instance
private Map<String, OffHeapUser> chronicleMap;
private static final Stri...
4.1. CHRONICLE MAP
Complex structures
public interface OffHeapUser {
String getUsername();
void setUsername(@MaxSize(30) S...
4.1. CHRONICLE MAP
Creating value instance
//fill the data
long accountValidUntil = System.currentTimeMillis() + YEAR_IN_M...
4.1. CHRONICLE MAP
Throughput
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was te...
4.1. CHRONICLE MAP
Memory usage
OutOfMemory
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
10 000 000 50 000 000 250 000 000 1 ...
4.1. CHRONICLE MAP
GC pauses
0,0
5,0
10,0
15,0
20,0
25,0
30,0
35,0
40,0
45,0
50,0
10 000 000 50 000 000 250 000 000 1 250 ...
Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (...
4.2. HAZELCAST
On-heap cache
Off-heap support (High Density Memory - commercial)
List, Map, Set, Queue
Topics
Executor Ser...
4.2. HAZELCAST
Example
@Test
public void hazelcastClusterTest(){
Config hazelcastConfig = new Config();
HazelcastInstance ...
4.3. REDIS
REmote DIctionary Server
Key/Value cache + store
ANSI C
Used by:
StackOverflow
GitHub
Twitter
Instagram
Alibaba...
4.3. REDIS
Redis data types
HashMaps (hset, hget)
LinkedList (lpush, ltrim, lrange)
Queue (lpush, rpop, brpop)
Topics (pub...
4.3. REDIS
Redis Java clients
Jedis
Redisson
Aredis
JDBC-Redis
Jredis
Lettuce
RedisClient
4.3. REDIS
Redis HashMap example
@Test
public void testRedisMap(){
Jedis jedis = new Jedis("localhost");
// Pipeline pipel...
5. COMPARISON
Chronicle Hazelcast Redis
Deployment model Embedded Embedded / Separate Separate
Replication Yes Yes Master/...
5. COMPARISON
R/W Performance
Chronicle Hazelcast Redis ConcurrentHashMap
Write (2mln entries) 2 555 583 178 491 110 668 1...
5. PERFORMANCE COMPARISON
GC Pauses
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role,...
Which cache should I use in my app ?
Questions?
3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes /
unallocated
byte[]
byte[]
Upcoming SlideShare
Loading in …5
×

JDD2015: On-heap cache vs Off-heap cache - Radek Grębski

155 views

Published on

ON-HEAP CACHE VS OFF-HEAP CACHE

W prezentacji przybliżę temat pamięci off-heap, czyli obszaru pamięci nie zarządzanej przez JVM. Przedstawię wady i zalety korzystania z tego rozwiązania, sposoby użycia klas ByteBuffer (direct/non-direct, big_endian/little_endian), Unsafe.allocateMemory oraz powiem czym jest memory mapped file.

Następnie omówię 3 rodzaje cache'e:
- Hazelcast (on heap cache)
- ChronicleMap (off-heap cache/store)
- Redis (napisany w C)
oraz przedstawię porównanie wydajności (zapis, odczyt, pauzy GC) z naciskiem na off-heapowe rozwiązanie.

Pokażę również fragmenty kodu ChronicleMap.

Published in: Software
  • Be the first to comment

  • Be the first to like this

JDD2015: On-heap cache vs Off-heap cache - Radek Grębski

  1. 1. On-heap cache vs Off-heap cache Radek Grębski @RadekGrebski https://github.com/rgrebski
  2. 2. PRESENTATION PLAN 1. Heap vs off-heap 2. Heap memory 3. Off-heap memory 3.1 Memory mapped file 3.2 Unsafe and ByteBuffers 3.3 Off-heap memory advantages 4. Cache: 4.1 Chronicle 4.2 Hazelcast 4.3 Redis 5. Comparison
  3. 3. 1. HEAP VS OFF-HEAP vs
  4. 4. 2. HEAP MEMORY JVM memory JDK8 S1Eden S0 Old Generation Heap (-Xmx) Metaspace (-XX:MaxMetaspaceSize) Native memory JVM Process memory OS memory Young Generation
  5. 5. Objects on heap Source: http://www.ibm.com/developerworks/library/j-codetoheap/ Integer (64-bit JVM): 7:1 (28 bytes) 9:1 (36 bytes) Integer (32-bit JVM): 3:1 overhead ratio 16 bytes String (32-bit JVM): 3.75:1 60 bytes int[1] 2. HEAP MEMORY
  6. 6. 3. OFF-HEAP MEMORY Off-heap (native) memoryJVM Process OS memory Other processes / unallocated byte[] byte[]
  7. 7. 3.1. MEMORY MAPPED FILE Off-heap memoryJVM Process OS memory JVM Process /tmp/myFile.dat byte[] byte[] byte[] byte[]
  8. 8. 3.2. UNSAFE AND BYTEBUFFERS How to allocate memory using standard Java classes? java.nio.ByteBuffer: HeapByteBuffer (on-heap, up to 2gb) DirectByteBuffer (off-heap, up to 2gb) MappedByteBuffer (off-heap, up to 2gb, persisted) sun.misc.Unsafe public native long allocateMemory(long allocationSizeInBytes);
  9. 9. 3.2. UNSAFE AND BYTEBUFFERS ByteBuffer.allocate ( <2GB ) JvmUtils.verifyJvmArgumentsPresent("-Xmx2g"); ByteBuffer byteBuffer = ByteBuffer.allocate((int) ByteUtil.GB); byteBuffer.putChar('a') //2bytes, position =0 .putInt(123) //4bytes, position = 2(0 + 2(char)) .put("test".getBytes("UTF-8")); //6 => 2(char) + 4(integer) byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length]; char charA = byteBuffer.getChar(/*address*/ 0); // 'a' int int123 = byteBuffer.getInt(/*address*/ 2); //123 byteBuffer.position(6); //set cursor position byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"
  10. 10. 1. HEAP VS OFF-HEAP vs
  11. 11. 3.2. UNSAFE AND BYTEBUFFERS Unallocating ByteBuffer memory public static void callCleaner(ByteBuffer byteBuffer){ //DirectByteBuffer.cleaner().clean() Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner"); cleanerMethod.setAccessible(true); Object cleaner = cleanerMethod.invoke(byteBuffer); Method cleanMethod = cleaner.getClass().getMethod("clean"); cleanMethod.setAccessible(true); cleanMethod.invoke(cleaner); }
  12. 12. 3.2. UNSAFE AND BYTEBUFFERS Allocating memory using sun.misc.Unsafe sun.misc.Unsafe::getUnsafe(): public static Unsafe getUnsafe() { Class cc = sun.reflect.Reflection.getCallerClass(2); if (cc.getClassLoader() != null) throw new SecurityException("Unsafe"); return theUnsafe; } Creating an instance of Unsafe: public static Unsafe createUnsafe() { Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor(); unsafeConstructor.setAccessible(true); return unsafeConstructor.newInstance(); } Memory allocation: long memorySizeInBytes = ByteUtil.GB * 4; long startAddress = unsafe.allocateMemory(memorySizeInBytes); unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0); unsafe.freeMemory(startAddress);
  13. 13. 3.2. UNSAFE AND BYTEBUFFERS Memory mapped file using Java API File mappedFile = new File("/tmp/mappedFile.tmp"); mappedFile.delete(); try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) { long buffer8MB = 8 * ByteUtil.MB; MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0, buffer8MB); long startAddress = 0; long elementsToPut = 200_000_000; for (long counter = 0; counter < elementsToPut; counter++) { if (!mappedByteBuffer.hasRemaining()) { startAddress += mappedByteBuffer.position(); mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress, buffer8MB); } mappedByteBuffer.putLong(counter); } } Time: 1,068 s Filesize: 1,49 GB
  14. 14. 3.2. UNSAFE AND BYTEBUFFERS Memory mapped file content
  15. 15. 3.2. UNSAFE AND BYTEBUFFERS Big endian vs Little endian 0xCAFEBABE Address 00 01 02 03 Big endian CA FE BA BE Address 00 01 02 03 Little endian BE BA FE CA
  16. 16. 3.2. UNSAFE AND BYTEBUFFERS ByteBuffer endianess @Test public void testEndianess() throws NoSuchMethodException, IllegalAccessException, InvocationTargetException { ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4); bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN); ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4); littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN); littleEndianByteBuffer.putInt(0xCAFEBABE); bigEndianByteBuffer.putInt(0xCAFEBABE); String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array()); String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array()); assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE"); assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA"); }
  17. 17. 3.3. OFF-HEAP MEMORY ADVANTAGES Off-heap storage Pros and cons No GC! Manual GC! Memory leaks Persistence IPC Latency Durability Scalability (1TB+)
  18. 18. 4. CACHE HashMap Collections.synchronizedMap(..) ConcurrentHashMap
  19. 19. 4.1. CHRONICLE http://chronicle.software Chronicle Map Chronicle Set Chronicle Queue Chronicle Logger Java Thread Affinity
  20. 20. 2. HEAP MEMORY JVM memory JDK8 S1Eden S0 Old Generation Heap (-Xmx) Metaspace (-XX:MaxMetaspaceSize) Native memory JVM Process memory OS memory Young Generation
  21. 21. 4.1. CHRONICLE MAP Creating off-heap map instance private Map<String, OffHeapUser> chronicleMap; private static final String KEY_SAMPLE = "12345678901234567890"; private static final long MAX_ENTRIES = 10_000_000; { chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class) .averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length) .constantValueSizeBySample(new OffHeapUserSample()) .createPersistedTo(new File("/tmp/mappedFile.bin")) .entries((long) (MAX_ENTRIES)) .create(); }
  22. 22. 4.1. CHRONICLE MAP Complex structures public interface OffHeapUser { String getUsername(); void setUsername(@MaxSize(30) String username); long getAccountValidUntil(); void setAccountValidUntil(long accountValidUntil); void setRoleAt(@MaxSize(2) int index, Role role); Role getRoleAt(@MaxSize(2) int index); static interface Role { String getRole(); void setRole(@MaxSize(10) String role); } }
  23. 23. 4.1. CHRONICLE MAP Creating value instance //fill the data long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS; String username = RandomStringUtils.randomAlphabetic(20); OffHeapUser offHeapUser = chronicleMap.newValueInstance(); offHeapUser.setAccountValidUntil(accountValidUntil); offHeapUser.setUsername(username); OffHeapUser.Role role0 = offHeapUser.getRoleAt(0); role0.setRole("Role0"); OffHeapUser.Role role1 = offHeapUser.getRoleAt(1); role0.setRole("Role1"); //put chronicleMap.put("someKey", offHeapUser); //get OffHeapUser offHeapUserActual = chronicleMap.get("someKey");
  24. 24. 4.1. CHRONICLE MAP Throughput Key = „u:0123456789”, value = counter *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap. Source: https://github.com/OpenHFT/Chronicle-Map 0 20 40 60 80 100 120 140 160 180 10 000 000 50 000 000 250 000 000 1 250 000 000 ThroughputMupd/s Map entries Throughput - ChronicleMap vs ConcurrentHashMap Cronicle Map ConcurrentHashMap OutOfMemory
  25. 25. 4.1. CHRONICLE MAP Memory usage OutOfMemory 0,0 20,0 40,0 60,0 80,0 100,0 120,0 140,0 10 000 000 50 000 000 250 000 000 1 250 000 000 MemoryinGB Map entries Memory used - ChronicleMap vs ConcurrentHashMap Cronicle Map ConcurrentHashMap Key = „u:0123456789”, value = counter *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap. Source: https://github.com/OpenHFT/Chronicle-Map
  26. 26. 4.1. CHRONICLE MAP GC pauses 0,0 5,0 10,0 15,0 20,0 25,0 30,0 35,0 40,0 45,0 50,0 10 000 000 50 000 000 250 000 000 1 250 000 000 WorstGCpauseinseconds Map entries Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap Cronicle Map ConcurrentHashMap OutOfMemory Key = „u:0123456789”, value = counter *ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap. Source: https://github.com/OpenHFT/Chronicle-Map
  27. 27. Objects on heap Source: http://www.ibm.com/developerworks/library/j-codetoheap/ Integer (64-bit JVM): 7:1 (28 bytes) 9:1 (36 bytes) Integer (32-bit JVM): 3:1 overhead ratio 16 bytes String (32-bit JVM): 3.75:1 60 bytes int[1] 2. HEAP MEMORY
  28. 28. 4.2. HAZELCAST On-heap cache Off-heap support (High Density Memory - commercial) List, Map, Set, Queue Topics Executor Service Dynamic clustering Transactional
  29. 29. 4.2. HAZELCAST Example @Test public void hazelcastClusterTest(){ Config hazelcastConfig = new Config(); HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig); HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig); Map<String, String> node1Map = hazelcastInstance1.getMap("someMapName"); Map<String, String> node2Map = hazelcastInstance2.getMap("someMapName"); node1Map.put("key", "value"); Assertions.assertThat(node2Map.get("key")).isEqualTo("value"); } Output: Members [2] { Member [192.168.1.23]:5701 this Member [192.168.1.23]:5702 }
  30. 30. 4.3. REDIS REmote DIctionary Server Key/Value cache + store ANSI C Used by: StackOverflow GitHub Twitter Instagram Alibaba Clients for almost all programming languages
  31. 31. 4.3. REDIS Redis data types HashMaps (hset, hget) LinkedList (lpush, ltrim, lrange) Queue (lpush, rpop, brpop) Topics (publish, subscribe) Set (sadd, srem) SortedSet (zadd, zrem)
  32. 32. 4.3. REDIS Redis Java clients Jedis Redisson Aredis JDBC-Redis Jredis Lettuce RedisClient
  33. 33. 4.3. REDIS Redis HashMap example @Test public void testRedisMap(){ Jedis jedis = new Jedis("localhost"); // Pipeline pipeline = jedis.pipelined(); // pipeline.multi(); jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek"); jedis.hset("user:1", "lastName", "Grebski"); jedis.hset("user:1", "email", "rgrebski@gmail.com"); // pipeline.sync() Map<String, String> mapFromRedis = jedis.hgetAll("user:1"); assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek"); assertThat(mapFromRedis) .hasSize(3) .contains(entry("firstName", "Radek")) .contains(entry("lastName", "Grebski")) .contains(entry("email", "rgrebski@gmail.com")); }
  34. 34. 5. COMPARISON Chronicle Hazelcast Redis Deployment model Embedded Embedded / Separate Separate Replication Yes Yes Master/Slave Topics (pub/sub) No Yes Yes Executor Service No Yes No Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically) Java collections support Yes Yes Yes (Redisson) Clustering No Yes Yes (since 3.0) Distributed events No Yes Yes Features
  35. 35. 5. COMPARISON R/W Performance Chronicle Hazelcast Redis ConcurrentHashMap Write (2mln entries) 2 555 583 178 491 110 668 1 100 715 Read (2mln entries) 937 207 214 638 156 177 4 092 769 0 500 000 1 000 000 1 500 000 2 000 000 2 500 000 3 000 000 3 500 000 4 000 000 4 500 000 Ops/s R/W Performance Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)
  36. 36. 5. PERFORMANCE COMPARISON GC Pauses Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long) Chronicle Hazelcast Redis ConcurrentHashMap GC pauses summarized 39 1452 480 726 GC pause max 8 28 15 360 0 200 400 600 800 1000 1200 1400 1600 Timeinmillis
  37. 37. Which cache should I use in my app ?
  38. 38. Questions?
  39. 39. 3. OFF-HEAP MEMORY Off-heap (native) memoryJVM Process OS memory Other processes / unallocated byte[] byte[]

×