The document compares on-heap and off-heap caching options. It discusses using heap memory within the JVM versus off-heap memory outside the JVM using memory-mapped files, ByteBuffers, and Unsafe. Popular caching libraries like Chronicle, Hazelcast, and Redis are also summarized. Chronicle uses memory-mapped files for off-heap caching while Hazelcast supports on and off-heap and Redis is separate from the JVM process. Performance tests show Chronicle generally outperforming ConcurrentHashMap and Redis for write and read throughput.

1. On-heap cache vs Off-heap cache
Radek Grębski
@RadekGrebski
https://github.com/rgrebski

2. PRESENTATION PLAN
1. Heap vs off-heap
2. Heap memory
3. Off-heap memory
3.1 Memory mapped file
3.2 Unsafe and ByteBuffers
3.3 Off-heap memory advantages
4. Cache:
4.1 Chronicle
4.2 Hazelcast
4.3 Redis
5. Comparison

3. 1. HEAP VS OFF-HEAP
vs

4. 2. HEAP MEMORY
JVM memory
JDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace
(-XX:MaxMetaspaceSize)
Native memory
JVM Process memory
OS memory
Young Generation

5. Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (36 bytes)
Integer (32-bit JVM):
3:1 overhead ratio
16 bytes
String (32-bit JVM):
3.75:1
60 bytes
int[1]
2. HEAP MEMORY

6. 3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes /
unallocated
byte[]
byte[]

7. 3.1. MEMORY MAPPED FILE
Off-heap memoryJVM Process
OS memory
JVM Process
/tmp/myFile.dat
byte[]
byte[] byte[]
byte[]

8. 3.2. UNSAFE AND BYTEBUFFERS
How to allocate memory using standard Java classes?
java.nio.ByteBuffer:
HeapByteBuffer (on-heap, up to 2gb)
DirectByteBuffer (off-heap, up to 2gb)
MappedByteBuffer (off-heap, up to 2gb, persisted)
sun.misc.Unsafe
public native long allocateMemory(long allocationSizeInBytes);

9. 3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer.allocate ( <2GB )
JvmUtils.verifyJvmArgumentsPresent("-Xmx2g");
ByteBuffer byteBuffer = ByteBuffer.allocate((int) ByteUtil.GB);
byteBuffer.putChar('a') //2bytes, position =0
.putInt(123) //4bytes, position = 2(0 + 2(char))
.put("test".getBytes("UTF-8")); //6 => 2(char) + 4(integer)
byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length];
char charA = byteBuffer.getChar(/*address*/ 0); // 'a'
int int123 = byteBuffer.getInt(/*address*/ 2); //123
byteBuffer.position(6); //set cursor position
byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"

10. 1. HEAP VS OFF-HEAP
vs

11. 3.2. UNSAFE AND BYTEBUFFERS
Unallocating ByteBuffer memory
public static void callCleaner(ByteBuffer byteBuffer){
//DirectByteBuffer.cleaner().clean()
Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(byteBuffer);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);
}

12. 3.2. UNSAFE AND BYTEBUFFERS
Allocating memory using sun.misc.Unsafe
sun.misc.Unsafe::getUnsafe():
public static Unsafe getUnsafe() {
Class cc = sun.reflect.Reflection.getCallerClass(2);
if (cc.getClassLoader() != null)
throw new SecurityException("Unsafe");
return theUnsafe;
}
Creating an instance of Unsafe:
public static Unsafe createUnsafe() {
Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();
unsafeConstructor.setAccessible(true);
return unsafeConstructor.newInstance();
}
Memory allocation:
long memorySizeInBytes = ByteUtil.GB * 4;
long startAddress = unsafe.allocateMemory(memorySizeInBytes);
unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0);
unsafe.freeMemory(startAddress);

13. 3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file using Java API
File mappedFile = new File("/tmp/mappedFile.tmp");
mappedFile.delete();
try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) {
long buffer8MB = 8 * ByteUtil.MB;
MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0,
buffer8MB);
long startAddress = 0;
long elementsToPut = 200_000_000;
for (long counter = 0; counter < elementsToPut; counter++) {
if (!mappedByteBuffer.hasRemaining()) {
startAddress += mappedByteBuffer.position();
mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress,
buffer8MB);
}
mappedByteBuffer.putLong(counter);
}
}
Time: 1,068 s
Filesize: 1,49 GB

14. 3.2. UNSAFE AND BYTEBUFFERS
Memory mapped file content

15. 3.2. UNSAFE AND BYTEBUFFERS
Big endian vs Little endian
0xCAFEBABE
Address 00 01 02 03
Big endian CA FE BA BE
Address 00 01 02 03
Little endian BE BA FE CA

16. 3.2. UNSAFE AND BYTEBUFFERS
ByteBuffer endianess
@Test
public void testEndianess() throws NoSuchMethodException, IllegalAccessException,
InvocationTargetException {
ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4);
bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN);
ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4);
littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN);
littleEndianByteBuffer.putInt(0xCAFEBABE);
bigEndianByteBuffer.putInt(0xCAFEBABE);
String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array());
String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array());
assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE");
assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA");
}

17. 3.3. OFF-HEAP MEMORY ADVANTAGES
Off-heap storage
Pros and cons
No GC!
Manual GC!
Memory leaks
Persistence
IPC
Latency
Durability
Scalability (1TB+)

18. 4. CACHE
HashMap
Collections.synchronizedMap(..)
ConcurrentHashMap

19. 4.1. CHRONICLE
http://chronicle.software
Chronicle Map Chronicle Set Chronicle Queue
Chronicle Logger Java Thread Affinity

20. 2. HEAP MEMORY
JVM memory
JDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace
(-XX:MaxMetaspaceSize)
Native memory
JVM Process memory
OS memory
Young Generation

21. 4.1. CHRONICLE MAP
Creating off-heap map instance
private Map<String, OffHeapUser> chronicleMap;
private static final String KEY_SAMPLE = "12345678901234567890";
private static final long MAX_ENTRIES = 10_000_000;
{
chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class)
.averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length)
.constantValueSizeBySample(new OffHeapUserSample())
.createPersistedTo(new File("/tmp/mappedFile.bin"))
.entries((long) (MAX_ENTRIES))
.create();
}

22. 4.1. CHRONICLE MAP
Complex structures
public interface OffHeapUser {
String getUsername();
void setUsername(@MaxSize(30) String username);
long getAccountValidUntil();
void setAccountValidUntil(long accountValidUntil);
void setRoleAt(@MaxSize(2) int index, Role role);
Role getRoleAt(@MaxSize(2) int index);
static interface Role {
String getRole();
void setRole(@MaxSize(10) String role);
}
}

23. 4.1. CHRONICLE MAP
Creating value instance
//fill the data
long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS;
String username = RandomStringUtils.randomAlphabetic(20);
OffHeapUser offHeapUser = chronicleMap.newValueInstance();
offHeapUser.setAccountValidUntil(accountValidUntil);
offHeapUser.setUsername(username);
OffHeapUser.Role role0 = offHeapUser.getRoleAt(0);
role0.setRole("Role0");
OffHeapUser.Role role1 = offHeapUser.getRoleAt(1);
role0.setRole("Role1");
//put
chronicleMap.put("someKey", offHeapUser);
//get
OffHeapUser offHeapUserActual = chronicleMap.get("someKey");

24. 4.1. CHRONICLE MAP
Throughput
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.
Source: https://github.com/OpenHFT/Chronicle-Map
0
20
40
60
80
100
120
140
160
180
10 000 000 50 000 000 250 000 000 1 250 000 000
ThroughputMupd/s
Map entries
Throughput - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory

25. 4.1. CHRONICLE MAP
Memory usage
OutOfMemory
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
10 000 000 50 000 000 250 000 000 1 250 000 000
MemoryinGB
Map entries
Memory used - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.
Source: https://github.com/OpenHFT/Chronicle-Map

26. 4.1. CHRONICLE MAP
GC pauses
0,0
5,0
10,0
15,0
20,0
25,0
30,0
35,0
40,0
45,0
50,0
10 000 000 50 000 000 250 000 000 1 250 000 000
WorstGCpauseinseconds
Map entries
Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.
Source: https://github.com/OpenHFT/Chronicle-Map

27. Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (36 bytes)
Integer (32-bit JVM):
3:1 overhead ratio
16 bytes
String (32-bit JVM):
3.75:1
60 bytes
int[1]
2. HEAP MEMORY

28. 4.2. HAZELCAST
On-heap cache
Off-heap support (High Density Memory - commercial)
List, Map, Set, Queue
Topics
Executor Service
Dynamic clustering
Transactional

29. 4.2. HAZELCAST
Example
@Test
public void hazelcastClusterTest(){
Config hazelcastConfig = new Config();
HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig);
HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig);
Map<String, String> node1Map = hazelcastInstance1.getMap("someMapName");
Map<String, String> node2Map = hazelcastInstance2.getMap("someMapName");
node1Map.put("key", "value");
Assertions.assertThat(node2Map.get("key")).isEqualTo("value");
}
Output:
Members [2] {
Member [192.168.1.23]:5701 this
Member [192.168.1.23]:5702
}

30. 4.3. REDIS
REmote DIctionary Server
Key/Value cache + store
ANSI C
Used by:
StackOverflow
GitHub
Twitter
Instagram
Alibaba
Clients for almost all programming languages

31. 4.3. REDIS
Redis data types
HashMaps (hset, hget)
LinkedList (lpush, ltrim, lrange)
Queue (lpush, rpop, brpop)
Topics (publish, subscribe)
Set (sadd, srem)
SortedSet (zadd, zrem)

32. 4.3. REDIS
Redis Java clients
Jedis
Redisson
Aredis
JDBC-Redis
Jredis
Lettuce
RedisClient

33. 4.3. REDIS
Redis HashMap example
@Test
public void testRedisMap(){
Jedis jedis = new Jedis("localhost");
// Pipeline pipeline = jedis.pipelined();
// pipeline.multi();
jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek");
jedis.hset("user:1", "lastName", "Grebski");
jedis.hset("user:1", "email", "rgrebski@gmail.com");
// pipeline.sync()
Map<String, String> mapFromRedis = jedis.hgetAll("user:1");
assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek");
assertThat(mapFromRedis)
.hasSize(3)
.contains(entry("firstName", "Radek"))
.contains(entry("lastName", "Grebski"))
.contains(entry("email", "rgrebski@gmail.com"));
}

34. 5. COMPARISON
Chronicle Hazelcast Redis
Deployment model Embedded Embedded / Separate Separate
Replication Yes Yes Master/Slave
Topics (pub/sub) No Yes Yes
Executor Service No Yes No
Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically)
Java collections support Yes Yes Yes (Redisson)
Clustering No Yes Yes (since 3.0)
Distributed events No Yes Yes
Features

35. 5. COMPARISON
R/W Performance
Chronicle Hazelcast Redis ConcurrentHashMap
Write (2mln entries) 2 555 583 178 491 110 668 1 100 715
Read (2mln entries) 937 207 214 638 156 177 4 092 769
0
500 000
1 000 000
1 500 000
2 000 000
2 500 000
3 000 000
3 500 000
4 000 000
4 500 000
Ops/s
R/W Performance
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

36. 5. PERFORMANCE COMPARISON
GC Pauses
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)
Chronicle Hazelcast Redis ConcurrentHashMap
GC pauses summarized 39 1452 480 726
GC pause max 8 28 15 360
0
200
400
600
800
1000
1200
1400
1600
Timeinmillis

37. Which cache should I use in my app ?

38. Questions?

39. 3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes /
unallocated
byte[]
byte[]