Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory

1,393 views

Published on

Presented by Jaromir Hamala, C2B2 Senior Consultant, at JDays Conference on the 27th of November 2013.

Cheap RAM means we can use more memory than an average disk size was not that long time ago. It's very tempting to increase size of our JVM heap and store more data in-memory. As always, there is a trade-off: Large heaps lead to higher latencies due to Garbage Collector overhead. This session will show different strategies for accessing off-heap memory and using the advantage of large RAM without paying the performance penalty.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,393
On SlideShare
0
From Embeds
0
Number of Embeds
124
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory

  1. 1. Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory Heap @author Jaromir Hamala, jhamala@c2b2.co.uk @author(type=Twitter) @jerrinot © C2B2 Consulting Limited 2013 All Rights Reserved
  2. 2. Who Am I? ■ Jaromir Hamala ■ A curious developer ■ Who likes to break play with stuff © C2B2 Consulting Limited 2013 All Rights Reserved
  3. 3. Safe Harbor This talk may contain non-sense! © C2B2 Consulting Limited 2013 All Rights Reserved
  4. 4. Facts • Memory is Cheap • We have a lot of data • Data in Memory == Fastest Possible Data (apart from CPU caches) http://www.npr.org/blogs/thetwo-way/2012/06/06/154416480/ho-hum-dull-and-boring-are-now-a-pair © C2B2 Consulting Limited 2013 All Rights Reserved
  5. 5. On-Heap Allocation • CoolObject cool = new CoolObject() • Great: – It’s easy and usually *very* fast! • Allocation can be actually faster than in C – GC goodies! • Not so great: – GC leads to Stop-the-World Pauses – Bigger Heap -> Longer Pauses © C2B2 Consulting Limited 2013 All Rights Reserved
  6. 6. © C2B2 Consulting Limited 2013 All Rights Reserved
  7. 7. © C2B2 Consulting Limited 2013 All Rights Reserved
  8. 8. Why to go Off-Heap? • Latency caused by GC • Locality • Sharing with non-Java http://digboston.com/boston-lulz/2013/08/trolley-trollop-first-comes-love/attachment/why-god-why-kitten1/ © C2B2 Consulting Limited 2013 All Rights Reserved
  9. 9. GC Implementations • (Old Gen) GC in HotSpot (Oracle/Sun JDK, OpenJDK) – ParOld – STW pauses by design! – CMS – fragmentation -> STW pause – G1 – Immature (yet?), not fulfilling the expectations -> occasional STW pauses © C2B2 Consulting Limited 2013 All Rights Reserved
  10. 10. Latency can kill you! © C2B2 Consulting Limited 2013 All Rights Reserved
  11. 11. What is Off-Heap? • Area of memory not maintained by GC – Manual memory control • No Notion of Java object – serialization http://nickshell1983.wordpress.com/2010/06/29/taking-a-god-nudged-leap-of-faith/ © C2B2 Consulting Limited 2013 All Rights Reserved
  12. 12. Off-Heap Options? • • • • • Direct Memory Buffer sun.misc.Unsafe JNI Memory Mapped File …. http://en.wikipedia.org/wiki/File:Road_to_Hell_One_Sheet.jpg © C2B2 Consulting Limited 2013 All Rights Reserved
  13. 13. Direct Memory Buffer • ByteBuffer directBuf = ByteBuffer.allocateDirect(int noBytes); • • • • Part of Java NIO Max. 2GB / buffer Pure Java = Portable getX()/putX() methods © C2B2 Consulting Limited 2013 All Rights Reserved
  14. 14. sun.misc.Unsafe – long addr = unsafe.allocateMemory(noBytes) – Not a standardized part of JDK! – Implemented by most JVM vendors • Sun / Oracle / IBM – It can disappear from JDK without warning! © C2B2 Consulting Limited 2013 All Rights Reserved
  15. 15. Cassandra I. public class NativeAllocator implements IAllocator { static final Unsafe unsafe; static { try { Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe"); field.setAccessible(true); unsafe = (sun.misc.Unsafe) field.get(null); } catch (Exception e) { throw new AssertionError(e); } } public long allocate(long size) { return unsafe.allocateMemory(size); } public void free(long peer) { unsafe.freeMemory(peer); } } © C2B2 Consulting Limited 2013 All Rights Reserved
  16. 16. JNI • Java Native Interface • Native library means worst portability between platforms • JNI Overhead • Can re-use already existing (native) allocator © C2B2 Consulting Limited 2013 All Rights Reserved
  17. 17. Cassandra II. • Java Native Interface via JNA public class JEMallocAllocator implements IAllocator { public interface JEMLibrary extends Library { long malloc(long size); void free(long pointer); } private final JEMLibrary library; public JEMallocAllocator() { library = (JEMLibrary) Native.loadLibrary("jemalloc", JEMLibrary.class); } public long allocate(long size) { return library.malloc(size); } public void free(long peer) { library.free(peer); } } © C2B2 Consulting Limited 2013 All Rights Reserved
  18. 18. Memory Mapped File • • • • Part of NIO Pure Java -> Portable Allows sharing data between processes Persisted by OS © C2B2 Consulting Limited 2013 All Rights Reserved
  19. 19. Javin Paul, http://javarevisited.blogspot.com/2012/01/memorymapped-file-and-io-in-java.html private static int count = 1010241024; //10 MB public static void main(String[] args) throws Exception { RandomAccessFile memoryMappedFile = new RandomAccessFile("largeFile.txt", "rw"); //Mapping a file into memory MappedByteBuffer out = memoryMappedFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, count); //Writing into Memory Mapped File for (int i = 0; i < count; i++) { out.put((byte) 'A'); } System.out.println("Writing to Memory Mapped File is completed"); //reading from memory file in Java for (int i = 0; i < 10 ; i++) { System.out.print((char) out.get(i)); } System.out.println("Reading from Memory Mapped File is completed"); } © C2B2 Consulting Limited 2013 All Rights Reserved
  20. 20. Use cases for Off-Heap? • Caches / Data Grids – A lot of data – Objects with well defined lifecycle • (Extremely) Latency-Sensitive tasks – High-Frequency trading… • Sharing Data with non-Java © C2B2 Consulting Limited 2013 All Rights Reserved
  21. 21. Apache DirectMemory • “…is a off-heap cache for the Java Virtual Machine” • Easy to integrate • Friendly License • Experimental • http://directmemory.apache.org/ © C2B2 Consulting Limited 2013 All Rights Reserved
  22. 22. DirectMemory Usage CacheService<Object, Object> cacheService = new DirectMemory<Object, Object>() .setNumberOfBuffers( 10 ) .setSize( 1000 ) .setInitialCapacity( 100000 ) .setConcurrencyLevel( 4 ) .newCacheService(); put( K key, V value ) put( K key, V value, int expiresIn ) © C2B2 Consulting Limited 2013 All Rights Reserved
  23. 23. Is Off-Heap The Future of Java? • Oh God! Please no! • R&D should be aimed to more effective GC – Zing JVM (based on HotSpot, C4 - the fully concurrent GC) – RedHat Shenandoah? © C2B2 Consulting Limited 2013 All Rights Reserved
  24. 24. Final Warning(s) • Stay On-Heap unless you have no other choice! • If you think you have no other choice, think twice • Do NOT re-invent the wheel! (Memory Management) • Measure, measure, measure! © C2B2 Consulting Limited 2013 All Rights Reserved
  25. 25. Further Sources • http://mechanical-sympathy.blogspot.com/ – Martin Thompson. There is a brilliant mailing list as well! • http://psy-lob-saw.blogspot.com – Nitsan Wakart • http://vanillajava.blogspot.com – Peter Lawrey • http://insightfullogic.com/blog/ – Richard Warburton • http://ashkrit.blogspot.com – Ashkrit © C2B2 Consulting Limited 2013 All Rights Reserved
  26. 26. © C2B2 Consulting Limited 2013 All Rights Reserved
  27. 27. Thank you © C2B2 Consulting Limited 2013 All Rights Reserved

×