This talk presents operating support that can dramatically improve the performance of garbage-collected applications. It describes a virtual memory manager that, combined with a collector-neutral heap sizing algorithm, ensures that garbage-collected applications run as fast as possible while avoiding paging.

1. CRAMM: Virtual Memory Support for Garbage-Collected Applications Ting Yang, Emery Berger, Scott Kaplan † , Eliot Moss Department of Computer Science Dept. of Math and Computer Science † University of Massachusetts Amherst College {tingy,emery,moss}@cs.umass.edu [email_address]

6. GC : How do we choose a good heap size?

7. GC: Collector-neutral model SemiSpace (copying) a ≈ ½ b ≈ JVM, code + app’s live size heapUtilFactor: constant dependent on GC algorithm Fixed overhead : Libraries, codes, copying (app’s live size)

8. GC: a collector-neutral WSS model SemiSpace (copying) MS (non-copying) a ≈ ½ b ≈ JVM, code + app’s live size a ≈ 1 b ≈ JVM, code heapUtilFactor: constant dependent on GC algorithm Fixed overhead : Libraries, codes, copying (app’s live size)

9. GC: Selecting new heap size GC: heapUtilFactor (a) & cur_heapSize VMM: WSS & available memory Set heap size so that working set just fits in current available memory

10. Heap Size vs. Execution time, WSS 1/x shape Y=0.99*X + 32.56 Linear shape

11. VM : How do we collect information to support heap size selection? (with low overhead) WSS, Available Memory

12. Calculating WSS w.r.t 5% Memory reference sequence LRU Queue Pages in Least Recently Used order Hit Histogram Fault Curve 1 14 5 1 1 14 11 4 Associated with each LRU position pages faults d e f g h i j k l m n c k l m n c b c d e f g h i j k l m n c k l m n a b a a b c d e f g h i j k l m n a b c d e f g h i j k l m n a b d e f g h i j c k l n m a b c d e f g h i j k m n l a b c d e f g h i j l m n k a b d e f g h i j k l m n c 4 n 3 2 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 m n l m n k l m n c k l m n a b c d e f g h i j k l m n c k l m n

14. Managing pages for a process Active (CLOCK) Inactive (LRU) Evicted (LRU) Major fault Evicted Refill & Adjustment Minor fault Pages protected by turning off permissions (minor fault) Pages evicted to disk. (major fault) Histogram Pages faults Header Page Des AVL node

15. Controlling overhead Buffer Active (CLOCK) Inactive (LRU) Evicted (LRU) Pages protected by turning off permissions (minor fault) Pages evicted to disk. (major fault) Histogram Pages faults control the boundary: 1% of execution time Header Page Des AVL node

17. Experimental Results

19. Dynamic Memory Pressure (1) stock w/o pressure 296.67 secs 1136 majflts CRAMM w/ pressure 302.53 secs 1613 majflts 98% CPU Stock w/ pressure 720.11 secs 39944 majflts 48% CPU I nitial heap size: 120MB Elapsed Time (seconds) GenMS – SPECjbb (Modified) w/ 160M memory s tock w/o pressure CRAMM w/ pressure # transactions finished (thousands) S tock w/ pressure

20. Dynamic Memory Pressure (2) SPECjbb (modified): Normalized Elapsed Time JRockit HotSpot CRAMM-GenMS CRAMM-MS CRAMM-SS HotSpot JRockit # transactions finished (thousands)

21. CRAMM VM: Efficiency Overhead: on average, 1% - 2.5% CRAMM VM Overhead 0 0.5 1 1.5 2 2.5 3 3.5 4 SPEC2Kint SPEC2Kfp Java- GenCopy Java- SemiSpace Java- MarkSweep Java-GenMS Java- CopyMS % Overhead Additional Overhead Histogram Collection

25. Characterizing Paging Behavior Memory reference sequence LRU Queue Pages in Least Recently Used order Hit Histogram Fault Curve 1 14 5 1 1 14 11 4 12 pages 5 pages Associated with each LRU position d e f g h i j k l m n c k l m n c b c d e f g h i j k l m n c k l m n a b a a b c d e f g h i j k l m n a b c d e f g h i j k l m n a b d e f g h i j c k l n m a b c d e f g h i j k m n l a b c d e f g h i j l m n k a b d e f g h i j k l m n c 4 n 3 2 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 m n l m n k l m n c k l m n a b c d e f g h i j k l m n c k l m n

26. Heap Size = 240Mb Memory = 145Mb # of Faults ≈ 1000 50 seconds extreme paging substantial paging: “ looping” behavior fits into memory Fault curve: Relationship of heap size, real memory & page faults Heap size= 0.5 second

28. VMM design: SegQ Active Inactive Evicted Active / Inactive Boundary CLOCK algorithm Strict LRU WSS What is the WSS w.r.t 5%? 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

29. CRAMM System: Demo JVM Memory (150MB) Other Apps Heap Size (120MB) Need 6 0MB memory Polling Memory Status GC: Memory exhausted, triggers a full collection … Collection Finished GC: Collection is finished; Memory is released VM: Calculates app’s WSS and Available memory WSS, Available Memory GC: Choose a new heap size using WSS model Heap Size ( 1 00 MB) Heap Size ( 90 MB) Heap Size ( 150 MB) VM M GC: Shrinks the heap size again Other apps finished GC: Grows the heap size to make better use of memory

32. Static Memory Pressure optimal

33. Dynamic Memory Pressure (1) Initial heap size: 120MB stock w/o pressure 336.17 secs 1136 majflts CRAMM w/ pressure 386.88 secs 1179 majflts 98% CPU Stock w/ pressure 928.49 secs 47941 majflts 36% CPU

34. Dynamic Memory Pressure (1) Available memory Heap size Sample after every collection adaptive

35. Dynamic Memory Pressure (3)

36. Appel _213_javac 60MB real memory Too small: GC a lot Too large: page a lot Optimal Problem & Motivation Heap size vs Running time

39. GC gives more choices ! Non-GCed Application GCed Application W(k,t) k k W(k,t) Heap: 20MB Heap: 30MB Heap: 45MB Heap: 65MB Working Set Size W(k, t) : at time t , the set of all pages used k most recent references Memory pressure , scan frequency , k , WSS , more pages can be evicted, page faults , running time Larger search space. Change heap size, change WSS, avoid page faults, less impact on running time Hmm… a search problem! Search Criteria Working Set Size: The amount of memory needed so that the time spent on page faults is lower than certain percent of total execution time. (typical value: 5%)