It's Dangerous to GC alone. Take this! IBM's talk on work integrating the OMR GC into Ruby. OMR preview: goo.gl/P3yXuy

1. It’s Dangerous to GC alone. Take this!
The OMR GC talk

2. Who are those handsome guys?
Robert Young
rwyoung@ca.ibm.com
@rwy0717
Craig Lehmann
craigl@ca.ibm.com
@CraigLehmann
2

3. IBM Runtime Technologies
3

4. 4
VisualAge Smalltalk
J9 JVM
1988
?
1993
2016

5. 5
VisualAge Smalltalk
J9 JVM
1988
OMR
1993
2016

6. An open source toolkit for
language runtime
technologies.
OMR

7. Execution Environment
7
Language-Agnostic Components
Platform Abstraction Layer
Garbage Collector
Diagnostic Services
Source Code Bytecode/AST
Compiler
Just-In-Time Compiler
InterpreterSource Bytecode
Compiler
Interpreter

8. Execution Environment
8
Language-Agnostic Components
Platform Abstraction Layer
Garbage Collector
Diagnostic Services
Source Code Bytecode/AST
Compiler
Just-In-Time Compiler
InterpreterSource Bytecode
Compiler
Interpreter

9. 9
A Runtime Toolkit for Many Languages
JVM
OMR
Java App
Python
OMR
Python App
MRI
OMR
Ruby App

10. In Ruby
OMR

11. 13
Getting started

12. 14
How do we start?
http://thefeedingdoctor.com/wp-content/uploads/2013/10/Kid-Confused.jpg

13. 15
How do we start?

14. 16

15. Read the source
17

16. Read the source
18

17. 19
Craig! Robert!
Tue Oct 5th - 1993

18. • Fantastic worldwide community
• Ruby has changed a lot over 20 years
• MRI is a complex virtual machine with a lot of history behind
it’s design decisions.
20

19. What we noticed

20. Objects
22
RVALUE
• Direct object pointers
• Conservative GC
Conservative
collection

21. 23
Dynamic memory
string arrayhash data …
Managed Memory
• Fixed sized objects
• What about variable sized objects

22. Dynamic memory
string arrayhash data …
Managed Memory
malloc malloc malloc malloc
Malloc Memory
24

23. Dynamic memory
string arrayhash data …
malloc malloc malloc malloc
25
• No heap fragmentation
• Easy conservative marking

24. Dynamic memory
string arrayhash data …
malloc malloc malloc malloc
26
• Heavyweight allocation
• Malloc fragmentation
• Poor cache locality
• Expensive sweeping

25. Ruby has an incremental GC
Mark Sweep
Ruby
27
• GC work is interleaved with execution
• Minimizes mutator pause times
• Single threaded
• Serialized with the mutator

26. What we did

27. Goals
 Make allocations faster
 Improve GC Time
 Improve mutator performance
 100% compatibility
29

28. 100% compatibility
 No C-extensions API changes
 Pass all extended tests
 Run Rails - unmodified
30

29. Let’s make it work
31

30. 32
Starting out with Mark Sweep
static int
gc_start(rb_objspace_t *objspace …)
{
…
OMR_GC_SystemCollect();
...
}
static int
newobj_of(rb_objspace_t *objspace …)
{
...
obj = rb_omr_get_freeobj(th, sizeof(RVALUE));
...
}

31. 33
Starting out with Mark Sweep
Need to support conservative collection
Conservative collection
Object Map (bit map)
• Object map allows us to support
conservative collection
• Bit Map used to keep track of objects
• Results in ~ 1.6% memory overhead

32. Let’s make it parallel
34

33. Multithreaded GC (Stop the world)
35
mark sweep
• Introducing Parallel Global GC
• Aggressively parallelized
• New APIs for parallelism
• Support for thread pooling and task
synchronization

34. Marking in parallel
36
mark sweep
Marking:
• Scan roots in parallel
• Break VM roots into subsets
• Complete marking in parallel

35. Sweeping in parallel
37
mark sweep
Sweeping:
• Free malloc space in parallel
• Clean up objects in parallel
• Move work out of finalization

36. Hey… It works!
38

37. New tools!
39

38. Garbage Collection Memory Visualizer for Ruby MRI
with zero changes to the tool
40

39. Health Center for Ruby MRI: Live GC visualization –
No code changes required
41

40. Health Center for Ruby MRI: Method profiling
42

41. How can we make CRuby faster?
43

42. Allocations!
44

43. Making allocation fast
Thread-local heaps:
• Cache regions of the heap per-thread
• Allocate without locking
• Better locality
45

44. Thread local heap
Object
TLH
Heap
Next Alloc
47
TLH

45. Thread local heap
TLH
Object
TLH
Heap
Next Alloc
48

46. Thread local heap
TLH
Object
TLH
Heap
Next Alloc
49

47. How can we make it faster?
50

48. Lets look at off heap memory!
51

49. How do we replace malloc/free?
• malloc/free callouts are expensive
• Rely on system for memory management concerns
• Still susceptible to fragmentation and concurrency
• Black box implementation
• Replace malloc and free with a new allocator
52

50. New built-in type!
55

51. New built-in type OMRBuffers
• Create a new, variable sized object type
• Allocate all buffers on the heap as objects
typedef struct OMRBuffer {
VALUE flags;
long size;
} OMRBuffer;
56
buffer

52. 58
OMRBuffers on the heap!
string arrayhash data …
Managed Memory
buffer buffer buffer buffer
Malloc Memory

53. 59
Shorter sweeping with OMRBuffers
mark sweep
GC Before
GC After
mark
sweep

54. Getting user defined types on heap
• RDatas are used to create C-extension types.
• Typed RDatas have a new flag:
RDATA_HEAP_ALLOCATED
• Automatically heap allocates the data buffer at allocation
• No free method results in no object finalization
• Allows for heap allocation in extensions
• Extremely easy to use
60

55. OMRBuffer wins
Improvements:
• Eliminated obj_free()
• Fast allocations
• Better cache locality
Challenges:
• Heap fragmentation
• Conservative collection
• Concurrency
61

56. Multithreaded allocations
• MRI does some work in backgrounded threads
• Use OMRBuffers in background threads
• Introduce finer grained locking than the GVL
• Allows for:
• multithreaded allocations
• GCing from a background thread
62

57. Is Ruby Fast Yet?
63
8% better throughput
122.57
112.84
0
20
40
60
80
100
120
140
2.2-omr 2.2.3

58. Experiments
64

59. Generational GC in MRI
• Experimenting with non-copying generational GC
• Marking is already fast
• Heap fragmentations issues
• High memory overhead
65

60. Segregated heap in MRI
• Introduced a segregated heap into ruby
• Heap divided into regions of fixed sized objects
• Bounds maximum heap fragmentation
66

61. 67
Concurrent GC in MRI
• Adding Concurrent GC to Ruby
• Ruby threads incrementally mark
• Background thread scans
• Parallelized sweep (STW)
sweepmark

62. What’s next
70

63. OMR GC in Ruby – what’s next
• Ongoing experimentation
• Balanced
• Semispace copying generational collection
• compaction
• Real time garbage collection
71

64. What’s next for OMR?
• Open source OMR
• Make our Ruby experiments available
• We want to hear from the experts (you)
• Lets make OMR and Ruby the best they can be
72

65. Ruby+OMR
Tech Preview
Download @
goo.gl/P3yXuy
• Releasing a
preview so you
can start test-
driving
• Send us
feedback!
73

66. 74

67. 75
We want to help

68. Thank You!
76

69. John Duimovich
CTO IBM Runtimes
duimovic@ca.ibm.com
@jduimovich
Ask us Anything!
Mark Stoodley
OMR Project Lead
mstoodle@ca.ibm.com
@mstoodle
Robert Young
OMR Developer
rwyoung@ca.ibm.com
@rwy0717
77
Charlie Gracie
OMR GC Architect
crgracie@ca.ibm.com
@crgracie
Craig Lehmann
OMR Developer
craigl@ca.ibm.com
@craiglehmann

70. The end!

71. Who were those handsome guys again?
Robert Young
rwyoung@ca.ibm.com
@rwy0717
Craig Lehmann
craigl@ca.ibm.com
@CraigLehmann
82

72. Trademarks, Copyrights, Disclaimers
88
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without
notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our
general product direction and it should not be relied on in making a purchasing decision. The information
mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may not be incorporated into
any contract. The development, release, and timing of any future features or functionality described for our
products remains at our sole discretion.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other
companies. A current list of other IBM trademarks is available on the web at "Copyright and trademark information" at
http://www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION
CONTAINED IN THIS PRESENTATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY
DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER
DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE
EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR
LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE
USE OF IBM PRODUCTS OR SOFTWARE.
© Copyright International Business Machines Corporation 2015. All rights reserved.

73. Additional Important Disclaimers
• THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
• WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
• ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A
CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE,
SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
• ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.
• IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT
PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.
• IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING
OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER
DOCUMENTATION.
• NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS
OR THEIR SUPPLIERS AND/OR LICENSORS
89