The document discusses learning Ruby by reading its source code. It provides an overview of Ruby's basic object structure, with every object having an RBasic struct containing flags and a klass pointer. It describes the various structs used to represent different Ruby object types like integers, floats, strings, arrays, and classes. It details how Ruby uses singleton classes to implement class and instance methods. The document also explores Ruby's approach to class inheritance and module inclusion by examining how method lookup and class hierarchies are represented internally. It encourages spelunking the C source code to better understand how Ruby works under the hood.

1. LEARNING RUBY BY
READING THE SOURCE
tw:@burkelibbey / gh:@burke

2. THESIS:
The best way to learn a piece of infrastructure is to learn about
how it’s implemented.

3. So let’s dig in to ruby’s source!

4. TOPICS
• Basic
Object Structure
• Class
inheritance
• Singleton
• Module
• MRI
classes
inheritance
Source spelunking

5. BASIC OBJECT STRUCTURE

6. Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}

7. flags stores information like whether the object is
frozen, tainted, etc.
struct RBasic {
VALUE flags;
VALUE klass;
}
It’s mostly internal stuff that you don’t think about
very often.

8. klass is a pointer to the class of the object
struct RBasic {
VALUE flags;
VALUE klass;
}
(or singleton class, which we’ll talk about later)

9. ...but what’s a VALUE?
struct RBasic {
VALUE flags;
VALUE klass;
}

10. VALUE is basically used as a void pointer.
typedef uintptr_t VALUE;
It can point to any ruby value.

11. You should interpret “VALUE” as:
“a (pointer to a) ruby object”

12. This is a Float.
struct RFloat {
struct RBasic basic;
double float_value;
}

13. Every type of object, including Float, has an RBasic.
struct RFloat {
struct RBasic basic;
double float_value;
}

14. And then, after the RBasic, type-specific info.
struct RFloat {
struct RBasic basic;
double float_value;
}

15. Ruby has quite a few types.
Each of them has their own
type-specific data fields.

16. But given a ‘VALUE’, we don’t
know which type we have.
How does ruby know?

17. Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}
And the object type is stored inside flags.

18. Given an object of unknown
type...
VALUE a
struct αѕgє‫נ‬qqωσ {
struct RBasic basic;
ιηт ‫נ‬ѕƒкq; // ???
ƒ‫נ‬є σтн‫¢נ‬є; // ???
}
We can extract the type from ‘basic’, which is
guaranteed to be the first struct member.

19. e.g. if the type is T_STRING,
struct RString {
struct RBasic basic;
union {
struct {
long len;
...
then we know it’s a `struct RString`.

20. Every* type corresponds to a
struct type, which ALWAYS
has an RBasic as the first
struct member.
* exceptions for immediate values

21. There are custom types for
primitives, mostly to make
them faster.

22. The special-case primitive
types aren’t particularly
surprising or interesting.

23. T_STRING => RString
RBasic, string data, length.
T_ARRAY => RArray
RBasic, array data, length.
T_HASH => RHash
RBasic, hashtable.
...and so on.

24. T_OBJECT (struct RObject)
is pretty interesting.
It’s what’s used for instances
of any classes you define, or
most of the standard library.

25. TL;DR: Instance Variables.
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}
This makes sense; an instance of a class has its own
data, and nothing else.

26. It stores the number of instance variables
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}

27. struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}
And a pointer to a hashtable containing the
instance variables

28. This is a shortcut to the class variables of the
object’s class.
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}
You could get the same result by looking it up on
basic.klass (coming up right away)

29. This definition is actually slightly simplified. I
omitted another performance optimization for
readability.
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}
Go read the full one after this talk if you’re so
inclined!

30. Class and Module types

31. Classes have instance variables (ivars),
class variables (cvars), methods, and a superclass.
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}

32. This is where the methods live.
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}
st_table is the hashtable implementation ruby
uses internally.

33. Class variables live in iv_index_tbl.
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}

34. struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}
struct rb_classext_struct {
VALUE super;
struct st_table *iv_tbl;
struct st_table *const_tbl;
}
typedef struct rb_classext_struct
rb_classext_t;

35. The superclass, instance variables, and constants
defined inside the class.
struct rb_classext_struct {
VALUE super;
struct st_table *iv_tbl;
struct st_table *const_tbl;
}

36. It ends up looking kinda like:
struct RClass {
struct RBasic basic;
VALUE super;
struct st_table *iv_tbl;
struct st_table *const_tbl;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}
...though this isn’t really valid because rb_classext_t
is referred to by a pointer.

37. So classes have:
struct RClass {
struct RBasic basic;
VALUE super;
(st) *iv_tbl;
(st) *const_tbl;
(st) *m_tbl;
(st) *iv_index_tbl;
}
*
*
*
*
*
*
RBasic
superclass
instance vars.
constants
methods
class vars.

38. Modules

39. Same underlying type (struct RClass) as a class
#define RCLASS(obj) (R_CAST(RClass)(obj))
#define RMODULE(obj) RCLASS(obj)
...just has different handling in a few code paths.

40. Immediate values

41. Sort of complicated.

42. For an integer N:
The fixnum representation is:
2N + 1

43. enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};
=
=
=
=
0x03,
0x01,
0x0e,
8

44. enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK = 0x03,
RUBY_FIXNUM_FLAG
= 0x01,
A RUBY_SYMBOL_FLAG a big integer, with a
pointer is basically just
= 0x0e,
number referring to a memory
RUBY_SPECIAL_SHIFT = 8 address.
};

45. enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
Remember how a VALUE is mostly a pointer?
RUBY_IMMEDIATE_MASK = 0x03,
These tiny addresses are in the0x01, space
RUBY_FIXNUM_FLAG
= kernel
in a process image, which means they’re
RUBY_SYMBOL_FLAG
= 0x0e,
unaddressable.
RUBY_SPECIAL_SHIFT = 8
};
So ruby uses them to refer to special values.

46. enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK = 0x03,
RUBY_FIXNUM_FLAG
= 0x01,
Any VALUE equal to 0 is false,0x0e,
2 is true, 4 is
RUBY_SYMBOL_FLAG
=
nil, and 6 is a special value only 8
RUBY_SPECIAL_SHIFT = used internally.
};

47. enum ruby_special_consts {
RUBY_Qfalse = 0, on the principle that
Integers and Symbols work
RUBY_Qtrue = allocated without 4-byte
2,
memory is never
RUBY_Qnil
= 4,
alignment.
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};
=
=
=
=
0x03,
0x01,
0x0e,
8

48. enum ruby_special_consts {
Any odd VALUE
RUBY_Qfalse = 0,> 0 is a Fixnum.
RUBY_Qtrue = 2,
An even VALUE not 4,
RUBY_Qnil
= divisible by 4 might be a
RUBY_Qundef =Symbol.
6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};
=
=
=
=
0x03,
0x01,
0x0e,
8

49. Symbols are just integers.

50. There is a global table mapping
Symbol IDs to the strings they
represent.

51. Symbols are immediates because
their IDs are stored in VALUE,
and looked up in the symbol
table for display.

52. CLASS INHERITANCE

53. We have a pretty good picture of how values are
represented; now we’re going to talk about how
they interact.

54. class Language
@@random_cvar = true
attr_reader :name
def initialize(name)
@name = name
end
end
basic.klass
Class
ptr->super
Object
iv_tbl
{}
const_tbl
{}
m_tbl
{name: #<M>, initialize: #<M>}
iv_index_tbl
{@@random_cvar: true}

55. class Ruby < Language
CREATOR = :matz
@origin = :japan
end
basic.klass
Class
ptr->super
Language
iv_tbl
{@origin: :japan}
const_tbl
{CREATOR: :matz}
m_tbl
{} # NB. Empty!
iv_index_tbl
{} # NB. Empty!

56. When you subclass, you create a
new RClass with
super=(parent) and
klass=Class

57. When you instantiate a class, you
create a new RObject with
klass=(the class)

61. Method lookup

63. Class methods
class Foo
def bar
:baz
end
end
class Foo
def self.bar
:baz
end
end
Foo.new.bar
Foo.baz
We know how
this works now.
But how does
this work?

64. SINGLETON CLASSES

65. class Klass
def foo; end
end
obj = Klass.new
def obj.bar ; end
ptr->super
(superclass)
basic.klass
(class)
Image borrowed from Ruby Hacking Guide

66. Singleton classes get type T_ICLASS.
T_ICLASS objects are never*
returned to ruby-land methods.
*for sufficiently loose definitions of “never”

67. class
def
end
class
def
end
A
foo ; end
B < A
self.bar ; end
ptr->super
(superclass)
basic.klass
(class)
Image borrowed from Ruby Hacking Guide

68. class
def
end
class
def
end
A
foo ; end
B < A
self.bar ; end
ptr->super
(superclass)
basic.klass
(class)
Image borrowed from Ruby Hacking Guide

69. MODULE INHERITANCE

73. MRI SOURCE SPELUNKING

74. First, check out the source

75. github.com/ruby/ruby

78. google “<your editor> ctags”

79. CASE STUDY:
How does Array#cycle work?

80. brb live demo

81. Builtin types have a <type>.c
(string.c, array.c, proc.c, re.c, etc.)

82. Interesting methods tend to be in
those files

83. They are always present inside
double quotes
(easy to search for)

84. The next parameter after the
string is the C function name

85. e.g. Search for “upcase” (with the
quotes) in string.c and follow the
chain.

86. Most of the supporting VM
internals are in vm_*.c

87. Garbage collection is in gc.c

88. Don’t look at parse.y.
Trust me.

89. Almost all of the stuff we’ve
looked at today is in object.c,
class.c, or ruby.h

90. I mostly look up definitions of
built-in methods

91. Further reading:
Ruby under a Microscope
http://patshaughnessy.net/ruby-under-a-microscope
Ruby Hacking Guide
http://ruby-hacking-guide.github.io/

92. Thanks, questions?