Learn Ruby by Reading the Source

LEARNING RUBY BY
READING THE SOURCE
tw:@burkelibbey / gh:@burke

THESIS:
The best way to learn a piece of infrastructure is to learn about
how it’s implemented.

So let’s dig in to ruby’s source!

TOPICS
• Basic

Object Structure

• Class

inheritance

• Singleton
• Module
• MRI

classes

inheritance

Source spelunking

Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}

ﬂags stores information like whether the object is
frozen, tainted, etc.
struct RBasic {
VALUE flags;
VALUE klass;
}

It’s mostly internal stuff that you don’t think about
very often.

klass is a pointer to the class of the object
struct RBasic {
VALUE flags;
VALUE klass;
}

(or singleton class, which we’ll talk about later)

...but what’s a VALUE?
struct RBasic {
VALUE flags;
VALUE klass;
}

VALUE is basically used as a void pointer.

typedef uintptr_t VALUE;

It can point to any ruby value.

You should interpret “VALUE” as:
“a (pointer to a) ruby object”

This is a Float.
struct RFloat {
struct RBasic basic;
double float_value;
}

Every type of object, including Float, has an RBasic.
struct RFloat {
double float_value;
}

And then, after the RBasic, type-speciﬁc info.
struct RFloat {
double float_value;
}

Ruby has quite a few types.
Each of them has their own
type-speciﬁc data ﬁelds.

But given a ‘VALUE’, we don’t
know which type we have.
How does ruby know?

Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}

And the object type is stored inside ﬂags.

Given an object of unknown
type...
VALUE a

struct αѕgє‫נ‬qqωσ {
ιηт ‫נ‬ѕƒкq; // ???
ƒ‫נ‬є σтн‫¢נ‬є; // ???
}

We can extract the type from ‘basic’, which is
guaranteed to be the ﬁrst struct member.

e.g. if the type is T_STRING,
struct RString {
union {
struct {
long len;
...

then we know it’s a `struct RString`.

Every* type corresponds to a
struct type, which ALWAYS
has an RBasic as the ﬁrst
struct member.

* exceptions for immediate values

There are custom types for
primitives, mostly to make
them faster.

The special-case primitive
types aren’t particularly
surprising or interesting.

T_STRING => RString
RBasic, string data, length.
T_ARRAY => RArray
RBasic, array data, length.
T_HASH => RHash
RBasic, hashtable.
...and so on.

T_OBJECT (struct RObject)
is pretty interesting.
It’s what’s used for instances
of any classes you deﬁne, or
most of the standard library.

TL;DR: Instance Variables.
struct RObject {
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}

This makes sense; an instance of a class has its own
data, and nothing else.

It stores the number of instance variables
struct RObject {
long numiv;
VALUE *ivptr;
}

struct RObject {
long numiv;
VALUE *ivptr;
}

And a pointer to a hashtable containing the
instance variables

This is a shortcut to the class variables of the
object’s class.
struct RObject {
long numiv;
VALUE *ivptr;
}

You could get the same result by looking it up on
basic.klass (coming up right away)

This deﬁnition is actually slightly simpliﬁed. I
omitted another performance optimization for
readability.
struct RObject {
long numiv;
VALUE *ivptr;
}

Go read the full one after this talk if you’re so
inclined!

Classes have instance variables (ivars),
class variables (cvars), methods, and a superclass.
struct RClass {
rb_classext_t *ptr;
struct st_table *m_tbl;
}

This is where the methods live.
struct RClass {
rb_classext_t *ptr;
}

st_table is the hashtable implementation ruby
uses internally.

Class variables live in iv_index_tbl.
struct RClass {
rb_classext_t *ptr;
}

struct RClass {
rb_classext_t *ptr;
}
struct rb_classext_struct {
VALUE super;
struct st_table *iv_tbl;
struct st_table *const_tbl;
}
typedef struct rb_classext_struct
rb_classext_t;

The superclass, instance variables, and constants
deﬁned inside the class.
struct rb_classext_struct {
VALUE super;
}

It ends up looking kinda like:
struct RClass {
VALUE super;
}

...though this isn’t really valid because rb_classext_t
is referred to by a pointer.

So classes have:
struct RClass {
VALUE super;
(st) *iv_tbl;
(st) *const_tbl;
(st) *m_tbl;
(st) *iv_index_tbl;
}

*
*
*
*
*
*

RBasic
superclass
instance vars.
constants
methods
class vars.

Same underlying type (struct RClass) as a class

#define RCLASS(obj) (R_CAST(RClass)(obj))
#define RMODULE(obj) RCLASS(obj)

...just has different handling in a few code paths.

For an integer N:
The ﬁxnum representation is:
2N + 1

enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};

=
=
=
=

0x03,
0x01,
0x0e,
8

RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK = 0x03,
RUBY_FIXNUM_FLAG
= 0x01,
A RUBY_SYMBOL_FLAG a big integer, with a
pointer is basically just
= 0x0e,
number referring to a memory
RUBY_SPECIAL_SHIFT = 8 address.
};

RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
Remember how a VALUE is mostly a pointer?
These tiny addresses are in the0x01, space
RUBY_FIXNUM_FLAG
= kernel
in a process image, which means they’re
RUBY_SYMBOL_FLAG
= 0x0e,
unaddressable.
RUBY_SPECIAL_SHIFT = 8
};
So ruby uses them to refer to special values.

RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_FIXNUM_FLAG
= 0x01,
Any VALUE equal to 0 is false,0x0e,
2 is true, 4 is
RUBY_SYMBOL_FLAG
=
nil, and 6 is a special value only 8
RUBY_SPECIAL_SHIFT = used internally.
};

RUBY_Qfalse = 0, on the principle that
Integers and Symbols work
RUBY_Qtrue = allocated without 4-byte
2,
memory is never
RUBY_Qnil
= 4,
alignment.
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};

=
=
=
=

0x03,
0x01,
0x0e,
8

Any odd VALUE
RUBY_Qfalse = 0,> 0 is a Fixnum.
RUBY_Qtrue = 2,
An even VALUE not 4,
RUBY_Qnil
= divisible by 4 might be a
RUBY_Qundef =Symbol.
6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNUM_FLAG
RUBY_SYMBOL_FLAG
RUBY_SPECIAL_SHIFT
};

=
=
=
=

0x03,
0x01,
0x0e,
8

There is a global table mapping
Symbol IDs to the strings they
represent.

Symbols are immediates because
their IDs are stored in VALUE,
and looked up in the symbol
table for display.

We have a pretty good picture of how values are
represented; now we’re going to talk about how
they interact.

class Language
@@random_cvar = true
attr_reader :name
def initialize(name)
@name = name
end
end
basic.klass

Class

ptr->super

Object

iv_tbl

{}

const_tbl

{}

m_tbl

{name: #<M>, initialize: #<M>}

iv_index_tbl

{@@random_cvar: true}

class Ruby < Language
CREATOR = :matz
@origin = :japan
end
basic.klass

Class

ptr->super

Language

iv_tbl

{@origin: :japan}

const_tbl

{CREATOR: :matz}

m_tbl

{} # NB. Empty!

iv_index_tbl

{} # NB. Empty!

When you subclass, you create a
new RClass with
super=(parent) and
klass=Class

When you instantiate a class, you
create a new RObject with
klass=(the class)

Class methods
class Foo
def bar
:baz
end
end

class Foo
def self.bar
:baz
end
end

Foo.new.bar

Foo.baz

We know how
this works now.

But how does
this work?

class Klass
def foo; end
end
obj = Klass.new
def obj.bar ; end

ptr->super
(superclass)

basic.klass
(class)
Image borrowed from Ruby Hacking Guide

Singleton classes get type T_ICLASS.
T_ICLASS objects are never*
returned to ruby-land methods.
*for sufﬁciently loose deﬁnitions of “never”

class
def
end
class
def
end

A
foo ; end
B < A
self.bar ; end

ptr->super
(superclass)

basic.klass
(class)
Image borrowed from Ruby Hacking Guide

google “<your editor> ctags”

CASE STUDY:
How does Array#cycle work?

Builtin types have a <type>.c
(string.c, array.c, proc.c, re.c, etc.)

Interesting methods tend to be in
those ﬁles

They are always present inside
double quotes
(easy to search for)

The next parameter after the
string is the C function name

e.g. Search for “upcase” (with the
quotes) in string.c and follow the
chain.

Most of the supporting VM
internals are in vm_*.c

Don’t look at parse.y.
Trust me.

Almost all of the stuff we’ve
looked at today is in object.c,
class.c, or ruby.h

I mostly look up deﬁnitions of
built-in methods

Further reading:
Ruby under a Microscope
http://patshaughnessy.net/ruby-under-a-microscope

Ruby Hacking Guide
http://ruby-hacking-guide.github.io/

Learn Ruby by Reading the Source

More Related Content

What's hot

Similar to Learn Ruby by Reading the Source

More from Burke Libbey

Recently uploaded

Learn Ruby by Reading the Source