Your SlideShare is downloading. ×
0
LEARNING RUBY BY
READING THE SOURCE
tw:@burkelibbey / gh:@burke
THESIS:
The best way to learn a piece of infrastructure is to learn about
how it’s implemented.
So let’s dig in to ruby’s source!
TOPICS
• Basic

Object Structure

• Class

inheritance

• Singleton
• Module
• MRI

classes

inheritance

Source spelunkin...
BASIC OBJECT STRUCTURE
Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}
flags stores information like whether the object is
frozen, tainted, etc.
struct RBasic {
VALUE flags;
VALUE klass;
}

It’s...
klass is a pointer to the class of the object
struct RBasic {
VALUE flags;
VALUE klass;
}

(or singleton class, which we’l...
...but what’s a VALUE?
struct RBasic {
VALUE flags;
VALUE klass;
}
VALUE is basically used as a void pointer.

typedef uintptr_t VALUE;

It can point to any ruby value.
You should interpret “VALUE” as:
“a (pointer to a) ruby object”
This is a Float.
struct RFloat {
struct RBasic basic;
double float_value;
}
Every type of object, including Float, has an RBasic.
struct RFloat {
struct RBasic basic;
double float_value;
}
And then, after the RBasic, type-specific info.
struct RFloat {
struct RBasic basic;
double float_value;
}
Ruby has quite a few types.
Each of them has their own
type-specific data fields.
But given a ‘VALUE’, we don’t
know which type we have.
How does ruby know?
Every object has an RBasic
struct RBasic {
VALUE flags;
VALUE klass;
}

And the object type is stored inside flags.
Given an object of unknown
type...
VALUE a

struct αѕgє‫נ‬qqωσ {
struct RBasic basic;
ιηт ‫נ‬ѕƒкq; // ???
ƒ‫נ‬є σтн‫¢נ‬є; ...
e.g. if the type is T_STRING,
struct RString {
struct RBasic basic;
union {
struct {
long len;
...

then we know it’s a `s...
Every* type corresponds to a
struct type, which ALWAYS
has an RBasic as the first
struct member.

* exceptions for immediat...
There are custom types for
primitives, mostly to make
them faster.
The special-case primitive
types aren’t particularly
surprising or interesting.
T_STRING => RString
RBasic, string data, length.
T_ARRAY => RArray
RBasic, array data, length.
T_HASH => RHash
RBasic, has...
T_OBJECT (struct RObject)
is pretty interesting.
It’s what’s used for instances
of any classes you define, or
most of the s...
TL;DR: Instance Variables.
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
...
It stores the number of instance variables
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table...
struct RObject {
struct RBasic basic;
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
}

And a pointer to a hasht...
This is a shortcut to the class variables of the
object’s class.
struct RObject {
struct RBasic basic;
long numiv;
VALUE *...
This definition is actually slightly simplified. I
omitted another performance optimization for
readability.
struct RObject ...
Class and Module types
Classes have instance variables (ivars),
class variables (cvars), methods, and a superclass.
struct RClass {
struct RBasic...
This is where the methods live.
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st...
Class variables live in iv_index_tbl.
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
str...
struct RClass {
struct RBasic basic;
rb_classext_t *ptr;
struct st_table *m_tbl;
struct st_table *iv_index_tbl;
}
struct r...
The superclass, instance variables, and constants
defined inside the class.
struct rb_classext_struct {
VALUE super;
struct...
It ends up looking kinda like:
struct RClass {
struct RBasic basic;
VALUE super;
struct st_table *iv_tbl;
struct st_table ...
So classes have:
struct RClass {
struct RBasic basic;
VALUE super;
(st) *iv_tbl;
(st) *const_tbl;
(st) *m_tbl;
(st) *iv_in...
Modules
Same underlying type (struct RClass) as a class

#define RCLASS(obj) (R_CAST(RClass)(obj))
#define RMODULE(obj) RCLASS(obj...
Immediate values
Sort of complicated.
For an integer N:
The fixnum representation is:
2N + 1
enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK
RUBY_FIXNU...
enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK = 0x03,
RU...
enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
Remember how a VALUE is mostly...
enum ruby_special_consts {
RUBY_Qfalse = 0,
RUBY_Qtrue = 2,
RUBY_Qnil
= 4,
RUBY_Qundef = 6,
RUBY_IMMEDIATE_MASK = 0x03,
RU...
enum ruby_special_consts {
RUBY_Qfalse = 0, on the principle that
Integers and Symbols work
RUBY_Qtrue = allocated without...
enum ruby_special_consts {
Any odd VALUE
RUBY_Qfalse = 0,> 0 is a Fixnum.
RUBY_Qtrue = 2,
An even VALUE not 4,
RUBY_Qnil
=...
Symbols are just integers.
There is a global table mapping
Symbol IDs to the strings they
represent.
Symbols are immediates because
their IDs are stored in VALUE,
and looked up in the symbol
table for display.
CLASS INHERITANCE
We have a pretty good picture of how values are
represented; now we’re going to talk about how
they interact.
class Language
@@random_cvar = true
attr_reader :name
def initialize(name)
@name = name
end
end
basic.klass

Class

ptr->s...
class Ruby < Language
CREATOR = :matz
@origin = :japan
end
basic.klass

Class

ptr->super

Language

iv_tbl

{@origin: :ja...
When you subclass, you create a
new RClass with
super=(parent) and
klass=Class
When you instantiate a class, you
create a new RObject with
klass=(the class)
Method lookup
Class methods
class Foo
def bar
:baz
end
end

class Foo
def self.bar
:baz
end
end

Foo.new.bar

Foo.baz

We know how
this ...
SINGLETON CLASSES
class Klass
def foo; end
end
obj = Klass.new
def obj.bar ; end

ptr->super
(superclass)

basic.klass
(class)
Image borrowe...
Singleton classes get type T_ICLASS.
T_ICLASS objects are never*
returned to ruby-land methods.
*for sufficiently loose defi...
class
def
end
class
def
end

A
foo ; end
B < A
self.bar ; end

ptr->super
(superclass)

basic.klass
(class)
Image borrowed...
class
def
end
class
def
end

A
foo ; end
B < A
self.bar ; end

ptr->super
(superclass)

basic.klass
(class)
Image borrowed...
MODULE INHERITANCE
MRI SOURCE SPELUNKING
First, check out the source
github.com/ruby/ruby
google “<your editor> ctags”
CASE STUDY:
How does Array#cycle work?
brb live demo
Builtin types have a <type>.c
(string.c, array.c, proc.c, re.c, etc.)
Interesting methods tend to be in
those files
They are always present inside
double quotes
(easy to search for)
The next parameter after the
string is the C function name
e.g. Search for “upcase” (with the
quotes) in string.c and follow the
chain.
Most of the supporting VM
internals are in vm_*.c
Garbage collection is in gc.c
Don’t look at parse.y.
Trust me.
Almost all of the stuff we’ve
looked at today is in object.c,
class.c, or ruby.h
I mostly look up definitions of
built-in methods
Further reading:
Ruby under a Microscope
http://patshaughnessy.net/ruby-under-a-microscope

Ruby Hacking Guide
http://ruby...
Thanks, questions?
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Learn Ruby by Reading the Source
Upcoming SlideShare
Loading in...5
×

Learn Ruby by Reading the Source

1,243

Published on

An updated version of a talk I've given a few times before; this one explains ruby's object model with a bit more lucidity and more C.

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,243
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Learn Ruby by Reading the Source"

  1. 1. LEARNING RUBY BY READING THE SOURCE tw:@burkelibbey / gh:@burke
  2. 2. THESIS: The best way to learn a piece of infrastructure is to learn about how it’s implemented.
  3. 3. So let’s dig in to ruby’s source!
  4. 4. TOPICS • Basic Object Structure • Class inheritance • Singleton • Module • MRI classes inheritance Source spelunking
  5. 5. BASIC OBJECT STRUCTURE
  6. 6. Every object has an RBasic struct RBasic { VALUE flags; VALUE klass; }
  7. 7. flags stores information like whether the object is frozen, tainted, etc. struct RBasic { VALUE flags; VALUE klass; } It’s mostly internal stuff that you don’t think about very often.
  8. 8. klass is a pointer to the class of the object struct RBasic { VALUE flags; VALUE klass; } (or singleton class, which we’ll talk about later)
  9. 9. ...but what’s a VALUE? struct RBasic { VALUE flags; VALUE klass; }
  10. 10. VALUE is basically used as a void pointer. typedef uintptr_t VALUE; It can point to any ruby value.
  11. 11. You should interpret “VALUE” as: “a (pointer to a) ruby object”
  12. 12. This is a Float. struct RFloat { struct RBasic basic; double float_value; }
  13. 13. Every type of object, including Float, has an RBasic. struct RFloat { struct RBasic basic; double float_value; }
  14. 14. And then, after the RBasic, type-specific info. struct RFloat { struct RBasic basic; double float_value; }
  15. 15. Ruby has quite a few types. Each of them has their own type-specific data fields.
  16. 16. But given a ‘VALUE’, we don’t know which type we have. How does ruby know?
  17. 17. Every object has an RBasic struct RBasic { VALUE flags; VALUE klass; } And the object type is stored inside flags.
  18. 18. Given an object of unknown type... VALUE a struct αѕgє‫נ‬qqωσ { struct RBasic basic; ιηт ‫נ‬ѕƒкq; // ??? ƒ‫נ‬є σтн‫¢נ‬є; // ??? } We can extract the type from ‘basic’, which is guaranteed to be the first struct member.
  19. 19. e.g. if the type is T_STRING, struct RString { struct RBasic basic; union { struct { long len; ... then we know it’s a `struct RString`.
  20. 20. Every* type corresponds to a struct type, which ALWAYS has an RBasic as the first struct member. * exceptions for immediate values
  21. 21. There are custom types for primitives, mostly to make them faster.
  22. 22. The special-case primitive types aren’t particularly surprising or interesting.
  23. 23. T_STRING => RString RBasic, string data, length. T_ARRAY => RArray RBasic, array data, length. T_HASH => RHash RBasic, hashtable. ...and so on.
  24. 24. T_OBJECT (struct RObject) is pretty interesting. It’s what’s used for instances of any classes you define, or most of the standard library.
  25. 25. TL;DR: Instance Variables. struct RObject { struct RBasic basic; long numiv; VALUE *ivptr; struct st_table *iv_index_tbl; } This makes sense; an instance of a class has its own data, and nothing else.
  26. 26. It stores the number of instance variables struct RObject { struct RBasic basic; long numiv; VALUE *ivptr; struct st_table *iv_index_tbl; }
  27. 27. struct RObject { struct RBasic basic; long numiv; VALUE *ivptr; struct st_table *iv_index_tbl; } And a pointer to a hashtable containing the instance variables
  28. 28. This is a shortcut to the class variables of the object’s class. struct RObject { struct RBasic basic; long numiv; VALUE *ivptr; struct st_table *iv_index_tbl; } You could get the same result by looking it up on basic.klass (coming up right away)
  29. 29. This definition is actually slightly simplified. I omitted another performance optimization for readability. struct RObject { struct RBasic basic; long numiv; VALUE *ivptr; struct st_table *iv_index_tbl; } Go read the full one after this talk if you’re so inclined!
  30. 30. Class and Module types
  31. 31. Classes have instance variables (ivars), class variables (cvars), methods, and a superclass. struct RClass { struct RBasic basic; rb_classext_t *ptr; struct st_table *m_tbl; struct st_table *iv_index_tbl; }
  32. 32. This is where the methods live. struct RClass { struct RBasic basic; rb_classext_t *ptr; struct st_table *m_tbl; struct st_table *iv_index_tbl; } st_table is the hashtable implementation ruby uses internally.
  33. 33. Class variables live in iv_index_tbl. struct RClass { struct RBasic basic; rb_classext_t *ptr; struct st_table *m_tbl; struct st_table *iv_index_tbl; }
  34. 34. struct RClass { struct RBasic basic; rb_classext_t *ptr; struct st_table *m_tbl; struct st_table *iv_index_tbl; } struct rb_classext_struct { VALUE super; struct st_table *iv_tbl; struct st_table *const_tbl; } typedef struct rb_classext_struct rb_classext_t;
  35. 35. The superclass, instance variables, and constants defined inside the class. struct rb_classext_struct { VALUE super; struct st_table *iv_tbl; struct st_table *const_tbl; }
  36. 36. It ends up looking kinda like: struct RClass { struct RBasic basic; VALUE super; struct st_table *iv_tbl; struct st_table *const_tbl; struct st_table *m_tbl; struct st_table *iv_index_tbl; } ...though this isn’t really valid because rb_classext_t is referred to by a pointer.
  37. 37. So classes have: struct RClass { struct RBasic basic; VALUE super; (st) *iv_tbl; (st) *const_tbl; (st) *m_tbl; (st) *iv_index_tbl; } * * * * * * RBasic superclass instance vars. constants methods class vars.
  38. 38. Modules
  39. 39. Same underlying type (struct RClass) as a class #define RCLASS(obj) (R_CAST(RClass)(obj)) #define RMODULE(obj) RCLASS(obj) ...just has different handling in a few code paths.
  40. 40. Immediate values
  41. 41. Sort of complicated.
  42. 42. For an integer N: The fixnum representation is: 2N + 1
  43. 43. enum ruby_special_consts { RUBY_Qfalse = 0, RUBY_Qtrue = 2, RUBY_Qnil = 4, RUBY_Qundef = 6, RUBY_IMMEDIATE_MASK RUBY_FIXNUM_FLAG RUBY_SYMBOL_FLAG RUBY_SPECIAL_SHIFT }; = = = = 0x03, 0x01, 0x0e, 8
  44. 44. enum ruby_special_consts { RUBY_Qfalse = 0, RUBY_Qtrue = 2, RUBY_Qnil = 4, RUBY_Qundef = 6, RUBY_IMMEDIATE_MASK = 0x03, RUBY_FIXNUM_FLAG = 0x01, A RUBY_SYMBOL_FLAG a big integer, with a pointer is basically just = 0x0e, number referring to a memory RUBY_SPECIAL_SHIFT = 8 address. };
  45. 45. enum ruby_special_consts { RUBY_Qfalse = 0, RUBY_Qtrue = 2, RUBY_Qnil = 4, RUBY_Qundef = 6, Remember how a VALUE is mostly a pointer? RUBY_IMMEDIATE_MASK = 0x03, These tiny addresses are in the0x01, space RUBY_FIXNUM_FLAG = kernel in a process image, which means they’re RUBY_SYMBOL_FLAG = 0x0e, unaddressable. RUBY_SPECIAL_SHIFT = 8 }; So ruby uses them to refer to special values.
  46. 46. enum ruby_special_consts { RUBY_Qfalse = 0, RUBY_Qtrue = 2, RUBY_Qnil = 4, RUBY_Qundef = 6, RUBY_IMMEDIATE_MASK = 0x03, RUBY_FIXNUM_FLAG = 0x01, Any VALUE equal to 0 is false,0x0e, 2 is true, 4 is RUBY_SYMBOL_FLAG = nil, and 6 is a special value only 8 RUBY_SPECIAL_SHIFT = used internally. };
  47. 47. enum ruby_special_consts { RUBY_Qfalse = 0, on the principle that Integers and Symbols work RUBY_Qtrue = allocated without 4-byte 2, memory is never RUBY_Qnil = 4, alignment. RUBY_Qundef = 6, RUBY_IMMEDIATE_MASK RUBY_FIXNUM_FLAG RUBY_SYMBOL_FLAG RUBY_SPECIAL_SHIFT }; = = = = 0x03, 0x01, 0x0e, 8
  48. 48. enum ruby_special_consts { Any odd VALUE RUBY_Qfalse = 0,> 0 is a Fixnum. RUBY_Qtrue = 2, An even VALUE not 4, RUBY_Qnil = divisible by 4 might be a RUBY_Qundef =Symbol. 6, RUBY_IMMEDIATE_MASK RUBY_FIXNUM_FLAG RUBY_SYMBOL_FLAG RUBY_SPECIAL_SHIFT }; = = = = 0x03, 0x01, 0x0e, 8
  49. 49. Symbols are just integers.
  50. 50. There is a global table mapping Symbol IDs to the strings they represent.
  51. 51. Symbols are immediates because their IDs are stored in VALUE, and looked up in the symbol table for display.
  52. 52. CLASS INHERITANCE
  53. 53. We have a pretty good picture of how values are represented; now we’re going to talk about how they interact.
  54. 54. class Language @@random_cvar = true attr_reader :name def initialize(name) @name = name end end basic.klass Class ptr->super Object iv_tbl {} const_tbl {} m_tbl {name: #<M>, initialize: #<M>} iv_index_tbl {@@random_cvar: true}
  55. 55. class Ruby < Language CREATOR = :matz @origin = :japan end basic.klass Class ptr->super Language iv_tbl {@origin: :japan} const_tbl {CREATOR: :matz} m_tbl {} # NB. Empty! iv_index_tbl {} # NB. Empty!
  56. 56. When you subclass, you create a new RClass with super=(parent) and klass=Class
  57. 57. When you instantiate a class, you create a new RObject with klass=(the class)
  58. 58. Method lookup
  59. 59. Class methods class Foo def bar :baz end end class Foo def self.bar :baz end end Foo.new.bar Foo.baz We know how this works now. But how does this work?
  60. 60. SINGLETON CLASSES
  61. 61. class Klass def foo; end end obj = Klass.new def obj.bar ; end ptr->super (superclass) basic.klass (class) Image borrowed from Ruby Hacking Guide
  62. 62. Singleton classes get type T_ICLASS. T_ICLASS objects are never* returned to ruby-land methods. *for sufficiently loose definitions of “never”
  63. 63. class def end class def end A foo ; end B < A self.bar ; end ptr->super (superclass) basic.klass (class) Image borrowed from Ruby Hacking Guide
  64. 64. class def end class def end A foo ; end B < A self.bar ; end ptr->super (superclass) basic.klass (class) Image borrowed from Ruby Hacking Guide
  65. 65. MODULE INHERITANCE
  66. 66. MRI SOURCE SPELUNKING
  67. 67. First, check out the source
  68. 68. github.com/ruby/ruby
  69. 69. google “<your editor> ctags”
  70. 70. CASE STUDY: How does Array#cycle work?
  71. 71. brb live demo
  72. 72. Builtin types have a <type>.c (string.c, array.c, proc.c, re.c, etc.)
  73. 73. Interesting methods tend to be in those files
  74. 74. They are always present inside double quotes (easy to search for)
  75. 75. The next parameter after the string is the C function name
  76. 76. e.g. Search for “upcase” (with the quotes) in string.c and follow the chain.
  77. 77. Most of the supporting VM internals are in vm_*.c
  78. 78. Garbage collection is in gc.c
  79. 79. Don’t look at parse.y. Trust me.
  80. 80. Almost all of the stuff we’ve looked at today is in object.c, class.c, or ruby.h
  81. 81. I mostly look up definitions of built-in methods
  82. 82. Further reading: Ruby under a Microscope http://patshaughnessy.net/ruby-under-a-microscope Ruby Hacking Guide http://ruby-hacking-guide.github.io/
  83. 83. Thanks, questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×