What lies beneath the beautiful code?

What lies beneath
the beautiful
code?

Ruby Conf India 2012

self.inspect

{

:name => “Niranjan Sarade”,

:role => “ruby developer @ TCS”,

:blog => “http://niranjansarade.blogspot.com”

:tweet => “twitter.com/nirusuma”,

:github => “github.com/NiranjanSarade”

}


Ruby

Beautiful

Pure object oriented

Interpreted


Matz’s Ruby Interpreter (MRI)

Koichi’s Ruby Interpreter (KRI)


Why
should
we
know?


Let’s dive in!


Why C?


TPI Ruby 1.8

Ruby
source
Tokenize
code (yylex)

Parse
Series of tokens (yacc)

Interpret
AST


TPCI Ruby 1.9

Ruby
source
Tokenize
code (yylex)

Parse
Series of tokens (Bison)

Compile
AST (compile.c)

Interpret
bytecode (YARV)

Tokenized

Parsed (AST)

bytecode


Ruby Source Overview

# README.EXT

ruby language core

class.c : classes and modules
error.c : exception classes and exception mechanism
gc.c : memory management
load.c : library loading
object.c : objects
variable.c : variables and constants

ruby syntax parser
parse.y
-> parse.c : automatically generated
keywords : reserved keywords
-> lex.c : automatically generated


ruby evaluator (a.k.a. YARV)

compile.c
eval.c
eval_error.c
eval_jump.c
eval_safe.c
insns.def : definition of VM instructions
iseq.c : implementation of VM::ISeq
thread.c : thread management and context swiching
thread_win32.c : thread implementation
thread_pthread.c : ditto
vm.c
vm_dump.c
vm_eval.c
vm_exec.c
vm_insnhelper.c
vm_method.c


regular expression engine (oniguruma)
regex.c
regcomp.c
regenc.c
regerror.c
regexec.c
regparse.c
regsyntax.c

utility functions
debug.c : debug symbols for C debuggger
dln.c : dynamic loading
st.c : general purpose hash table
strftime.c : formatting times
util.c : misc utilities


ruby interpreter implementation

dmyext.c
dmydln.c
dmyencoding.c
id.c
inits.c
main.c
ruby.c
version.c

multilingualization
encoding.c : Encoding
transcode.c : Encoding::Converter
enc/*.c : encoding classes
enc/trans/* : codepoint mapping tables


class library

array.c : Array numeric.c : Numeric, Integer, Fixnum,
bignum.c : Bignum
compar.c : Comparable Float
complex.c : Complex pack.c : Array#pack, String#unpack
cont.c : Fiber, Continuation proc.c : Binding, Proc
dir.c : Dir process.c : Process
enum.c : Enumerable random.c : random number
enumerator.c : Enumerator range.c : Range
file.c : File rational.c : Rational
hash.c : Hash re.c : Regexp, MatchData
io.c : IO signal.c : Signal
marshal.c : Marshal sprintf.c :
math.c : Math string.c : String
struct.c : Struct
time.c : Time


ruby.h

Struct Rbasic Struct RRegexp

Struct RObject Struct RHash

Struct RClass Struct RFile

Struct RFloat Struct RBignum

Struct RString Struct RArray


RObject, RBasic and RClass

struct RObject { struct RClass {
struct RBasic basic; struct RBasic basic;
union { rb_classext_t *ptr;
struct { struct st_table *m_tbl;
long numiv; struct st_table *iv_index_tbl;
VALUE *ivptr; };
struct st_table *iv_index_tbl;
} heap;
} as;
};

struct RBasic {
VALUE flags;
VALUE klass;
};


Instance specific behavior

my_obj = Object.new

def my_obj.hello
p “hello”
end

my_obj.hello
#=> hello

Object.new.hello

# NoMethodError: # undefined method `hello' for #<Object:0x5418467>


Conceptual sketch

Object
my_obj
klass *m_tbl

Object
*m_tbl

‘my_obj
my_obj *super
klass *m_tbl
-hello


#class.c
VALUE
make_singleton_class(VALUE obj)
{
VALUE orig_class = RBASIC(obj)->klass;
VALUE klass = rb_class_boot(orig_class);

FL_SET(klass, FL_SINGLETON);
RBASIC(obj)->klass = klass;

return klass;
}


Am I Immediate Object or Pointer ?

VALUE


typedef unsigned long VALUE

C type for referring to arbitrary ruby objects

Stores immediate values of :-
Fixnum
Symbols
True
False
Nil
Undef

Bit test :
If the LSB = 1, it is a Fixnum.

If the VALUE is equal to 0,2,4, or 6 it is a special constant:
false, true, nil, or undef.

If the lower 8 bits are equal to '0xe', it is a Symbol.

Otherwise, it is an Object Reference

RString

#1.8.7 # 1.9.3
struct RString { #define RSTRING_EMBED_LEN_MAX ((int)
struct RBasic basic; ((sizeof(VALUE)*3)/sizeof(char)-1))
long len; struct RString {
char *ptr; struct RBasic basic;
union { union {
long capa; struct {
VALUE shared; long len;
} aux; char *ptr;
}; union {
long capa;
VALUE shared;
} aux;
} heap;
char ary[RSTRING_EMBED_LEN_MAX + 1];
} as;
};


Images created using wordle.net

Heap Strings

Heap
str

RString

char *ptr “This is a very very very
long len = 46 very very long string”

str2


Shared Strings
str = "This is a very very very very very long string"
str2 = String.new(str)
#str2 = str.dup

Heap
RString
char *ptr
str2
long len = 46
VALUE shared

“This is a very very very
very very long string”

RString

str char *ptr
long len = 46


Copy on Write

str2 = str.dup
str2.upcase!

Heap
RString
str char *ptr
“This is a very very very very very
long string”
long len = 46

RString
“THIS IS A VERY VERY VERY
str2 char *ptr VERY VERY LONG STRING”
long len = 46


Embedded Strings

str2 = str[0..3]
#str2 = “This”

Heap
RString
str char *ptr
“This is a very very very very very
long string”
long len = 46

Rstring

str2 long len = 4
char ary[] = “This”


Shared Strings with slice

str2 = str[1..-1]
#str2 = str[22..-1]
# 0 <= start_offset < 46-23

RString
Heap

str char *ptr
long len = 46
VALUE shared
T h i . . i n g

RString

str2 char *ptr
long len = 45


String.new(“learning”)

Creating a string 23 characters or less is fastest

Creating a substring running to the end of the target string is also fast

When sharing same string data, memory and execution time is saved

Creating any other long substring or string, 24 or more bytes, is slower.


RHash
1.8.7 :002 > {1 => "a", "f" => "b", 2 => "c"}
=> {1=>"a", 2=>"c", "f"=>"b"}

1.9.3p0 :001 > {1 => "a", "f" => "b", 2 => "c"}
=> {1=>"a", "f"=>"b", 2=>"c"}

#1.8.7 #1.9.3
struct RHash { struct RHash {
struct RBasic basic; struct RBasic basic;
struct st_table *tbl; struct st_table *ntbl;
int iter_lev; int iter_lev;
VALUE ifnone; VALUE ifnone;
}; };

struct st_table { struct st_table {
struct st_hash_type *type; const struct st_hash_type *type;
int num_bins; st_index_t num_bins;
int num_entries; ...
struct st_table_entry **bins; struct st_table_entry **bins;
}; struct st_table_entry *head, *tail;
};
struct st_table_entry { struct st_table_entry {
st_data_t key; st_data_t key;
st_data_t record; st_data_t record;
st_table_entry *next; st_table_entry *next;
}; st_table_entry *fore, *back;
};

RHash 1.8.7
st_table_entries

key1 value key3 value x

st_table
key2 value x
num_entries = 4

num_bins = 5

**bins

key4 value x

hash buckets - slots


RHash 1.9.3
st_table_entries

1x key1 value key2 value 3
4x 2

st_table
3 key3 value 4
2 3
num_entries = 4

num_bins = 5

**bins

*head

*tail

4 key4 value 1x
3 4x

hash buckets - slots


C Extensions – why and when ?

Performance

Using C libraries from ruby applications

Using ruby gems with native C extensions

e.g. mysql, nokogiri, eventmachine, RedCloth, Rmagick, libxml-ruby, etc

Since ruby interpreter is implemented in C, its API can be used


My fellow ist
Patrick Shaughnessy


Image Credits

http://pguims-random-science.blogspot.in/2011/08/ten-benefits-of-scuba-diving.html

http://www.istockphoto.com/stock-illustration-7620122-tree-roots.php

http://horror.about.com/od/horrortoppicklists/tp/familyfriendlyhorror.01.htm

http://www.creditwritedowns.com/2011/07/european-monetary-union-titanic.html

http://irvine-orthodontist.com/wordpress/for-new-patients/faqs


Thank you all for being patient
and hearing me out !

Hope this helps you !


What lies beneath the beautiful code?

In this document