Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Ruby meets Go
Dec. 12, 2015
Masaki Matsushita
About Me
● Masaki Matsushita
● CRuby Committer
○ 138 Commits
■ Mainly for performance improvement
■ Marshal.load, Hash#fla...
Today’s Topic
● Go 1.5 Feature: buildmode “c-shared”
○ Cgo Basics
● Using Go Function from Ruby
○ FFI and Fiddle without r...
Buildmode “c-shared”
● Go 1.5 relased in August 2015
● Buildmode “c-shared” was introduced
○ go build -buildmode c-shared
...
Cgo Example: Say hello with puts() in C
package main
/*
#include <stdlib.h>
#include <stdio.h>
*/
import "C"
import "unsaf...
Cgo Example: define and use C function
package main
/*
char *hello(void) {
return "Hello, world!";
}
*/
import "C"
import ...
Try c-shared: add.go
package main
import "C"
//export add
func add(a C.int, b C.int) C.int {
return a + b
}
func main() {}...
Load c-shared Libraries
● ruby-ffi
○ https://github.com/ffi/ffi
○ gem install ffi
● fiddle
○ Standard ext library
● useful...
Call Go Function from Ruby: ruby-ffi
require "ffi"
module Int
extend FFI::Library
ffi_lib "int.so"
attach_function :add, [...
Call Go Function from Ruby: fiddle
require "fiddle/import"
module Int
extend Fiddle::Importer
dlload "int.so"
extern "int ...
Go String and C String: str.so
package main
import "C"
import "fmt"
//export hello
func hello(cstr *C.char) {
str := C.GoS...
Returning String: ruby-ffi
require "ffi"
module Hello
extend FFI::Library
ffi_lib "str.so"
attach_function :hello, [:strin...
Returning String: fiddle
require "fiddle/import"
module Hello
extend Fiddle::Importer
dlload "str.so"
extern "void hello(c...
Cgo Functions to Convert String
● C.CString(goString string) *C.char
○ copy Go String to C String
○ Users are responsible ...
Writing Extension Library with Go
● Naruse-san’s Amazing Talk:
“Writing extension libraries in Go”
at OedoRubyKaigi 05
htt...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
C Extension Library Basics
static VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo...
Minimal Go Ext Example?
//export rb_magic_num
func rb_magic_num(self C.VALUE) C.VALUE {
return INT2NUM(42)
}
//export Init...
Writing Extension Library with Go
● Wrapper Function equivalent to C Macro
○ C macros can’t be used by Cgo
● Convert Go St...
C Macros for Ruby Extention Libraries
● Useful C macros are defined in ruby.h
○ INT2NUM: C int to Ruby Numeric
○ NIL_P: tr...
Use Equivalent C Function
func LONG2NUM(n C.long) C.VALUE {
return C.rb_long2num_inline(n)
}
func NUM2LONG(n C.VALUE) C.lo...
Wrap C macros with C function
package main
/*
long rstring_len(VALUE str) {
return RSTRING_LEN(str);
}
*/
import "C"
func ...
Convert Go String into Ruby without Copy
● Go String -> C String -> Ruby String
● C.CString(goString string) *C.char
○ cop...
Basic Usage of C.CString()
// go/doc/progs/cgo4.go
func Print(s string) {
cs := C.CString(s)
defer C.free(unsafe.Pointer(c...
Basic Usage of C.CString()
str := "Hello, world!"
// Copy #1
cstr := C.CString(str) // will be discarded soon
// Copy #2
r...
Avoid Copy of Strings
● Get *C.char from Go String without Copy
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte...
Avoid Copy of Strings
● Technique to Get []byte from Go w/o Copy
http://qiita.com/mattn/items/176459728ff4f854b165
func GO...
Avoid Copy of Strings
● Get *C.char from []byte
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte)(unsafe.Pointer...
Example Usage of GOSTRING_PTR()
func RbString(str string) C.VALUE {
if len(str) == 0 { return C.rb_utf8_str_new(nil, C.lon...
Propagate Ruby Reference to Go
● Go’s GC doesn’t know refs from Ruby
● Go obj referenced from Ruby can be collected
● We h...
Propagate Ruby Reference to Go
var objects = make(map[interface{}]int)
//export goobj_retain
func goobj_retain(obj unsafe....
Propagate Ruby Reference to Go
static const rb_data_type_t go_type = {
"GoStruct",
{NULL, goobj_free, NULL},
0, 0, RUBY_TY...
Create Gem including Go code
● Directory Structure
● Rakefile
● extconf.rb
Directory Structure
● Use “bundle gem --ext foo”
├── ext
│ └── foo
│ ├── extconf.rb // configured to use go build
│ ├── fo...
Rakefile
require 'bundler'
Bundler::GemHelper.install_tasks
require 'rake/extensiontask'
task :default => [:compile]
spec ...
extconf.rb
require 'mkmf'
find_executable('go')
$objs = []
def $objs.empty?; false ;end
create_makefile("memberlist/member...
extconf.rb
File.open('Makefile', 'a') do |f|
f.write <<-EOS.gsub(/^ {8}/, "t")
$(DLLIB): Makefile $(srcdir)/memberlist.go ...
Ruby meets Go
● Buildmode “c-shared” and Cgo Basics
● Using Go Function from Ruby
○ FFI and Fiddle without ruby.h
● Writin...
Upcoming SlideShare
Loading in …5
×

Ruby meets Go

23,047 views

Published on

Presentation in RubyKaigi 2015

Published in: Technology

Ruby meets Go

  1. 1. Ruby meets Go Dec. 12, 2015 Masaki Matsushita
  2. 2. About Me ● Masaki Matsushita ● CRuby Committer ○ 138 Commits ■ Mainly for performance improvement ■ Marshal.load, Hash#flatten, etc. ● Software Engineer at NTT Communications ○ Contribution to OpenStack ○ Slide at OpenStack Summit Tokyo http://goo.gl/OXTYor ● Twitter: @_mmasaki Github: mmasaki
  3. 3. Today’s Topic ● Go 1.5 Feature: buildmode “c-shared” ○ Cgo Basics ● Using Go Function from Ruby ○ FFI and Fiddle without ruby.h ● Writing Extension Library with Go (and C) ○ Define Functions Equivalent to C Macros ○ Avoid Copy of Strings ○ Propagate Reference from Ruby to Go ○ Creating Gem including Go code
  4. 4. Buildmode “c-shared” ● Go 1.5 relased in August 2015 ● Buildmode “c-shared” was introduced ○ go build -buildmode c-shared ○ Build C shared library with cgo ● cgo enables: ○ Refer to C functions, types and variables ○ Export Go functions for use by C
  5. 5. Cgo Example: Say hello with puts() in C package main /* #include <stdlib.h> #include <stdio.h> */ import "C" import "unsafe" func main() { cstr := C.CString("Hello, world!") defer C.free(unsafe.Pointer(cstr)) C.puts(cstr) } Include C header file Convert Go string into C String
  6. 6. Cgo Example: define and use C function package main /* char *hello(void) { return "Hello, world!"; } */ import "C" import "fmt" func main() { cstr := C.hello() fmt.Println(C.GoString(cstr)) } Define C Function Convert into Go String Call C Function from Go
  7. 7. Try c-shared: add.go package main import "C" //export add func add(a C.int, b C.int) C.int { return a + b } func main() {} ● go build -buildmode c-shared -o add.so add.go Export Go Function for use by C
  8. 8. Load c-shared Libraries ● ruby-ffi ○ https://github.com/ffi/ffi ○ gem install ffi ● fiddle ○ Standard ext library ● useful to call Go functions simply (without ruby.h)
  9. 9. Call Go Function from Ruby: ruby-ffi require "ffi" module Int extend FFI::Library ffi_lib "int.so" attach_function :add, [:int, :int], :int end p Int.add(15, 27) #=> 42 Load c-shared library Add Go Function to Module
  10. 10. Call Go Function from Ruby: fiddle require "fiddle/import" module Int extend Fiddle::Importer dlload "int.so" extern "int add(int, int)" end p Int.add(15, 27) #=> 42
  11. 11. Go String and C String: str.so package main import "C" import "fmt" //export hello func hello(cstr *C.char) { str := C.GoString(cstr) fmt.Println("Hello, " + str) } func main() {} Receive C String Convert to Go String
  12. 12. Returning String: ruby-ffi require "ffi" module Hello extend FFI::Library ffi_lib "str.so" attach_function :hello, [:string], :void end Hello.hello("world") #=> "Hello, world" Ruby String can be passed
  13. 13. Returning String: fiddle require "fiddle/import" module Hello extend Fiddle::Importer dlload "str.so" extern "void hello(char *str)" end Hello.hello("world") #=> "Hello, world"
  14. 14. Cgo Functions to Convert String ● C.CString(goString string) *C.char ○ copy Go String to C String ○ Users are responsible to free C String ● C.GoString(cString *C.char) string ● C.GoStringN(cString *C.char, length C.int) string ○ copy C String to Go String
  15. 15. Writing Extension Library with Go ● Naruse-san’s Amazing Talk: “Writing extension libraries in Go” at OedoRubyKaigi 05 https://speakerdeck.com/naruse/writing-extension-libraries-in-go ● gohttp: https://github.com/nurse/gohttp ○ Implementation of extension library in Go
  16. 16. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); }
  17. 17. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Method Implementation
  18. 18. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Using C Macro
  19. 19. C Extension Library Basics static VALUE rb_magic_number(VALUE self) { return INT2NUM(42); } void Init_test(void) { rb_cFoo = rb_define_class("Foo"); rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0); } Function Pointer
  20. 20. Minimal Go Ext Example? //export rb_magic_num func rb_magic_num(self C.VALUE) C.VALUE { return INT2NUM(42) } //export Init_foo func Init_foo() { rb_cFoo = rb_define_class("Foo", C.rb_cObject) rb_define_method(rb_cFoo, "magic_num", C.rb_magic_num, 0) }
  21. 21. Writing Extension Library with Go ● Wrapper Function equivalent to C Macro ○ C macros can’t be used by Cgo ● Convert Go String into Ruby without Copy ● Propagate Ruby Reference to Go ● Create gem including Go code ○ Modify Rakefile and extconf.rb
  22. 22. C Macros for Ruby Extention Libraries ● Useful C macros are defined in ruby.h ○ INT2NUM: C int to Ruby Numeric ○ NIL_P: true if obj is nil ○ RSTRING_PTR: pointer to buffer of String ○ RSTRING_LEN: lengh of String ● These macros can’t be used from Cgo… ● Define Go functions equivalent to C macros ○ Use equivalent C function ○ Wrap C macros with C function
  23. 23. Use Equivalent C Function func LONG2NUM(n C.long) C.VALUE { return C.rb_long2num_inline(n) } func NUM2LONG(n C.VALUE) C.long { return C.rb_num2long(n) }
  24. 24. Wrap C macros with C function package main /* long rstring_len(VALUE str) { return RSTRING_LEN(str); } */ import "C" func RSTRING_LEN(str C.VALUE) C.long { return C.rstring_len(str) }
  25. 25. Convert Go String into Ruby without Copy ● Go String -> C String -> Ruby String ● C.CString(goString string) *C.char ○ copy Go String to C String ○ Users are responsible to free C String ● VALUE rb_str_new(const char *ptr, long len) ○ copy C String to Ruby String
  26. 26. Basic Usage of C.CString() // go/doc/progs/cgo4.go func Print(s string) { cs := C.CString(s) defer C.free(unsafe.Pointer(cs)) C.fputs(cs, (*C.FILE)(C.stdout)) } ● Call C func and discard C str soon
  27. 27. Basic Usage of C.CString() str := "Hello, world!" // Copy #1 cstr := C.CString(str) // will be discarded soon // Copy #2 rbstr := C.rb_str_new(cstr, C.long(len(str))) ● Need to copy twice!
  28. 28. Avoid Copy of Strings ● Get *C.char from Go String without Copy func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) } // example of use cstr := GOSTRING_PTR(str) C.rb_utf8_str_new (cstr, C.long(len(str)))
  29. 29. Avoid Copy of Strings ● Technique to Get []byte from Go w/o Copy http://qiita.com/mattn/items/176459728ff4f854b165 func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) }
  30. 30. Avoid Copy of Strings ● Get *C.char from []byte func GOSTRING_PTR(str string) *C.char { bytes := *(*[]byte)(unsafe.Pointer(&str)) return (*C.char)(unsafe.Pointer(&bytes[0])) } Cast to char
  31. 31. Example Usage of GOSTRING_PTR() func RbString(str string) C.VALUE { if len(str) == 0 { return C.rb_utf8_str_new(nil, C.long(0)) } return C.rb_utf8_str_new(GOSTRING_PTR(str), GOSTRING_LEN(str)) } func rb_define_class(name string, parent C.VALUE) C.VALUE { return C.rb_define_class(GOSTRING_PTR(name), parent) } func rb_define_method(klass C.VALUE, name string, fun unsafe.Pointer, args int) { cname := GOSTRING_PTR(name) C.rb_define_method(klass, cname, (*[0]byte)(fun), C.int(args)) }
  32. 32. Propagate Ruby Reference to Go ● Go’s GC doesn’t know refs from Ruby ● Go obj referenced from Ruby can be collected ● We have to propagate Ruby Refs to Go ● Use Map to keep reference to Go Objects
  33. 33. Propagate Ruby Reference to Go var objects = make(map[interface{}]int) //export goobj_retain func goobj_retain(obj unsafe.Pointer) { objects[obj]++ // increment reference count } //export goobj_free func goobj_free(obj unsafe.Pointer) { objects[obj]-- // decrement reference count if objects[obj] <= 0 { delete(objects, obj) } }
  34. 34. Propagate Ruby Reference to Go static const rb_data_type_t go_type = { "GoStruct", {NULL, goobj_free, NULL}, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY|RUBY_TYPED_WB_PROTECTED }; VALUE NewGoStruct(VALUE klass, void *p) { goobj_retain(p); return TypedData_Wrap_Struct((klass), &go_type, p); } Increment Reference Count Decrement Reference Count
  35. 35. Create Gem including Go code ● Directory Structure ● Rakefile ● extconf.rb
  36. 36. Directory Structure ● Use “bundle gem --ext foo” ├── ext │ └── foo │ ├── extconf.rb // configured to use go build │ ├── foo.c // helper functions for use by Go │ └── foo.h // export helper functions │ ├── foo.go // created by hand │ └── wrapper.go // created by hand └── lib
  37. 37. Rakefile require 'bundler' Bundler::GemHelper.install_tasks require 'rake/extensiontask' task :default => [:compile] spec = eval File.read('foo.gemspec') Rake::ExtensionTask.new('foo', spec) do |ext| ext.lib_dir = File.join(*['lib', 'foo', ENV['FAT_DIR']].compact) ext.ext_dir = 'ext/foo' ext.source_pattern = "*.{c,cpp,go}" end ● Add .go into source_pattern
  38. 38. extconf.rb require 'mkmf' find_executable('go') $objs = [] def $objs.empty?; false ;end create_makefile("memberlist/memberlist") case `#{CONFIG['CC']} --version` when /Free Software Foundation/ ldflags = '-Wl,--unresolved-symbols=ignore-all' when /clang/ ldflags = '-undefined dynamic_lookup' end ● Some techniques to build successful
  39. 39. extconf.rb File.open('Makefile', 'a') do |f| f.write <<-EOS.gsub(/^ {8}/, "t") $(DLLIB): Makefile $(srcdir)/memberlist.go $(srcdir)/wrapper.go CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' go build -p 4 -buildmode=c-shared -o $(DLLIB) EOS end ● Modify Makefile to use go build
  40. 40. Ruby meets Go ● Buildmode “c-shared” and Cgo Basics ● Using Go Function from Ruby ○ FFI and Fiddle without ruby.h ● Writing Extension Library with Go ○ Define Functions Equivalent to C Macros ○ Avoid Copy of Strings ○ Propagate Reference from Ruby to Go ○ Creating Gem including Go code ● Let’s Hack Go for Ruby together!

×