Your SlideShare is downloading. ×
0
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Hacking with ruby2ruby
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Hacking with ruby2ruby

2,382

Published on

Gisting is an implementation of Google\'s MapReduce framework for processing and extracting useful information from very large data sets. At the time of this writing, the code is available for …

Gisting is an implementation of Google\'s MapReduce framework for processing and extracting useful information from very large data sets. At the time of this writing, the code is available for PREVIEW at http://github.com/mchung/gisting. I am currently working to release this framework for general usage.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,382
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Hacking with ruby2ruby Marc Chung OpenRain blog.marcchung.com blog.openrain.com @heisenthought
  • 2. Hacking with ruby2ruby Marc Chung OpenRain blog.marcchung.com blog.openrain.com @heisenthought
  • 3. ruby2ruby Ruby2Ruby Walks the Ruby AST To emit Ruby
  • 4. sudo gem install ruby2ruby
  • 5. Limitations • Only Ruby 1.8 • Not 1.9, yet • Built into Rubinius
  • 6. function rhino_rocks() { return “Rhino, rocks!”; }
  • 7. js> rhino_rocks(); Rhino, rocks!
  • 8. js> rhino_rocks.toString(); function rhino_rocks() { return “Rhino, rocks!”; }
  • 9. js> eval(rhino_rocks.toString()).call(); Rhino, rocks!
  • 10. ruby_rocks = proc { return “Ruby, rocks”; }
  • 11. irb> ruby_rocks.call; Ruby, rocks!
  • 12. irb> require “ruby2ruby” irb> require “parse_tree_extensions” irb> ruby_rocks.to_ruby proc { return “Ruby, rocks” }
  • 13. irb> eval(ruby_rocks.to_ruby).call Ruby, rocks!
  • 14. define_method class Voice define_method(:shout) do |word| word.upcase! end end puts Ruby2Ruby.translate(Voice)
  • 15. class Voice < Object def yell(word) word.upcase! end end
  • 16. alias class AliasClass def old_busted quot;42quot; end alias_method :new_hotness, :old_busted end puts Ruby2Ruby.translate(AliasClass)
  • 17. class AliasClass < Object def new_hotness quot;42quot; end def old_busted quot;42quot; end end
  • 18. #module_function module HelperModule def help puts quot;Help mequot; end module_function : help end puts Ruby2Ruby.translate(HelperModule)
  • 19. module HelperModule def help puts(quot;Help mequot;) end def self. help puts(quot;Help mequot;) end end
  • 20. ActiveRecord Models class Doctor < ActiveRecord::Base has_many :patients end
  • 21. See, Ruby, Run! • Drmap • git clone git://github.com/mchung/drmap.git • Optionally, install Journeta
  • 22. Under the hood • Abstract syntax trees • S-expressions • ParseTree
  • 23. class Book def title “Ruby programming” end end
  • 24. case NODE_BREAK: case NODE_NEXT: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals); break; case NODE_YIELD: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals); if (node->nd_stts && (nd_type(node->nd_stts) == NODE_ARRAY || nd_type(node->nd_stts) == NODE_ZARRAY) && !node->nd_state) rb_ary_push(current, Qtrue); break;
  • 25. [:class, :Book, [:const, :Object], [:defn, :title, [:scope, [:block, [:args], [:str, quot;Ruby programmingquot;]]]]]
  • 26. class Book < Object def title “Ruby programming” end end
  • 27. /opt/local/usr/bin • parse_tree_show • r2r_show
  • 28. class Book def title “Ruby programming” end end
  • 29. $ parse_tree_show book.rb
  • 30. s(:class, :Book, nil, s(:scope, s(:defn, :title, s(:args), s(:scope, s(:block, s(:str, quot;Ruby programmingquot;))))))
  • 31. $ r2r_show book.rb
  • 32. class Book def title() “Ruby programming” end end
  • 33. Operator Precedence with Guy Decoux: a b c, d [[:fcall, :a, [:array, [:fcall, :b, [:array, [:vcall, :c], [:vcall, :d]]]]]] ParseTree.new.parse_tree_for_string(“a b c, d”)
  • 34. ruby2java
  • 35. class JavaClass def self.main puts “Java, rocks!” end end
  • 36. [:class, :JavaClass, [:const, :Object], [:defs, [:self], :main, [:scope, [:block, [:args], [:fcall, :puts, [:array, [:str, quot;Hello, Javaquot;]]]]]]]
  • 37. class Ruby2Java < SexpProcessor def process_class(exp) “public class #{exp.shift} #{next_token(exp, true)}” end def process_const(exp) “extends #{exp.shift}” end .... end
  • 38. public class JavaClass { public static void main(String argv[]) { return “Hello, Java”; } }
  • 39. Distributing Ruby • drmap - multi-machine worker queue • gisting - map/reduce framework
  • 40. drmap irb> fn = proc {|x| x**10} irb> YAML::dump(fn) => quot;--- !ruby/object:Proc {}nnquot; irb> Marshal.dump(fn) TypeError: no marshal_dump is defined for class Proc irb> YAML::dump(fn.to_ruby) => quot;--- proc { |x| (x ** 10) }nquot; irb> Marshal.dump(fn.to_ruby) => quot;004bquot;eproc { |x| (x ** 10) }quot;
  • 41. Enumerable#drmap
  • 42. module Enumerable def drmap(&block) pool = Drmap::BeanstalkPool.new jid = rand(100) each_with_index do |element, idx| pool.put_job(jid, block.to_ruby, element) end results = [] while results.size < length results << pool.next_result(jid) end results end end
  • 43. def process loop do begin job_payload = next_job fn = job_payload[:proc] data = job_payload[:data] result = eval(fn).call(data) save(job_payload[:jid], result) success! rescue => e fail!(e) end end end
  • 44. drmap • Trivial • Distributed • Powerful
  • 45. Demo time!
  • 46. Gisting • MapReduce in Ruby • http://github.com/mchung/gisting
  • 47. map / reduce • Functional programming • Iterating over collections • (5..10).map {|x| x+1} #=> [6, 7, 8, 9, 10, 11] • (5..10).reduce(:+) #=> 45
  • 48. Google MapReduce • Modeled after map() and reduce() • Programming model • Programs are trivially parallelizable
  • 49. Demo time!
  • 50. Gisting::Spec
  • 51. input.file_pattern = quot;file.txtquot; input.map do |map_input| words = map_input.strip.split(quot;tquot;) Emit(words[1], quot;1quot;) end
  • 52. output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count) end
  • 53. inputs = args spec = Gisting::Spec.new inputs.each do |file_input| input = spec.add_input input.file_pattern = file_input input.map do |map_input| # 2722 mailbox 2006-05-23 00:08:39 words = map_input.strip.split(quot;tquot;) Emit(words[1], quot;1quot;) end end output = spec.output output.filebase = quot;/Users/mchung/Public/datasets/outputquot; output.num_tasks = 2 output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count) end result = MapReduce(spec) pp result
  • 54. Gisting • MapReduce programming model in Ruby • Uses ruby2ruby to serialize procs • Uses EventMachine for Map/ReduceServer
  • 55. • -: 94978 • mailbox: 20872 • google: 6758 • ebay: 2832 • yahoo: 2674 • yahoo.com: 2198 • myspace.com: 1916 • google.com: 1882
  • 56. self.give(:thanks) OpenRain openrain.com My Git Repo github.com/mchung Marc Chung blog.marcchung.com Ruby::AZ rubyaz.org
  • 57. Resources Code Generation: The Safety Scissors Of Metaprogramming mwrc2008.confreaks.com/03bowkett.html Forbidden Fruit: A Taste of Ruby's Parse Tree goruco2008.confreaks.com/03_wanstrath.html Ruby2Ruby seattlerb.rubyforge.org/ruby2ruby/ ParseTree parsetree.rubyforge.org

×