Successfully reported this slideshow.

Three Ruby usages

1

Share

Upcoming SlideShare
Python VS GO
Python VS GO
Loading in …3
×
1 of 56
1 of 56

Three Ruby usages

1

Share

Download to read offline

This talk describes three Ruby usages:

* Implementing high-level interface by Ruby.
* Using Ruby as a glue language.
* Embedding Ruby into a C program for flexibility.

All of them have pros and cons. They are trade-off. If a case can
ignore cons, we will have pros for the case.

Most of Rubyists implement their Ruby applications only by Ruby. This
talk shows another options to use Ruby as your convenient tool. If you
know another options and trade-offs of them, you will implement your
Ruby applications more effectively.

This talk uses Droonga, a distributed full-text search engine, as a
sample application to describe these Ruby usages. Droonga uses these
three Ruby usages.

This talk describes three Ruby usages:

* Implementing high-level interface by Ruby.
* Using Ruby as a glue language.
* Embedding Ruby into a C program for flexibility.

All of them have pros and cons. They are trade-off. If a case can
ignore cons, we will have pros for the case.

Most of Rubyists implement their Ruby applications only by Ruby. This
talk shows another options to use Ruby as your convenient tool. If you
know another options and trade-offs of them, you will implement your
Ruby applications more effectively.

This talk uses Droonga, a distributed full-text search engine, as a
sample application to describe these Ruby usages. Droonga uses these
three Ruby usages.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Three Ruby usages

  1. 1. Three Ruby usages Kouhei Sutou ClearCode Inc. RubyKaigi 2014 2014/09/20 Three Ruby usages Powered by Rabbit 2.1.4
  2. 2. Silver sponsor Three Ruby usages Powered by Rabbit 2.1.4
  3. 3. Goal You know three Ruby usages ✓High-level interface ✓Glue ✓Embed ✓ ✓You can remember them later Three Ruby usages Powered by Rabbit 2.1.4
  4. 4. Targets High-level interface ✓Pure Rubyists ✓ Glue ✓Rubyists who can write C/C++ ✓ Embed ✓Rubyists who also write C/C++ ✓ Three Ruby usages Powered by Rabbit 2.1.4
  5. 5. Case study Implement distributed full-text search engine in Ruby Abbreviation: DFTSE = Distributed Full-Text Search Engine Three Ruby usages Powered by Rabbit 2.1.4
  6. 6. DFTSE? Distrubuted Full- Text Search Engine Full- Text Search Engine 1: Full-text search 2: Distribute sub requests 3: Merge responses Three Ruby usages Powered by Rabbit 2.1.4
  7. 7. Why do we use DFTSE? I'm developing Droonga (A DFTSE implementation in Ruby)  Three Ruby usages Powered by Rabbit 2.1.4
  8. 8. High-level interface Three Ruby usages High-level interface Target: ✓ Pure Rubyists ✓ ✓Glue ✓Embed Three Ruby usages Powered by Rabbit 2.1.4
  9. 9. High-level interface Provides lower layer feature to higher layer ✓ With simpler/✓ convenience API Three Ruby usages Powered by Rabbit 2.1.4
  10. 10. High-level interface Higher layer users High-level interface Feature by Yukihiro Matsumoto Application/Library Three Ruby usages Powered by Rabbit 2.1.4
  11. 11. Example Vagrant Active Record Developers Vagrantfile Build development environment Vagrant b y Y u k i h i r o M a t s u m o t o Developers Object based API b y Y u k i h i r o M a t s u m o t o Access data in RDBMS Active Record Three Ruby usages Powered by Rabbit 2.1.4
  12. 12. Droonga: High-level IF DFTSE components Full-✓ text search engine ✓Messaging system ✓Cluster management ✓Process management Three Ruby usages Powered by Rabbit 2.1.4
  13. 13. Messaging system 1: Full-text search DTFSE FTSE 2: Distribute sub requests 3: Merge responses Worker process Messaging system Three Ruby usages Powered by Rabbit 2.1.4
  14. 14. Messaging system Provides distributed search feature ✓Plan how to search ✓Distribute requests ✓Merge responses ✓ ✓Users don't know details Three Ruby usages Powered by Rabbit 2.1.4
  15. 15. Characteristic Plan how to search May speed up/✓ down over 100 times ✓ Distribute requests ✓Network bound operation ✓ Merge responses ✓CPU and network bound operation ✓ Three Ruby usages Powered by Rabbit 2.1.4
  16. 16. Point Algorithm is important Need to find new/existing better algorithm ✓ "Rapid prototype and measure" feedback loop is helpful ✓ Ruby is ✓ good at rapid dev. ✓ Three Ruby usages Powered by Rabbit 2.1.4
  17. 17. Glue Three Ruby usages ✓High-level interface Glue Target: Rubyists who can write C/C++ ✓ ✓ ✓Embed Three Ruby usages Powered by Rabbit 2.1.4
  18. 18. Glue Export a feature Feature Combine features Glue Ruby Other Language Three Ruby usages Powered by Rabbit 2.1.4
  19. 19. Example mysql2 gem Vagrant Access to MySQL VM Provision libmysqlclient.so (VirtualBox) (Chef) Feature Active Record Glue Three Ruby usages Powered by Rabbit 2.1.4
  20. 20. Why do we glue? Reuse ✓ existing features Three Ruby usages Powered by Rabbit 2.1.4
  21. 21. How to glue Use external library Implement ✓ bindings (mysql2 gem) ✓ Use external command ✓Spawn command (Vagrant) ✓ Use external service ✓Implement client ✓ Three Ruby usages Powered by Rabbit 2.1.4
  22. 22. Glue in Droonga Rroonga: Groonga bindings Groonga: ✓ FTSE C library (and server) ✓ Cool.io: libev bindings libev: Event loop C library (Based on I/O multiplexing and non-blocking I/O) ✓ ✓ ✓Serf: Clustering tool (in Droonga) Three Ruby usages Powered by Rabbit 2.1.4
  23. 23. Rroonga in Droonga 1: Full-text search DTFSE FTSE 2: Distribute sub requests 3: Merge responses Worker process Messaging system Three Ruby usages Powered by Rabbit 2.1.4
  24. 24. FTSE in Droonga ✓Must be fast! ✓CPU bound processing Three Ruby usages Powered by Rabbit 2.1.4
  25. 25. For fast Rroonga Do heavy processing in C Nice to ✓ have Ruby-ish API ✓ Less memory allocation ✓Cache internal buffer ✓ Multiprocessing ✓Groonga supports multiprocessing ✓ Three Ruby usages Powered by Rabbit 2.1.4
  26. 26. Search Groonga::Database.open(ARGV[0]) entries = Groonga["Entries"] entries.select do |record| record.description =~ "Ruby" end Three Ruby usages Powered by Rabbit 2.1.4
  27. 27. Search - Pure Ruby (ref) Groonga::Database.open(ARGV[0]) entries = Groonga["Entries"] entries.find_all do |record| # This block is evaluated for each record /Ruby/ =~ record.description end Three Ruby usages Powered by Rabbit 2.1.4
  28. 28. Search impl. # (2) Evaluate expression in C entries.select do |record| # (1) Build expression in Ruby # This block is evaluated only once record.description =~ "Ruby" end Three Ruby usages Powered by Rabbit 2.1.4
  29. 29. Search impl. - Fig. Ruby by Groonga project C entries.select do |record| record.description =~ "Ruby" end Build Search request Expression Result set Evaluate expression by Groonga project Three Ruby usages Powered by Rabbit 2.1.4
  30. 30. Search - Benchmark ✓Ruby (It's already showed) ✓C Three Ruby usages Powered by Rabbit 2.1.4
  31. 31. Search - C grn_obj *expr; grn_obj *variable; const gchar *filter = "description @ "Ruby""; grn_obj *result; GRN_EXPR_CREATE_FOR_QUERY(&ctx, table, expr, variable); grn_expr_parse(&ctx, expr, filter, strlen(filter), NULL, GRN_OP_MATCH, GRN_OP_AND, GRN_EXPR_SYNTAX_SCRIPT); result = grn_table_select(&ctx, table, expr, NULL, GRN_OP_OR); grn_obj_unlink(&ctx, expr); grn_obj_unlink(&ctx, result); Three Ruby usages Powered by Rabbit 2.1.4
  32. 32. Search - Benchmark Ruby impl. is fast enough  Impl. Elapsed time C 0.6ms Ruby 0.8ms (Full-text search with "Ruby" against 72632 records) Three Ruby usages Powered by Rabbit 2.1.4
  33. 33. Embed Three Ruby usages ✓High-level interface ✓Glue Embed Target: Rubyists who also write C/C++ ✓ ✓ Three Ruby usages Powered by Rabbit 2.1.4
  34. 34. Embed Internal engine Interface C/C++ application C/C++ library Implement some features in Ruby Plugin API Conifugration by Yukihiro Matsumoto by Yukihiro Matsumoto C/C++ application C/C++ library Three Ruby usages Powered by Rabbit 2.1.4
  35. 35. Examples Internal engine Interface Implement query optimizer in Ruby vim-ruby by Yukihiro Matsumoto by Yukihiro Matsumoto VIM Three Ruby usages Powered by Rabbit 2.1.4
  36. 36. Embed in Droonga by Yukihiro Matsumoto by Yukihiro Matsumoto ... by The Groonga Project by The Groonga Project by The Groonga Project mruby by Yukihiro Matsumoto Three Ruby usages Powered by Rabbit 2.1.4
  37. 37. CRuby vs. mruby CRuby ✓Full featured! ✓Signal handler isn't needed  ✓ mruby ✓Multi-interpreters in a process! ✓You may miss some features  ✓ Three Ruby usages Powered by Rabbit 2.1.4
  38. 38. mruby in Groonga ✓Query optimizer Command interface (plan) ✓Interface and also high-level interface! ✓ Plugin API (plan) ✓Interface! ✓ Three Ruby usages Powered by Rabbit 2.1.4
  39. 39. Query optimizer Query Optimize Optimized Query Optimizer query Full-text search Evaluator by Yukihiro Matsumoto Result set Three Ruby usages Powered by Rabbit 2.1.4
  40. 40. Query optimizer Plan how to search ✓It's a bother  ✓ ✓Light operation than FTS Depends on data (Choose effective index, use table scan and so on) ✓ Three Ruby usages Powered by Rabbit 2.1.4
  41. 41. Example rank < 200 && rank > 100 Three Ruby usages Powered by Rabbit 2.1.4
  42. 42. Simple impl. rank 1 2 ... 100 ... 200 ... ... 10000 rank > 100 rank < 200 101 199 && 101 ... 199 rank < 200 && rank > 100 Three Ruby usages Powered by Rabbit 2.1.4
  43. 43. Simple impl. Slow against many out of range data ✓ Three Ruby usages Powered by Rabbit 2.1.4
  44. 44. Optimized impl. rank 1 2 ... 100 101 ... 199 200 ... ... 10000 100 < rank < 200 101 ... 199 rank < 200 && rank > 100 Three Ruby usages Powered by Rabbit 2.1.4
  45. 45. Is embedding reasonable? Measure Three Ruby usages Powered by Rabbit 2.1.4
  46. 46. Measure ✓mruby overhead ✓Speed-up by optimization Three Ruby usages Powered by Rabbit 2.1.4
  47. 47. Overhead Small overhead: Reasonable # conds mruby Elapsed 1 ○ 0.24ms 1 × 0.16ms 4 ○ 0.45ms 4 × 0.19ms Three Ruby usages Powered by Rabbit 2.1.4
  48. 48. Speed-up Fast for many data:Reasonable # records mruby no mruby 1000 0.29ms 0.31ms 10000 0.31ms 2.3ms 100000 0.26ms 21.1ms 1000000 0.26ms 210.2ms Three Ruby usages Powered by Rabbit 2.1.4
  49. 49. Note Embedding needs many works Write bindings, import mruby your build system and ... ✓ ✓ How to test your mruby part? ✓And how to debug? ✓ Three Ruby usages Powered by Rabbit 2.1.4
  50. 50. Conclusion
  51. 51. Conclusion 1 Describe three Ruby usages ✓High-level interface ✓Glue ✓Embed ✓ Three Ruby usages Powered by Rabbit 2.1.4
  52. 52. Conclusion 2 High-level interface ✓ Target: Pure Rubyists ✓ Provides lower layer feature to higher layer w/ usable interface ✓Ruby's flexibility is useful ✓ Three Ruby usages Powered by Rabbit 2.1.4
  53. 53. Conclusion 3 Glue Target: Rubyists who can write C/C++ ✓ Why: Reuse ✓ existing feature ✓To be fast, do the process in C ✓ Three Ruby usages Powered by Rabbit 2.1.4
  54. 54. Conclusion 4 Embed Target: Rubyists who also write C/C++ ✓ Why: Avoid bother programming by Ruby ✓ ✓ Three Ruby usages Powered by Rabbit 2.1.4
  55. 55. Conclusion 5 Embed Is it reasonable ✓ for your case? ✓You need many works ✓ Very powerful if your case is reasonable ✓ Three Ruby usages Powered by Rabbit 2.1.4
  56. 56. Announcement ClearCode Inc. ✓A silver sponsor ✓Is recruiting ✓Will do readable code workshop ✓ The next Groonga conference ✓It's held at 11/29 ✓ Three Ruby usages Powered by Rabbit 2.1.4

×