Tew4 Yatce presentation

  • 2,349 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,349
On Slideshare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide









































Transcript

  • 1. Erlang Tightly coupled Erlang Yatce: Yet Another TC-Erlang binding 4th Tokyo Erlang Workshop in Aoyama, 2010/2/26 @kuenishi http://creativecommons.org/licenses/by-sa/2.1/jp/
  • 2. @kuenishi http://www.google.com/profiles/kuenishi : an employee Distributed System Developer, C/C++ OCaml …
  • 3. Why YATCE?
  • 4. Erlang
  • 5. I/O
  • 6. mnesia dets
  • 7. TokyoCabinet
  • 8. TokyoCabinet Erlang Erlang
  • 9. Erlang C
  • 10. C node
  • 11. C node A process behaves like an Erlang node, BERT/TCP connected, Failure doesn’t involve the other. main(){ ... TCP Erlang C node ... node }
  • 12. Port Driver
  • 13. Port Driver ==Popen3, just an IPC, BERT/stdin/stdout connected, tolerant for Child process’ failure. fork/exec stdin Erlang any CUI process stdout
  • 14. tcerl Pioneer in TC-Erlang! Port driver, “I couldn’t make it work”, - matthew, seems not being maintained.
  • 15. Linkedin Driver
  • 16. Linked-in Driver in-process erlang-port, % acts like an Erlang process % Message can be sent/recv’d, less failure-separeted. Erlang process output(..){ ?DEMO. ... }
  • 17. message passing to port Process Process Process ! or port_command/3 ! or return port socket
  • 18. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • 19. everything wrapped OS driver_alloc, driver_free driver_enq, driver_deq driver_monitor_process erl_driver_select erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ... s/erl_drv_/p/g driver_set_timer, driver_cancel_timer, driver_read_timer R13B04 API
  • 20. Toke Most famous in TC-Erlang, Linked-in driver, Simple: just what matthew/LShift wants, to be used in RabbitMQ?.
  • 21. Yatce Made in Japan by me, Linked-in driver (0.10) port_control instead of port_command interface based on tokyocabinet.idl, http://bitbucket.org/kuenishi/yatce
  • 22. Yatce 0.10 R13B02 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 3000 2250 average response time (us) 1500 750 0 0 1750 3500 5250 7000 Thruput (qps) ets dets yatce tcerl
  • 23. one more problem...
  • 24. port locking
  • 25. Python GIL Ruby GVL driver
  • 26. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • 27. ERTS Worker Worker Worker tcadb
  • 28. TC
  • 29. driver port port level locking TC DB port …
  • 30. R13B03
  • 31. Native Interface Functions Since R13B03, Beta, Built-in code runs inside ERTS, easy way to write Erlang module in C, Nothing separated.
  • 32. it’s just a function pointer calling inside ERTS nif_func_in_c(...){ call_nif()-> nif_func_in_c(..), hoge, done. return term; } Erlang C
  • 33. ERTS Thread Thread Thread tcadb
  • 34. Yatce 0.20 R13B03 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 700 525 Response time (us) 350 175 0 0 5000 10000 15000 20000 Throughput (qps) ets dets yatce
  • 35. NIF R13B04 erlang-patches erlang-questions GitHub erlang/otp NIF Erlang VM IO linkedin driver I/O pthread driver_select Yatce 0.20 TC TC NIF Yatce Erlang VM
  • 36. Choose what fits to you! Slow C node Safe Port driver Linked-in driver port_command/3 port_control/3 Fast NIF Difficult
  • 37. Any Questions? or come on to the party and ask me!
  • 38. References ERTS: Interoperability tutorial guide http://www.erlang.org/doc/tutorial/overview.html BERT/BERT-RPC http://github.com/blog/531-introducing-bert-and-bert-rpc http://bert-rpc.org/ Erlang external term format http://erlang.org/doc/apps/erts/erl_ext_dist.html NIF http://erlang.org/doc/man/erl_nif.html Toke, vs tcerl http://bit.ly/8nPhm2