• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Tew4 Yatce presentation
 

Tew4 Yatce presentation

on

  • 2,979 views

 

Statistics

Views

Total Views
2,979
Views on SlideShare
2,539
Embed Views
440

Actions

Likes
0
Downloads
0
Comments
0

19 Embeds 440

http://kuenishi.blogspot.com 170
http://d.hatena.ne.jp 102
http://kuenishi.blogspot.jp 97
http://kuenishi.hatenadiary.jp 32
http://www.slideshare.net 7
http://kuenishi.blogspot.de 7
http://kuenishi.blogspot.co.uk 5
http://kuenishi.blogspot.in 4
http://webcache.googleusercontent.com 3
http://flavors.me 3
http://kuenishi.blogspot.se 2
http://kuenishi.blogspot.ch 1
http://www.linkedin.com 1
http://kuenishi.blogspot.ca 1
http://kuenishi.blogspot.pt 1
http://kuenishi.blogspot.ie 1
http://kuenishi.blogspot.hk 1
http://kuenishi.blogspot.com.es 1
http://kuenishi.blogspot.it 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Tew4 Yatce presentation Tew4 Yatce presentation Presentation Transcript

  • Erlang Tightly coupled Erlang Yatce: Yet Another TC-Erlang binding 4th Tokyo Erlang Workshop in Aoyama, 2010/2/26 @kuenishi http://creativecommons.org/licenses/by-sa/2.1/jp/
  • @kuenishi http://www.google.com/profiles/kuenishi : an employee Distributed System Developer, C/C++ OCaml …
  • Why YATCE?
  • Erlang
  • I/O
  • mnesia dets
  • TokyoCabinet
  • TokyoCabinet Erlang Erlang
  • Erlang C
  • C node
  • C node A process behaves like an Erlang node, BERT/TCP connected, Failure doesn’t involve the other. main(){ ... TCP Erlang C node ... node }
  • Port Driver
  • Port Driver ==Popen3, just an IPC, BERT/stdin/stdout connected, tolerant for Child process’ failure. fork/exec stdin Erlang any CUI process stdout
  • tcerl Pioneer in TC-Erlang! Port driver, “I couldn’t make it work”, - matthew, seems not being maintained.
  • Linkedin Driver
  • Linked-in Driver in-process erlang-port, % acts like an Erlang process % Message can be sent/recv’d, less failure-separeted. Erlang process output(..){ ?DEMO. ... }
  • message passing to port Process Process Process ! or port_command/3 ! or return port socket
  • Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • everything wrapped OS driver_alloc, driver_free driver_enq, driver_deq driver_monitor_process erl_driver_select erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ... s/erl_drv_/p/g driver_set_timer, driver_cancel_timer, driver_read_timer R13B04 API
  • Toke Most famous in TC-Erlang, Linked-in driver, Simple: just what matthew/LShift wants, to be used in RabbitMQ?.
  • Yatce Made in Japan by me, Linked-in driver (0.10) port_control instead of port_command interface based on tokyocabinet.idl, http://bitbucket.org/kuenishi/yatce
  • Yatce 0.10 R13B02 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 3000 2250 average response time (us) 1500 750 0 0 1750 3500 5250 7000 Thruput (qps) ets dets yatce tcerl
  • one more problem...
  • port locking
  • Python GIL Ruby GVL driver
  • Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • ERTS Worker Worker Worker tcadb
  • TC
  • driver port port level locking TC DB port …
  • R13B03
  • Native Interface Functions Since R13B03, Beta, Built-in code runs inside ERTS, easy way to write Erlang module in C, Nothing separated.
  • it’s just a function pointer calling inside ERTS nif_func_in_c(...){ call_nif()-> nif_func_in_c(..), hoge, done. return term; } Erlang C
  • ERTS Thread Thread Thread tcadb
  • Yatce 0.20 R13B03 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 700 525 Response time (us) 350 175 0 0 5000 10000 15000 20000 Throughput (qps) ets dets yatce
  • NIF R13B04 erlang-patches erlang-questions GitHub erlang/otp NIF Erlang VM IO linkedin driver I/O pthread driver_select Yatce 0.20 TC TC NIF Yatce Erlang VM
  • Choose what fits to you! Slow C node Safe Port driver Linked-in driver port_command/3 port_control/3 Fast NIF Difficult
  • Any Questions? or come on to the party and ask me!
  • References ERTS: Interoperability tutorial guide http://www.erlang.org/doc/tutorial/overview.html BERT/BERT-RPC http://github.com/blog/531-introducing-bert-and-bert-rpc http://bert-rpc.org/ Erlang external term format http://erlang.org/doc/apps/erts/erl_ext_dist.html NIF http://erlang.org/doc/man/erl_nif.html Toke, vs tcerl http://bit.ly/8nPhm2