Erlang
Tightly coupled Erlang

Yatce: Yet Another TC-Erlang binding

     4th Tokyo Erlang Workshop
        in Aoyama, 201...
@kuenishi

http://www.google.com/profiles/kuenishi

           : an employee

Distributed System Developer, C/C++



   OC...
Why YATCE?
Erlang
I/O
mnesia
dets
TokyoCabinet
TokyoCabinet   Erlang


    Erlang
Erlang   C
C node
C node
     A process behaves like an Erlang node,

     BERT/TCP connected,

     Failure doesn’t involve the other.


ma...
Port
Driver
Port Driver
==Popen3, just an IPC,

BERT/stdin/stdout connected,

tolerant for Child process’ failure.

           fork/ex...
tcerl

Pioneer in TC-Erlang!

Port driver,

“I couldn’t make it work”, - matthew,

seems not being maintained.
Linkedin
 Driver
Linked-in Driver
in-process erlang-port,

  % acts like an Erlang process

  % Message can be sent/recv’d,

less failure-s...
message passing
                                       to port

                            Process

             Process
...
Port!{self(), {command, Data}}
Process
                   or port_command/2


                   port


                  ...
everything wrapped

OS
driver_alloc, driver_free

driver_enq, driver_deq

driver_monitor_process

erl_driver_select

erl_d...
Toke

Most famous in TC-Erlang,

Linked-in driver,

Simple: just what matthew/LShift wants,

to be used in RabbitMQ?.
Yatce

Made in Japan by me,

Linked-in driver (0.10)

port_control instead of port_command

interface based on tokyocabine...
Yatce 0.10
                                                                                                               ...
one more
problem...
port
locking
Python   GIL


Ruby   GVL


       driver
Port!{self(), {command, Data}}
Process
                   or port_command/2


                   port


                  ...
ERTS
                 Worker
Worker


          Worker




         tcadb
TC
driver        port   port level locking
  TC     DB      port
                                          …
R13B03
Native Interface
     Functions

Since R13B03, Beta,

Built-in code runs inside ERTS,

easy way to write Erlang module in ...
it’s just a function pointer
             calling inside ERTS


                         nif_func_in_c(...){


call_nif()-...
ERTS




Thread   Thread   Thread




         tcadb
Yatce 0.20
                                                                                                               ...
NIF


                R13B04
           erlang-patches        erlang-questions   GitHub   erlang/otp
          NIF    Erla...
Choose what fits to
        you!
Slow   C node
                          Safe

       Port driver

       Linked-in driver...
Any Questions?
          or
come on to the party
     and ask me!
References
ERTS: Interoperability tutorial guide
      http://www.erlang.org/doc/tutorial/overview.html

BERT/BERT-RPC
   ...
Tew4 Yatce presentation
Tew4 Yatce presentation
Tew4 Yatce presentation
Upcoming SlideShare
Loading in...5
×

Tew4 Yatce presentation

2,431

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,431
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide









































  • Transcript of "Tew4 Yatce presentation"

    1. 1. Erlang Tightly coupled Erlang Yatce: Yet Another TC-Erlang binding 4th Tokyo Erlang Workshop in Aoyama, 2010/2/26 @kuenishi http://creativecommons.org/licenses/by-sa/2.1/jp/
    2. 2. @kuenishi http://www.google.com/profiles/kuenishi : an employee Distributed System Developer, C/C++ OCaml …
    3. 3. Why YATCE?
    4. 4. Erlang
    5. 5. I/O
    6. 6. mnesia dets
    7. 7. TokyoCabinet
    8. 8. TokyoCabinet Erlang Erlang
    9. 9. Erlang C
    10. 10. C node
    11. 11. C node A process behaves like an Erlang node, BERT/TCP connected, Failure doesn’t involve the other. main(){ ... TCP Erlang C node ... node }
    12. 12. Port Driver
    13. 13. Port Driver ==Popen3, just an IPC, BERT/stdin/stdout connected, tolerant for Child process’ failure. fork/exec stdin Erlang any CUI process stdout
    14. 14. tcerl Pioneer in TC-Erlang! Port driver, “I couldn’t make it work”, - matthew, seems not being maintained.
    15. 15. Linkedin Driver
    16. 16. Linked-in Driver in-process erlang-port, % acts like an Erlang process % Message can be sent/recv’d, less failure-separeted. Erlang process output(..){ ?DEMO. ... }
    17. 17. message passing to port Process Process Process ! or port_command/3 ! or return port socket
    18. 18. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
    19. 19. everything wrapped OS driver_alloc, driver_free driver_enq, driver_deq driver_monitor_process erl_driver_select erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ... s/erl_drv_/p/g driver_set_timer, driver_cancel_timer, driver_read_timer R13B04 API
    20. 20. Toke Most famous in TC-Erlang, Linked-in driver, Simple: just what matthew/LShift wants, to be used in RabbitMQ?.
    21. 21. Yatce Made in Japan by me, Linked-in driver (0.10) port_control instead of port_command interface based on tokyocabinet.idl, http://bitbucket.org/kuenishi/yatce
    22. 22. Yatce 0.10 R13B02 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 3000 2250 average response time (us) 1500 750 0 0 1750 3500 5250 7000 Thruput (qps) ets dets yatce tcerl
    23. 23. one more problem...
    24. 24. port locking
    25. 25. Python GIL Ruby GVL driver
    26. 26. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
    27. 27. ERTS Worker Worker Worker tcadb
    28. 28. TC
    29. 29. driver port port level locking TC DB port …
    30. 30. R13B03
    31. 31. Native Interface Functions Since R13B03, Beta, Built-in code runs inside ERTS, easy way to write Erlang module in C, Nothing separated.
    32. 32. it’s just a function pointer calling inside ERTS nif_func_in_c(...){ call_nif()-> nif_func_in_c(..), hoge, done. return term; } Erlang C
    33. 33. ERTS Thread Thread Thread tcadb
    34. 34. Yatce 0.20 R13B03 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 700 525 Response time (us) 350 175 0 0 5000 10000 15000 20000 Throughput (qps) ets dets yatce
    35. 35. NIF R13B04 erlang-patches erlang-questions GitHub erlang/otp NIF Erlang VM IO linkedin driver I/O pthread driver_select Yatce 0.20 TC TC NIF Yatce Erlang VM
    36. 36. Choose what fits to you! Slow C node Safe Port driver Linked-in driver port_command/3 port_control/3 Fast NIF Difficult
    37. 37. Any Questions? or come on to the party and ask me!
    38. 38. References ERTS: Interoperability tutorial guide http://www.erlang.org/doc/tutorial/overview.html BERT/BERT-RPC http://github.com/blog/531-introducing-bert-and-bert-rpc http://bert-rpc.org/ Erlang external term format http://erlang.org/doc/apps/erts/erl_ext_dist.html NIF http://erlang.org/doc/man/erl_nif.html Toke, vs tcerl http://bit.ly/8nPhm2

    ×