Misultin 0.7.1 Presentation

  • 1,709 views
Uploaded on

Misultin (pronounced mee-sool-téen) is an Erlang library for building fast lightweight HTTP(S) servers, which also supports websockets. …

Misultin (pronounced mee-sool-téen) is an Erlang library for building fast lightweight HTTP(S) servers, which also supports websockets.

https://github.com/ostinelli/misultin

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,709
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
41
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Http Process survives to handle persistent connections\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Misultin [pron.: Mee-sool-téen]An Erlang library for building fast lightweight HTTP(S) servers>-|-|(°> Erlang User Group London - May 16th, 2011
  • 2. Who am I?Technologist and Entrepreneur. I amparticularly interested in Erlang, Python,scalability, research, development andintegration of the multiple technologiesbehind the creation of ubiquitouscomputing. I’m currently co-founderand CTO of WideTag Inc [widetag.com].e: roberto@ostinelli.nett: @ostinelli
  • 3. <1>
  • 4. What are Misultins?Misultin are a culinary specialty of the Lake of Como.They are lake fishes that go by the latin name of Alosafallax lacustris, treated and cooked in a traditionaland specific way.
  • 5. What is Misultin?Misultin (pronounced mee-sool-téen) is an Erlanglibrary for building fast lightweight HTTP(S) servers,which also supports websockets. >-|-|(°>
  • 6. Why another lib?When do programmers start a new lib? ‣ Nothing similar exists ‣ Different Approach / Focus ‣ Total Control / Knowledge ‣ Experiment/Hack/Academic
  • 7. Features v0.7.1‣ Embedded, not application‣ ~ HTTP/1.1‣ Support for GET, POST, HEAD, PUT, DELETE, TRACE and CONNECT‣ SSL‣ Output compression [gzip, deflate]‣ Chunked / Stream / File sending [attachment, visualization]‣ Customizable parameters [max_connections, post_max_size, get_url_max_size, ...]‣ Multiple servers on a single node [custom name / nameless]‣ Websockets [also on SSL]‣ Trapping of client closing a browser in Comet applications‣ With / without Parametrized Modules
  • 8. </1>
  • 9. <2>
  • 10. Code | “Hello World”Simple “Hello World” using parametrized modules.
  • 11. Code | “Hello World”Simple “Hello World” not using parametrized modules.
  • 12. Code | File SendingSimple file sending.
  • 13. Code | REST
  • 14. Code | Websocket1. Start Misultin specifying a Websocket loop.
  • 15. Code | Websocket2. Send the HTML + Js page.
  • 16. Code | Websocket3. Handle Websocket Data on server.
  • 17. Code | WebsocketLive Demo.
  • 18. Code | Server Optionsstart_link(Options) -> ResultOptions = [Option]Option = {ip, list() | tuple()} | {port, integer()} | {name, atom()|false} | {max_connections, integer()} | {post_max_size, integer()} | {get_url_max_size, integer()} | {compress, true|false} | {loop, fun()} | {autoexit, true|false} | {ws_loop, fun()|none} | {ws_autoexit, true|false} | {backlog, integer()} | {acceptors_poolsize, integer()} | {recv_timeout, integer()} | {ssl, SslOptions} SslOptions = [SslOption(§)]
  • 19. </2>
  • 20. <3>
  • 21. Design | Overview proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 Acceptor . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process Http Process Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 22. Design | Crash: Acceptor proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 23. Design | Crash: Acceptor proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 24. Design | Crash: Acceptor proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] EXIT message: . erlang:spawn/1 gen_server respawns . gen_tcp:controlling_process/2 a new acceptor . call to gen_server to:Main gen_server - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 25. Design | Crash: HttpProcess proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 26. Design | Crash: HttpProcess proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count 1. Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 27. Design | Crash: HttpProcess proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count 1. Http Process HTTP Logic erlang:spawn_link/1 2. HttpLoop Application Logic
  • 28. Design | Crash: HttpProcess proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count 3. DOWN message: 1. decrease open connections counter Http Process HTTP Logic erlang:spawn_link/1 2. HttpLoop Application Logic
  • 29. Design | Crash: HttpLoop proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 30. Design | Crash: HttpLoop proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 31. Design | Crash: HttpLoop proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic EXIT message: HttpProcess sends HTTP 500 error erlang:spawn_link/1 message to client HttpLoop Application Logic
  • 32. Design | Crash: gen_server proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 33. Design | Crash: gen_server 1. proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 34. Design | Crash: gen_server 1. 2. proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref - erlang:monitor/2: for gen_server count Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 35. Design | Crash: gen_server 1. 2. proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref terminate(_,_) -> - erlang:monitor/2: for gen_server count exit(HttpPid, kill). 3. Http Process HTTP Logic erlang:spawn_link/1 HttpLoop Application Logic
  • 36. Design | Crash: gen_server 1. 2. proc_lib:spawn_link/3 gen_server Acceptor TCP Acceptor [misultin] . erlang:spawn/1 . gen_tcp:controlling_process/2Main gen_server . call to gen_server to: - http_pid_ref_add/2: add new pid to ref terminate(_,_) -> - erlang:monitor/2: for gen_server count exit(HttpPid, kill). 3. Http Process HTTP Logic erlang:spawn_link/1 4. HttpLoop Application Logic
  • 37. </3>
  • 38. <4>
  • 39. Benchmarks | Foreword‣ No such thing as generic benchmarks: specific case [fast application times, loads of connections, small data]‣ ‘Fast’ is only 1 of the ‘n’ features you want [stability, features, ease of maintenance, low standard deviation, code usability, ...]‣ This is no ‘pissing’ contest. *Not* what I wanted to see:
  • 40. Benchmarks | Test‣ Not only Erlang‣ Chosen libraries are all lightweight & dynamic [compare apple with apples: no Yaws, no Nginx]‣ Periodically close down the TCP connection‣ All tests run on a single CPU‣ No RAM/CPU considerations
  • 41. Benchmarks | Test‣ ‘Dynamic’ tests: no “Hello world”The test basically asks servers to: • check if a GET variable is set • if the variable is not set, reply with an XML stating the error: <http_test><error>no value specified</error></http_test> • if the variable is set, echo it inside an XML <http_test><value>MYVALUE</value></http_test>
  • 42. Benchmarks | Setup‣ Patched HttPerf [for higher file handling]‣ Virtualized Up-to-date Ubuntu 10.04 LTS, 2CPU, 1.5G Ram [tuning of /etc/sysctl.conf and /etc/security/limits.conf]‣ Libraries: • Misultin 0.7.1 (Erlang R14B02) • Mochiweb 1.5.2 (Erlang R14B02) • Cowboy master 420f5ba (Erlang R14B02) • NodeJS 0.4.7 • Tornadoweb 1.2.1 (Python 2.6.5)‣ All the libraries have been run with the standard settings. Erlang was launched with Kernel Polling enabled, and with SMP disabled so that a single CPU was used by all the libraries.
  • 43. Benchmarks | Results
  • 44. Benchmarks | Results
  • 45. Benchmarks | Results
  • 46. Benchmarks | Results
  • 47. </4>
  • 48. <5>
  • 49. Comet | Practical issues HTTP Logic Http Processspawn_link/1 HttpLoop Application Logic
  • 50. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3spawn_link/1 HttpLoop Application Logic
  • 51. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3 Actual TCP sendingspawn_link/1 Data to be Sent HttpLoop Application Logic
  • 52. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3 Actual TCP sending => monitor Socket while in socket_loop?spawn_link/1 Data to be Sent HttpLoop Application Logic
  • 53. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3 Actual TCP sending => monitor Socket while in socket_loop?spawn_link/1 => get data from Socket while in socket_loop? Data to be Sent HttpLoop Application Logic
  • 54. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3 Actual TCP sending => monitor Socket while in socket_loop?spawn_link/1 => get data from Socket while in socket_loop? Data to be Sent CONFLICT: • Catch Client’s Socket closed event HttpLoop • Support HTTP pipelining Application Logic
  • 55. Comet | Practical issues HTTP Logic Http Process 1. Building Req: {active, once} | gen_tcp:recv/3 2. Spawn HttpLoop, enter socket_loop/3 Actual TCP sending => monitor Socket while in socket_loop?spawn_link/1 => get data from Socket while in socket_loop? Data to be Sent CONFLICT: • Catch Client’s Socket closed event HttpLoop • Support HTTP pipelining SOLUTION: Application Logic handle_http(Req) -> Req:options([{comet, true}]), ...
  • 56. </5>
  • 57. Thank youe: roberto@ostinelli.nett: @ostinelli >-|-|(°>