Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Published in: Technology, News & Politics
  • Be the first to comment


  1. 3. Erlang + EDTK + BDB: Disrupting the Web Margo Seltzer Architect
  2. 4. What kind of talk is this? <ul><li>Evangelism </li></ul><ul><li>FUD </li></ul><ul><li>Fun </li></ul>
  3. 5. Erlang + EDTK + BDB: Huh? <ul><li>General purpose programming language </li></ul><ul><li>Runtime system </li></ul><ul><li>Developed by Ericsson </li></ul><ul><li>Open source </li></ul><ul><li>Key feature: designed for ease of developing highly-reliable, highly-concurrent systems </li></ul><ul><li>Enables highly robust applications: </li></ul><ul><ul><li>The AXD 301 includes 2 million lines of Erlang </li></ul></ul><ul><ul><li>Reliability: NINE 9’s </li></ul></ul>
  4. 6. Erlang + EDTK + BDB: Huh? <ul><li>Erlang Driver Toolkit </li></ul><ul><li>Declarative APIs to external library </li></ul><ul><ul><li>Automatic wrapper generation </li></ul></ul><ul><li>Tracks library resources and cleans up </li></ul><ul><li>Enhanced for BDB support: </li></ul><ul><ul><li>Supports Erlang’s “crash early” error-handling model (supervisors) </li></ul></ul><ul><ul><li>Meshes Erlang-style concurrency with BDB concurrency (private thread pools) </li></ul></ul><ul><ul><li>Adds significant convenience layers for configration and replication </li></ul></ul><ul><ul><li>Minimal overhead </li></ul></ul>
  5. 7. Erlang + EDTK + BDB : Huh? <ul><li>Berkeley Database </li></ul><ul><li>Database functionality … </li></ul><ul><ul><li>Transactions </li></ul></ul><ul><ul><li>Recovery </li></ul></ul><ul><ul><li>Replication </li></ul></ul><ul><li>… in a different package </li></ul><ul><ul><li>Library linked directly into an application </li></ul></ul><ul><ul><li>Programmatic APIs </li></ul></ul><ul><ul><li>Schemaless: key/data pairs </li></ul></ul>
  6. 8. Disruptive Technology <ul><li>“ a technological innovation, product, or service that eventually overturns the existing dominant technology or status quo product in the market.” -- Wikipedia </li></ul>
  7. 9. Outline <ul><li>Interpreting the Title </li></ul><ul><li>Disecting an Internet Service </li></ul><ul><li>Erlang, it’s philosophy and runtime </li></ul><ul><li>Putting it all together </li></ul><ul><li>Q&A </li></ul>
  8. 10. An Internet Service Clients Load Balancer .NET Java The Internet App Servers Database Servers
  9. 11. An Internet Service Clients Load Balancer .NET Java CGI PHP LISP Perl The Internet Database Servers
  10. 12. An Internet Service Clients Load Balancer BDB CGI Perl Application Code The Internet Servers
  11. 13. The Software Architecture <ul><li>In any case, your software is a mess </li></ul>Gazillion Active Ports Fraction of a gazillion threads A relatively small number of disks
  12. 14. Outline <ul><li>Interpreting the Title </li></ul><ul><li>Disecting an Internet Service </li></ul><ul><li>Erlang, it’s philosophy and runtime </li></ul><ul><li>Putting it all together </li></ul><ul><li>Q&A </li></ul>
  13. 15. The Erlang Approach <ul><li>Don’t fight the problem </li></ul><ul><li>Don’t fight the medium (the network) </li></ul><ul><li>Don’t fight the medium (distributed software) </li></ul>
  14. 16. Don’t Fight the Problem <ul><li>1:1 concurrency with the problem/solution domain </li></ul><ul><li>Explicit lightweight stateful conversations (addressable processes) </li></ul><ul><li>Arbitrarily rich messages </li></ul>
  15. 17. Don’t Fight the the Network <ul><li>Asynchronous send (location-agnostic send and pray) </li></ul><ul><li>Ordered inbox per process </li></ul><ul><li>Blocking receive with timeout (message-selection via pattern-matching) </li></ul>
  16. 18. Don’t Fight Distributed Software <ul><li>No shared memory: no mutexes, no mutation </li></ul><ul><ul><li>True loose-coupling: processes are free to migrate </li></ul></ul><ul><ul><li>Safe to kill any process any time </li></ul></ul><ul><li>Recovery-oriented computing (before it was fashionable) </li></ul><ul><ul><li>Let it crash </li></ul></ul><ul><ul><li>Propagate exceptions </li></ul></ul><ul><ul><li>Know how to recover </li></ul></ul>
  17. 19. Erlang Tricks: Processes <ul><li>Belong to the language (runtime), not OS </li></ul><ul><li>Very lightweight </li></ul><ul><li>Immutable data </li></ul><ul><li>Asynchronous message passing </li></ul><ul><li>Upgrade application on live, running system </li></ul><ul><li>Implication: </li></ul><ul><ul><li>No big deal if a process dies </li></ul></ul>
  18. 20. Erlang Tricks: Links <ul><li>Links connect processes </li></ul><ul><li>On process crash, all linked processes get message. </li></ul><ul><li>Notified processes can clean up, takeover, do whatever is necessary. </li></ul><ul><li>Easy to create supervisors </li></ul><ul><li>No recovery: just crash, cleanup and restart </li></ul>
  19. 21. Erlang Tricks: Concurrency <ul><li>Use concurrency to structure the application </li></ul><ul><ul><li>“ My first message is that concurrency is best regarded as a program structuring principle” </li></ul></ul><ul><ul><ul><ul><ul><li>Tony Hoare, 2001 </li></ul></ul></ul></ul></ul><ul><li>Concurrency-oriented programming </li></ul><ul><ul><li>Share nothing </li></ul></ul><ul><ul><li>Pure message passing </li></ul></ul><ul><ul><li>Let it crash </li></ul></ul>
  20. 22. Erlang Nuts and Bolts <ul><li>Hello World </li></ul><ul><ul><li>-module(hello). </li></ul></ul><ul><ul><li>-export(hello_world/0). </li></ul></ul><ul><ul><li>hello_world() -> </li></ul></ul><ul><ul><li>io:format(“Hello world.~n”, []). </li></ul></ul><ul><li>Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false] </li></ul><ul><li>Eshell V5.5.5 (abort with ^G) </li></ul><ul><li>1> c(hello). </li></ul><ul><li>{ok,hello} </li></ul><ul><li>2> hello:hello_world(). </li></ul><ul><li>Hello world. </li></ul><ul><li>ok </li></ul><ul><li>3> </li></ul>
  21. 23. Spawning Processes <ul><li>Pid = spawn(module, function, args). </li></ul><ul><li>For example: </li></ul><ul><ul><ul><li>Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false] </li></ul></ul></ul><ul><ul><ul><li>Eshell V5.5.5 (abort with ^G) </li></ul></ul></ul><ul><ul><ul><li>1> spawn(hello, hello_world, []). </li></ul></ul></ul><ul><ul><ul><li><0.32.0>Hello world. </li></ul></ul></ul><ul><ul><ul><li>2> </li></ul></ul></ul>
  22. 24. Message send and receive <ul><li>Send: Pid ! message. </li></ul><ul><li>Receive: </li></ul><ul><ul><li>receive </li></ul></ul><ul><ul><li>Pattern1 [when Guard1] -> </li></ul></ul><ul><ul><li>Expression1; </li></ul></ul><ul><ul><li>Pattern2 [when Guard2] -> </li></ul></ul><ul><ul><li>Expression2; </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>end </li></ul></ul>
  23. 25. Send/Receive Example <ul><li>-module(food). </li></ul><ul><li>-export([dinner/0]). </li></ul><ul><li>dinner() -> </li></ul><ul><li>receive </li></ul><ul><li>sunday -> </li></ul><ul><li>io:format(&quot;Sunday is fish.~n&quot;, []), </li></ul><ul><li>dinner(); </li></ul><ul><li>monday -> </li></ul><ul><li>io:format(&quot;Monday is chicken.~n&quot;, []), </li></ul><ul><li>dinner(); </li></ul><ul><li>tuesday -> </li></ul><ul><li>io:format(&quot;Tuesday is beef.~n&quot;, []), </li></ul><ul><li>dinner(); </li></ul><ul><li>Other -> </li></ul><ul><li>io:format(&quot;Time to go home.~n&quot;, []) </li></ul><ul><li>end. </li></ul>
  24. 26. Send/Receive Execution <ul><ul><li>Eshell V5.5.5 (abort with ^G) </li></ul></ul><ul><ul><li>1> Pid = spawn(food, dinner, []). </li></ul></ul><ul><ul><li><0.32.0> </li></ul></ul><ul><ul><li>2> Pid ! tuesday. </li></ul></ul><ul><ul><li>tuesdayTuesday is beef. </li></ul></ul><ul><ul><li>3> Pid ! sunday. </li></ul></ul><ul><ul><li>sundaySunday is fish. </li></ul></ul><ul><ul><li>4> Pid ! wednesday. </li></ul></ul><ul><ul><li>wednesdayTime to go home. </li></ul></ul><ul><ul><li>5> Pid ! monday. </li></ul></ul><ul><ul><li>monday </li></ul></ul><ul><ul><li>6> </li></ul></ul>
  25. 27. BDB in Erlang <ul><li>Erlang: </li></ul><ul><ul><li>Is Functional (not procedural) </li></ul></ul><ul><ul><li>Communicates via messages </li></ul></ul><ul><ul><li>Communicates asynchronously </li></ul></ul><ul><li>BDB: </li></ul><ul><ul><li>Is Procedural </li></ul></ul><ul><ul><li>Communicates via shared memory </li></ul></ul><ul><ul><li>Blocks on locks/IO/etc </li></ul></ul>
  26. 28. EDTK fixes mismatch <ul><li>Erlang interfaces to outside world via ports. </li></ul><ul><li>EDTK automatically generates code that wraps library API, making library look like Erlang process(es). </li></ul><ul><li>Provides framework to: </li></ul><ul><ul><li>Clean up BDB resources after a crash </li></ul></ul><ul><ul><li>Manage threadpools to deal with BDB blocking </li></ul></ul><ul><ul><li>Manage administrative processes </li></ul></ul><ul><ul><li>Manage replication groups </li></ul></ul>
  27. 29. And the code looks like BDB <ul><ul><li>DB = ?BDB:db_create(Port, []). </li></ul></ul><ul><ul><li>?BDB:db_open(Port, DB, void, “database”, “”, bdb_DB_BTREE, [bdb_DB_CREATE, 8#644). </li></ul></ul><ul><ul><li>?BDB:db_put(Port, DB, void “foo”, “foodata”, []). </li></ul></ul><ul><ul><li>{Key, Data} = ?BDB:db_get(Port, </li></ul></ul><ul><ul><li>DB, void, “foo”, <<>>, []). </li></ul></ul><ul><ul><li>?BDB:db_close(Port, DB, []). </li></ul></ul>
  28. 30. Outline <ul><li>Interpreting the Title </li></ul><ul><li>Disecting an Internet Service </li></ul><ul><li>Erlang, it’s philosophy and runtime </li></ul><ul><li>Putting it all together </li></ul><ul><li>Q&A </li></ul>
  29. 31. An Internet Service Clients Load Balancer Servers The Internet
  30. 32. How does it Perform? From:
  31. 33. Acknowledgements <ul><li>Joe Armstrong (Erlang) </li></ul><ul><li>Scott Lystig Fritchie (EDTK) </li></ul><ul><li>Chris Newcombe (EDTK extensions for BDB) </li></ul>
  32. 34. A Q &