Massively Scalable, Massively Fault Tolerant

1,047
-1

Published on

This talk covers the basics of Erlang/OTP, a framework for highly scalable, highly fault tolerant applications.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,047
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Massively Scalable, Massively Fault Tolerant

    1. 1. MASSIVELY SCALEABLEMASSIVELY FAULT-TOLERANT Trotter Cashion Philly Lambda, 2011
    2. 2. ErlangBUT FIRST... AND OTP IN ACTION Buy this Book!http://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
    3. 3. ErlangNO SERIOUSLY... IN ACTION AND OTP http://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
    4. 4. ErlangAVAILABLE IN PDF... AND OTP IN ACTIONhttp://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
    5. 5. AND NOW FOR ME...
    6. 6. CO-FOUNDER
    7. 7. @CASHION http://www.charlesheflin.com/wp-content/uploads/2010/06/fail-whale.jpg
    8. 8. github.com/trotter
    9. 9. CHLOE• Realtime Web Server• Proxies WebSocket (and fallback) connections to your app• http://github.com/mashion/chloe• Written in Erlang/OTP (plus a lot of JavaScript)
    10. 10. BAI PHILLY! Leaving in July... http://www.flickr.com/photos/dennis/156048151/
    11. 11. HAI CALIFORNIA! See you in August... http://www.flickr.com/photos/salim/402618628/
    12. 12. AN ERLANG PRIMER Yoinked from the Erlang Movie. Go Watch It!!!!
    13. 13. THE BASICS• Functional• Single Assignment Variables• Pattern Matching is Big Here• Everything’s a Process• One Module Per File
    14. 14. FUNCTIONALsay_hello() ->  io:fwrite("Ohai!n").
    15. 15. ANONYMOUS FUNCTIONSmake_hello() ->  fun () -> io:fwrite("Ohai!n")end.Hello = make_hello().Hello().
    16. 16. SINGLE ASSIGNMENT VARS1> Hello = 3.32> Hello = 2.** exception error: no match of right hand side value 23> f().ok4> Hello = 2.2
    17. 17. OH NOES! WHAT WENT WRONG?destructuring_bind() ->  [{Name, Title}, _] = [{"Trotter", "Speaker"},                        {"Aaron", "Organizer"}],  io:fwrite("~pn", [Name]), %% => Trotter  io:fwrite("~pn", [Title]). %% => Speaker
    18. 18. PATTERN MATCHING Erlang Rubymatch_me(1) -> def match_me(val)  loneliest;   case valmatch_me(2) ->   when 1  next_to_loneliest;     :loneliestmatch_me(friend) ->   when 2  all_ok;     :next_to_loneliestmatch_me(_) ->   when :friend  what_is_this.     :all_ok   else     :what_is_this   end end
    19. 19. MAKING A PROCESS process.erlconcurrent_stuffs() ->  spawn(fun () -> timer:sleep(1000),                  io:fwrite("So slown") end),  spawn(fun () -> io:fwrite("Super fastn") end). erlang shell5> process:concurrent_stuffs().Super fast<0.39.0>So slow
    20. 20. RECEIVE LOOPSfood_loop() ->  receive    pizza -> io:fwrite("lets eatn"),             food_loop();    death -> io:fwrite("all done heren");    _ -> io:fwrite("Nothing to don"),             food_loop()  end.
    21. 21. SENDING MESSAGESmake_process() ->  spawn(?MODULE, food_loop, []).pizza_death() ->  Proc = make_process(),  Proc ! pizza,  Proc ! hi,  Proc ! death,  %% Goes into the void  Proc ! huh.
    22. 22. HOW THAT WORKS process 1pizza_death
    23. 23. HOW THAT WORKS process 1 spawn/3pizza_death
    24. 24. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop
    25. 25. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox
    26. 26. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox
    27. 27. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza
    28. 28. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi
    29. 29. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death
    30. 30. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    31. 31. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    32. 32. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    33. 33. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza “let’s eatn” hi death huh
    34. 34. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza “let’s eatn” hi death huh
    35. 35. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    36. 36. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi “Nothing to don” death huh
    37. 37. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi “Nothing to don” death huh
    38. 38. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    39. 39. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death “all done heren” huh
    40. 40. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death “all done heren” huh
    41. 41. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    42. 42. HOW THAT WORKS process 1 process 2 spawn/3pizza_death food_loop food_loop’s Mailbox pizza hi death huh
    43. 43. HOW THAT WORKS process 1 spawn/3pizza_death food_loop’s Mailbox pizza hi death huh
    44. 44. HOW THAT WORKS process 1 spawn/3pizza_death pizza hi death huh
    45. 45. HOW THAT WORKS process 1 spawn/3pizza_death pizza hi death
    46. 46. BUT DOES IT SCALE? Hells yea! http://www.flickr.com/photos/24736216@N07/4808606178/
    47. 47. TWO WAYS TO SCALE Name ImplementationHorizontally Buy a lot of machines Vertically Buy a really big machine
    48. 48. ERLANG SCALES WELL ON BOTHWITH NEARLY ZERO CODE CHANGE
    49. 49. BOOM!
    50. 50. VERTICAL SCALING• Your processes will take advantage of all cores available• Yay! No code changes required
    51. 51. HORIZONTAL SCALING• New nodes added to the cluster can be auto-detected• Code must know that a service could live on another node• You’ll need resource discovery
    52. 52. CONNECTING NODES$ erl -name b1> AHost = a@awesome-town.local.a@awesome-town.local2> io:fwrite("Current nodes: ~p~n", [nodes()]).Current nodes: []3> io:fwrite("Ping? ~p~n", [net_adm:ping(AHost)]).Ping? pong4> io:fwrite("Current nodes: ~p~n", [nodes()]).Current nodes: [a@awesome-town.local5> q().
    53. 53. SERVER REFERENCES Type Definition <Pid> globally unique process identifier RegisteredName locally registered process process locally registered on{RegisteredName, Node} another node {global, GlobalName} globally registered process
    54. 54. LISTEN FOR INCOMING MESSAGElisten() ->  register(listener, self()),  listen_loop().listen_loop() ->  receive    {From, pizza} -> From ! "Nom noms", listen_loop();    {From, _} -> From ! "Boo!", listen_loop();    _ -> io:fwrite("Listen is dying!")  end.
    55. 55. SENDING MESSAGE TO NODES$ erl -name b1> AHost = a@awesome-town.local.a@awesome-town.local2> {listener, AHost} ! {self(), pizza}.ok3> receive PizzaResp -> PizzaResp end.
    56. 56. RESOURCE DISCOVERY Method Purpose add_target_resource(Type) specify a resource type you want to useadd_local_resource(Type, Pid) specify a resource type you have fetch_resources(Type) fetch all resources of specified type trade_resources() trade resource information See Chapter 8 of Erlang And OTP In Action
    57. 57. HORIZONTAL CAVEATS• Passing large amounts of data between nodes is inefficient• Too many nodes will impact performance (Keep it < 100)• There are ways around both these problems
    58. 58. OTP BASICSCause managing all the processes by hand is dumb Thanks, Homer
    59. 59. OTPWTFBBQ?• Open Telecom Platform• Abstracts common systems behavior (servers, supervisors, ...)• Easier than pure Erlang!• Gives Erlang its reputation for fault tolerance
    60. 60. BASIC CONSTRUCTSConstruct Purpose Encapsulates a large set of codeApplication meant to run as one unit.Supervisor Starts and restarts workers. Worker Actually does stuff
    61. 61. HTTP://GITHUB.COM/MASHION/CHLOE Chloe (Application) chloe_sup (Supervisor) websocket_sup session_sup session_manager Other Stuffs (Supervisor) (Supervisor) (Worker)
    62. 62. WORKERS• Use the `gen_server` behaviour (for the most part)• Contain your application logic• You define a module with callbacks• OTP runs a server calls back to module when messages arrive
    63. 63. HOW WORKERS WORK client process (your code) gen_server functions gen_server code (part of OTP) gen_server callbacks worker code (your code) worker process
    64. 64. SENDING MESSAGES TO WORKERSsay_hello_sync(Name) ->  gen_server:call(?SERVER, {say_hello, Name}).say_hello_async() ->  gen_server:cast(?SERVER, say_hello). gen_server_examples.erl
    65. 65. EXAMPLE CALLBACKShandle_call({say_hello, Name}, _From, State) ->  Response = string:concat("Hello, ", Name),  {reply, {ok, Response}, State}.handle_cast(say_hello, State) ->  io:fwrite("Hello world!n"),  {noreply, State}. gen_server_examples.erl
    66. 66. WORKER CALLBACKS Callback Purpose init(Args) Initialize the server processhandle_call(Request, From, State) Handle a synchronous call handle_cast(Request, State) Handle an asynchronous cast handle_info(Info, State) For timeouts and random messages terminate(Reason, State) Clean up the process when it diescode_change(OldVsn, State, Extra) Called when upgrading code
    67. 67. WORKER FUNCTIONS Callback Purposestart_link(Module, Args, Options) Start a supervised worker start(Module, Args, Options) Start an unsupervised workercall(ServerRef, Request, Timeout) Synchronously call a worker cast(ServerRef, Request) Asynchronously cast to a worker man gen_server
    68. 68. SUPERVISORS• Use the `supervisor` behaviour• Contain little to no application logic• Know how to start and restart failed workers• Keep your application from ever fully dying
    69. 69. CHILD SPECS{Id, StartFunc, Restart, Shutdown, Type, Modules} Unique name used to identify child process.
    70. 70. CHILD SPECS{session_manager, StartFunc, Restart, Shutdown, Type, Modules} Unique name used to identify child process.
    71. 71. CHILD SPECS{session_manager, StartFunc, Restart, Shutdown, Type, Modules} Tuple {M, F, A} used to start the child.
    72. 72. CHILD SPECS{session_manager, {session_manager, start_link, []}, Restart, Shutdown, Type, Modules} Tuple {M, F, A} used to start the child.
    73. 73. CHILD SPECS{session_manager, {session_manager, start_link, []}, Restart, Shutdown, Type, Modules} Determines when a terminated process should be restarted.
    74. 74. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, Shutdown, Type, Modules} Determines when a terminated process should be restarted.
    75. 75. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, Shutdown, Type, Modules} Determines how to kill child processes.
    76. 76. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, 5000, Type, Modules} Determines how to kill child processes.
    77. 77. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, 5000, Type, Modules} Is this child a worker or a supervisor?
    78. 78. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, 5000, worker, Modules} Is this child a worker or a supervisor?
    79. 79. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, 5000, worker, Modules} Used by code reloading. Make it the same as the other module name.
    80. 80. CHILD SPECS{session_manager, {session_manager, start_link, []}, permanent, 5000, worker, [session_manager]} Used by code reloading. Make it the same as the other module name.
    81. 81. RESTART STRATEGIES Strategy When a child dies.... one_for_one restart it one_for_all restart all children restart it and all children after it rest_for_one in child list restart it (also allows you tosimple_one_for_one dynamically add children)
    82. 82. ONE_FOR_ONE SupervisorChild 1 Child 2 Child 3
    83. 83. ONE_FOR_ONE Supervisor Child 2 Child 3
    84. 84. ONE_FOR_ONE SupervisorChild 1 Child 2 Child 3
    85. 85. ONE_FOR_ALL SupervisorChild 1 Child 2 Child 3
    86. 86. ONE_FOR_ALL Supervisor Child 2 Child 3
    87. 87. ONE_FOR_ALL Supervisor Child 3
    88. 88. ONE_FOR_ALL Supervisor
    89. 89. ONE_FOR_ALL SupervisorChild 1
    90. 90. ONE_FOR_ALL SupervisorChild 1 Child 2
    91. 91. ONE_FOR_ALL SupervisorChild 1 Child 2 Child 3
    92. 92. REST_FOR_ONE SupervisorChild 1 Child 2 Child 3
    93. 93. REST_FOR_ONE SupervisorChild 1 Child 3
    94. 94. REST_FOR_ONE SupervisorChild 1
    95. 95. REST_FOR_ONE SupervisorChild 1 Child 2
    96. 96. REST_FOR_ONE SupervisorChild 1 Child 2 Child 3
    97. 97. SIMPLE_ONE_FOR_ONE Supervisor
    98. 98. SIMPLE_ONE_FOR_ONE SupervisorChild 1
    99. 99. SIMPLE_ONE_FOR_ONE SupervisorChild 1 Child 2
    100. 100. SIMPLE_ONE_FOR_ONE SupervisorChild 1 Child 2 Child 3
    101. 101. SIMPLE_ONE_FOR_ONE Supervisor Child 2 Child 3
    102. 102. SIMPLE_ONE_FOR_ONE SupervisorChild 1 Child 2 Child 3
    103. 103. STARTING A SUPERVISORinit([]) ->  Worker = {gen_server_examples, {gen_server_examples, start_link, []},            permanent, 5000, worker, [gen_server_examples]},  RestartStrategy = {one_for_one, 5, 30},  {ok, {RestartStrategy, [Worker]}}.
    104. 104. SUPERVISOR CALLBACKcallback purpose return data structure telling whatinit(Args) to supervise
    105. 105. SUPERVISOR FUNCTIONS Callback Purpose start_link(Module, Args) Start a supervised supervisorstart_child(SupRef, ChildSpec) Start a child under the supervisor man supervisor
    106. 106. FAULT TOLERANCE• When you hit an error, let the worker die• Supervisor will restart it• Either try what killed it again or move on• Life is grand
    107. 107. LINKS• http://bit.ly/erlang-otp-gist• http://bit.ly/otp-book
    108. 108. THANK YOU!• Questions?• http://bit.ly/erlang-otp-gist• http://bit.ly/otp-book• http://mashion.net
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×