Comet for highly scalable applications

14,784
-1

Published on

MIchael Carter and Arthur Lee's talk about Comet (why it's good, how to implement, and some specifics about the Orbited comet server)

Published in: Technology
1 Comment
35 Likes
Statistics
Notes
  • Do you mind if I pblish this at http://highlyscalable.com/ ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
14,784
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
1
Likes
35
Embeds 0
No embeds

No notes for slide

Comet for highly scalable applications

  1. 1. Comet for Highly- Scalable Applications Michael Carter and Arthur Lee diagrams by Jacob Rus 24 October 2007 The Ajax Experience Boston
  2. 2. What is Comet? “Comet” describes a model of user interaction Pushing real-time updates to web browsers Renkoo Gmail Chat Meebo
  3. 3. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  4. 4. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  5. 5. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  6. 6. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  7. 7. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  8. 8. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  9. 9. Polling Overview Old and busted Pros: • Easy to implement • Client side • Server side Cons: • Wasted requests • Additional latency
  10. 10. Enter Long-Polling The new hotness Pros: • Latency—gone! • Client side still easy Cons: • Difficult to scale server-side • Conceptually more complicated
  11. 11. Vertical Scaling: How do you scale comet on a single server? photo by Michael (mx5tx)
  12. 12. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  13. 13. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  14. 14. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  15. 15. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  16. 16. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  17. 17. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Compared to polling: Latency reduced for very light load Avg. latency depends on the order of events As much as 7× the CPU for moderate load CPU-bound at moderate load
  18. 18. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  19. 19. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  20. 20. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  21. 21. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Compared to polling: Decreased latency and CPU usage Compared to threaded Comet: Vastly decreased CPU usage Order of events irrelevant Performance degrades gracefully
  22. 22. Horizontal Scaling: How do you scale comet across many servers? photo by ulybug
  23. 23. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Embed publish-subscribe architecture into comet server Keep application state within comet nodes Allow comet nodes to communicate with each-other
  24. 24. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application Node scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  25. 25. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “A” sends message Node to “E” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  26. 26. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “E” sends response Node to “A” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  27. 27. Horizontal Scaling: How do you scale comet across many servers? Right way: Share nothing between nodes • No CPU or I/O overhead of sharing state Application Node • Comet CPU usage Nodes scales linearly and latency remains low Browsers
  28. 28. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Application Hash Pre-defined Nodes Function Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  29. 29. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “B” sends Application Hash Pre-defined Nodes message Function to “E” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  30. 30. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “E” sends Application Hash Pre-defined Nodes response Function to “B” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  31. 31. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates
  32. 32. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates real-time chat
  33. 33. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates stock tickers real-time chat
  34. 34. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single comet node, which dispatches events to multiple users in multiple groups then how do we scale up to multiple comet nodes?
  35. 35. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single node, which dispatches events to multiple users in multiple groups… A single comet Application Node node serves all users and all Comet Dispatcher groups Node Browsers
  36. 36. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Either we arrange Application Node nodes by group: All of a group’s Comet Dispatcher Nodes events are sent by one node Browsers
  37. 37. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Or we arrange Application Node nodes by user: A user receives Comet Dispatcher Nodes all events from the same node Browsers
  38. 38. Scaling Pub/Sub How do you scale publish/subscribe horizontally? But how, specifically? Split the architecture into layers Do not mix pub/sub layer App Publish/Subscribe Cloud with comet dispatcher Comet Dispatcher Nodes Treat pub/sub layer as a black box Browsers
  39. 39. Pub/Sub Options Publish/subscribe already has solutions Don’t reinvent the wheel: Many well-understood, scalable publish/subscribe solutions exist • Internet Relay Chat: Nearly 20 years old. Scales well for large groups of users. • XMPP (Jabber): Flexible protocol. Does presence well, in addition to messaging. • Java Message Service: It’s enterprise, baby.
  40. 40. Pub/Sub with Orbited Orbited can be easily added to a pub/sub stack Scaling through modularity: Its simplicity makes Orbited easy to add to any web app architecture Orbited handles Orbited comet dispatch Pub/Sub Stack Internet IRC or Jabber Relay IRC Bridge Chat handles publish/ Orbited subscribe Application / App servers do Web Server Browser everything else
  41. 41. Conclusion: • Polling latency unacceptable • Comet solves this problem • Impossible to scale Comet with threaded design • Event-based architecture is necessary • Horizontal scaling requires “share nothing” design • Publish/subscribe is a difficult problem • Scaling pub/sub is easiest with layered architecture
  42. 42. Michael Carter CarterMichael@gmail.com http://www.orbited.org/ 24 September 2007 AJAXWorld

×