Comet for highly scalable applications

  • 14,354 views
Uploaded on

MIchael Carter and Arthur Lee's talk about Comet (why it's good, how to implement, and some specifics about the Orbited comet server)

MIchael Carter and Arthur Lee's talk about Comet (why it's good, how to implement, and some specifics about the Orbited comet server)

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Do you mind if I pblish this at http://highlyscalable.com/ ?
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
14,354
On Slideshare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
0
Comments
1
Likes
34

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Comet for Highly- Scalable Applications Michael Carter and Arthur Lee diagrams by Jacob Rus 24 October 2007 The Ajax Experience Boston
  • 2. What is Comet? “Comet” describes a model of user interaction Pushing real-time updates to web browsers Renkoo Gmail Chat Meebo
  • 3. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 4. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 5. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 6. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 7. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 8. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  • 9. Polling Overview Old and busted Pros: • Easy to implement • Client side • Server side Cons: • Wasted requests • Additional latency
  • 10. Enter Long-Polling The new hotness Pros: • Latency—gone! • Client side still easy Cons: • Difficult to scale server-side • Conceptually more complicated
  • 11. Vertical Scaling: How do you scale comet on a single server? photo by Michael (mx5tx)
  • 12. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  • 13. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  • 14. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  • 15. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  • 16. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  • 17. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Compared to polling: Latency reduced for very light load Avg. latency depends on the order of events As much as 7× the CPU for moderate load CPU-bound at moderate load
  • 18. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  • 19. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  • 20. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  • 21. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Compared to polling: Decreased latency and CPU usage Compared to threaded Comet: Vastly decreased CPU usage Order of events irrelevant Performance degrades gracefully
  • 22. Horizontal Scaling: How do you scale comet across many servers? photo by ulybug
  • 23. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Embed publish-subscribe architecture into comet server Keep application state within comet nodes Allow comet nodes to communicate with each-other
  • 24. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application Node scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  • 25. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “A” sends message Node to “E” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  • 26. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “E” sends response Node to “A” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  • 27. Horizontal Scaling: How do you scale comet across many servers? Right way: Share nothing between nodes • No CPU or I/O overhead of sharing state Application Node • Comet CPU usage Nodes scales linearly and latency remains low Browsers
  • 28. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Application Hash Pre-defined Nodes Function Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  • 29. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “B” sends Application Hash Pre-defined Nodes message Function to “E” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  • 30. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “E” sends Application Hash Pre-defined Nodes response Function to “B” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  • 31. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates
  • 32. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates real-time chat
  • 33. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates stock tickers real-time chat
  • 34. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single comet node, which dispatches events to multiple users in multiple groups then how do we scale up to multiple comet nodes?
  • 35. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single node, which dispatches events to multiple users in multiple groups… A single comet Application Node node serves all users and all Comet Dispatcher groups Node Browsers
  • 36. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Either we arrange Application Node nodes by group: All of a group’s Comet Dispatcher Nodes events are sent by one node Browsers
  • 37. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Or we arrange Application Node nodes by user: A user receives Comet Dispatcher Nodes all events from the same node Browsers
  • 38. Scaling Pub/Sub How do you scale publish/subscribe horizontally? But how, specifically? Split the architecture into layers Do not mix pub/sub layer App Publish/Subscribe Cloud with comet dispatcher Comet Dispatcher Nodes Treat pub/sub layer as a black box Browsers
  • 39. Pub/Sub Options Publish/subscribe already has solutions Don’t reinvent the wheel: Many well-understood, scalable publish/subscribe solutions exist • Internet Relay Chat: Nearly 20 years old. Scales well for large groups of users. • XMPP (Jabber): Flexible protocol. Does presence well, in addition to messaging. • Java Message Service: It’s enterprise, baby.
  • 40. Pub/Sub with Orbited Orbited can be easily added to a pub/sub stack Scaling through modularity: Its simplicity makes Orbited easy to add to any web app architecture Orbited handles Orbited comet dispatch Pub/Sub Stack Internet IRC or Jabber Relay IRC Bridge Chat handles publish/ Orbited subscribe Application / App servers do Web Server Browser everything else
  • 41. Conclusion: • Polling latency unacceptable • Comet solves this problem • Impossible to scale Comet with threaded design • Event-based architecture is necessary • Horizontal scaling requires “share nothing” design • Publish/subscribe is a difficult problem • Scaling pub/sub is easiest with layered architecture
  • 42. Michael Carter CarterMichael@gmail.com http://www.orbited.org/ 24 September 2007 AJAXWorld