Comet for Highly-
Scalable Applications
  Michael Carter and Arthur Lee
         diagrams by Jacob Rus



           24 Oc...
What is Comet?
 “Comet” describes a model of user interaction

 Pushing real-time updates to web browsers
                ...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Old Method: Polling
Otherwise known as “hammering the server”


                               App


       (Event Dispatc...
Polling Overview
                   Old and busted

Pros:
 •   Easy to implement
     •   Client side
     •   Server side...
Enter Long-Polling
                 The new hotness

Pros:
•   Latency—gone!
•   Client side still easy
Cons:
•   Difficult...
Vertical Scaling:
How do you scale comet on a single server?




                          photo by Michael (mx5tx)
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
       ...
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
       ...
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
       ...
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
       ...
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
       ...
Vertical Scaling:
 How do you scale comet on a single server?

Wrong way: Use a threaded server like Apache
or IIS
Compare...
Vertical Scaling:
 How do you scale comet on a single server?

Right way: Use an event-based architecture
 •   Supports as...
Vertical Scaling:
 How do you scale comet on a single server?

Right way: Use an event-based architecture
 •   Supports as...
Vertical Scaling:
 How do you scale comet on a single server?

Right way: Use an event-based architecture
 •   Supports as...
Vertical Scaling:
 How do you scale comet on a single server?

Right way: Use an event-based architecture
Compared to poll...
Horizontal Scaling:
How do you scale comet across many servers?




                              photo by ulybug
Horizontal Scaling:
How do you scale comet across many servers?

Wrong way: Build the whole software stack into
the comet ...
Horizontal Scaling:
How do you scale comet across many servers?

Wrong way: Build the whole software stack into
the comet ...
Horizontal Scaling:
How do you scale comet across many servers?

Wrong way: Build the whole software stack into
the comet ...
Horizontal Scaling:
How do you scale comet across many servers?

Wrong way: Build the whole software stack into
the comet ...
Horizontal Scaling:
How do you scale comet across many servers?

Right way: Share nothing between nodes
 •   No CPU or I/O...
Horizontal Scaling:
How do you scale comet across many servers?

Right way: Use Orbited «wink»
   Distributed
   hash tabl...
Horizontal Scaling:
How do you scale comet across many servers?

Right way: Use Orbited «wink»
   Distributed
   hash tabl...
Horizontal Scaling:
How do you scale comet across many servers?

Right way: Use Orbited «wink»
   Distributed
   hash tabl...
Publish / Subscribe
Pushing published events to groups of browsers

What’s it good for? Applications for which
multiple us...
Publish / Subscribe
Pushing published events to groups of browsers

What’s it good for? Applications for which
multiple us...
Publish / Subscribe
Pushing published events to groups of browsers

What’s it good for? Applications for which
multiple us...
Scaling Pub/Sub
How do you scale publish/subscribe horizontally?

If we start with a single comet node, which
dispatches e...
Scaling Pub/Sub
How do you scale publish/subscribe horizontally?

  If we start with a single node, which dispatches
  eve...
Scaling Pub/Sub
How do you scale publish/subscribe horizontally?

  …then how do we scale up to a pair of nodes?


       ...
Scaling Pub/Sub
How do you scale publish/subscribe horizontally?

  …then how do we scale up to a pair of nodes?


       ...
Scaling Pub/Sub
  How do you scale publish/subscribe horizontally?

    But how, specifically? Split the architecture into
...
Pub/Sub Options
     Publish/subscribe already has solutions

Don’t reinvent the wheel: Many well-understood,
scalable pub...
Pub/Sub with Orbited
Orbited can be easily added to a pub/sub stack

Scaling through modularity: Its simplicity makes
Orbi...
Conclusion:
•   Polling latency unacceptable
•   Comet solves this problem
•   Impossible to scale Comet with threaded des...
Michael Carter

CarterMichael@gmail.com
 http://www.orbited.org/

       24 September 2007
           AJAXWorld
Upcoming SlideShare
Loading in...5
×

Comet for highly scalable applications

14,671

Published on

MIchael Carter and Arthur Lee's talk about Comet (why it's good, how to implement, and some specifics about the Orbited comet server)

Published in: Technology
1 Comment
35 Likes
Statistics
Notes
  • Do you mind if I pblish this at http://highlyscalable.com/ ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
14,671
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
1
Likes
35
Embeds 0
No embeds

No notes for slide

Comet for highly scalable applications

  1. 1. Comet for Highly- Scalable Applications Michael Carter and Arthur Lee diagrams by Jacob Rus 24 October 2007 The Ajax Experience Boston
  2. 2. What is Comet? “Comet” describes a model of user interaction Pushing real-time updates to web browsers Renkoo Gmail Chat Meebo
  3. 3. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  4. 4. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  5. 5. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  6. 6. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  7. 7. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  8. 8. Old Method: Polling Otherwise known as “hammering the server” App (Event Dispatcher (Event Queues inside) Browsers
  9. 9. Polling Overview Old and busted Pros: • Easy to implement • Client side • Server side Cons: • Wasted requests • Additional latency
  10. 10. Enter Long-Polling The new hotness Pros: • Latency—gone! • Client side still easy Cons: • Difficult to scale server-side • Conceptually more complicated
  11. 11. Vertical Scaling: How do you scale comet on a single server? photo by Michael (mx5tx)
  12. 12. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  13. 13. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  14. 14. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  15. 15. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  16. 16. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Threaded Comet Dispatcher • Supports a few Browsers Event Connect dozen users Queue Queue App • CPU usage and latency go through the roof as users are added
  17. 17. Vertical Scaling: How do you scale comet on a single server? Wrong way: Use a threaded server like Apache or IIS Compared to polling: Latency reduced for very light load Avg. latency depends on the order of events As much as 7× the CPU for moderate load CPU-bound at moderate load
  18. 18. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  19. 19. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  20. 20. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture • Supports as Event-Based Comet Dispatcher many users as Browsers Event Connect CPU and I/O Queue Pool App can handle • CPU usage scales linearly and latency remains low
  21. 21. Vertical Scaling: How do you scale comet on a single server? Right way: Use an event-based architecture Compared to polling: Decreased latency and CPU usage Compared to threaded Comet: Vastly decreased CPU usage Order of events irrelevant Performance degrades gracefully
  22. 22. Horizontal Scaling: How do you scale comet across many servers? photo by ulybug
  23. 23. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server Embed publish-subscribe architecture into comet server Keep application state within comet nodes Allow comet nodes to communicate with each-other
  24. 24. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application Node scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  25. 25. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “A” sends message Node to “E” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  26. 26. Horizontal Scaling: How do you scale comet across many servers? Wrong way: Build the whole software stack into the comet server • Design is inflexible, can’t Application “E” sends response Node to “A” scale for some Comet Nodes uses • Hard to scale to many nodes A B C D E F in the best case Browsers
  27. 27. Horizontal Scaling: How do you scale comet across many servers? Right way: Share nothing between nodes • No CPU or I/O overhead of sharing state Application Node • Comet CPU usage Nodes scales linearly and latency remains low Browsers
  28. 28. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table Application Hash Pre-defined Nodes Function Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  29. 29. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “B” sends Application Hash Pre-defined Nodes message Function to “E” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  30. 30. Horizontal Scaling: How do you scale comet across many servers? Right way: Use Orbited «wink» Distributed hash table “E” sends Application Hash Pre-defined Nodes response Function to “B” Comet hash function Nodes Leave authenti- cation to app layer A B C D E F Browsers
  31. 31. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates
  32. 32. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates real-time chat
  33. 33. Publish / Subscribe Pushing published events to groups of browsers What’s it good for? Applications for which multiple users need the same live updates stock tickers real-time chat
  34. 34. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single comet node, which dispatches events to multiple users in multiple groups then how do we scale up to multiple comet nodes?
  35. 35. Scaling Pub/Sub How do you scale publish/subscribe horizontally? If we start with a single node, which dispatches events to multiple users in multiple groups… A single comet Application Node node serves all users and all Comet Dispatcher groups Node Browsers
  36. 36. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Either we arrange Application Node nodes by group: All of a group’s Comet Dispatcher Nodes events are sent by one node Browsers
  37. 37. Scaling Pub/Sub How do you scale publish/subscribe horizontally? …then how do we scale up to a pair of nodes? Or we arrange Application Node nodes by user: A user receives Comet Dispatcher Nodes all events from the same node Browsers
  38. 38. Scaling Pub/Sub How do you scale publish/subscribe horizontally? But how, specifically? Split the architecture into layers Do not mix pub/sub layer App Publish/Subscribe Cloud with comet dispatcher Comet Dispatcher Nodes Treat pub/sub layer as a black box Browsers
  39. 39. Pub/Sub Options Publish/subscribe already has solutions Don’t reinvent the wheel: Many well-understood, scalable publish/subscribe solutions exist • Internet Relay Chat: Nearly 20 years old. Scales well for large groups of users. • XMPP (Jabber): Flexible protocol. Does presence well, in addition to messaging. • Java Message Service: It’s enterprise, baby.
  40. 40. Pub/Sub with Orbited Orbited can be easily added to a pub/sub stack Scaling through modularity: Its simplicity makes Orbited easy to add to any web app architecture Orbited handles Orbited comet dispatch Pub/Sub Stack Internet IRC or Jabber Relay IRC Bridge Chat handles publish/ Orbited subscribe Application / App servers do Web Server Browser everything else
  41. 41. Conclusion: • Polling latency unacceptable • Comet solves this problem • Impossible to scale Comet with threaded design • Event-based architecture is necessary • Horizontal scaling requires “share nothing” design • Publish/subscribe is a difficult problem • Scaling pub/sub is easiest with layered architecture
  42. 42. Michael Carter CarterMichael@gmail.com http://www.orbited.org/ 24 September 2007 AJAXWorld

×