1. Chris Wanstrath
hi everyone, i’m chris wanstrath
2. i love randy rhoads
3. and kurt vonnegut
4. i live in san francisco
5. and work at github
6. (which is written in rails)
7. but! i’m not gonna talk about any of that stuff
8. today i want to talk about python
9. and comet
(among other things)
web (And other buzzwords)
By Chris Wanstrath
this talk is titled “the real-time web (and other buzzwords)”
11. so what is the “real-time” web?
12. techcrunch and readwriteweb would have you believe it’s a way to get your RSS stories
14. but is that it?
16. it’s all about pushing
17. instead of polling
18. it’s getting told what’s new
instead of asking for what’s new
20. instant chat in the browser
21. and information the moment it’s available
one persistent connection
instead of many, short lived connections
24. right now when we say “real-time web” we usually mean one of three things
26. ﬂash’s XMLSocket
27. or HTML5’s WebSocket
28. let’s start with comet
how many people know what comet is?
how many people know how it differs from XML socket or WebSocket?
good! for a long time i had no idea what comet was
30. i think it has a big marketing problem.
31. see, comet is a cleaning product.
32. just like ajax
33. so it’s kind of like,
34. Me, too!
35. I, too, am a
technique you didn’t know
existed and can start using
today that will forever change
the way you imagine the
“I, too, am a revolutionary web technique you didn’t know existed and can start using today
that will forever change the way you think about the web experience.”
and you’re like
39. if i had my way...
40. i’d call it something else
41. like maybe asteroid
42. i dunno, there’s some cool imagery there
43. i dunno, there’s some cool imagery there
44. i dunno, there’s some cool imagery there
45. i dunno, there’s some cool imagery there
46. i dunno, there’s some cool imagery there
47. i dunno, there’s some cool imagery there
48. i dunno, there’s some cool imagery there
49. anything but comet
50. anyway, comet is any standards compliant technique which pushes or streams data to the
browser over HTTP
with comet you can essentially fake a socket connection between a browser and a backend
51. how about an example
well, for this year’s django dash
53. alex gaynor... whoops...
54. alex gaynor
55. and leah culver
56. ( World’s smallest park )
(seen here next to the world’s smallest park)
57. built leafychat
irc in your browser using comet
58. you can connect to freenode channels
59. see who’s online
60. and do the irc thing
right in your browser (demo?)
61. all without ajax
63. well, there are a few components at play here
ﬁrst you’ve got the browser (naturally)
65. Browser Apache
then we have apache
sitting behind apache is django (via mod_wsgi or whatever)
the browser hits leafychat.com, port 80, which hits apache. apache sees it’s a django request
and hands off the request.
once django generates a response, apache takes it and delivers it
back to the browser. that’s, more or less, our HTTP request cycle
comet connections work in a similar fashion
72. Browser Orbited
instead of apache, we have Orbited sitting on port 8100
73. orbited is an open source, python comet server powered...
74. by twisted.
it does all of the comet heavy-lifting for us
75. Browser Orbited
sitting behind orbited we have our app-speciﬁc comet code. for leafy chat it was a twisted-
based daemon named zeddicus. it handled all the IRC stuff - connecting to channels, sending
messages, receiving private messages, logging, all that
89. with json.
orbited supports STOMP, XMPP, and raw IRC, too, but JSON is the way to go.
why do i say that?
90. well, unfortunately this is where orbited does not rock
91. orbited only cares about giving you a socket, you have to do everything else yourself.
that’s the deal
92. so you need to design your own protocol...
93. how it interacts with your back-end daemon
94. how events are ﬁred and responded to
95. all that boring stuff
96. we ended up using a combination of django signals and json
so this is our overview, except with one small change
these are actually persistent connections
between orbited and zeddicus it’s a tcpsocket
between the browser and orbited it’s a comet technique
we jump through all these hoops because it allows us to write our app as if the browser is
connecting DIRECTLY to zeddicus via a tcpsocket
which is what the “real time” web is all about and what comet gives us
the browser writes to and reads from a socket, our back-end daemon does the same.
103. fast communication
alright, so let’s talk about this part a bit more
the comet part
there are a few different ways to fake a persistent connection to a server with modern
106. xhr long polling
- hang an xhr request until we get a response or 30s, then re-open and wait again
script tag long polling
- dynamically create a script tag pointing to a url that hangs or times out, rinse and repeat
- open a Content-Encoded: chunked url in an iframe, each chunk is a <script> tag that runs
109. xhr streaming
- set content-encoding: chunked and trigger onreadystate callback with each chunk
110. so now you know our white lie: we don’t have real persistence, or a real socket.
we fake it at both ends.
111. or, really, orbited fakes it at both ends
112. all that to get irc in your browser
113. well, kinda
see, with orbited, you can connect directly to an irc server
and why not? it’s just a socket connection
115. in fact you can demo a (somewhat functional) irc connection on the orbited website
but with leafychat we wrote our own backend daemon that connected to irc.
if we loaded our django code into zeddicus, we could easily log irc chats you’re interested in
based on your session id.
it works very well.
119. Django Orbited
at this point i should mention the older tutorials online explaining how to load orbited into
120. Apache Django
but i the best (and simplest) way is to let each component be its own daemon
it works great for production as well as dev mode - the django dash judges were able to start
and run our app locally, despite the number of daemons that needed to run
121. if you want to get a comet app up and running locally, check out orbited
it supports a ton of comet transports and is actively maintained
122. using java? jetty has comet support
123. the ruby world has juggernaut
124. while perl has meteor
125. there’s the interesting in-progress Ajax Push Engine
126. and in erlang there’s erlycomet (built on mochiweb)
127. so, ﬂash’s XMLSocket
it turns this
130. how? ﬂash allows you to make tcp connections in actionscript.
socket connections from the browser
131. no lying needed
132. of course, it may be non-standard and might not work great across ﬁrewalls. but if you can
use it, it’s pretty great
133. a popular technique is to attempt to open a ﬂash socket, then fall back to standards based
comet methods if it fails
it’s good backup
134. there are a few nice libraries for xml socket on github
135. tmm1 / jssocket
my favorite is tmm1’s jssocket
136. defunkt / jssocket
(i have a fork which removes the jquery dependencies)
137. ﬁnally: HTML5’s WebSocket
138. it’s still a proposed draft
but it’ll let you open a socket to any serve that speaks the special WebSocket protocol
servers need to speak the WebSocket protocol - you can’t open arbitrary connections to irc or
this plugs up the obvious security holes but also makes it a bit harder to implement than
something like XMLSocket
your server needs to speak ws
140. so, in review
142. should be called asteroid
143. ﬂash’s XMLSocket
144. is non-standard but nice
145. HTML5’s WebSocket
146. not yet here
147. you should give orbited a shot
148. with json
probably null terminated
150. Django Orbited
don’t load orbited in django