1. Chris Wanstrath
http://defunkt.github.com
hi everyone, i’m chris wanstrath

2. i love randy rhoads

3. and kurt vonnegut

4. i live in san francisco

5. and work at github

6. (which is written in rails)

7. but! i’m not gonna talk about any of that stuff

8. today i want to talk about python

9. and comet
(among other things)

10. the
real-time
web (And other buzzwords)
By Chris Wanstrath
this talk is titled “the real-time web (and other buzzwords)”

11. so what is the “real-time” web?
?

12. techcrunch and readwriteweb would have you believe it’s a way to get your RSS stories

13. faster

15. but is that it?

16. no.

17. it’s all about pushing

18. instead of polling

19. it’s getting told what’s new
!

20. ?
instead of asking for what’s new

21. instant chat in the browser

22. and information the moment it’s available

23. Server
Client
one persistent connection

24. Client
Server
instead of many, short lived connections

25. right now when we say “real-time web” we usually mean one of three things

26. comet

27. flash’s XMLSocket

28. or HTML5’s WebSocket

29. let’s start with comet

30. ?
how many people know what comet is?
how many people know how it differs from XML socket or WebSocket?
good! for a long time i had no idea what comet was

31. i think it has a big marketing problem.
!

32. see, comet is a cleaning product.

33. just like ajax

34. so it’s kind of like,

35. Me, too!
“me too”

36. I, too, am a
revolutionary web
technique you didn’t know
existed and can start using
today that will forever change
the way you imagine the
web experience.
“I, too, am a revolutionary web technique you didn’t know existed and can start using today
that will forever change the way you think about the web experience.”

37. ...
and you’re like

38. Uhh...
uh...

39. AJAX!!!

40. if i had my way...

41. i’d call it something else

42. like maybe asteroid

43. i dunno, there’s some cool imagery there

44. i dunno, there’s some cool imagery there

45. i dunno, there’s some cool imagery there

46. i dunno, there’s some cool imagery there

47. i dunno, there’s some cool imagery there

48. i dunno, there’s some cool imagery there

49. i dunno, there’s some cool imagery there

50. anything but comet

51. anyway, comet is any standards compliant technique which pushes or streams data to the
browser over HTTP
with comet you can essentially fake a socket connection between a browser and a backend
server

52. how about an example
well, for this year’s django dash

53. me

54. alex gaynor... whoops...

55. alex gaynor

56. and leah culver

57. ( World’s smallest park )
(seen here next to the world’s smallest park)

58. built leafychat
irc in your browser using comet

59. you can connect to freenode channels

60. see who’s online

61. and do the irc thing
right in your browser (demo?)

62. all without ajax

63. how?
?

64. well, there are a few components at play here

65. Browser
first you’ve got the browser (naturally)

66. Browser Apache
80
then we have apache

67. Django
8000
Browser Apache
80
sitting behind apache is django (via mod_wsgi or whatever)

68. Django
8000
Browser Apache
80
the browser hits leafychat.com, port 80, which hits apache. apache sees it’s a django request

69. Django
8000
Browser Apache
80
and hands off the request.

70. Django
8000
Browser Apache
80
once django generates a response, apache takes it and delivers it

71. Django
8000
Browser Apache
80
back to the browser. that’s, more or less, our HTTP request cycle

72. Browser
comet connections work in a similar fashion

73. Browser Orbited
8100
instead of apache, we have Orbited sitting on port 8100

74. orbited is an open source, python comet server powered...

75. by twisted.
it does all of the comet heavy-lifting for us

76. Browser Orbited
8100
so.

77. Zeddicus
8200
Browser Orbited
8100
sitting behind orbited we have our app-specific comet code. for leafy chat it was a twisted-
based daemon named zeddicus. it handled all the IRC stuff - connecting to channels, sending
messages, receiving private messages, logging, all that

78. Orbited
8100
orbited handles generic browser stuff...

79. Zeddicus
8200
while our app-specific daemon (zeddicus) deals with the business logic.
in this case, irc stuff

80. Zeddicus
8200
Browser Orbited
8100
does this look familiar?

81. Django
8000
Browser Apache
80
no? ok.

82. Zeddius
Zeddicus
8200
Browser Orbited
8100
anyway, the browser (using orbited’s bundled js library) makes a request...

83. Zeddius
Zeddicus
8200
Browser Orbited
8100
to port 8100. orbited sees it’s a request for zeddicus and hands it off to our backend

84. Zeddius
Zeddicus
8200
Browser Orbited
8100
zeddicus sees a new socket connection open, does some stuff, then writes to the socket

85. Zeddius
Zeddicus
8200
Browser Orbited
8100
orbited reads what zeddicus wrote to the socket connection they share

86. Zeddius
Zeddicus
8200
Browser Orbited
8100
then sends it back the browser via comet

87. Django
8000
Browser Apache
80
just like before

88. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<script>
// session data
var session_nick = ""
var session_channels = []
</script>
<script type="text/javascript" src="/static/js/soundmanager2/soundmanager2-nodebug-jsmin.js"></script
<script type="text/javascript" src="/static/js/audio-player.js"></script>
<title>Leafy Chat</title>
<link rel="icon" href="/static/img/favicon.ico"/>
<link rel="stylesheet" href="/static/css/reset.css" type="text/css" media="screen" />
<link rel="stylesheet" href="/static/css/facebox.css" type="text/css" media="screen/>
<link rel="stylesheet" href="/static/css/base.css" type="text/css" media="screen" />
<script> document.domain = document.domain; </script>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js">
<script type="text/javascript" src="http://leafychat.com:8100/static/Orbited.js"></script>
<script type="text/javascript" src="/static/js/facebox.js" charset="utf-8"></script>
<script type="text/javascript" src="/static/js/cookie.js" charset="utf-8"></script>
<script type="text/javascript" src="/static/js/leafy.js" charset="utf-8"></script>
<script type="text/javascript" src="/static/js/kahlan.js" charset="utf-8"></script>
<div id="main">
<div id="header">
<div id="navigation" class="rounded">
<ul class="nav-list">
<!-- this should be a list -->
<li>Hi <span id="welcome-user">there</span>!</li>
but while apache and django deal with html

89. Zeddius
Zeddicus
8200
Browser Orbited
8100
zeddicus, our back-end daemon, concerns itself only...

90. with json.
orbited supports STOMP, XMPP, and raw IRC, too, but JSON is the way to go.
why do i say that?

91. well, unfortunately this is where orbited does not rock

92. orbited only cares about giving you a socket, you have to do everything else yourself.
that’s the deal

93. so you need to design your own protocol...

94. how it interacts with your back-end daemon

95. how events are fired and responded to

96. all that boring stuff

97. we ended up using a combination of django signals and json

98. Zeddius
Zeddicus
8200
Browser Orbited
8100
so this is our overview, except with one small change

99. Zeddius
Zeddicus
8200
Browser Orbited
8100
these are actually persistent connections

100. Zeddius
Zeddicus
8200
Browser Orbited
8100
between orbited and zeddicus it’s a tcpsocket

101. Zeddius
Zeddicus
8200
Browser Orbited
8100
between the browser and orbited it’s a comet technique

102. Zeddicus
Browser
8200
we jump through all these hoops because it allows us to write our app as if the browser is
connecting DIRECTLY to zeddicus via a tcpsocket

103. Zeddicus
Browser
8200
which is what the “real time” web is all about and what comet gives us
the browser writes to and reads from a socket, our back-end daemon does the same.

104. fast communication

105. Zeddius
Zeddicus
8200
Browser Orbited
8100
alright, so let’s talk about this part a bit more
the comet part

106. Zeddius
Zeddicus
8200
Browser Orbited
8100
there are a few different ways to fake a persistent connection to a server with modern
browsers

107. xhr long polling
- hang an xhr request until we get a response or 30s, then re-open and wait again

108. <script>
script tag long polling
- dynamically create a script tag pointing to a url that hangs or times out, rinse and repeat

109. <iframe>
forever frame
- open a Content-Encoded: chunked url in an iframe, each chunk is a <script> tag that runs

110. xhr streaming
- set content-encoding: chunked and trigger onreadystate callback with each chunk

111. so now you know our white lie: we don’t have real persistence, or a real socket.
we fake it at both ends.

112. or, really, orbited fakes it at both ends

113. all that to get irc in your browser

114. well, kinda

115. Zeddius
freenode
Zeddicus
8200
6667
Browser Orbited
8100
see, with orbited, you can connect directly to an irc server
and why not? it’s just a socket connection

116. in fact you can demo a (somewhat functional) irc connection on the orbited website

117. Zeddius
Zeddicus
8200
Browser Orbited
8100
Zeddius
freenode
Zeddicus
8200
6667
but with leafychat we wrote our own backend daemon that connected to irc.
why?

118. logging.

119. Zeddius
Zeddicus
8200
Browser Orbited
8100
if we loaded our django code into zeddicus, we could easily log irc chats you’re interested in
based on your session id.
it works very well.

120. Django Orbited
8000
Browser Apache
80
Zeddius
8200
at this point i should mention the older tutorials online explaining how to load orbited into
django.

121. Apache Django
80 8000
Browser
Orbited Zeddicus
8100 8200
but i the best (and simplest) way is to let each component be its own daemon
it works great for production as well as dev mode - the django dash judges were able to start
and run our app locally, despite the number of daemons that needed to run

122. if you want to get a comet app up and running locally, check out orbited
it supports a ton of comet transports and is actively maintained

123. using java? jetty has comet support

124. the ruby world has juggernaut

125. while perl has meteor

126. there’s the interesting in-progress Ajax Push Engine

127. and in erlang there’s erlycomet (built on mochiweb)

128. so, flash’s XMLSocket

129. Zeddius
Zeddicus
8200
Browser Orbited
8100
it turns this

130. Zeddicus
Browser
8200
into this

131. how? flash allows you to make tcp connections in actionscript.
by providing a javascript api to those tcp connections, we can use flash to create persistent,
socket connections from the browser

132. no lying needed

133. of course, it may be non-standard and might not work great across firewalls. but if you can
use it, it’s pretty great

134. a popular technique is to attempt to open a flash socket, then fall back to standards based
comet methods if it fails
it’s good backup

135. there are a few nice libraries for xml socket on github

136. tmm1 / jssocket
my favorite is tmm1’s jssocket

137. defunkt / jssocket
(i have a fork which removes the jquery dependencies)

138. finally: HTML5’s WebSocket

139. it’s still a proposed draft
but it’ll let you open a socket to any serve that speaks the special WebSocket protocol

140. ws://
servers need to speak the WebSocket protocol - you can’t open arbitrary connections to irc or
xmpp gateways
this plugs up the obvious security holes but also makes it a bit harder to implement than
something like XMLSocket
your server needs to speak ws

141. so, in review
?

142. comet

143. should be called asteroid

144. flash’s XMLSocket

145. is non-standard but nice

146. HTML5’s WebSocket

147. not yet here
but futuristic

148. you should give orbited a shot

149. with json

150. 0
probably null terminated

151. Django Orbited
8000
don’t load orbited in django

152. Orbited
8100
use them alongside each other

153. is that it?
?

154. that’s it
!

155. Thank You

157. http://www.flickr.com/photos/mojombo/3785549701/sizes/l/
http://www.flickr.com/photos/voteprime/2361330726/sizes/o/
http://www.flickr.com/photos/johnkerr/2371310025/sizes/l/
http://www.flickr.com/photos/h19/182898904/
http://www.flickr.com/photos/ukinindia/3595042998/sizes/l/
http://www.flickr.com/photos/scott1027/3189137578/sizes/l/
http://www.flickr.com/photos/foxypar4/2124673642/sizes/l/
http://www.flickr.com/photos/nickwheeleroz/2166114756/sizes/l/
http://www.flickr.com/photos/nickwheeleroz/2178146080/sizes/l/
http://www.flickr.com/photos/diyromarcade/3006368260/sizes/o/
http://www.flickr.com/photos/boopsiedaisy/3611187885/sizes/o/
http://www.flickr.com/photos/bastispicks/2834869959/sizes/l/
http://www.flickr.com/photos/courtenay/2536259393/sizes/l/
http://www.flickr.com/photos/jardinle/3335907363/sizes/o/
http://www.flickr.com/photos/tim-miley/3569809538/
http://www.flickr.com/photos/dexterousartisan/3209508363/sizes/l/
http://www.flickr.com/photos/equanimity/3763158824/sizes/l/
http://www.flickr.com/photos/24617281@N04/2329660856/sizes/o/
http://farm4.static.flickr.com/3555/3767120120_8f43f885e1.jpg
http://www.flickr.com/photos/raffaella/64701476/
http://www.flickr.com/photos/monicareyes/405402858/sizes/o/
http://www.flickr.com/photos/11016633@N07/2232831953/
flickr