SocketStream

3,107 views

Published on

A talk given at Node.js Cambridge about SocketStream, a realtime framework for single page apps.

https://socketstream.com

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,107
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SocketStream

  1. 1. A web framework for single page apps
  2. 2. Created by Owen Barnes @temporalwave ! ! (my boss at a former company)
  3. 3. About me Paul Jensen @paulbjensen ! ! (I’m the new lead developer)
  4. 4. Where to begin?
  5. 5. Why Single Page Apps?
  6. 6. With a traditional web app, the user has to refresh the page to see new information
  7. 7. Client Server Time
  8. 8. GET/football/live Client Server Time HTTP/1.1
  9. 9. HTTP/1.1 200 OK Client Server Time
  10. 10. 20 seconds later… Client Server Time
  11. 11. I wonder what the latest score is… Let’s reload the page Client Server Time
  12. 12. GET/football/live Client Server Time HTTP/1.1
  13. 13. HTTP/1.1 304 Not Modified Client Server Time
  14. 14. Client Server Time
  15. 15. The user had to press F5 to get at any new information Client Server Time
  16. 16. Client Server Time Even though there was no new information, the server still had to serve the HTTP request
  17. 17. This is not a fun experience
  18. 18. How do we make this better?
  19. 19. Client Server Time
  20. 20. We could use AJAX to update the page
  21. 21. We’d save the user having to press the F5 key
  22. 22. What else can we do?
  23. 23. Client Server Time
  24. 24. Optimise the response
  25. 25. GZIP the response data, and …
  26. 26. Avoid sending data we already have on the client
  27. 27. We could also separate the HTML from the data
  28. 28. Reuse the HTML on the client
  29. 29. …and use the server to provide you with just data
  30. 30. And the web site becomes a client Web App Native App Server API User
  31. 31. The server is just an API
  32. 32. A beautiful separation of concerns
  33. 33. Overview • The server becomes a REST API serving JSON • HTML compilation is done on the client • As a result, less processing & bandwidth is consumed by the server
  34. 34. Why Realtime?
  35. 35. Client Server Time
  36. 36. Polling the server every [n] seconds for new data is redundant
  37. 37. There has to be a better way
  38. 38. What if the server could send its client(s) new data as soon as it came about?
  39. 39. We can, thanks to WebSockets
  40. 40. WebSockets allows data to be sent both ways
  41. 41. Client Server Time
  42. 42. Client Server Time Goal
  43. 43. The server sends a message to the client that an action has occurred Client Server Time Goal
  44. 44. We eliminate the need to poll the server for new data
  45. 45. Overview • We can replace AJAX polling with WebSockets, and provide a better user experience as a result • We also remove the need to make redundant polling requests back to the server. • We use WebSockets for sending/receiving JSON
  46. 46. Single Page Apps + The Realtime Web
  47. 47. There are many ways to build this kind of app
  48. 48. You could build it mostly from scratch, and use Express + Socket.io
  49. 49. Or alternatively, you could use a web framework like Meteor or Firebase
  50. 50. SocketStream is somewhere in-between these 2 approaches
  51. 51. It provides tools to help with building realtime single page apps...
  52. 52. ... Whilst trying not to restrict what technologies you can use with your app
  53. 53. For example, we don't provide an ORM. Instead, you choose the database & the ORM
  54. 54. Also, we don't mandate using a specific client-side framework ! You can use BackBone, Angular, Ember, or something else, that is entirely your choice.
  55. 55. What we focus on instead are these things:
  56. 56. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  57. 57. I'll run through each of these, 1-by-1. But first, let's look at how to use SocketStream
  58. 58. Getting started npm install -g socketstream ! socketstream new my_app
  59. 59. Getting started ! ! ! ! Success! Created app 'my_app' with: ✓ Basic chat demo (-m for minimal install) ✓ Javascript example code (-c if you prefer CoffeeScript) ✓ Plain HTML for views (-j if you prefer Jade) Next, run the following commands: cd my_app [sudo] npm install To start your app: node app.js

  60. 60. Here's what the initial file/ folder structure looks like
  61. 61. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  62. 62. Client code is organised into 5 sub-folders
  63. 63. Client side code organisation • CODE stores client side JavaScript files and libraries • CSS stores CSS files • STATIC stores public files like images, font files, and other assets • TEMPLATES stores HTML templates for the single page app to render on the client • VIEWS stores HTML files that are rendered from the server for the initial page
  64. 64. Those sub-folders have subfolders, but are optional
  65. 65. This is how we load them // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  66. 66. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  67. 67. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  68. 68. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  69. 69. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  70. 70. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  71. 71. // My SocketStream 0.3 app ! var http = require('http'), ss = require('socketstream'); ! // Define a single-page client called 'main' ss.client.define('main', { view: 'app.html', css: ['libs/reset.css', 'app.styl'], code: ['libs/jquery.min.js', 'app'], tmpl: '*' }); ! // Serve this client on the root URL ss.http.route('/', function(req, res){ res.serveClient('main'); });
  72. 72. SocketStream uses Browserify to handle requiring JS files
  73. 73. Browserify allows us to use a Node.js style of requiring JS files
  74. 74. // This file automatically gets called first by SocketStream and must always exist ! // Make 'ss' available to all modules and the browser console window.ss = require('socketstream'); ! ss.server.on('disconnect', function(){ console.log('Connection down :-('); }); ! ss.server.on('reconnect', function(){ console.log('Connection back up :-)'); }); ! ss.server.on('ready', function(){ ! ! ! // Wait for the DOM to finish loading jQuery(function(){ // Load app require('/app'); }); });
  75. 75. // This file automatically gets called first by SocketStream and must always exist ! // Make 'ss' available to all modules and the browser console window.ss = require('socketstream'); ! ss.server.on('disconnect', function(){ console.log('Connection down :-('); }); ! ss.server.on('reconnect', function(){ console.log('Connection back up :-)'); }); ! ss.server.on('ready', function(){ ! ! ! // Wait for the DOM to finish loading jQuery(function(){ // Load app require('/app'); }); });
  76. 76. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  77. 77. Over the years, developers have come up with new languages to generate HTML, CSS, and JavaScript
  78. 78. SocketStream allows developers to use these code preprocessors in their apps
  79. 79. Adding a preprocessor is simple // Code Formatters ss.client.formatters.add(require('ss-stylus'));
  80. 80. For Javascript • SS-COFFEE - supports using CoffeeScript • SS-GORILLA - supports using GorillaScript
  81. 81. For CSS • SS-STYLUS - supports using Stylus • SS-LESS - supports using Less
  82. 82. For HTML Views • SS-JADE - supports using Jade
  83. 83. For HTML Templating • SS-HOGAN - supports using Twitter's Hogan.js • SS-COFFEEKUP - supports using CoffeeKup
  84. 84. Setting a Templating engine // Use server-side compiled Hogan (Mustache) templates. Others engines available ss.client.templateEngine.use(require('ss-hogan'));
  85. 85. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  86. 86. Having to press F5 to reload the page in order to view changes to HTML/CSS/JS...
  87. 87. ... is not a fun experience
  88. 88. In development mode, SocketStream will watch the client files for changes, and reload the page when they occur
  89. 89. In the case of CSS, SocketStream will apply the changes without reloading the page
  90. 90. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  91. 91. Client-side HTML templates are made available to the browser via the ss.tmpl object
  92. 92. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  93. 93. When you're building a single page app, you'll have a lot of JS files, and maybe a few CSS files
  94. 94. But serving a HTML page with lots of these files can take time, and is inefficient
  95. 95. SocketStream provides a way to concatenate, minify, and GZip these files into 1 JS and 1 CSS file
  96. 96. This saves bytes being transferred, as well as reducing the number of HTTP requests you make
  97. 97. Also, you can tell SocketStream to load these files from a CDN
  98. 98. Setting a Templating engine // Minimize and pack assets if you type: SS_ENV=production node app.js if (ss.env === 'production') ss.client.packAssets();
  99. 99. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  100. 100. Web Workers are handy for intensive client-side JS operations
  101. 101. SocketStream provides support for using Web Workers in your app
  102. 102. First, create a folder
  103. 103. Next, put your web worker files in that folder
  104. 104. Then, load the worker in a client code file, and enjoy
  105. 105. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  106. 106. SocketStream uses Connect middleware to support HTTP features
  107. 107. SocketStream uses the following middleware by default: • compress - for GZipping assets • cookieParser - for handling user tracking • favicon - for serving a favicon.ico file • session - for handling sessions • static - for serving static assets
  108. 108. SocketStream uses the following middleware by default: • compress middleware is loaded first, before all other middleware • static middleware is loaded last, after all other middleware
  109. 109. SocketStream provides a way to load custom middleware into the connect stack
  110. 110. ss.http.middleware.prepend() ss.http.middleware.append()
  111. 111. This allows you to use all of the connect middleware out there today, i.e. EveryAuth
  112. 112. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  113. 113. We use connect’s session middleware, so authentication can be done with either EveryAuth, PassportJS, or you can roll your own.
  114. 114. We also recommend using connect-redis
  115. 115. Both HTTP and WebSocket interfaces can get/set the session data
  116. 116. Via HTTP // app.js ss.http.router.on('/updateSession', function(req, res) { req.session.myVar = 4321; res.end('req.session.myVar has been updated to', req.session.myVar); });
  117. 117. Via WebSockets
  118. 118. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  119. 119. RPC is a common pattern for clients requesting data from the server
  120. 120. SocketStream provides a way to construct RPC APIs with flexibility
  121. 121. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  122. 122. PubSub is a great pattern for Single Page Apps
  123. 123. SocketStream handles this in various ways:
  124. 124. 1 - Publishing to everyone viewing the app right now Server ss.publish.all('newMessage', message); // Broadcast the message to everyone Client // Listen out for newMessage events coming from the server ss.event.on('newMessage', function(message) { // do something with the message });
  125. 125. 2 - Sending to private channels Server (subscribe/unsubscribe the session ) // in a /server/rpc file after calling req.use('session') middleware ! req.session.channel.subscribe('disney') ! req.session.channel.unsubscribe('kids') ! req.session.channel.reset() // unsubscribes the session from every channel req.session.channel.list() // shows what channels are subscribed to !
  126. 126. 2 - Sending to private channels Server (publish to channel) // in a /server/rpc file ss.publish.channel('disney', 'chatMessage', {from: 'jerry', message: 'Has anyone seen Tom?'}); Client (receive channel message) // in a /client/code file ss.event.on('chatMessage', function(msg, channelName){ console.log('The following message was sent to the ' + channelName + ' channel:', msg); });
  127. 127. 3 - Sending to users Server // in a /server/rpc file ss.publish.user('fred', 'specialOffer', 'Here is a special offer just for you!');
  128. 128. 4 - Sending to a browser tab Server // in a /server/rpc file ss.publish.socketId('254987654324567', 'justForMe', 'Just for one tab');
  129. 129. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  130. 130. On top of RPC and PubSub, SocketStream provides you with a way to create custom request responders
  131. 131. Request Response is basically a WebSocket message handler
  132. 132. It allows you to write message handling for games, where every byte matters
  133. 133. HTML / CSS / JS code preprocessing Minifying CSS/JS for production use Client-side code organisation HTML Templates WebSocket Management Live Reload Building RPC APIs Building PubSub APIs Session Management Building custom APIs on top of WS Web Workers Connect middleware compatibility
  134. 134. WebSockets are not immortal…
  135. 135. They are mangled by mobile networks…
  136. 136. Blocked by firewalls…
  137. 137. Routed to dead ends by proxy servers
  138. 138. And severed by train tunnels
  139. 139. Also, browser support for WebSockets isn’t guaranteed
  140. 140. You need a transport strategy
  141. 141. Originally, SocketStream used Socket.io
  142. 142. But Socket.io asserted that if a browser supported WebSockets, then it would work
  143. 143. They learned from this, by building Engine.io
  144. 144. I created the transport wrapper for Engine.io in SocketStream for Bechtel & Dashku
  145. 145. And designed it to reconnect the client when severed
  146. 146. Months later, it made it’s way into SocketStream’s core.
  147. 147. SocketStream let’s you use this, alongside SockJS
  148. 148. …and that is SocketStream in a nutshell. Whew!
  149. 149. Let’s look at some SocketStream apps in the wild
  150. 150. Hollow hollowdocumentary.com
  151. 151. Vmux vmux.co
  152. 152. Dashku dashku.com
  153. 153. SocketStream plugins
  154. 154. SS-BACKBONE
  155. 155. SS-ANGULAR
  156. 156. SS-CUCUMBER
  157. 157. Tips for deploying SocketStream in production
  158. 158. 1 - Check your server’s ulimit configuration (This can bite you hard)
  159. 159. I learned this when Dashku went #1 on Hacker News in 45min
  160. 160. 2 - Use HTTPS, but handle it at the load balancer level rather than at the app level
  161. 161. HTTPS helps to improve the stability of WebSocket connections, especially on mobile devices
  162. 162. But Node’s HTTPS implementation is noticeably slower than using HAProxy or Nginx
  163. 163. Where is SocketStream going next?
  164. 164. We’re in the process of getting SocketStream’s test coverage up
  165. 165. We’re also trying to close some ancient bugs
  166. 166. We also need better documentation
  167. 167. We’re giving the web site an overhaul
  168. 168. And we want to document how SocketStream’s internals function, to help build 0.4
  169. 169. but what about 0.4?
  170. 170. …0.4 is starting to look like these:
  171. 171. I promise you all, it’s coming in June 2014
  172. 172. Thank You

×