Building a multi protocol broker for the internet of things using nodejs

12,738 views

Published on

Have you ever wondered how to interconnect your apps with physical things? Have you ever felt that the request/response pattern of HTTP is not enough? What about a binary protocol? In this talk you will discover the internal of the open source QEST broker, a Node.js-based broker for the Internet of Things that implements a classic publish/subscribe pattern, while making it accessible from HTTP and MQTT, an ultra-fast binary protocol.

Published in: Technology
0 Comments
29 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
12,738
On SlideShare
0
From Embeds
0
Number of Embeds
4,064
Actions
Shares
0
Downloads
156
Comments
0
Likes
29
Embeds 0
No embeds

No notes for slide

Building a multi protocol broker for the internet of things using nodejs

  1. 1. @matteocollina matteocollina.com Building a multi-protocol broker for the Internet of Things using node.jsNode.js Conference - Brescia 2012/11/09
  2. 2. How many people will owna smartphone by 2014 1.000.000.000
  3. 3. Did you seeit coming http://www.flickr.com/photos/12738000@N00/360231193/
  4. 4. They didn’t! ...
  5. 5. Whats next (hint: its a big number) 50.000.000.000
  6. 6. 50.000.000.0001.000.000.000
  7. 7. "Things"
  8. 8. Lets experiment
  9. 9. Goal: I want to chat with my house
  10. 10. >: hi househi matteo>: whats thetemperature?20
  11. 11. >: 4 8 15 16 23 42 If you don’t understand this, you are not a geek!
  12. 12. Enter Hubot Hubot © 2012 GitHub Inc. All rights
  13. 13. A programmable robotthat is controlled throughchat Hubot © 2012 GitHub Inc. All rights
  14. 14. We can supercharge our house with hubot
  15. 15. We can supercharge our house with hubot Howmodule.exports = (robot) -> robot.respond /what’s the temperature?/i, (msg) -> msg.http("http://mchouse.it/temperature") .header("Accept", "application/json") .get() (err, res, body) -> msg.send JSON.parse(body)
  16. 16. In order to ask our tempto hubot, we need to: 1. sense the temp 2. get the temp on the web 3. build a web API
  17. 17. We are building an API In Italy, “API” means “bees” http://www.flickr.com/photos/theseanster93/4056815767
  18. 18. What do we need tosense my housetemperature
  19. 19. We add a sensor to an Arduino TMP 36 TMP 36
  20. 20. We add an ethernet shield to connect to the Internet TMP 36 http://www.flickr.com/photos/snootlab/6052465980/
  21. 21. What protocol do we useto push our temperatureto our API?
  22. 22. HTTP is slow and safe http://www.flickr.com/photos/clearlyambiguous/48185613/
  23. 23. We need afast, binaryprotocol http://www.flickr.com/photos/grrphoto/305649629
  24. 24. M2M protocols are “Things” should interactmostly ad-hoc, and with our lives, and all theresearchers and technology should bebusinesses focus on low built to make them easylevel problems. to use.
  25. 25. • “things” exposed • “things” exposed with binary protocol to the web• publish/subscribe • request/response• topics as the naming • URIs as the system naming system
  26. 26. DiscoverQESTqest.me
  27. 27. HTTP Clients MQTT Clients QEST REST Server MQTT Server• MQTT broker QEST• REST interface• HTTP semantics Data Layer• no QoS• built on node.js and redis Redis
  28. 28. state-of-art state-of-art QEST-based QEST-based approach to IoT apps IoT apps approach to solution to IoT apps solution to IoT apps Web App Web App Web App Bridge Web App QEST Bridge QEST IoT Broker Device IoT Device 3 2 1 0 9 8 7 6 5 4 3 2 1 0 GND SCL AREF SDA 1 1 1 1 DIGITAL RX TX PWM PWM PWM PWM PWM PWM L TX Arduino UNO ON Broker RX 1 ICSP www.arduino.cc RESET IOREF POWER ANALOG IN 3V3 5V Gnd Vin 0 1 2 3 4 5 Device 3 2 1 0 9 8 7 6 5 4 3 2 1 0 GND SCLAREF SDA 1 1 1 1 DIGITAL RX TX PWM PWM PWM PWM PWM PWM L TX RX Arduino UNO ON 1 ICSP www.arduino.cc RESET IOREF POWER ANALOG IN 3V3 5V Gnd Vin 0 1 2 3 4 5 Device
  29. 29. QEST : MQTT to REST• retains every message received client = PubSubClient(server, 1883, callback); client.publish("temp", "30");• every topic has its own URI: /topics/<NAME> curl -H "Accept: txt" http://qest.me/topics/temp
  30. 30. QEST : REST to MQTT• transform every HTTP PUT received to a MQTT message curl -X PUT -d { "payload": 42 } -H "Content-Type: application/json" http://qest.me/topics/temp• devices can listen directly to MQTT topics void callback(char* topic, byte* payload, int length) { ... } PubSubClient(server, 1883, callback); client.subscribe("temp");
  31. 31. HTTP/MQTT ClientsHow to Load Balancerscale REST Server MQTT Server REST Server MQTT Server QEST Data Layer ... QEST Data Layer Redis
  32. 32. How muchtime took meto write thefirst versionof QEST?
  33. 33. How much A day.time took meto write thefirst version Thanksof QEST?
  34. 34. A day. • Express • Socket.io • node_redis Thanks • MQTT.js • Coffeescript
  35. 35. What happens if youwrite a broker in a day? (Hint)
  36. 36. Spaghetti code!
  37. 37. How to untangle it?
  38. 38. http://www.flickr.com/photos/adewale_oshineye/2933030620/Refactoring!!!
  39. 39. 1. A single file can go really far! Well, the first improvement were two files
  40. 40. Coffee-scriptis awesome for prototyping!
  41. 41. Coffee-scriptis a real pain to debug
  42. 42. 2. Extracting a data layer! HTTP Clients MQTT Clients REST Server MQTT Server QEST Data Layer Redis
  43. 43. Discover my roots
  44. 44. Ruby Language
  45. 45. Ruby LanguageRuby on Rails
  46. 46. Ruby on Rails The ActiveRecord pattern is AWESOME!
  47. 47. ActiveRecord1. Best pattern for storing data2. Its main responsibilities are CRUD operations3. I used that pattern to build a cross-process pub/sub system
  48. 48. Hubot © 2012 GitHub Inc. All rights
  49. 49. 3. Extracting an inter process pub/sub library
  50. 50. Discover Ascoltatorigithub.com/mcollina/ascoltatori
  51. 51. ir to lta • Redisco is a multi- • ZeroMQ process pub/subAs library backed by • RabbitMQ • MQTT (Mosquitto) • Memory (EventEmitter)
  52. 52. r i to ltaco var  ascoltatori  =  require(ascoltatori); //  you  can  use  RedisAscoltatore,  ZeromqAscoltatore,As //  RabbitAscoltatore,  MQTTAscoltatore var  ascoltatore  =  new  ascoltatori.MemoryAscoltatore(); ascoltatore.subscribe("hello/*",  function()  {    //  this  will  print  {  0:  "hello/42",  1:  "a  message"  }    console.log(arguments);      process.exit(0); }); ascoltatore.publish("hello/42",  "a  message",  function()  {    console.log("message  published"); });
  53. 53. Memory Redis ZeroMQ RabbitMQ Mosquitto r i to lta100.000 us co 10.000 usAs 1.000 us 100 us 10 us 1 us 1 10 100 1000 Listeners
  54. 54. i r to ltaco allowed me toAs refactor QEST as just an MQTT wrapper!
  55. 55. ut ing s rib ek or nt seco m Ia github.com/mcollina/qest github.com/mcollina/ascoltatori
  56. 56. DEMO! Source NASA
  57. 57. Matteo CollinaSoftware Engineer @MavigexPh.D. Student @University of Bolognamatteocollina.com
  58. 58. @matteocollinaThank You! Matteo Collina (matteo.collina2@unibo.it) http://www.flickr.com/photos/axel-d/479627824/

×