Building a multi protocol broker for the internet of things using nodejs

@matteocollina
matteocollina.com

Building a multi-protocol
broker for the Internet of
Things using node.js

Node.js Conference - Brescia 2012/11/09

How many people will own
a smartphone by 2014

1.000.000.000

Did you see
it coming

http://www.ﬂickr.com/photos/12738000@N00/360231193/

What's next
(hint: it's a big number)

50.000.000.000

50.000.000.000

1.000.000.000

Goal:

I want to
chat with
my house

>: hi house
hi matteo

>: what's the
temperature?
20

>: 4 8 15 16 23 42

If you don’t
understand
this, you are
not a geek!

Enter Hubot

Hubot © 2012 GitHub Inc. All rights

A programmable robot
that is controlled through
chat


We can supercharge our house with hubot

We can supercharge our house with hubot

How
module.exports = (robot) ->

robot.respond /what’s the temperature?/i, (msg) ->
msg.http("http://mchouse.it/temperature")
.header("Accept", "application/json")
.get() (err, res, body) ->
msg.send JSON.parse(body)

In order to ask our temp
to hubot, we need to:

1. sense the temp
2. get the temp on the web
3. build a web API

We are building an API

In Italy,
“API” means
“bees”
http://www.flickr.com/photos/theseanster93/4056815767

What do we need to
sense my house
temperature

We add a sensor to an Arduino

TMP
36

TMP
36

We add an ethernet shield to
connect to the Internet

TMP
36

http://www.flickr.com/photos/snootlab/6052465980/

What protocol do we use
to push our temperature
to our API?

HTTP is slow and safe

http://www.flickr.com/photos/clearlyambiguous/48185613/

We need a
fast, binary
protocol

http://www.flickr.com/photos/grrphoto/305649629

M2M protocols are “Things” should interact
mostly ad-hoc, and with our lives, and all the
researchers and technology should be
businesses focus on low built to make them easy
level problems. to use.

• “things” exposed • “things” exposed
with binary protocol to the web

• publish/subscribe • request/response

• topics as the naming • URIs as the
system naming system

HTTP Clients MQTT Clients

QEST
REST Server MQTT Server
• MQTT broker
QEST
• REST interface

• HTTP semantics Data Layer

• no QoS

• built on node.js
and redis Redis

state-of-art
state-of-art QEST-based QEST-based
approach to IoT apps IoT apps
approach to solution to IoT apps
solution to IoT apps

Web App Web App
Web App
Bridge
Web App

QEST
Bridge

QEST
IoT
Broker Device
IoT
Device
3 2 1 0 9 8 7 6 5 4 3 2 1 0

GND
SCL

AREF
SDA
1 1 1 1 DIGITAL

RX
TX
PWM
PWM
PWM

PWM
PWM

PWM
L

TX
Arduino UNO ON

Broker
RX

1
ICSP

www.arduino.cc

RESET
IOREF
POWER ANALOG IN

3V3
5V Gnd Vin 0 1 2 3 4 5

Device
3 2 1 0 9 8 7 6 5 4 3 2 1 0
GND
SCL

AREF
SDA

1 1 1 1 DIGITAL
RX
TX
PWM
PWM
PWM

PWM
PWM

PWM

L

TX
RX
Arduino UNO ON
1

ICSP

www.arduino.cc
RESET
IOREF

POWER ANALOG IN
3V3

5V Gnd Vin 0 1 2 3 4 5

Device

QEST : MQTT to REST
• retains every message received

client = PubSubClient(server, 1883, callback);
client.publish("temp", "30");

• every topic has its own URI: /topics/<NAME>

curl -H "Accept: txt" http://qest.me/topics/temp

QEST : REST to MQTT
• transform every HTTP PUT received to a MQTT message

curl -X PUT -d '{ "payload": 42 }'
-H "Content-Type: application/json"
http://qest.me/topics/temp

• devices can listen directly to MQTT topics
void callback(char* topic, byte*
payload, int length) {
...
}
PubSubClient(server, 1883, callback);
client.subscribe("temp");

HTTP/MQTT Clients

How to Load Balancer

scale

REST Server MQTT Server REST Server MQTT Server

QEST
Data Layer
... QEST
Data Layer

Redis

How much
time took me
to write the
ﬁrst version
of QEST?

How much A day.
time took me
to write the
ﬁrst version Thanks
of QEST?

A day. • Express

• Socket.io

• node_redis

Thanks • MQTT.js

• Coffeescript

What happens if you
write a broker in a day?
(Hint)

http://www.ﬂickr.com/photos/adewale_oshineye/2933030620/

Refactoring!!!

1.
A single file
can go really
far!

Well, the first
improvement
were two files

Coﬀee-script
is awesome for prototyping!

Coﬀee-script
is a real pain to debug

2.
Extracting a
data layer!
HTTP Clients MQTT Clients

REST Server MQTT Server
QEST

Data Layer

Redis

Ruby Language

Ruby on Rails

Ruby on Rails
The
ActiveRecord
pattern is
AWESOME!

ActiveRecord

1. Best pattern for storing data
2. Its main responsibilities are
CRUD operations
3. I used that pattern to build a
cross-process pub/sub
system

3.
Extracting an
inter process
pub/sub
library

Discover

Ascoltatori
github.com/mcollina/ascoltatori

ir
to
lta • Redis
co
is a multi- • ZeroMQ
process pub/sub
As

library backed by
• RabbitMQ

• MQTT
(Mosquitto)

• Memory
(EventEmitter)

r i
to
lta
co
var
ascoltatori
=
require('ascoltatori');

//
you
can
use
RedisAscoltatore,
ZeromqAscoltatore,
As

//
RabbitAscoltatore,
MQTTAscoltatore
var
ascoltatore
=
new
ascoltatori.MemoryAscoltatore();

ascoltatore.subscribe("hello/*",
function()
{

//
this
will
print
{
'0':
"hello/42",
'1':
"a
message"
}

console.log(arguments);

process.exit(0);
});

ascoltatore.publish("hello/42",
"a
message",
function()
{

console.log("message
published");
});

Memory Redis ZeroMQ RabbitMQ Mosquitto

r i
to
lta
100.000 us
co
10.000 us
As

1.000 us

100 us

10 us

1 us
1 10 100 1000
Listeners

i r
to
lta
co
allowed me to
As

refactor QEST as
just an MQTT
wrapper!

ut ing
s
rib ek
or
nt se
co m
Ia

github.com/mcollina/qest

github.com/mcollina/ascoltatori

Matteo Collina

Software Engineer @
Mavigex

Ph.D. Student @
University of Bologna

matteocollina.com

@matteocollina

Thank You!
Matteo Collina (matteo.collina2@unibo.it)

http://www.ﬂickr.com/photos/axel-d/479627824/

Building a multi protocol broker for the internet of things using nodejs

Building a multi protocol broker for the internet of things using nodejs

More Related Content

Similar to Building a multi protocol broker for the internet of things using nodejs

More from Matteo Collina

Recently uploaded

Building a multi protocol broker for the internet of things using nodejs