Arduino and the real time web


Published on

Using real time aspects of the web and Socket IO (Django SocketIO) to pull data from an Arduino or to control one.

Published in: Technology, Education
  • @hernanchodiaz Probably better to raise an issue on my github there are definitely some changes as a result of Socket.IO and the python / Django wrappers changing on that.
    Are you sure you want to  Yes  No
    Your message goes here
  • the ArduinoWebsocketClient library have changed 'update' and current android code dont work show many error, I cant suscribe channel 'tempsensor' I search for hours in internet but cant find someting than help me :(
    Are you sure you want to  Yes  No
    Your message goes here
  • @xaral well really the benefit comes from having the arduino networked. If you wanted to use XBee you could do a point to point via Xbee and then use a bridge out to the Internet. Would slow up quite a bit I expect.
    Are you sure you want to  Yes  No
    Your message goes here
  • @andrea.reginato apologies only just saw this now. The full WS spec isn't available yet so the way I would do it is via tokenisation on the initial request. So you subscribe to a channel, pass a token (like an API key) and then the server will allow access to the channel on it or not.

    The full WS spec makes provision for this but that won't be handled by arduino for a while due to weight of the library.
    Are you sure you want to  Yes  No
    Your message goes here
  • One question. It is possible to create a real time web even if we don't use a ethernet shield or wifi, and just a xbee connected to our arduino.?? This xbee will send the data wirelessly to another xbee connected to a computer.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Thanks Jon – Good afternoon everyone and thanks for coming along for at the end of the day. My session is going to be a little different from some of the others this afternoon as it's a little more software and network oriented than the others.
  • Housekeeping wise – my presso will be up at slideshare with all my notes and all my code is in github so don't feel like you need to write much of this down. It's the end of what's been a great day so sit back, relax and just think about how you can play around with some of what I'm going to show you this afternoon.
  • So before we get started let's get a sense of knowledge level. Who has done some sort of web development involving server side coding? Okay who has played with Arduino's ethernet library? Arduino Ethernet Server? Arduino Ethernet Client? Okay so there's a few that haven't used the Ethernet library at all
  • Very quickly, arduino's can connect to the network with an ethernet shield or using an etherten or similar like this. If you haven't played with the library think of it like a networked version of a serial connection you can pass data back and forth and your arduino can be either a server or a client.
  • To get up to speed with this I'd thoroughly working through and digesting the various examples in the Arduino IDE and trawling all the API documentation on the site. Don't worry too much though the principles and architecture I'll be talking about are just as important as the code so you'll follow along even if you haven't played with it much.
  • There are some great ways to connect an arduino to the web and you might find some of these useful for your particular needs before we get into realtime. .
  • If all you want to do is get some data to present onto a single web page nice and simply then I can suggest you check out the examples in the Arduino Networking library. If you want something a little more robust but still quite simple check our ArduServer These are a bit limited in terms of interaction but they are quick to produce
  • If you're looking to control IO pins using URLs in a more restful manner then check out restduino or webduino. These are more fully featured web servers – Webduino is getting pretty clever now. These types of applications generally give you a lot of URLs to play with, often customisable, you can serve images, turn pins on or off and read state and serve content as HTML or JSON.
  • So there's clearly lots of ways to use an arduino with the web already. Given that, why would I want to hook it up to the real time web? Surely using one of these methods already should be sufficient.
  • Quite simply it's so you can create your robot army to take over the world that you control via the web browser on your mobile phone.
  • Realistically they probably look a bit more like this – but conquest will happen – just really slowly. Actually in the traditional web model we face some limitations when we start playing with arduinos.
  • Arduinos don't scale well with concurrent users. If you try and build a web app on your arduino you end up with thousands of lines of code in your program. Likewise processing thousands of lines of HTML can be error prone and result in buffer overruns and crashes. Many projects offload this processing to another machine and then communicate via serial but clearly that's going to affect your mobility. So to deal with this we have to get a method that works consistently knowing these limitations.
  • So the architecture for interaction with the web in real time is a little more complex than the stock request response method and at the heart of how I do this is to use Web Sockets. Who here is familiar with how web sockets work? Very briefly for those that aren't – web sockets are a full duplex bidirectional connection over a single TCP connection. In practice this usually means a client – typically a web browser connects to a server and holds open a connection that data is streamed back and forwards through
  • There's some good resources for how this all works and I won't explain the detail now but suffice to say Web Sockets allows us to hold a single connection open between the client and server and stream data back and forth in real time which provides an efficient method of communication over HTTP without the overhead of lots of handshaking and header passing for every request.
  • There are three main components First is the web socket server which provides application logic – it could service other functions as well such as authentication, hooking to messaging systems or other web application components or a database. The next part is the Web Browser Client – this provides the UI component and is typically a mix of JS and HTML. Finally is the client that sits on the arduino that can processes messages coming in. Web scokets are generally set up as a pub / sub model on channels so you push data to a channel and listen for messages which you can action.
  • To show you how this works then in some code. I'm a python guy so I use Django Socket IO for this but the same principles hold for things like Node and what not as well. You can see here I have some code that sets up a channel that clients can subscribe to. After that I then have some code that executes when it receives message events which process the message and do something with it. I can send messages back to the originating client or I can broadcast messages to everyone on a channel as well.
  • Here's the web browser code. I have some JS here that is connecting to the channel and you can see here I have code that sends a message and receives one back. Web sockets typically use JSON in order to send messages back and forth so that makes it very suitable for use in web pages and JSON plays nicely with obviously Node and Python which is why these two stacks tend to get used for Web Sockets most often.
  • This then is the arduino client. This is an adaptation of the Arduino Web Sockets library by Kevin Rohling to work with Socket IO. It's worth noting that this version of the library now only works with Arduino 1.0. Again I subscribe to a channel like the others. Note a couple of the pauses I do to let everything have time to catch up. Once I'm subscribed I create a delegate function that handles my messages which in this case is just going to print a message
  • So in my main loop I then call monitor to just check if a message has come in and in this instance I'm going to just send a message off to the server every second on the channel.
  • Now we understand the building blocks let's put them together in an example. We're going to take data from two independent sensors on two arduinos and get it to come together in real time to a display that you'll all be able to view on your web browsers if you join my network.
  • Let's start with the arduino code which is pretty simple. I have two ether tens connected to the network. Attached to them are simply a temperature sensor using a LM335 sensor. I won't go through the temperature code as it's irrelevant but here you can see my loop just calls the web sockets send command with the current state of the temperature and the node id every second. Note that I haven't calibrated these sensors either as it's just a quick and dirty example.
  • Now on the server, we take the message sent to the channel and we process it to grab the node ID and the temperature value. This is then broadcast in a package to any client that's subscribed to listen on a broadcast channel. The reason we do this as well is to stop an echo chamber from developing where you flood the sender back with echoes of it's own messages. It's just a nice way to limit the amount of data you're pushing around.
  • On the browser side we then process the message as it comes in and hand that data off to a graphing library called smoothie which makes pretty charts from web socket streamed data. Pretty straight forward huh?
  • Now let's see this working – assuming my code works. You can actually see this if you jump on my network and then hit this IP address. It should work with just about all modern browsers and works okay on iPhone and newish Android devices Switch to screen <RIGHT> So you can see here the two charts from the two sensors. If I hold one to heat it up you can see it goes up, then down as I let it go. You get the average between them below as well in the chart below. So this is a pretty basic example but you can see how this can start to work – hopefully your brains are starting to shift into project applications now.
  • So you now you understand the principles involved there's lots of applications for this. You could scale a sensor mesh quite easily using this set up or else you could flip it and use a web browser as the controller or input which then gets piped to one or many arduinos to react in some fashion – this is how you'd make your mobile phone browser controlled robot army. Similarly for M2M there are some good options here as because of the pub/sub model a subscriber for data could be an arduino, a web service or some other application, so could a publisher. Using the same stack across the whole network creates new opportunities for data to move around. So you can see it's possible to create some interesting interactions using the real time aspects of modern web browsers - all with some web tech and an arduino and it's getting a lot easier to do it.
  • Now if you don't want to muck around with setting up Node or Django servers there are some other options available. There's a site called that provides web sockets infrastructure. There's a pusher port of the web sockets library as well so that makes it very simple. It is a paid service if you want more than 20 connections or 100 thousand messages per day though. Also Socket.IO is the library I use which is a very good web sockets library with a lot of fall backs for older browsers and handles things like different media types very well also.
  • So thanks very much for your attention at the end of the day - I think we have time for probably a question or two. I'm around after this so if you want to grab me please feel free. My details are all up here too so get in touch to talk further about your project or application.
  • Arduino and the real time web

    1. 1. Arduino and the real time webLinux Conf AU – Arduino Miniconf16 January, 2012@ajfisher
    2. 2. Slides, notes &
    3. 3. Arduino ❤ Web
    4. 4. Arduino networking introductionPhoto (CC): Flickr rfranklinaz
    5. 5. Arduino networking resourcesArduino IDEExamples/Ethernet/Web Client & Web ServerArduino
    6. 6. Sensor serving doesnt have to be plain @ajfisher @superhighfives
    7. 7. Simple web servingArduino web server examplesExamples/Ethernet/Web
    8. 8. Web interaction using an network
    9. 9. Why is real time webinteraction important for an arduino?
    10. 10. World domination by arduinoPhoto (CC): Flickr marcus ramberg
    11. 11. @kevinrohling s web sockets controlled robot
    12. 12. Difficult?Photo (CC): Flickr Vrogy
    13. 13. Photo (CC): Flickr InertiaCreeps
    14. 14. Web sockets resourcesW3C web sockets (JS Library) Socket.IO (Django app)
    15. 15. Real time architecture Web Sockets Server
    16. 16. Web serverfrom django_socketio import events, broadcast,broadcast_channel@events.on_subscribe(channel="channel-name")def channel_subscription(request, socket, context,channel): #do some stuff related to a subscription@events.on_message(channel="^channel-name")def message_processor(request, socket, context,message): message = message[0] foo = message["foo"] bar = message["bar"] #do some processing socket.send({"value":some_value}) socket.broadcast({"foo": foo, "value": some_value})
    17. 17. Web browser clientvar room = channel-name;var socket;$(function() { socket = new io.Socket(); socket.connect(); socket.on(connect, function() { socket.subscribe(room); }); socket.on(message, function(data) { console.log(data.value); }); socket.send({room: room, foo: foo, bar: bar});});
    18. 18. Arduino client#include <WebSocketClient.h>// defsWebSocketClient client(server, "/",80);void setup() { Ethernet.begin(mac, ip); delay(1000); if(client.connect()) { client.setDataArrivedDelegate(dataArrived); client.subscribe("channel-name"); delay(1000); } else { while(1) {} }}void dataArrived(WebSocketClient client, String data) { Serial.println("Data Arrived: " + data);}
    19. 19. Arduino clientvoid loop() { client.monitor(); // send a message String message = "{"room":"channel-name", ""foo": bar }"; client.send(message); delay(1000);}
    20. 20. Example – real time sensor data  display Stream live temperature data from two distinct sensors on the network to a web based display that overlays the dataFull code available at
    21. 21. Arduino sensorvoid loop() { char _s[8]; String message = "{"room":"tempsensor", "; message += ""sensor":"; message += sensor_id; message += ", "value":"; message += dtostrf(get_temp(0), 8, 3, _s); message += "}"; client.send(message); delay(100);}
    22. 22. Web serverfrom django_socketio import events, broadcast_channel#other views@events.on_message(channel="^tempsensor")def get_temperature(request, socket, context,message): message = message[0] value = message["value"] sensor_id = message["sensor"] socket.broadcast_channel( {"sensor": sensor_id, "value":value}, channel="tempvalues" )
    23. 23. Discussion$(function() { socket = new io.Socket(); socket.connect(); socket.on(connect, function() { socket.subscribe(tempvalues); CreateTimeline(); }); socket.on(message, function(data) { tempdata[data.sensor].append( new Date().getTime(), data.value ); });});
    24. 24. Static view of data that was presented locally live
    25. 25. Other applicationsScale out sensor networkWeb controlled installationsM2M
    26. 26. Things to try outPusherpusher.comSocket IOsocket.ioArduino Web Sockets
    27. 27. Arduino and the real time