Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building droids with JavasSript


Published on

JavaScript is finding its way further and further out of the browser. Only a couple of years ago, if someone had said they wanted to build robots only using JS you'd think they were crazy. Having tried it at the time those naysayers were correct - it was a disaster.

Recently, particularly as a result of the nodebots project, JS Robotics has started to come of age and it's now possible to build simple robots using JavaScript for the majority of the stack - from everything from control and sensors to motors to lights, AI and computer vision.

This talk will give an overview of what's currently possible, where the current gotchas are, how to get started and have some interactive elements that can be played with during or after the session.

This talk was given at MelbJS as well as NodeBots Day Sydney in August 2013.

It is aimed at web developers who may be very confident with JavaScript but who have never touched hardware to show how straightforward it is to start playing with hardware now.

Published in: Technology
  • Be the first to comment

Building droids with JavasSript

  1. 1. Building droids with JavaScript Andrew Fisher @ajfisher MelbJS, 14 August 2013 Hi! My name is Andrew Fisher and I’m an interaction developer. Tonight I want to talk to you about playing with hardware and building droids with javascript for the next 15 minutes or so.
  2. 2. Atwood’s Law We all laugh about the observation behind Atwood’s Law - mostly because we’ve all been culprits of it at one point or another and I’m no different - but the reason for this is that JS is becoming more and more capable as time goes on. Historically when it comes to hardware however that hasn’t been the case. So if you wanted to work with hardware you really had to go back to languages like C to work there. As someone who grew up writing C though, I never really had a problem with this, I’d just switch back and forth between languages depending on what part of the hardware stack I was working on.
  3. 3. JS all the REAL things flickr (cc) Quasimondo Recently though, some great work has been done in the node community getting JS to work with hardware - to the point where working with hardware using javascript is now extremely viable. So tonight I want to talk to you about that and how all of you can all start working with hardware with JS and along the way we’ll bump into some robots. Image:
  4. 4. TODO 1. Different ways to interface with hardware 2. The JS / hardware stack 3.Applications and examples So to do this, we’ll look at. How we can interact with hardware. Some code to show what a common JS hardware stack starts to look like Then we’ll play with some examples.
  5. 5. Working with hardware flickr (cc) Oskay Let’s start by looking conceptually at how we develop with hardware. I think about this at sort of three levels. At the metal At the device And somewhere in between. Image:
  6. 6. Working with the metal flickr (cc) Wonderlane When we work with the metal we are usually writing code specifically for a chip or board at a very low level. It usually means writing C or C++. You can get a bit of abstraction with hardware libraries but even with good libraries you need to write a lot of code. How many of you have written C++? How many of you do for work or for “real”? So pretty much no one. Does anyone find it fun? No didn’t think so.... The other thing to consider here is you have insane limitations. It’s like going back to the 80s - an arduino for example has TWO KILOBYTES of RAM. Image:
  7. 7. Limitations { "motor": 100, "id": 1, "dir": "forward", "period": 10, “position”: 78, “voltmax”: 5.3, } This means that for example, a JSON string like this: Will probably overrun a buffer in memory if you try and parse it as actual json object in a programme and not character by character. So there’s some serious limitations here at the metal. However these sorts of constraints are really good for your coding discipline and making you think about performance at a very low level so I would thoroughly encourage you to play here as I know both my JS and Python have both improved by working with such extreme limitations.
  8. 8. Working low level flickr (cc) lisovy Playing at the chip level gives you masses of control but your iteration cycles take time and you’re constantly thinking about things like memory and garbage collection and not frying a chip. Debugging is also generally a nightmare. Image:
  9. 9. Device hacking flickr (cc) Roo Reynolds At the other end of the spectrum there are devices that are already made like this little printer from Berg. This is hardware with APIs - maybe they are via the network, maybe via an embedded web server or some sort of serial protocol. However it works, you’re basically interacting with the hardware given to you as a service. You can build clever ways of working with the service but fundamentally you can’t change the way the hardware works. IMage:
  10. 10. AR.Drone flickr (cc) neeravbhatt A good example of this is the AR Drone - you can play with using a node library to interact with the copter directly to do some really interesting things but you can’t fundamentally change the hardware - you can’t suddenly change this helicopter and turn it into a submarine for example. Having said that, people have done some fantastic things at this level with this device.
  11. 11. Blink(1) flickr (cc) todbot The Blink(1) is an interactive light which can be worked with via a library with various applications that you might want to create as well such as task or status indicators. These guys are doing a new version on kickstarter at the moment as well if you’re interested. So there’s plenty of very interesting devices you can grab hold of and start playing about with really easily.
  12. 12. Hacking in comfort flickr (cc) Zack Hoeken In between these extremes though is a new category where you can work with and change the hardware but you can still use high level languages to do it. This is a great combo as it allows for rapid prototyping but also makes hardware more accessible to non-embedded application designers. A raft of options over the last 12-18 months have appeared in this space such as: Image:
  13. 13. Raspberry PI flickr (cc) GijsbertPeijs The Raspberry PI which seems to have found a firm home within the python community. image:
  14. 14. Beaglebone black flickr (cc) stfnix The beagle bone which is still in its early days but seems to be aiming firmly at JS devs. It even ships with nodejs installed and running by default with Cloud9 IDE available to hack away on it directly. Hardware connectivity is a little light but once that is a bit better I think this may become the tool of choice for web devs crossing over to hardware. Image:
  15. 15. Arduino But the device that has the best support with a huge community and masses of compatible hardware is the arduino using a protocol layer called Firmata which we’ll look at in more detail in a moment.
  16. 16. Sketching in hardware flickr (cc) Camille Moussette So these approaches give you the ability to prototype rapidly as well as work almost directly with the hardware. There are some limitations around some hardware but to start off with and for some prototyping that doesn’t matter - let’s make something.
  17. 17. First steps We’re going to focus on arduino and a tool called firmata. This is an arduino - they cost about $30, they are awesome and come in many different forms for different applications from small to large. They run off USB or a battery so it’s hard to blow them up and even harder to electrocute yourself. There is also huge amounts of community information about them. Firmata is a program or sketch in arduino-speak that you load onto the arduino which allows you to communicate with it over USB serial and tell it to do things like turn a pin on or off, take a reading etc. Firmata is a neat idea as it essentially exposes most of the features of the arduino but via a protocol so now you can control it from somewhere else.
  18. 18. How to Get an arduino ( Install the arduino IDE ( Load the firmata firmware sketch (from the IDE) Install nodejs ( Install node firmata (npm install firmata) Write an application (vim something...) So here’s a quick run through how we’ll make our first hardware application. We’ll get an arduino, install the arduino IDE and load the firmata sketch onto the arduino. On the node side, we install firmata so we can communicate with the arduino. I won’t do these as I expect you can all buy things online and install software that’s well documented. Once we have all of this we can get on with making something.
  19. 19. Hardware hello world flickr (cc) pj_vanf So we plug in an LED (this one is bigger than normal so you can see it easily) and then we’ll need to write some code. I’ve got a little bit of scaffolding here so I don’t have to type everything. But you can see it just creates a board and connects it. Now it’s connected we can tell the board we want to assign a pin as an output then we’ll tell it to make the pin go HIGH which means send it some volts - we do that with the digitalWrite command. Then we can make it go LOW and as you can see as I do this it turns the LED on and off. Code at
  20. 20. Web thing hello world Demoed live So we’re all web devs here so let’s not stray too far away from our comfort zone. Let’s wrap a web interface around this light so we can turn it on and off with a click of a button on a page. We can’t do this interactively very easily so I’ll show you some code. This is a bit of overkill but we’re going to use express and web sockets so it’s a bit more realtime but also it will lay the foundation for what I’ll show you next. We set up the socket messages on the server to switch things on and off . On the client side all our HTML and JS is doing is just sending messages on click so nothing too interesting. And there we go - button click to turn a light on and off via a web browser. I can hear the whirring of brains considering home interaction interfaces as I stand here.
  21. 21. The JS hardware stack Arduino (Sensors and actuators) Firmata (Communications protocol) NodeJS (Application logic) WS/HTTP (Network & security protocols) Client (UI, Input, Visualisation) So this is what the JS hardware stack looks like. We’ve got an arduino with sensors and actuators. Firmata which provides the communications protocol. The NodeJS application gives us application logic and integration with other libraries such as johnny-five which allows us to turn hardware into objects. Networking is provided over http or websockets and this can give us security and encryption. Finally the web page for the client gives as input methods, a data viz layer as well as user interface.
  22. 22. Easy install npm install johnny-five express So this stack can be created with pretty much just this command plus an arduino with firmata on it in about 2 minutes. Which is pretty cool. And it’s robust enough now that it’s most likely going to work the first time you try it.
  23. 23. Looking for droids? flickr (cc) solo So let’s look at an application of this. I don’t have any flying robots due to space constraints and not wanting to take out any eyes but go get an ar-drone and check out for some fun. Here’s a ground based option however. Image:
  24. 24. Nodebot This is a bot I built from scratch - The core was just some metal and some motors. Originally last year I built it all in C. Then I layered a web interface onto it earlier this year for control via a browser and did that using node talking to my C code. Over the last couple of months I’ve ported it to be 100% javascript.
  25. 25. Nodebot architecture RPiArduino Battery 4WD motor control Sensors Firmata NodeJS - Express - Johnny Five - Open CV - Socket IO WiFi Camera 4WD motor control Architecturally this is what it looks like: The arduino deals with the hardware using firmata. A raspberry pi runs a node server and communicates to the arduino over a serial connection. The webcam provides a video feed using node-opencv as a stream A web interface shows the video and provides input control to drive the bot using the device API and the gyro in your phone or tablet or keyboard on your desktop.
  26. 26. Nodebot code var motor = new five.Motor(pins: {dir:11, pwm: 3}); motor.forward(150); motor.stop(); motor.reverse(150); The code uses Johnny Five which is a node library on top of firmata that abstracts common hardware into objects to make it super simple to work with. For example you can define a motor then tell it to go forwards or backwards. Very very highly recommended. Besides that, we set up the code, wait for someone to connect over web sockets when they do we send data from the sensors and the webcam to the web client. And we send control messages to the the motors from the gyro in the phone.
  27. 27. Let’s drive Demoed live In action then. Here we go.
  28. 28. Let’s drive The audience at MelbJS So this is pretty much 100% web tech driving this robot around.
  29. 29. Go and make (start here for everything) Freetronics (supportAussie / Melbourne hardware designers) Make magazine (Great ideas and skill development) Johnny Five (Brilliant JS library to work with hardware) So there’s plenty of ways all of you can start playing with hardware. You can go low level with things like C or high level with already built devices. But you can also sit in the middle and prototype as well. The stack I’ve talked about is very solid, it works really well and will allow you to experiment very efficiently. You can all do this and I’m sure build a robot much better than this one I’ve got here. Don’t be worried about playing with hardware - it’s loads of fun - just stay away from mains electricity. These links should get you started.
  30. 30. Nodebots Day Melbs Saturday October 12, 2013 Play with hardware Build web controlled things Now if this has got you interested in playing with hardware I have some news for you. A couple of weeks ago we ran a Nodebots day in Sydney with about 40 web devs who all came together to learn about and play with hardware. As it was really successful - you can see some pics there, we’re going to run one down here in Melbourne as well. It is aimed at devs who haven’t had much hardware experience and will be a whole day event on Saturday October 12. Thanks to Mark, Seek has provided us some space to use. I will have some more info soon so watch MelbJS and my twitter feeds for info over the next week around pricing and availability etc.
  31. 31. Building droids with JavaScript Andrew Fisher @ajfisher (twitter, github, adn etc) MelbJS, 14 August 2013 So thanks a lot and if you have any questions now or after please let me know.