JavaScript is finding its way further and further out of the browser. Only a couple of years ago, if someone had said they wanted to build robots only using JS you'd think they were crazy. Having tried it at the time those naysayers were correct - it was a disaster. Recently, particularly as a result of the nodebots project, JS Robotics has started to come of age and it's now possible to build simple robots using JavaScript for the majority of the stack - from everything from control and sensors to motors to lights, AI and computer vision. This talk will give an overview of what's currently possible, where the current gotchas are, how to get started and have some interactive elements that can be played with during or after the session. This talk was given at MelbJS as well as NodeBots Day Sydney in August 2013. It is aimed at web developers who may be very confident with JavaScript but who have never touched hardware to show how straightforward it is to start playing with hardware now.

1. Building droids with
JavaScript
Andrew Fisher @ajfisher
MelbJS, 14 August 2013
Hi! My name is Andrew Fisher and I’m an interaction developer. Tonight I want to talk to you
about playing with hardware and building droids with javascript for the next 15 minutes or
so.

2. Atwood’s Law
We all laugh about the observation behind Atwood’s Law - mostly because we’ve all been
culprits of it at one point or another and I’m no different - but the reason for this is that JS is
becoming more and more capable as time goes on. Historically when it comes to hardware
however that hasn’t been the case. So if you wanted to work with hardware you really had to
go back to languages like C to work there. As someone who grew up writing C though, I
never really had a problem with this, I’d just switch back and forth between languages
depending on what part of the hardware stack I was working on.

3. JS all the REAL things
flickr (cc) Quasimondo
Recently though, some great work has been done in the node community getting JS to work
with hardware - to the point where working with hardware using javascript is now extremely
viable.
So tonight I want to talk to you about that and how all of you can all start working with
hardware with JS and along the way we’ll bump into some robots.
Image: http://www.flickr.com/photos/quasimondo/5203908319

4. TODO
1. Different ways to interface with hardware
2. The JS / hardware stack
3.Applications and examples
So to do this, we’ll look at.
How we can interact with hardware.
Some code to show what a common JS hardware stack starts to look like
Then we’ll play with some examples.

5. Working with hardware
flickr (cc) Oskay
Let’s start by looking conceptually at how we develop with hardware. I think about this at sort
of three levels.
At the metal
At the device
And somewhere in between.
Image: http://www.flickr.com/photos/oskay/2310115216/

6. Working with the metal
flickr (cc) Wonderlane
When we work with the metal we are usually writing code specifically for a chip or board at a
very low level. It usually means writing C or C++. You can get a bit of abstraction with
hardware libraries but even with good libraries you need to write a lot of code.
How many of you have written C++? How many of you do for work or for “real”? So pretty
much no one. Does anyone find it fun? No didn’t think so....
The other thing to consider here is you have insane limitations. It’s like going back to the 80s
- an arduino for example has TWO KILOBYTES of RAM.
Image: http://www.flickr.com/photos/wonderlane/3198166347

7. Limitations
{
"motor": 100,
"id": 1,
"dir": "forward",
"period": 10,
“position”: 78,
“voltmax”: 5.3,
}
This means that for example, a JSON string like this:
Will probably overrun a buffer in memory if you try and parse it as actual json object in a
programme and not character by character. So there’s some serious limitations here at the
metal.
However these sorts of constraints are really good for your coding discipline and making you
think about performance at a very low level so I would thoroughly encourage you to play here
as I know both my JS and Python have both improved by working with such extreme
limitations.

8. Working low level
flickr (cc) lisovy
Playing at the chip level gives you masses of control but your iteration cycles take time and
you’re constantly thinking about things like memory and garbage collection and not frying a
chip. Debugging is also generally a nightmare.
Image: http://www.flickr.com/photos/lisovy/4677688431/

9. Device hacking
flickr (cc) Roo Reynolds
At the other end of the spectrum there are devices that are already made like this little printer
from Berg. This is hardware with APIs - maybe they are via the network, maybe via an
embedded web server or some sort of serial protocol.
However it works, you’re basically interacting with the hardware given to you as a service.
You can build clever ways of working with the service but fundamentally you can’t change the
way the hardware works.
IMage: http://www.flickr.com/photos/rooreynolds/9350631793

10. AR.Drone
flickr (cc) neeravbhatt
A good example of this is the AR Drone - you can play with using a node library to interact
with the copter directly to do some really interesting things but you can’t fundamentally
change the hardware - you can’t suddenly change this helicopter and turn it into a submarine
for example.
Having said that, people have done some fantastic things at this level with this device.
http://www.flickr.com/photos/neeravbhatt/6885424870

11. Blink(1)
flickr (cc) todbot
The Blink(1) is an interactive light which can be worked with via a library with various
applications that you might want to create as well such as task or status indicators. These
guys are doing a new version on kickstarter at the moment as well if you’re interested.
So there’s plenty of very interesting devices you can grab hold of and start playing about with
really easily.
http://www.flickr.com/photos/todbot/8245679238

12. Hacking in comfort
flickr (cc) Zack Hoeken
In between these extremes though is a new category where you can work with and change
the hardware but you can still use high level languages to do it. This is a great combo as it
allows for rapid prototyping but also makes hardware more accessible to non-embedded
application designers.
A raft of options over the last 12-18 months have appeared in this space such as:
Image: http://www.flickr.com/photos/hoeken/3519955473/

13. Raspberry PI
flickr (cc) GijsbertPeijs
The Raspberry PI which seems to have found a firm home within the python community.
image: http://www.flickr.com/photos/gijsbertpeijs/7988262046

14. Beaglebone black
flickr (cc) stfnix
The beagle bone which is still in its early days but seems to be aiming firmly at JS devs. It
even ships with nodejs installed and running by default with Cloud9 IDE available to hack
away on it directly. Hardware connectivity is a little light but once that is a bit better I think
this may become the tool of choice for web devs crossing over to hardware.
Image: http://www.flickr.com/photos/stfnix/9176951660/sizes/o/in/photostream/

15. Arduino
But the device that has the best support with a huge community and masses of compatible
hardware is the arduino using a protocol layer called Firmata which we’ll look at in more
detail in a moment.

16. Sketching in hardware
flickr (cc) Camille Moussette
So these approaches give you the ability to prototype rapidly as well as work almost directly
with the hardware. There are some limitations around some hardware but to start off with
and for some prototyping that doesn’t matter - let’s make something.
http://www.flickr.com/photos/9225693@N08/6051548279

17. First steps
We’re going to focus on arduino and a tool called firmata.
This is an arduino - they cost about $30, they are awesome and come in many different
forms for different applications from small to large. They run off USB or a battery so it’s hard
to blow them up and even harder to electrocute yourself. There is also huge amounts of
community information about them.
Firmata is a program or sketch in arduino-speak that you load onto the arduino which allows
you to communicate with it over USB serial and tell it to do things like turn a pin on or off,
take a reading etc. Firmata is a neat idea as it essentially exposes most of the features of the
arduino but via a protocol so now you can control it from somewhere else.

18. How to
Get an arduino (littlebitdelectronics.com.au)
Install the arduino IDE (arduino.cc)
Load the firmata firmware sketch (from the IDE)
Install nodejs (nodejs.org)
Install node firmata (npm install firmata)
Write an application (vim something...)
So here’s a quick run through how we’ll make our first hardware application.
We’ll get an arduino, install the arduino IDE and load the firmata sketch onto the arduino.
On the node side, we install firmata so we can communicate with the arduino.
I won’t do these as I expect you can all buy things online and install software that’s well
documented.
Once we have all of this we can get on with making something.

19. Hardware hello world
flickr (cc) pj_vanf
So we plug in an LED (this one is bigger than normal so you can see it easily) and then we’ll
need to write some code.
I’ve got a little bit of scaffolding here so I don’t have to type everything. But you can see it just
creates a board and connects it. Now it’s connected we can tell the board we want to assign a
pin as an output then we’ll tell it to make the pin go HIGH which means send it some volts -
we do that with the digitalWrite command. Then we can make it go LOW and as you can see as
I do this it turns the LED on and off.
Code at http://github.com/ajfisher/nbs
http://www.flickr.com/photos/vanf/5210360116

20. Web thing hello world
http://github.com/ajfisher/nbs
Demoed live
So we’re all web devs here so let’s not stray too far away from our comfort zone. Let’s wrap a
web interface around this light so we can turn it on and off with a click of a button on a page.
We can’t do this interactively very easily so I’ll show you some code.
This is a bit of overkill but we’re going to use express and web sockets so it’s a bit more
realtime but also it will lay the foundation for what I’ll show you next.
We set up the socket messages on the server to switch things on and off . On the client side
all our HTML and JS is doing is just sending messages on click so nothing too interesting.
And there we go - button click to turn a light on and off via a web browser. I can hear the
whirring of brains considering home interaction interfaces as I stand here.

21. The JS hardware stack
Arduino (Sensors and actuators)
Firmata (Communications protocol)
NodeJS (Application logic)
WS/HTTP (Network & security protocols)
Client (UI, Input, Visualisation)
So this is what the JS hardware stack looks like.
We’ve got an arduino with sensors and actuators. Firmata which provides the
communications protocol. The NodeJS application gives us application logic and integration
with other libraries such as johnny-five which allows us to turn hardware into objects.
Networking is provided over http or websockets and this can give us security and encryption.
Finally the web page for the client gives as input methods, a data viz layer as well as user
interface.

22. Easy install
npm install johnny-five express socket.io
So this stack can be created with pretty much just this command plus an arduino with firmata
on it in about 2 minutes.
Which is pretty cool. And it’s robust enough now that it’s most likely going to work the first
time you try it.

23. Looking for droids?
flickr (cc) solo
So let’s look at an application of this.
I don’t have any flying robots due to space constraints and not wanting to take out any eyes
but go get an ar-drone and check out nodecopter.org for some fun.
Here’s a ground based option however.
Image: http://www.flickr.com/photos/donsolo/3768623542/

24. Nodebot
This is a bot I built from scratch - The core was just some metal and some motors.
Originally last year I built it all in C. Then I layered a web interface onto it earlier this year for
control via a browser and did that using node talking to my C code. Over the last couple of
months I’ve ported it to be 100% javascript.

25. Nodebot architecture
RPiArduino
Battery
4WD motor control
Sensors
Firmata
NodeJS
- Express
- Johnny Five
- Open CV
- Socket IO
WiFi
Camera
4WD motor control
Architecturally this is what it looks like:
The arduino deals with the hardware using firmata.
A raspberry pi runs a node server and communicates to the arduino over a serial connection.
The webcam provides a video feed using node-opencv as a stream
A web interface shows the video and provides input control to drive the bot using the device
API and the gyro in your phone or tablet or keyboard on your desktop.

26. Nodebot code
var motor = new five.Motor(pins: {dir:11, pwm: 3});
motor.forward(150);
motor.stop();
motor.reverse(150);
The code uses Johnny Five which is a node library on top of firmata that abstracts common
hardware into objects to make it super simple to work with. For example you can define a
motor then tell it to go forwards or backwards. Very very highly recommended.
Besides that, we set up the code, wait for someone to connect over web sockets when they do
we send data from the sensors and the webcam to the web client. And we send control
messages to the the motors from the gyro in the phone.

27. Let’s drive
http://github.com/ajfisher/ajnodebot
http://www.youtube.com/watch?v=Jkbcn1vA7yk
Demoed live
In action then. Here we go.

28. Let’s drive
The audience at MelbJS
So this is pretty much 100% web tech driving this robot around.

29. Go and make
Arduino.cc
(start here for everything)
Freetronics
(supportAussie / Melbourne hardware designers)
Make magazine
(Great ideas and skill development)
Johnny Five
(Brilliant JS library to work with hardware)
So there’s plenty of ways all of you can start playing with hardware.
You can go low level with things like C or high level with already built devices. But you can
also sit in the middle and prototype as well.
The stack I’ve talked about is very solid, it works really well and will allow you to experiment
very efficiently. You can all do this and I’m sure build a robot much better than this one I’ve
got here.
Don’t be worried about playing with hardware - it’s loads of fun - just stay away from mains
electricity.
These links should get you started.

30. Nodebots Day Melbs
Saturday October 12, 2013
Play with hardware
Build web controlled things
Now if this has got you interested in playing with hardware I have some news for you.
A couple of weeks ago we ran a Nodebots day in Sydney with about 40 web devs who all
came together to learn about and play with hardware. As it was really successful - you can
see some pics there, we’re going to run one down here in Melbourne as well.
It is aimed at devs who haven’t had much hardware experience and will be a whole day event
on Saturday October 12.
Thanks to Mark, Seek has provided us some space to use. I will have some more info soon so
watch MelbJS and my twitter feeds for info over the next week around pricing and availability
etc.

31. Building droids with
JavaScript
Andrew Fisher @ajfisher (twitter, github, adn etc)
MelbJS, 14 August 2013
So thanks a lot and if you have any questions now or after please let me know.