0
Advanced Programming with
#nodecopter
Laurent Eschenauer
#njugbe – August 2013
Laurent Eschenauer
http://eschnou.com
@eschnou
Entrepreneur, software hacker & rock climber. Co-
founder/CTO of @comodit. ...
Parrot AR Drone 2.0 http://ardrone2.parrot.com/
https://www.youtube.com/watch?v=jl5v3bsMH_E
var arDrone = require('ar-drone');
var client = arDrone.createClient();
client.takeoff();
client
.after(5000, function() {...
Nodecopter Berlin, Oct 2012 – Photo by Jan Marsch
Javascript
and
Robotics
client.front(speed)
client.goto(x,y,z,callback)
#1
state estimation
client.on('navdata', callback);
{
"controlState": "CTRL_LANDED",
"flyState": "FLYING_OK",
"batteryPercentage": 59,
"rotation": {
"pitch": -0.165,
"roll": ...
roll
yaw
pitch
vx
vz
vy
distance = speed x time
var dx = vx * dt;
var dy = vy * dt;
x = x + dx * Math.cos(yaw) - dy * Math.sin(yaw);
y = y + dx * Math.sin(yaw) + dy * Mat...
Estimating state by integrating the drone odometry
How to improve?
client.config('detect:detect_type', 12);
{
"visionDetect": {
"xc": [
259
],
"yc": [
233
],
"width": [
340
],
"height": [
454
],
"dist": [
40
],
"orientationAngle":...
From pixels to
coordinates
Back projection from image coordinates to real world
Planar mapping from world to image space
http://www.dandiggins.co.uk/...
1. Predict position
2. Predict tag position
3. Observe real position (if tag visible)
4. Measure error
5. Correct pose est...
Extended Kalman Filter
// Linear algebra in Javascript is possible :-)
var sylvester = require('sylvester');
var Matrix = sylvester.Matrix;
var V...
Odometry only With correction step
How to improve?
VSLAM (Visual - Simultaneous localization and mapping)
Tum AR Drone, autonomous flight with PTAM-based visual navigation f...
#2
motion control
Model of a PID controller
Wikipedia, PID Controller
https://en.wikipedia.org/wiki/File:PID_en_updated_feedback.svg
Shower Nr2 by PhotoAtelier, CC BY 2.0
http://www.flickr.com/photos/glenbledsoe/5845417144/
PID.prototype.getCommand = function(e) {
var time = Date.now();
var dt = (time - this._last_time) / 1000
var de = 0;
// Co...
Moving the drone forward 1 meter with a PID controller
Hovering above a tag using PID contol
https://www.youtube.com/watch?v=PA0voAWISYo
#3
path planning
mission.takeoff()
.hover(500)
.altitude(2)
.forward(2)
.right(2)
.backward(2)
.go({x:0, y:0})
.hover(500)
.land();
mission...
Demo of a fully autonomous flight
https://www.youtube.com/watch?v=wPXsG_fjncM
#4
objects detection
& tracking
Open Source Computer Vision
var cv = require('opencv');
var face_cascade = new cv.CascadeClassifier('frontalface.xml');
function detectFaces() {
cv.re...
Face tracking in WebFlight
https://github.com/eschnou/webflight-copterface
one more thing...
// Land on ctrl-c
process.on('SIGINT', function() {
mission.control().disable();
mission.client().land(function() {
proces...
References
node-ar-drone library
https://github.com/felixge/node-ar-drone
Ardrone 2.0 SDK
https://projects.ardrone.org/
Vi...
Advanced programming with #nodecopter
Advanced programming with #nodecopter
Upcoming SlideShare
Loading in...5
×

Advanced programming with #nodecopter

14,230

Published on

A presentation given on August 7th 2013 at the Belgian NodeJS user group. Discussing autonomous flights topics for the AR Drone 2.0 controlled via NodeJS (nodecopter.com).

Published in: Technology, Art & Photos

Transcript of "Advanced programming with #nodecopter"

  1. 1. Advanced Programming with #nodecopter Laurent Eschenauer #njugbe – August 2013
  2. 2. Laurent Eschenauer http://eschnou.com @eschnou Entrepreneur, software hacker & rock climber. Co- founder/CTO of @comodit. Builder of the #indieweb with @storytlr. Flying #nodecopter robots with #nodejs. ardrone-webflight Browser-based GCS (Ground Control Station) http://eschnou.github.io/ardrone-webflight/ ardrone-autonomy Library to support autonomous flights https://github.com/eschnou/ardrone-autonomy
  3. 3. Parrot AR Drone 2.0 http://ardrone2.parrot.com/
  4. 4. https://www.youtube.com/watch?v=jl5v3bsMH_E
  5. 5. var arDrone = require('ar-drone'); var client = arDrone.createClient(); client.takeoff(); client .after(5000, function() { this.clockwise(0.5); }) .after(3000, function() { this.animate('flipLeft', 15); }) .after(1000, function() { this.stop(); this.land(); });
  6. 6. Nodecopter Berlin, Oct 2012 – Photo by Jan Marsch
  7. 7. Javascript and Robotics
  8. 8. client.front(speed)
  9. 9. client.goto(x,y,z,callback)
  10. 10. #1 state estimation
  11. 11. client.on('navdata', callback);
  12. 12. { "controlState": "CTRL_LANDED", "flyState": "FLYING_OK", "batteryPercentage": 59, "rotation": { "pitch": -0.165, "roll": 0.151, "yaw": -139.978, }, "velocity": { "x": -1.1502487330672384e-11, "y": -5.620504929526593e-12, "z": 0 } }
  13. 13. roll yaw pitch
  14. 14. vx vz vy
  15. 15. distance = speed x time
  16. 16. var dx = vx * dt; var dy = vy * dt; x = x + dx * Math.cos(yaw) - dy * Math.sin(yaw); y = y + dx * Math.sin(yaw) + dy * Math.cos(yaw);
  17. 17. Estimating state by integrating the drone odometry
  18. 18. How to improve?
  19. 19. client.config('detect:detect_type', 12);
  20. 20. { "visionDetect": { "xc": [ 259 ], "yc": [ 233 ], "width": [ 340 ], "height": [ 454 ], "dist": [ 40 ], "orientationAngle": [ 3.7453956604003906 ] }
  21. 21. From pixels to coordinates
  22. 22. Back projection from image coordinates to real world Planar mapping from world to image space http://www.dandiggins.co.uk/arlib-6.html
  23. 23. 1. Predict position 2. Predict tag position 3. Observe real position (if tag visible) 4. Measure error 5. Correct pose estimate
  24. 24. Extended Kalman Filter
  25. 25. // Linear algebra in Javascript is possible :-) var sylvester = require('sylvester'); var Matrix = sylvester.Matrix; var Vector = sylvester.Vector; s = Matrix.I(3).subtract(K.multiply(H)).multiply(s);
  26. 26. Odometry only With correction step
  27. 27. How to improve?
  28. 28. VSLAM (Visual - Simultaneous localization and mapping) Tum AR Drone, autonomous flight with PTAM-based visual navigation for the Parrot AR.Drone. http://www.ros.org/wiki/tum_ardrone
  29. 29. #2 motion control
  30. 30. Model of a PID controller Wikipedia, PID Controller https://en.wikipedia.org/wiki/File:PID_en_updated_feedback.svg
  31. 31. Shower Nr2 by PhotoAtelier, CC BY 2.0 http://www.flickr.com/photos/glenbledsoe/5845417144/
  32. 32. PID.prototype.getCommand = function(e) { var time = Date.now(); var dt = (time - this._last_time) / 1000 var de = 0; // Compute de (error derivation) if (this._last_error < Infinity) { de = (e - this._last_error) / dt; } // Integrate error this._error_sum += e * dt; // Update our trackers this._last_time = time; this._last_error = e; // Compute commands var command = this._kp * e + this._ki * this._error_sum + this._kd * de; return command; }
  33. 33. Moving the drone forward 1 meter with a PID controller
  34. 34. Hovering above a tag using PID contol https://www.youtube.com/watch?v=PA0voAWISYo
  35. 35. #3 path planning
  36. 36. mission.takeoff() .hover(500) .altitude(2) .forward(2) .right(2) .backward(2) .go({x:0, y:0}) .hover(500) .land(); mission.run(function (err, result) { if (err) { mission.client().stop(); mission.client().land(); } else { process.exit(0); } })
  37. 37. Demo of a fully autonomous flight https://www.youtube.com/watch?v=wPXsG_fjncM
  38. 38. #4 objects detection & tracking
  39. 39. Open Source Computer Vision
  40. 40. var cv = require('opencv'); var face_cascade = new cv.CascadeClassifier('frontalface.xml'); function detectFaces() { cv.readImage( lastPng, function(err, im) { var opts = {}; face_cascade.detectMultiScale(im, function(err, faces) { var face; var biggestFace; for(var k = 0; k < faces.length; k++) { face = faces[k]; if( !biggestFace || biggestFace.width < face.width ) { biggestFace = face; } } io.sockets.emit('face', face); }
  41. 41. Face tracking in WebFlight https://github.com/eschnou/webflight-copterface
  42. 42. one more thing...
  43. 43. // Land on ctrl-c process.on('SIGINT', function() { mission.control().disable(); mission.client().land(function() { process.exit(0); }); });
  44. 44. References node-ar-drone library https://github.com/felixge/node-ar-drone Ardrone 2.0 SDK https://projects.ardrone.org/ Visual Navigation for Flying Robots http://vision.in.tum.de/teaching/ss2013/visnav2013
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×