Advanced programming with #nodecopter

18,097 views

Published on

A presentation given on August 7th 2013 at the Belgian NodeJS user group. Discussing autonomous flights topics for the AR Drone 2.0 controlled via NodeJS (nodecopter.com).

Published in: Technology, Art & Photos

Advanced programming with #nodecopter

  1. 1. Advanced Programming with #nodecopter Laurent Eschenauer #njugbe – August 2013
  2. 2. Laurent Eschenauer http://eschnou.com @eschnou Entrepreneur, software hacker & rock climber. Co- founder/CTO of @comodit. Builder of the #indieweb with @storytlr. Flying #nodecopter robots with #nodejs. ardrone-webflight Browser-based GCS (Ground Control Station) http://eschnou.github.io/ardrone-webflight/ ardrone-autonomy Library to support autonomous flights https://github.com/eschnou/ardrone-autonomy
  3. 3. Parrot AR Drone 2.0 http://ardrone2.parrot.com/
  4. 4. https://www.youtube.com/watch?v=jl5v3bsMH_E
  5. 5. var arDrone = require('ar-drone'); var client = arDrone.createClient(); client.takeoff(); client .after(5000, function() { this.clockwise(0.5); }) .after(3000, function() { this.animate('flipLeft', 15); }) .after(1000, function() { this.stop(); this.land(); });
  6. 6. Nodecopter Berlin, Oct 2012 – Photo by Jan Marsch
  7. 7. Javascript and Robotics
  8. 8. client.front(speed)
  9. 9. client.goto(x,y,z,callback)
  10. 10. #1 state estimation
  11. 11. client.on('navdata', callback);
  12. 12. { "controlState": "CTRL_LANDED", "flyState": "FLYING_OK", "batteryPercentage": 59, "rotation": { "pitch": -0.165, "roll": 0.151, "yaw": -139.978, }, "velocity": { "x": -1.1502487330672384e-11, "y": -5.620504929526593e-12, "z": 0 } }
  13. 13. roll yaw pitch
  14. 14. vx vz vy
  15. 15. distance = speed x time
  16. 16. var dx = vx * dt; var dy = vy * dt; x = x + dx * Math.cos(yaw) - dy * Math.sin(yaw); y = y + dx * Math.sin(yaw) + dy * Math.cos(yaw);
  17. 17. Estimating state by integrating the drone odometry
  18. 18. How to improve?
  19. 19. client.config('detect:detect_type', 12);
  20. 20. { "visionDetect": { "xc": [ 259 ], "yc": [ 233 ], "width": [ 340 ], "height": [ 454 ], "dist": [ 40 ], "orientationAngle": [ 3.7453956604003906 ] }
  21. 21. From pixels to coordinates
  22. 22. Back projection from image coordinates to real world Planar mapping from world to image space http://www.dandiggins.co.uk/arlib-6.html
  23. 23. 1. Predict position 2. Predict tag position 3. Observe real position (if tag visible) 4. Measure error 5. Correct pose estimate
  24. 24. Extended Kalman Filter
  25. 25. // Linear algebra in Javascript is possible :-) var sylvester = require('sylvester'); var Matrix = sylvester.Matrix; var Vector = sylvester.Vector; s = Matrix.I(3).subtract(K.multiply(H)).multiply(s);
  26. 26. Odometry only With correction step
  27. 27. How to improve?
  28. 28. VSLAM (Visual - Simultaneous localization and mapping) Tum AR Drone, autonomous flight with PTAM-based visual navigation for the Parrot AR.Drone. http://www.ros.org/wiki/tum_ardrone
  29. 29. #2 motion control
  30. 30. Model of a PID controller Wikipedia, PID Controller https://en.wikipedia.org/wiki/File:PID_en_updated_feedback.svg
  31. 31. Shower Nr2 by PhotoAtelier, CC BY 2.0 http://www.flickr.com/photos/glenbledsoe/5845417144/
  32. 32. PID.prototype.getCommand = function(e) { var time = Date.now(); var dt = (time - this._last_time) / 1000 var de = 0; // Compute de (error derivation) if (this._last_error < Infinity) { de = (e - this._last_error) / dt; } // Integrate error this._error_sum += e * dt; // Update our trackers this._last_time = time; this._last_error = e; // Compute commands var command = this._kp * e + this._ki * this._error_sum + this._kd * de; return command; }
  33. 33. Moving the drone forward 1 meter with a PID controller
  34. 34. Hovering above a tag using PID contol https://www.youtube.com/watch?v=PA0voAWISYo
  35. 35. #3 path planning
  36. 36. mission.takeoff() .hover(500) .altitude(2) .forward(2) .right(2) .backward(2) .go({x:0, y:0}) .hover(500) .land(); mission.run(function (err, result) { if (err) { mission.client().stop(); mission.client().land(); } else { process.exit(0); } })
  37. 37. Demo of a fully autonomous flight https://www.youtube.com/watch?v=wPXsG_fjncM
  38. 38. #4 objects detection & tracking
  39. 39. Open Source Computer Vision
  40. 40. var cv = require('opencv'); var face_cascade = new cv.CascadeClassifier('frontalface.xml'); function detectFaces() { cv.readImage( lastPng, function(err, im) { var opts = {}; face_cascade.detectMultiScale(im, function(err, faces) { var face; var biggestFace; for(var k = 0; k < faces.length; k++) { face = faces[k]; if( !biggestFace || biggestFace.width < face.width ) { biggestFace = face; } } io.sockets.emit('face', face); }
  41. 41. Face tracking in WebFlight https://github.com/eschnou/webflight-copterface
  42. 42. one more thing...
  43. 43. // Land on ctrl-c process.on('SIGINT', function() { mission.control().disable(); mission.client().land(function() { process.exit(0); }); });
  44. 44. References node-ar-drone library https://github.com/felixge/node-ar-drone Ardrone 2.0 SDK https://projects.ardrone.org/ Visual Navigation for Flying Robots http://vision.in.tum.de/teaching/ss2013/visnav2013

×