Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Robocalypse: Controlling Robots with a Kinect

147 views

Published on

Building Arduino robots is pretty cool, but being able to program a robot in Javascript is even better. However, what is even more awesome is the ability to control a robot with Kinect gestures. This presentation provides attendees with a brief introduction to the awesome world of NodeJs, Johnny-Five and Socket.io. The session will deliver an overview of a Kinect gesture detection library and how to connect it to a node-bot.

Published in: Engineering

Robocalypse: Controlling Robots with a Kinect

  1. 1. Smart Homes Made Smarter With JS Joel Lord Guidebook app: NEPHP2016
  2. 2. www.spiria.com Robocalypse: Controlling Nodebots with a Kinect PresentedBy JOEL LORD North EastPHP 2016,Charlottetown,PE August 5th,2016
  3. 3. @joel__lord #NEPHP JOEL LORD About me,eh? • Javascript Junkie • Technology enthusiast • Tinkerer
  4. 4. @joel__lord #NEPHP AGENDA Why The Goals What Exists What I wanted Gestures 101 Demo
  5. 5. @joel__lord #NEPHP MY NODEBOT Say hello to my little friend
  6. 6. Why? CONTROLLING A ROBOT WITH A KINECT
  7. 7. @joel__lord #NEPHP Why • Worked with Nodebots for a while now @joel__lord
  8. 8. @joel__lord #NEPHP Why • Worked with Nodebots for a while now • Wanted to explore other possibilities of NodeJs @joel__lord
  9. 9. @joel__lord #NEPHP Why • Worked with Nodebots for a while now • Wanted to explore other possibilities of NodeJs • Because controlling a robot with a Kinect is pretty cool @joel__lord
  10. 10. @joel__lord #NEPHP Why • Worked with Nodebots for a while now • Wanted to explore other possibilities of NodeJs • Because controlling a robot with a Kinect is pretty cool • But mostly… for fame ! @joel__lord
  11. 11. Goals CONTROLLING A ROBOT WITH A KINECT
  12. 12. @joel__lord #NEPHP Goals • Needed a way to use events to tell the robot to perform an action @joel__lord
  13. 13. @joel__lord #NEPHP Goals • Needed a way to use events to tell the robot to perform an action • Gestures seemed the best way to achieve this @joel__lord
  14. 14. What already exists CONTROLLING A ROBOT WITH A KINECT
  15. 15. @joel__lord #NEPHP TheNodebot Stack • NodeJs (https://nodejs.org/) @joel__lord
  16. 16. @joel__lord #NEPHP TheNodebot Stack • NodeJs (https://nodejs.org/) • Socket.io (http://socket.io/) @joel__lord
  17. 17. @joel__lord #NEPHP TheNodebot Stack • NodeJs (https://nodejs.org/) • Socket.io (http://socket.io/) • Johnny-Five (http://johnny-five.io/) @joel__lord
  18. 18. A little parenthesis JOHNNY-FIVE?
  19. 19. @joel__lord #NEPHP Johnny-Five • Uses Javascript to control an Arduino (or other boards) via Standard Firmata • Also works with a bunch of platforms like Tessel, Raspberry Pi, Particle • Uses a standard API for all the platforms @joel__lord
  20. 20. @joel__lord #NEPHP Johnny-Five’s Hello World var five = require("johnny-five"); var board = new five.Board(); board.on("ready", function() { // Create a standard `led` // component instance var led = new five.Led(13); // "blink" the led in 500ms // on-off phase periods led.blink(500); }); @joel__lord
  21. 21. Back to our regular programming
  22. 22. @joel__lord #NEPHP OpenNI and node-openni • Connects the Kinect to NodeJs using OpenNI @joel__lord
  23. 23. @joel__lord #NEPHP OpenNI and node-openni • Connects the Kinect to NodeJs using OpenNI • Very basic event for almost everything @joel__lord
  24. 24. @joel__lord #NEPHP OpenNI and node-openni • Connects the Kinect to NodeJs using OpenNI • Very basic event for almost everything • No gestures ! @joel__lord
  25. 25. What I needed CONTROLLING A ROBOT WITH A KINECT
  26. 26. @joel__lord #NEPHP What Ineeded • Support for gestures @joel__lord
  27. 27. @joel__lord #NEPHP What Ineeded • Support for gestures • Support for a full « skeleton » @joel__lord
  28. 28. Gestures 101 CONTROLLING A ROBOT WITH A KINECT
  29. 29. @joel__lord #NEPHP Gestures 101 • Is the initial condition met? @joel__lord
  30. 30. @joel__lord #NEPHP Gestures 101 • Is the initial condition met? • Is the condition still met? @joel__lord
  31. 31. @joel__lord #NEPHP Gestures 101 • Is the initial condition met? • Is the condition still met? • Is the final condition met? @joel__lord
  32. 32. @joel__lord #NEPHP Gestures 101 - Needs • A skeleton with an event when it changes @joel__lord
  33. 33. @joel__lord #NEPHP Gestures 101 - Needs • A skeleton with an event when it changes • Know the position of the « COM » @joel__lord
  34. 34. @joel__lord #NEPHP Gestures 101 - Needs • A skeleton with an event when it changes • Know the position of the « COM » • Have a « base unit » @joel__lord
  35. 35. @joel__lord #NEPHP Gestures 101 –Introducing kinect-gestures • Gesture detection using a base class @joel__lord
  36. 36. @joel__lord #NEPHP Gestures 101 –Introducing kinect-gestures • Gesture detection using a base class • Tracks the skeleton for an initial condition @joel__lord
  37. 37. @joel__lord #NEPHP Gestures 101 –Introducing kinect-gestures • Gesture detection using a base class • Tracks the skeleton for an initial condition • Checks every 100ms to see if the condition is still met @joel__lord
  38. 38. @joel__lord #NEPHP Gestures 101 –Introducing kinect-gestures • Gesture detection using a base class • Tracks the skeleton for an initial condition • Checks every 100ms to see if the condition is still met • If the final condition is met, trigger an event @joel__lord
  39. 39. @joel__lord #NEPHP Gestures 101 –Introducing kinect-gestures • Gesture detection using a base class • Tracks the skeleton for an initial condition • Checks every 100ms to see if the condition is still met • If the final condition is met, trigger an event • Other classes are used to define the actual gestures @joel__lord
  40. 40. IT’S CODING TIME! LET’S GET SERIOUS
  41. 41. IT’S CODING TIME! LET’S GET SERIOUS
  42. 42. @joel__lord #NEPHP Next few steps 8/5/2016 @joel__lord 42 • Return a more friendly coordinate system for skeleton • Add more information to the returned event (velocity?) • Add more gestures (jump, wave, kick)
  43. 43. DOCUMENT CONFIDENTIEL, TOUT DROIT RÉSERVÉ PRESENTED BY The End ! Questions ? JOEL LORD August 5th 2016 TWITTER: @JOEL__LORD GITHUB: HTTP://GITHUB.COM/JOELLORD

×