©SIProp Project, 2006-2008<br />1<br />Auto Chasing Turtle<br />HirotakaNiisato<br />Noritsuna Imamura<br />
About ourselves<br />Ourselves<br />HirotakaNiisato(@hirotakaster)<br />Noritsuna Imamura(@noritsuna)<br />About SIProp Pr...
Agenda<br />Summary<br />Part “Software”<br />Detect the face<br />Calculate the course<br />Calculate the distance<br />P...
Summary<br />This product is "Auto Chasing Turtle".<br />By autonomous control, this robot recognizes people's face and ap...
Robot behavior<br />Rotate and look for the human who becomes a target.<br />Detect the human with recognizing people's fa...
Movie<br />
YouTube<br />http://www.youtube.com/watch?v=8EgfAk5RBVo<br />Source Code & detail explanation<br />http://www.siprop.org/j...
complete set parts<br />Robot<br />KONDO Animal 01<br />Controllers<br />For servo<br />RCB3<br />For application<br />Bea...
complete set parts<br />
Why this project?<br />DENSUKE project<br />Mobile Internet Device with Cyber-Pet User Interface<br />
DENSUKE project<br />OESF(Open Embedded Software Foundation) Future Systems WG’s mission<br />http://fswg.oesf.biz/<br />P...
Detail of “Kinect Turtle world”<br />
Why KINECT and Robot ?<br />Application have changed <br />Game, Motion Controller, Nunchuk<br />Cell Phone, Tablet<br />
Why KINECT and Robot ?<br />KINECT show the next future<br />Camera, Depth censor.<br />Recognition of human’s data.<br />...
Why KINECT and Robot ?<br />KINECT on Robot is result of mobility problem.<br />sensory system<br />drive system<br />Robo...
Part “Software”<br />
ofxDroidKinect<br />openFrameworks<br />x<br />Kinect<br />x<br />Android<br />
About ofxDroidKinect<br />
About ofxDroidKinect<br />This is the Android Application Framework which runs on openFrameworks & uses Kinect.<br />Using...
Using Kinect<br />OpenKinect<br />Open Source Free Library<br />openFrameworks<br />OpenNI<br />PrimeSense<br />Initial su...
Control Robot and KINECT<br />ofxDroidKinect<br />USB<br />Serial<br />openFrameworks<br />Linaro/Android<br />Wifi<br />
Analyze KINECT data<br />Detect the Face<br />Calculate the course<br />Calculate the distance<br />1. detect<br />3. dist...
Detect the face<br />What’s data ?<br />640x480 RGB image, using KINECT’s camera, <br />Characteristics of human face.<br ...
Detect the face<br />Not all frame, but realtime analyzing.<br />Auto search around people.<br />analyze thread<br />10fps...
Calculate the course<br />Using width.<br />Specify centerposition of face.<br />Specify position of the face in 4-section...
Calculate the course<br />Using height.<br />Change Kinect’s angle.<br />Specify position of the face in image relavive to...
Calculate the distance<br />Mesure the depth of specified point using KINECT.<br />Robots walk forward or back, depending ...
Performance problem<br />Data analyze and image processing<br />Face recognition<br />More resource !!<br />GPU, Multi cor...
Part “Hardware”<br />
How to control the robot?<br />Only control servos’ angle.<br />Accuracy<br />Reaction velocity<br />Torque<br />Compact(S...
How to control a servo?<br />RCB series<br />24 controls pins<br />Control commands<br />Control all servo at same time !<...
More control<br />Do you know this IC?<br />
How to process more sensors?<br />Analog data is stream data!<br />Need powerful processor<br />Need to save power for Mob...
How to optimize for each sensors?<br />         http://linaro.org/<br />Mission<br />Optimize for each SoC & Platform<br /...
Campus Party Colombia 2011<br />
MAKE:style<br />Hardware<br />beagleboard-xM<br />KONDO Animal<br />Kinect<br />Software<br />ofxDroidKinect<br />Linaro K...
Upcoming SlideShare
Loading in …5
×

Androidで出来る!! KinectとiPadを使った亀ロボ

3,853 views
3,735 views

Published on

Android Bazaar and Conference 2011 Summer.
robot track.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,853
On SlideShare
0
From Embeds
0
Number of Embeds
53
Actions
Shares
0
Downloads
39
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Androidで出来る!! KinectとiPadを使った亀ロボ

  1. 1. ©SIProp Project, 2006-2008<br />1<br />Auto Chasing Turtle<br />HirotakaNiisato<br />Noritsuna Imamura<br />
  2. 2. About ourselves<br />Ourselves<br />HirotakaNiisato(@hirotakaster)<br />Noritsuna Imamura(@noritsuna)<br />About SIProp Project<br />Open Source Software Project<br />The world of media (communication medium) is extended.<br />
  3. 3. Agenda<br />Summary<br />Part “Software”<br />Detect the face<br />Calculate the course<br />Calculate the distance<br />Part “Hardware”<br />How to control the robot<br />How to process the analog data<br />Need more power supply<br />
  4. 4. Summary<br />This product is "Auto Chasing Turtle".<br />By autonomous control, this robot recognizes people's face and approaches to the detected human. <br />
  5. 5. Robot behavior<br />Rotate and look for the human who becomes a target.<br />Detect the human with recognizing people's face by using RGB camera of Kinect.<br />If it can detect the human, it calculates the course which it should follow. And It direction is changed.<br />Calculate the distance to target by using Z(depth) camera of Kinect.<br />Walk toward to target.<br />2-5 are repeated until it becomes a suitable distance. And if losts target, returns to 1.<br />The scene that it is working in real time can be seen by iPad.<br />
  6. 6. Movie<br />
  7. 7. YouTube<br />http://www.youtube.com/watch?v=8EgfAk5RBVo<br />Source Code & detail explanation<br />http://www.siprop.org/ja/2.0/index.php?product%2FAutoChasingTurtle<br />Movie<br />
  8. 8. complete set parts<br />Robot<br />KONDO Animal 01<br />Controllers<br />For servo<br />RCB3<br />For application<br />Beagleboard-xM<br />Linaro-Kernel<br />Android(Embedded Master)<br />Sensors<br />Kinect<br />RGBcamera<br />Z(Depth)camera<br />Connect to outside<br />Radio wave<br />Wi-Fi router<br />Viewer<br />iPad<br />Power supply<br />12V1A output<br />10V1A output<br />5V3A output<br />
  9. 9. complete set parts<br />
  10. 10. Why this project?<br />DENSUKE project<br />Mobile Internet Device with Cyber-Pet User Interface<br />
  11. 11. DENSUKE project<br />OESF(Open Embedded Software Foundation) Future Systems WG’s mission<br />http://fswg.oesf.biz/<br />Project leader<br />Mr. Ohtsuki<br />The Father of AIBO<br />Partners<br />
  12. 12. Detail of “Kinect Turtle world”<br />
  13. 13. Why KINECT and Robot ?<br />Application have changed <br />Game, Motion Controller, Nunchuk<br />Cell Phone, Tablet<br />
  14. 14. Why KINECT and Robot ?<br />KINECT show the next future<br />Camera, Depth censor.<br />Recognition of human’s data.<br />Mobility Problem ...<br />
  15. 15. Why KINECT and Robot ?<br />KINECT on Robot is result of mobility problem.<br />sensory system<br />drive system<br />Robot movement<br />Camera data<br />Depth censor<br />Analyze<br />Grey Walter's turtle robot in 1949.<br />
  16. 16. Part “Software”<br />
  17. 17. ofxDroidKinect<br />openFrameworks<br />x<br />Kinect<br />x<br />Android<br />
  18. 18. About ofxDroidKinect<br />
  19. 19. About ofxDroidKinect<br />This is the Android Application Framework which runs on openFrameworks & uses Kinect.<br />Using softwares<br />openFrameworks for Android<br />Linaro Android<br />
  20. 20. Using Kinect<br />OpenKinect<br />Open Source Free Library<br />openFrameworks<br />OpenNI<br />PrimeSense<br />Initial support for the Android & ARM platforms.<br />Kinect for Windows SDK<br />Kinect Services for Microsoft Robotisc Developer Studio<br />
  21. 21. Control Robot and KINECT<br />ofxDroidKinect<br />USB<br />Serial<br />openFrameworks<br />Linaro/Android<br />Wifi<br />
  22. 22. Analyze KINECT data<br />Detect the Face<br />Calculate the course<br />Calculate the distance<br />1. detect<br />3. distance<br />2. course<br />
  23. 23. Detect the face<br />What’s data ?<br />640x480 RGB image, using KINECT’s camera, <br />Characteristics of human face.<br />KINECT Image<br />
  24. 24. Detect the face<br />Not all frame, but realtime analyzing.<br />Auto search around people.<br />analyze thread<br />10fps<br />image<br />thread<br />FaceDetector detector = new FaceDetector(w, h, faces.length);<br />intnumFaces = detector.findFaces(bitmap, faces);<br />if (numFaces > 0) {<br /> // Calculate the course, Calculate the distance<br />} else {<br /> // search around<br />}<br />
  25. 25. Calculate the course<br />Using width.<br />Specify centerposition of face.<br />Specify position of the face in 4-sections relavive to KINECT’s image.<br />
  26. 26. Calculate the course<br />Using height.<br />Change Kinect’s angle.<br />Specify position of the face in image relavive to KINECT’s image.<br />if (pointX > 0 && pointX < w/4) {<br />DroidBot.getInstance().turnRight2(); // right position<br />} else if (pointX >= w/4 && pointX <= 3*w/4) {<br /> ; // center position<br />} else if (pointX > 3*w/4 && pointX <= w) {<br />DroidBot.getInstance().turnLeft2(); // left position<br />}<br />int angle = 30 - pointY*30/h;<br />if (angle > 0 && angle <= 30)<br />OFAndroid.setAngle(angle);<br />void Java_cc_openframeworks_OFAndroid_setAngle(JNIEnv* env, jobject<br />thiz, jint angle) {<br /> if (angle > 30) angle = 30;<br /> else if (angle < 0) angle = 0;<br />kinectApp->kinect.setCameraTiltAngle(angle);<br />}<br />
  27. 27. Calculate the distance<br />Mesure the depth of specified point using KINECT.<br />Robots walk forward or back, depending on the mesured distance.<br />int dist = OFAndroid.getDistance(pointX, pointY);<br />if (dist < 100) DroidBot.getInstance().walkBack4();<br />else if (dist >= 100 && dist < 150) DroidBot.getInstance().walkToward4();<br />else if (dist >= 150 && dist < 200) DroidBot.getInstance().walkToward8();<br />else if (dist >= 200 && dist < 300) DroidBot.getInstance().walkToward16();<br />else if (dist >= 300) DroidBot.getInstance().walkToward32();<br />JNIEXPORT<br />jint<br />JNICALL Java_cc_openframeworks_OFAndroid_getDistance(JNIEnv* env,<br />jobjectthiz, jintobjx, jintobjy){<br /> return (int)kinectApp->kinect.getDistanceAt(objx, objy);<br />}<br />
  28. 28. Performance problem<br />Data analyze and image processing<br />Face recognition<br />More resource !!<br />GPU, Multi core.<br />
  29. 29. Part “Hardware”<br />
  30. 30.
  31. 31. How to control the robot?<br />Only control servos’ angle.<br />Accuracy<br />Reaction velocity<br />Torque<br />Compact(Size, Weight)<br />
  32. 32. How to control a servo?<br />RCB series<br />24 controls pins<br />Control commands<br />Control all servo at same time !<br />
  33. 33. More control<br />Do you know this IC?<br />
  34. 34. How to process more sensors?<br />Analog data is stream data!<br />Need powerful processor<br />Need to save power for Mobile<br />Now Solution<br />ARM base Processor<br />dual Cortex A9<br />MALI 400 GPU<br />Next trend<br />GPGPU<br />FPGA<br />
  35. 35. How to optimize for each sensors?<br />         http://linaro.org/<br />Mission<br />Optimize for each SoC & Platform<br />We are committer!<br />
  36. 36. Campus Party Colombia 2011<br />
  37. 37. MAKE:style<br />Hardware<br />beagleboard-xM<br />KONDO Animal<br />Kinect<br />Software<br />ofxDroidKinect<br />Linaro Kernel<br />Android(Embedded Master)<br />  Do It Yourself ⇒Do It With Others!<br />

×