Android and OpenNI - NUI Application Treasure Hunter Robot

5,151 views

Published on

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,151
On SlideShare
0
From Embeds
0
Number of Embeds
2,238
Actions
Shares
0
Downloads
54
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Android and OpenNI - NUI Application Treasure Hunter Robot

  1. 1. Android and OpenNI - NUI Application Treasure Hunter Robot Hirotaka Niisato @hirotakaster 1 ©SIProp Project, 2006-2012 1
  2. 2. About Myself Special Server/Network Technology Community (OSS) SIProp.org Company GMO Internet, Inc ©SIProp Project, 2006-2012 2
  3. 3. NUI - Natural User Interface ©SIProp Project, 2006-2012 3
  4. 4. NUI Device ©SIProp Project, 2006-2012 4
  5. 5. Device Specifications Kinect Xtion Pro Live (Windows)RGB Camera ○ ○Depth Sensor ○ ○Audio ○ ○Commercial Use ○ ○Library Kinect for OpenNI Windows SDK (GPL/LPGL) ©SIProp Project, 2006-2012 5
  6. 6. Shop ©SIProp Project, 2006-2012 6
  7. 7. Robotics ©SIProp Project, 2006-2012 7
  8. 8. others ©SIProp Project, 2006-2012 8
  9. 9. OpenNI Open Source Environment ● OS:Windows/Linux(+ARM)/MacOS/Android ● Java/C++/C# Modules ● OpenNI ● Sensor ● Middleware(NITE) ©SIProp Project, 2006-2012 9
  10. 10. OpenNI with Android Application Pattern.1 USB ©SIProp Project, 2006-2012 10
  11. 11. OpenNI with Android Application Pattern.2 USB NW Android : Toshiba T-01C(Regza Phone) ©SIProp Project, 2006-2012 11
  12. 12. Treasure Hunting Robot ©SIProp Project, 2006-2012 12
  13. 13. Summary This is an "AR(augmented reality) Treasure Hunting Game“ You get virtual treasures by controlling real robot! ©SIProp Project, 2006-2012 13
  14. 14. Manual Look at radar window like dragon radar. Show some treasure on radar. Center is a place in which a robot is present. N(North) is a robots front. If you move the robot to a treasure place, you g et some point. Controlling the robot is by brain wave. ©SIProp Project, 2006-2012 14
  15. 15. How to control Keep 3 plots/5 plots in same area. Upper(exciting) area Turn left Middle(active) area Go toward Lower(relax) area Turn right ©SIProp Project, 2006-2012 15
  16. 16. How to make ©SIProp Project, 2006-2012 16
  17. 17. NUI Devices ©SIProp Project, 2006-2012 17
  18. 18. Hardwares & Softwares Hardwares Softwares Base computer Ubuntu & Android Pandaboard Linaro11.11 Brain Wave Sensor Depth Sensor MindWave OpenNI NeuroSky Bone skeleton tracker Depth Sensor NITE for ARM Xtion pro live Only binary… ASUS UI Framework Display openFrameworks AiRscoter Android Brother Industries Walking Robot KHR-3WL Kondo science ©SIProp Project, 2006-2008 18
  19. 19. Hardwares’ photo ©SIProp Project, 2006-2012 19
  20. 20. Softwares’ photo ©SIProp Project, 2006-2012 20
  21. 21. Calculate robot position Recognizing a robot by bone skeleton. Using OpenNI Calculate direction & position of robot. Calculate by moving robot. Using depth camera. ©SIProp Project, 2006-2012 21
  22. 22. Recognizing bone skeleton NITE is NOT OSS! Only binary for ARM/Ubuntu. Want to use Android&openFrameworks UI. Made dual boot OS for Ubuntu&Android. UserLand: Linaro/Android (Run on Ubuntu by chroot) UserLand: Linaro/Ubuntu (OpenNI runs on this userland) Linaro Kernel for Ubuntu&Android on Pandaboard ©SIProp Project, 2006-2012 22
  23. 23. Recognizing bone skeleton NITE can NOT recognize our robot… Head is too SMALL as human. Legs & arms are too THIN as human. Upright pose is not like human. ©SIProp Project, 2006-2012 23
  24. 24. Calculate robot position Calculate a current center position. Moving distance is calculated by using depth camera. Calculate a robots front. This robot has just 3 ways action. Go toward, Turn left, Turn right Direction of “Go toward” is the robots front. ©SIProp Project, 2006-2012 24
  25. 25. How to control Keep 3 plots in same area. Upper(exciting) area Turn left Middle(active) area Go toward Lower(relax) area Turn right ©SIProp Project, 2006-2012 25
  26. 26. Brain wave 1/2 What wave is What feeling? γ(gamma) wave is exciting. β(beta) wave is active. α(alpha) wave is relax. θ(theta ) wave is sleep. δ(delta) wave is deep sleep. δ θ α β γRelax Exciting ©SIProp Project, 2006-2012 26
  27. 27. Brain wave 2/2 Upper area γ(gamma) wave is exciting. Middle area β(beta) wave is active. Lower area α(alpha) wave is relax. δ θ α β γRelax Exciting ©SIProp Project, 2006-2012 27
  28. 28. Download source code We release all source code on our site. http://www.siprop.org/ja/2.0/index.php?product%2FTreasureHuntingRobot  Do It Yourself ⇒Do It With Others! ©SIProp Project, 2006-2012 28
  29. 29. Thank you! ©SIProp Project, 2006-2012 29

×