Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Android and OpenNI - NUI Application Treasure Hunter Robot Hirotaka Niisato @hirotakaster                                 ...
About Myself Special   Server/Network Technology Community (OSS)   SIProp.org Company   GMO Internet, Inc                 ...
NUI - Natural User Interface                        ©SIProp Project, 2006-2012   3
NUI Device             ©SIProp Project, 2006-2012   4
Device Specifications                         Kinect      Xtion Pro Live                        (Windows)RGB Camera       ...
Shop       ©SIProp Project, 2006-2012   6
Robotics           ©SIProp Project, 2006-2012   7
others         ©SIProp Project, 2006-2012   8
OpenNI  Open Source  Environment   ●       OS:Windows/Linux(+ARM)/MacOS/Android   ●       Java/C++/C#  Modules   ●       O...
OpenNI with Android Application Pattern.1                          USB                                            ©SIProp ...
OpenNI with Android Application Pattern.2                 USB                             NW               Android : Toshi...
Treasure Hunting Robot                         ©SIProp Project, 2006-2012   12
Summary This is an "AR(augmented reality) Treasure Hunting Game“    You get virtual treasures by controlling real robot!  ...
Manual Look at radar window like dragon radar.   Show some treasure on radar.   Center is a place in which a robot is pres...
How to control Keep 3 plots/5 plots in same area.   Upper(exciting) area      Turn left   Middle(active) area      Go towa...
How to make              ©SIProp Project, 2006-2012   16
NUI Devices              ©SIProp Project, 2006-2012   17
Hardwares & Softwares Hardwares                     Softwares   Base computer                 Ubuntu & Android      Pandab...
Hardwares’ photo                   ©SIProp Project, 2006-2012   19
Softwares’ photo                   ©SIProp Project, 2006-2012   20
Calculate robot position Recognizing a robot by bone skeleton.   Using OpenNI Calculate direction & position of robot.   C...
Recognizing bone skeleton NITE is NOT OSS!   Only binary for ARM/Ubuntu.   Want to use Android&openFrameworks UI. Made dua...
Recognizing bone skeleton NITE can NOT recognize our robot…   Head is too SMALL as human.   Legs & arms are too THIN as hu...
Calculate robot position Calculate a current center position.   Moving distance is calculated by using depth camera. Calcu...
How to control Keep 3 plots in same area.   Upper(exciting) area      Turn left   Middle(active) area      Go toward   Low...
Brain wave 1/2        What wave is What feeling?          γ(gamma) wave is exciting.          β(beta) wave is active.     ...
Brain wave 2/2        Upper area          γ(gamma) wave is exciting.        Middle area          β(beta) wave is active.  ...
Download source code We release all source code on our site.   http://www.siprop.org/ja/2.0/index.php?product%2FTreasureHu...
Thank you!             ©SIProp Project, 2006-2012   29
Upcoming SlideShare
Loading in …5
×

Android and OpenNI - NUI Application Treasure Hunter Robot

5,397 views

Published on

Published in: Technology

Android and OpenNI - NUI Application Treasure Hunter Robot

  1. 1. Android and OpenNI - NUI Application Treasure Hunter Robot Hirotaka Niisato @hirotakaster 1 ©SIProp Project, 2006-2012 1
  2. 2. About Myself Special Server/Network Technology Community (OSS) SIProp.org Company GMO Internet, Inc ©SIProp Project, 2006-2012 2
  3. 3. NUI - Natural User Interface ©SIProp Project, 2006-2012 3
  4. 4. NUI Device ©SIProp Project, 2006-2012 4
  5. 5. Device Specifications Kinect Xtion Pro Live (Windows)RGB Camera ○ ○Depth Sensor ○ ○Audio ○ ○Commercial Use ○ ○Library Kinect for OpenNI Windows SDK (GPL/LPGL) ©SIProp Project, 2006-2012 5
  6. 6. Shop ©SIProp Project, 2006-2012 6
  7. 7. Robotics ©SIProp Project, 2006-2012 7
  8. 8. others ©SIProp Project, 2006-2012 8
  9. 9. OpenNI Open Source Environment ● OS:Windows/Linux(+ARM)/MacOS/Android ● Java/C++/C# Modules ● OpenNI ● Sensor ● Middleware(NITE) ©SIProp Project, 2006-2012 9
  10. 10. OpenNI with Android Application Pattern.1 USB ©SIProp Project, 2006-2012 10
  11. 11. OpenNI with Android Application Pattern.2 USB NW Android : Toshiba T-01C(Regza Phone) ©SIProp Project, 2006-2012 11
  12. 12. Treasure Hunting Robot ©SIProp Project, 2006-2012 12
  13. 13. Summary This is an "AR(augmented reality) Treasure Hunting Game“ You get virtual treasures by controlling real robot! ©SIProp Project, 2006-2012 13
  14. 14. Manual Look at radar window like dragon radar. Show some treasure on radar. Center is a place in which a robot is present. N(North) is a robots front. If you move the robot to a treasure place, you g et some point. Controlling the robot is by brain wave. ©SIProp Project, 2006-2012 14
  15. 15. How to control Keep 3 plots/5 plots in same area. Upper(exciting) area Turn left Middle(active) area Go toward Lower(relax) area Turn right ©SIProp Project, 2006-2012 15
  16. 16. How to make ©SIProp Project, 2006-2012 16
  17. 17. NUI Devices ©SIProp Project, 2006-2012 17
  18. 18. Hardwares & Softwares Hardwares Softwares Base computer Ubuntu & Android Pandaboard Linaro11.11 Brain Wave Sensor Depth Sensor MindWave OpenNI NeuroSky Bone skeleton tracker Depth Sensor NITE for ARM Xtion pro live Only binary… ASUS UI Framework Display openFrameworks AiRscoter Android Brother Industries Walking Robot KHR-3WL Kondo science ©SIProp Project, 2006-2008 18
  19. 19. Hardwares’ photo ©SIProp Project, 2006-2012 19
  20. 20. Softwares’ photo ©SIProp Project, 2006-2012 20
  21. 21. Calculate robot position Recognizing a robot by bone skeleton. Using OpenNI Calculate direction & position of robot. Calculate by moving robot. Using depth camera. ©SIProp Project, 2006-2012 21
  22. 22. Recognizing bone skeleton NITE is NOT OSS! Only binary for ARM/Ubuntu. Want to use Android&openFrameworks UI. Made dual boot OS for Ubuntu&Android. UserLand: Linaro/Android (Run on Ubuntu by chroot) UserLand: Linaro/Ubuntu (OpenNI runs on this userland) Linaro Kernel for Ubuntu&Android on Pandaboard ©SIProp Project, 2006-2012 22
  23. 23. Recognizing bone skeleton NITE can NOT recognize our robot… Head is too SMALL as human. Legs & arms are too THIN as human. Upright pose is not like human. ©SIProp Project, 2006-2012 23
  24. 24. Calculate robot position Calculate a current center position. Moving distance is calculated by using depth camera. Calculate a robots front. This robot has just 3 ways action. Go toward, Turn left, Turn right Direction of “Go toward” is the robots front. ©SIProp Project, 2006-2012 24
  25. 25. How to control Keep 3 plots in same area. Upper(exciting) area Turn left Middle(active) area Go toward Lower(relax) area Turn right ©SIProp Project, 2006-2012 25
  26. 26. Brain wave 1/2 What wave is What feeling? γ(gamma) wave is exciting. β(beta) wave is active. α(alpha) wave is relax. θ(theta ) wave is sleep. δ(delta) wave is deep sleep. δ θ α β γRelax Exciting ©SIProp Project, 2006-2012 26
  27. 27. Brain wave 2/2 Upper area γ(gamma) wave is exciting. Middle area β(beta) wave is active. Lower area α(alpha) wave is relax. δ θ α β γRelax Exciting ©SIProp Project, 2006-2012 27
  28. 28. Download source code We release all source code on our site. http://www.siprop.org/ja/2.0/index.php?product%2FTreasureHuntingRobot  Do It Yourself ⇒Do It With Others! ©SIProp Project, 2006-2012 28
  29. 29. Thank you! ©SIProp Project, 2006-2012 29

×