The not so short

1,722 views

Published on

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,722
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The not so short

  1. 1. Hang xiesonicisdreaming@gmail.comhttp://Xhbang.com
  2. 2. Chongqing University bachelor(单身汉)A boy couldnt stop dreamingBike travelling
  3. 3. •Dr.•Sensor related•Poet(魔幻现实主义)
  4. 4. •50% Idle•25% Computer•20% Programmer•5% Student/Paper
  5. 5. Programming dummy?Linux || Windows?Understand 3D?Work with the Kinect?
  6. 6. 体感操作,Kinect,OPENNI介绍基本程序设计Working with Depth ImageWorking with Point Clouds(*)Working with Skeleton Data & TrackingKinect Hacks
  7. 7. 昨夜西风凋碧树,独上高楼,望尽天涯路
  8. 8. Buy it!
  9. 9. TeardownPrincipleHardwarePerformace
  10. 10. Hardware & teardown
  11. 11. http://www.ifixit.com/Teardown/Microsoft-Kinect-Teardown
  12. 12. •RGB camera•IR depth sensor•IR projector•Audio device
  13. 13. Noise at edges(shoulders)Depth shadowsMisalignment between the color and depth imagesFrame rate(30 vs 60)
  14. 14. Behind Kinect
  15. 15. •Time of Flight•以色列公司3DV( 09年微软收购)•光飞行时间,飞秒(1e−15)级快门•砷化镓,使其时钟频率提高到上百•高成本,难以适应消费电子品•Natal并不是基于ToF的原理
  16. 16. •连续的照明(而非脉冲) •普通的CMOS感光芯片(低成本) •结构光技术(三维纵深的“体编码” ) •激光散斑(laser speckle)http://www.freepatentsonline.com/7433024.pdfhttp://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036http://www.javaforge.com/wiki/103639
  17. 17. •机器学习系统•TB计的数据训练集1. 寻找移动部位(背景分离,分割遮罩)2. 辨认身体部位可能性3. 模型匹配:生成骨架系统
  18. 18. •Something we forget•Supported by OpenNI, not bySensorKinect
  19. 19. •“Sound Wave”•微软研究院和华盛顿大学•利用多普勒效应,使用笔记本内置的扬声器和麦克风开发出类似Kinect的姿势感知系统•低成本•识别精度能达到90%以上•用手势玩俄罗斯方块游戏(视频)
  20. 20. Concepts
  21. 21. http://en.wikipedia.org/wiki/Natural_user_interface
  22. 22. Next generation interface
  23. 23. Wii Remote
  24. 24. ASUS Xtion
  25. 25. Kinect
  26. 26. Project Glass
  27. 27. Development
  28. 28. http://www.openni.org/
  29. 29. http://code.google.com/p/simple-openni/
  30. 30. https://github.com/OpenKinect/libfreenecthttp://openkinect.org/wiki/Main_Page
  31. 31. Architecture Quick view(*)
  32. 32. •Short for “Open Natural Interface”•Multi-langue•Cross-platform•Standard & Infrastructure
  33. 33. 使用openNI开发的程序在以后的版本中都可以继续使用,因为开发人员可以毫无顾虑的更新openNI到最新的版本。
  34. 34. OpenNI Framework(1)
  35. 35. •Top: 代表软件,在OpenNI顶部,实现了自然交互的应用程序。•Middle: 代表OpenNI,提供通信接口,同时连接传感器和中间件组件,后者分析来自传感器的数据。•Bottom:硬件设备,捕捉现场的视频和音频内容。
  36. 36. OpenNI Framework(2)
  37. 37. •一个USB抽象访问层•一些基本数据类型的实现(包括列表,哈希,等等)•系统日志和转储•内存和性能分析•活动(授权要注册的回调函数到一个特定的事件)•任务调度•不是标准OpenNI的一部分(*)
  38. 38. •函数指针•注册•事件响应
  39. 39.  Generator Metadata Capability
  40. 40. Generator注:绿色已经有,红色计划中,后略
  41. 41. •单位集•生产性角色•使用较低水平的生产节点•“有意义的三位数据”
  42. 42. •Device•Depth Generator•Image Generator•IR Generator•Audio Generator
  43. 43. •Gesture Generator•Scene Analyzer•Hands Generator•User Generator•//provided by NITE
  44. 44. •Recoder•Player•Codec
  45. 45. Matadata
  46. 46. Capability
  47. 47. SessionDetector
  48. 48. •Full body analysis•Hand point analysis•Gesture detection•Scene Analyzer
  49. 49. Session
  50. 50. Detector
  51. 51. 衣带渐宽终不悔,为伊消得人憔悴。
  52. 52. •Preparation•The program
  53. 53. •OpenNI+NITE •Avin’s SensorKinect •Processing(*) •SimpleOpenNI(*)https://github.com/avin2/SensorKinect
  54. 54. Here’s the code,too young too simple
  55. 55. import SimpleOpenNI.*;SimpleOpenNI kinect;void setup(){ size(640*2, 480); kinect = new SimpleOpenNI(this); kinect.enableDepth(); kinect.enableRGB();}void draw(){ kinect.update(); image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0);}
  56. 56. Understand the code
  57. 57. import SimpleOpenNI.*;SimpleOpenNI kinect;•Import the library•Declare the SimpleOpenNI object
  58. 58. void setup(){ size(640*2, 480); kinect = new SimpleOpenNI(this); kinect.enableDepth(); kinect.enableRGB();}•Declare the size of app•Instantiate the object•Call two method
  59. 59. void draw(){ kinect.update(); image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0);}•Get fresh data from kinect•Place the image
  60. 60. •R G B•纯黑代表无穷远,纯白代表无穷近•Demo:ex03_basic_depth_plus_rgb
  61. 61. •Projective•Realworld•Z值相同,代表深度,单位mm•ConvertProjectiveToRealWorld()
  62. 62. The Kinect coordinate system
  63. 63. DemochPC_ex12_obj_in_point_cloud
  64. 64. •最近距离:about 0.5(?)•最远距离: 10m•GetDeviceMaxDepth()•官方建议:1.2 – 3.6米•近远精确性比较
  65. 65. Welcome to the Third Dimension 3d reconstruction E.g:KinectFusion PCLhttp://pointclouds.org/
  66. 66. DEMOchPC_ex05_rotating_point_cloud
  67. 67. •User Detection•Joints & Skeleton•Distance in 3D•Scene Map, Com•Tracking
  68. 68. •Calibration or not•Stages in Process
  69. 69. •Joints•Distance between two joints in 3D
  70. 70. WindowsSDK: 20 joints(all)
  71. 71. OpenNI: 24 joints
  72. 72. DEMOchSK_ex02_skel_anatomy
  73. 73. •Background remove•User pixels•Center of Mass
  74. 74. •chSK_scene_map•chSK_scene_image•chSK_center_of_mass
  75. 75. •kinect.enableScene();•sceneMap = kinect.sceneMap();•if(sceneMap[i] == n)
  76. 76. Demo:PointViewer
  77. 77. 众里寻他千百度,蓦然回首, 那人却在,灯火阑珊处。
  78. 78. Apps out of boxCreate your own cool stuffsLiving coding with VS & Cpp
  79. 79. Demo: chDE_face_changing
  80. 80. •Collision detection•kinect.getJointPositionSkeleton()•kinect.convertRealWorldToProjective()•Picture scala
  81. 81. Demo:Minority Report(*)Demo:2D TrackPadDemo:Hotpoint 3D
  82. 82. •Key event mapping •Game playinghttp://projects.ict.usc.edu/mxr/faast/
  83. 83. •DEMO: 宅男妄想地带•Microsoft: NO!!
  84. 84. Sorry, Bill
  85. 85. •Map(Joint position + gesture, key event)
  86. 86. http://www.kinectdevs.com/forums/kinect-virtual-camera-kinect-webcam-f17
  87. 87. The following demo need abetter computer for higher GPU support
  88. 88. 一秒钟凹凸曼合体!完成你童年 时候的梦想 Videohttp://code.google.com/p/kinect-ultra/
  89. 89. 初音妹子Video
  90. 90. What do you want to make?http://www.kinect-hacks.com/openni
  91. 91. reference•http://viml.nchc.org.tw•Visualization and Interactive Media Laboratory of NCHC•国家科學視算與互動媒體實驗室•http://www.javaforge.com/wiki/103532•Kinect for PC Wiki•中文
  92. 92. Hang xiesonicisdreaming@gmail.comhttp://www.xhbang.com
  93. 93. Do not delete this page

×