This document introduces the Kinect and provides an overview of its capabilities and applications. It discusses the Kinect hardware, development environments like the Kinect SDK and OpenNI, and basic processes for building Kinect applications using techniques like computer vision, image processing, and skeleton tracking. Example applications demonstrated include 3D scanning, motion capture in Blender, and interactive games using the Blender game engine and Python scripts. Helpful additional programs mentioned are NI Mate for transferring motion data and Skanect for 3D scanning.

2. 강사 소개
• Kincet 관련 작업
– 키넥트 펀치백 테스트
– 키넥트 기능성 게임 달걀 받기, 두더쥐 잡기
– 키넥트 농사 체험 (중앙 박물관 어린이 박물관)
– 키넥트 울트라 빔, 포스, 날아다니기등 예제
– 3D 프레젠테이션 데모

3. Contents
• Overview
• Device
• Driver, Library and Software
• Kinect Application Basic
• Application

4. OVERVIEW
키넥트란 무엇인가?

5. Kinect
• Natural User Interface
– Motion sensing
– Speech Recognition
적외선 발사 반사된 적외선
수신

6. Natural User Interface?
• NUI http://en.wikipedia.org/wiki/Natural_user_interface

7. How does it work?
Labeled
Depth Image
Image
이미지 (Image)
깊이 이미지
(Depth Image)
Labeling
Skeleton Tracking
Etc..

8. Demo
• NI MATE
– 키넥트로 이미지, 깊이 이미지등을 받아
“뼈대(Skeleton)”등을 구성한 후 다른
프로그램에 전달 해줌
– http://www.ni-mate.com/
• Skanect
– 키넥트를 통해 3차원 스캔 가능
– http://skanect.manctl.com/

9. 키넥트가 인식하는 뼈대 구조
• 20 parts

10. Kinect 응용 분야
• Kinect Games
– http://www.youtube.com/watch?v=7jEJS9rkc48
– http://starwars.com/watch/vg_kinect_trailer.html
• Kinect for Education
– https://www.youtube.com/watch?v=V0nv1A9nFy8
– http://marketplace.xbox.com/en-US/Product/Kinect-
Sesame-Street-TV/66acd000-77fe-1000-9115-
d8024d5309da

11. Kinect 응용 분야
• 3D Scan
– http://skanect.manctl.com/
– http://reconstructme.net/
• Motion Capture
– http://www.ni-mate.com/
– http://ipisoft.com/download/
• You can do crazy things!!
– http://code.google.com/p/kinect-ultra/
– http://www.youtube.com/watch?v=jhoITmb4518

12. Other NUIs
• Microsoft PixelSense
– http://www.youtube.com/user/mspixelsense
• 3D Immersive Touch (e.g. Apple’s Devices)
– Multi User
– Multi Touch
– Interact with 3D Virtual Environments
• Kinect like Devices
• LeapMotion
• MYO

13. DEVICES
기넥트 기기에 대해 자세히 알아보자

14. Kinect or similar devices
• Kinect for Xbox 360
– The original(old) one
– Developed for Xbox360
– Can be used in other env (Mac, PC)
• Kinect for Windows
– Include Developer License
– Additional Charge
– Support Near Mode
• Xtion (ASUS)
– Smaller, Lighter
– Less quality, No Motor
• Xtion Live (ASUS)
http://www.microsoft.com/en-us/kinectforwindows/news/faq.aspx

15. Coming Soon!!
• Prime Sense Capri

16. Assembly
• http://www.ifixit.com/Teardown/Microsoft+Kinect+Teardown/4066/3

17. DRIVER, LIBRARY AND SOFTWARE
필요한 라이브러리 및 소프트웨어들의 구성을 알아보자

18. Development Environments
• MS Kinect SDK
– Only Support Kinect for Windows and Xbox360 (?)
– Support all Kinect functions
• Motor, Speech, Near Mode and etc.
– Good Examples (Face, eye tracking etc)
• OpenNI
– Open Source
– Supported by PrimeSense
– Support all Kinect Like Devices
– PrimeSense
• Israeli company provides 3D sensing technology for Kinect

19. OpenNI
• Architecture

20. OpenNI API
• OpenNI
– Voice
– Voice Command
– Hand Gesture
– Body Motion Tracking
– Not full function support for Kinect (?)

21. Middleware for OpenNI
• NITE (PrimeSense)
Physical Device
Voice, Hand, Motion APIs

22. MS Kinect SDK

23. Summarize @_@
• Complex
Kinect SDK OpenNI
Windows MacOS Linux

24. 설치– 기본 드라이버 및
라이브러리
• MS Kinect SDK 또는 OpenNI 설치 (우리는
OpenNI만) – 설치시 기기는 분리
– MS Kinect Runtime 1.6 :
• http://www.microsoft.com/en-
us/download/details.aspx?id=34811
– OpenNI v2.1 :
• http://www.openni.org/openni-sdk/

25. 설치할 것들 - 미들웨어들
• 적당히 필요한 것들 있으면 설치
– http://www.openni.org/software/?pg=1&cat_slug
=file-cat1#.UY6bLCuYFa8

26. KINECT APPLICATION BASICS

27. 키넥트 가지고 놀때 필요한
기술들
• AI (Artificial Intelligence)
– Neural Net
– Support Vector Machine (SVM)
– …
• Computer Vision
• Image Processing
• Skeleton Animation (with skin)
• FSM (Finite State Machine)
• 3D Graphics

28. Basic Process
• 1. Recognition Setting / Start
– Initial Pose or Etc
• 2. Input Data (Depth, Image, Skeleton, Hand
Point, …)
• 3. Process Data
– 이 부분을 이전 장의 기술들로 처리

29. Recognition Setting
• Hand point or Skeleton Mode
– Hand point
• Start with gesture (wave hand)
• Unstable
– Skeleton
• Initial Pose
• Stable when all bones are visible from Kinect
• Mirror Mode Support
• MS Kinect only
– Seated Mode (Unstable)
– Kinect for windows only
• Near Mode

30. 주요 Input Data
• Depth Image
• Image
• Hand points
• Skeleton
– Joint position
– Joint rotation
– Root position
• Voice and voice position

31. Output
• Screen 상에 물체의 위치, 색, 기타속성등을
변경
• Motor Control

32. APPLICATION
Python Programming, 3D Scan & Motion Capture

33. 응용 해보기 예제 소개
1. 간단한 조작 해보기
– Blender + Python
– Blender Game Engine + Python
– OpenNI -> NI Mate -> Blender Game Engine Python
코딩
2. Motion Capture
– OpenNI -> NI Mate -> Blender -> .bvh 파일 생성
3. 3D Scan
– OpenNI/MS SDK -> SKNECT -> .ply등 모델 파일 생성

34. 플러그인들 및 프로그램
• NI Mate : 모션 데이터등 응용프로그램에 전달
– http://www.ni-mate.com/download/
– 설치 후 Trial 버전으로 인증하는 과정이 필요함
– Blender Plugin 설치
• http://www.ni-mate.com/use/blender/
• 3D 파일 및 애니메이션 편집 툴
– Blender 2.66 버전으로 설치 (2.67은 문제 있음)
• http://download.blender.org/release/Blender2.66/
• SKANECT: 3D Scan
– http://skanect.manctl.com/download/

35. Blender Python
• Blender Python Tutorial
– http://cgcookie.com/blender/2011/08/26/introductio
n-to-scripting-with-python-in-blender/
• Basics
1. 새로운 창 생성
2. Text Editor, Syntax Highlighting
3. New Text
4. 코딩 하기
5. Run Script
6. Save File / Open File

36. Blender Python Ex 1
1. Open Python Console
2. bpy.context.active_object
• ctrl + space : 자동 완성
3. bpy.context.active_object.location
4. bpy.context.active_object.location.x = 2
5. Obj = bpy.data.objects*“Cube”+
6. Obj.location.z = 3
7. 기타 물체 속성 테스트

37. Blender Python Ex 2

38. Blender Python Ex 3

39. Blender Python & Game Engine
1. Open Logic Editor
2. Sensors -> Add Sensor -> Always
– Activate TRUE level triggering
3. Controllers -> Add Controller -> Python Script
– Set Python Script
4. Property -> Physics tab
– Physics Test
• Tutorials
– http://cgcookie.com/blender/2011/09/22/introduction-to-
scripting-in-the-blender-game-engine-part-01/
– http://cgcookie.com/blender/2011/09/26/introduction-to-
scripting-in-the-blender-game-engine-part-02/

40. Blender Python Game Engine Ex 1
• 물체를 위쪽 화살표키로 움직이기

41. Blender Python Game Engine Ex 2
• 부닫치면 “Hit” 메세지 표시하기

42. NI Mate & Blender Tutorial
• Blender, NI Mate Tutorial
– Basic
• http://www.youtube.com/watch?feature=player_embedded&v=b
QhS0SFQ8Gg
– Motion Capture
• http://www.youtube.com/watch?v=bQhS0SFQ8Gg
• http://www.youtube.com/watch?v=-nN0neRp444
• http://www.youtube.com/watch?v=EogegvTG3Po
– Model Download: http://www.blendswap.com/blends/view/7093
– Bone, Skinning
• http://www.youtube.com/watch?v=8aFbTy5nDeo

43. NI Mate & Blender Game Engine
• 아바타 조종해서 물방울 터트리기
– 스스로 해봅시다

44. NI Mate + Blender Game Engine +
SKNECT(3D Scan)
• Don’t Touch Me! 프로그램 만들어 보기
– 키넥트를 사용해 모델을 건드리면 뒤로
물러나게 하기
• 준비
– SKNECT
• 3D 모델 스캔
• 텍스쳐 맵핑기능은 아직 지원 안함
• 버텍스 칼라 지원

45. Other Helpful Programs
• 3D Scan
– ReconstructMe
• http://reconstructme.net/
– SCENECT
• http://3d-app-center.faro.com/index.php/stand-alone-apps-
faro-scenect
– 123D Catch: 아이폰 만으로 3차원 모델 생성가능
• http://www.123dapp.com/catch
• Motion Capture
– iPi Motion Capture: 여러 개의 키넥트/카메라로
정교한 모션 캡쳐 가능
• http://ipisoft.com/