Bodily Interaction                         Lecture 1, 01.11.2011                              Ferhat Şen04 Kasım 2011 Cuma
Course                     • develop gesture-based Natural User                       Interfaces (NUI)                    ...
Course In General                     • the course is not to teach Processing, Pure                       data or Unity   ...
Schedule              •      1 Introduction              •      2 Kinect with Processing              •      3 Kinect with...
Small Project                     • Very simple project                     • Done in Processing, PD, Unity or QC         ...
Final Project                     •   Concept Development                  Virtual Reality                                ...
Programming                     • Processing?                     • Pure Data?                     • Unity?               ...
User Interface           the system by which           users (people) interact (communicate) with a machine.04 Kasım 2011 ...
Components of UI                     •   The user interface includes                         •   hardware (physical)      ...
Interaction                     •   indicates the means by which user inputs changes                         to the system...
Interaction                             • How do you do?                             • How do you feel?                   ...
Timeline of UI’s              Command-line Interface04 Kasım 2011 Cuma
04 Kasım 2011 Cuma
Timeline of UI’s              Command-line Interface   Graphical User Interface04 Kasım 2011 Cuma
WIMP Paradigm                     WINDOW                     ICON                     MENU                     POINTER04 K...
04 Kasım 2011 Cuma
Timeline of UI’s                                                                  Post WIMP Era                           ...
tangible                                gestural04 Kasım 2011 Cuma
Natural Interaction                     •   Experience (Human-Computer --> Human-Human )                         •   Peopl...
Why?                     • Less cognitive load                     • Simpler (for certain applications)                   ...
Bodily Interaction     • whole body in context     • multi-modality        • human=multi-sensory                ?     • us...
Bodily Input Modalities                     •   Skeletal: Hand,Fingers, Head, Leg, Feet                     •   Sonic: Spe...
Put-that-There, 1979                     •   voice and gesture at the graphics interface                         pioneerin...
Kinect Sensor                     IR Laser Projector    IR Camera04 Kasım 2011 Cuma
Step 1: Depth Sensing                               Kinect’s IR-Grid                      Watch the Video   Watch the Vide...
04 Kasım 2011 Cuma
Step 2: Tracking Algorithms                           Skeletal Tracking                     http://www.youtube.com/watch? ...
Alternative Devices                         Watch the Video04 Kasım 2011 Cuma
04 Kasım 2011 Cuma
4 Tracking Possibilities                     • Depth Filtering                     • User Tracking                     • H...
Depth Filtering04 Kasım 2011 Cuma
Hand Tracking                       Watch the Video04 Kasım 2011 Cuma
User+Skeletal Tracking                                Watch the Video04 Kasım 2011 Cuma
04 Kasım 2011 Cuma
Technical Specs of                          Kinect     •      Field of View            Horizontal field of view: 57 degrees...
04 Kasım 2011 Cuma
IR Projector   IR Camera04 Kasım 2011 Cuma
Examples                     http://nui.mlog.taik.fi/interesting-projects04 Kasım 2011 Cuma
Upcoming SlideShare
Loading in...5
×

Lecture 1: Bodily Interaction Autumn2011

2,401
-1

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,401
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture 1: Bodily Interaction Autumn2011

  1. 1. Bodily Interaction Lecture 1, 01.11.2011 Ferhat Şen04 Kasım 2011 Cuma
  2. 2. Course • develop gesture-based Natural User Interfaces (NUI) • in various interactive application platforms for sonic, musical, 2D or 3D interfaces04 Kasım 2011 Cuma
  3. 3. Course In General • the course is not to teach Processing, Pure data or Unity • to develop applications involving bodily interaction by using these programming environments and Kinect sensor. • prerequisite is to know basics of one of them. you will learn by doing more about them04 Kasım 2011 Cuma
  4. 4. Schedule • 1 Introduction • 2 Kinect with Processing • 3 Kinect with Pure Data • 4 Skeletal Tracking with Processing, Pure Data and Kinect SDK • 1 Small Project Presentation • 2 Concept Idea Presentation/Group Formation • 3 Hands on Project Work/Tutoring • 4 Hands on Project Work/Tutoring • 1 Hands on Project Work/Tutoring • 2 Hands on Project Work/Tutoring • 3 Hands on Project Work/Tutoring • 4 Final Project Presentation04 Kasım 2011 Cuma
  5. 5. Small Project • Very simple project • Done in Processing, PD, Unity or QC • Some interaction with body data • Individually • Presented in the class04 Kasım 2011 Cuma
  6. 6. Final Project • Concept Development Virtual Reality Augmented Reality Puppetry • Sonic/Musical, 2D, 3D applications Multiuser interactive environment Augmented interactive dance performance (from image viewer to media art) Gestural musical instrument Architectural Projection • Groups of 2 people (strongly suggested) • Exhibition • Public space in TaiK • Venue outside (public space outside) Spring2011 • Spring Demoday ()04 Kasım 2011 Cuma
  7. 7. Programming • Processing? • Pure Data? • Unity? • Any other? • Level of knowledge? (0-3) • Online Survey04 Kasım 2011 Cuma
  8. 8. User Interface the system by which users (people) interact (communicate) with a machine.04 Kasım 2011 Cuma
  9. 9. Components of UI • The user interface includes • hardware (physical) • software (logical) components. • Provide a means of: • Input, allowing the users to manipulate a system, and/or • Output, allowing the system to indicate the effects of the users manipulation.04 Kasım 2011 Cuma
  10. 10. Interaction • indicates the means by which user inputs changes to the system and the feedback supplied by the system04 Kasım 2011 Cuma
  11. 11. Interaction • How do you do? • How do you feel? • How do you know?04 Kasım 2011 Cuma
  12. 12. Timeline of UI’s Command-line Interface04 Kasım 2011 Cuma
  13. 13. 04 Kasım 2011 Cuma
  14. 14. Timeline of UI’s Command-line Interface Graphical User Interface04 Kasım 2011 Cuma
  15. 15. WIMP Paradigm WINDOW ICON MENU POINTER04 Kasım 2011 Cuma
  16. 16. 04 Kasım 2011 Cuma
  17. 17. Timeline of UI’s Post WIMP Era Tangible User Interface WIMP Gesture-Based Interface Command-line Interface Graphical User Interface Natural User Interface04 Kasım 2011 Cuma
  18. 18. tangible gestural04 Kasım 2011 Cuma
  19. 19. Natural Interaction • Experience (Human-Computer --> Human-Human ) • People communicate thru • gestures • expressions • movements • People discover by • looking around • manipulating physical stuff04 Kasım 2011 Cuma
  20. 20. Why? • Less cognitive load • Simpler (for certain applications) • typing??? • changing slide, navigating in 3D VR • Direct manipulation04 Kasım 2011 Cuma
  21. 21. Bodily Interaction • whole body in context • multi-modality • human=multi-sensory ? • user as an input modality • direct input from user’s sensory modalities • manipulate digital data with body04 Kasım 2011 Cuma
  22. 22. Bodily Input Modalities • Skeletal: Hand,Fingers, Head, Leg, Feet • Sonic: Speech,Voice characteristics (volume, frequency), Body-made sounds • Biofeatures: Breath, Sweat, Heartbeat, Blood Pressure • Touch • Facial muscular activation • Presence in space • Multiple bodies04 Kasım 2011 Cuma
  23. 23. Put-that-There, 1979 • voice and gesture at the graphics interface pioneering multimodal application combined speech and gesture recognition. • Put-that-There, MIT, 1979 • Put-that-there, MIT 1983 • Richard A. Bolt "Put-That-There":Voice and Gesture at the Graphics Interface (pdf) SIGGRAPH 8004 Kasım 2011 Cuma
  24. 24. Kinect Sensor IR Laser Projector IR Camera04 Kasım 2011 Cuma
  25. 25. Step 1: Depth Sensing Kinect’s IR-Grid Watch the Video Watch the Video 204 Kasım 2011 Cuma
  26. 26. 04 Kasım 2011 Cuma
  27. 27. Step 2: Tracking Algorithms Skeletal Tracking http://www.youtube.com/watch? v=ifEVIsdBQr4&feature=related04 Kasım 2011 Cuma
  28. 28. Alternative Devices Watch the Video04 Kasım 2011 Cuma
  29. 29. 04 Kasım 2011 Cuma
  30. 30. 4 Tracking Possibilities • Depth Filtering • User Tracking • Hand Tracking • Skeletal Tracking04 Kasım 2011 Cuma
  31. 31. Depth Filtering04 Kasım 2011 Cuma
  32. 32. Hand Tracking Watch the Video04 Kasım 2011 Cuma
  33. 33. User+Skeletal Tracking Watch the Video04 Kasım 2011 Cuma
  34. 34. 04 Kasım 2011 Cuma
  35. 35. Technical Specs of Kinect • Field of View Horizontal field of view: 57 degrees Vertical field of view: 43 degrees Physical tilt range: ± 27 degrees Depth sensor range: 1.2m – 3.5m (10m) • Skeletal Tracking System Tracks up to 6 people, including 2 active players Tracks 20 joints per active player • Works in complete darkness, problems in direct sunlight04 Kasım 2011 Cuma
  36. 36. 04 Kasım 2011 Cuma
  37. 37. IR Projector IR Camera04 Kasım 2011 Cuma
  38. 38. Examples http://nui.mlog.taik.fi/interesting-projects04 Kasım 2011 Cuma

×