Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lecture 1: Bodily Interaction Autumn2011

2,761 views

Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Lecture 1: Bodily Interaction Autumn2011

  1. 1. Bodily Interaction Lecture 1, 01.11.2011 Ferhat Şen04 Kasım 2011 Cuma
  2. 2. Course • develop gesture-based Natural User Interfaces (NUI) • in various interactive application platforms for sonic, musical, 2D or 3D interfaces04 Kasım 2011 Cuma
  3. 3. Course In General • the course is not to teach Processing, Pure data or Unity • to develop applications involving bodily interaction by using these programming environments and Kinect sensor. • prerequisite is to know basics of one of them. you will learn by doing more about them04 Kasım 2011 Cuma
  4. 4. Schedule • 1 Introduction • 2 Kinect with Processing • 3 Kinect with Pure Data • 4 Skeletal Tracking with Processing, Pure Data and Kinect SDK • 1 Small Project Presentation • 2 Concept Idea Presentation/Group Formation • 3 Hands on Project Work/Tutoring • 4 Hands on Project Work/Tutoring • 1 Hands on Project Work/Tutoring • 2 Hands on Project Work/Tutoring • 3 Hands on Project Work/Tutoring • 4 Final Project Presentation04 Kasım 2011 Cuma
  5. 5. Small Project • Very simple project • Done in Processing, PD, Unity or QC • Some interaction with body data • Individually • Presented in the class04 Kasım 2011 Cuma
  6. 6. Final Project • Concept Development Virtual Reality Augmented Reality Puppetry • Sonic/Musical, 2D, 3D applications Multiuser interactive environment Augmented interactive dance performance (from image viewer to media art) Gestural musical instrument Architectural Projection • Groups of 2 people (strongly suggested) • Exhibition • Public space in TaiK • Venue outside (public space outside) Spring2011 • Spring Demoday ()04 Kasım 2011 Cuma
  7. 7. Programming • Processing? • Pure Data? • Unity? • Any other? • Level of knowledge? (0-3) • Online Survey04 Kasım 2011 Cuma
  8. 8. User Interface the system by which users (people) interact (communicate) with a machine.04 Kasım 2011 Cuma
  9. 9. Components of UI • The user interface includes • hardware (physical) • software (logical) components. • Provide a means of: • Input, allowing the users to manipulate a system, and/or • Output, allowing the system to indicate the effects of the users manipulation.04 Kasım 2011 Cuma
  10. 10. Interaction • indicates the means by which user inputs changes to the system and the feedback supplied by the system04 Kasım 2011 Cuma
  11. 11. Interaction • How do you do? • How do you feel? • How do you know?04 Kasım 2011 Cuma
  12. 12. Timeline of UI’s Command-line Interface04 Kasım 2011 Cuma
  13. 13. 04 Kasım 2011 Cuma
  14. 14. Timeline of UI’s Command-line Interface Graphical User Interface04 Kasım 2011 Cuma
  15. 15. WIMP Paradigm WINDOW ICON MENU POINTER04 Kasım 2011 Cuma
  16. 16. 04 Kasım 2011 Cuma
  17. 17. Timeline of UI’s Post WIMP Era Tangible User Interface WIMP Gesture-Based Interface Command-line Interface Graphical User Interface Natural User Interface04 Kasım 2011 Cuma
  18. 18. tangible gestural04 Kasım 2011 Cuma
  19. 19. Natural Interaction • Experience (Human-Computer --> Human-Human ) • People communicate thru • gestures • expressions • movements • People discover by • looking around • manipulating physical stuff04 Kasım 2011 Cuma
  20. 20. Why? • Less cognitive load • Simpler (for certain applications) • typing??? • changing slide, navigating in 3D VR • Direct manipulation04 Kasım 2011 Cuma
  21. 21. Bodily Interaction • whole body in context • multi-modality • human=multi-sensory ? • user as an input modality • direct input from user’s sensory modalities • manipulate digital data with body04 Kasım 2011 Cuma
  22. 22. Bodily Input Modalities • Skeletal: Hand,Fingers, Head, Leg, Feet • Sonic: Speech,Voice characteristics (volume, frequency), Body-made sounds • Biofeatures: Breath, Sweat, Heartbeat, Blood Pressure • Touch • Facial muscular activation • Presence in space • Multiple bodies04 Kasım 2011 Cuma
  23. 23. Put-that-There, 1979 • voice and gesture at the graphics interface pioneering multimodal application combined speech and gesture recognition. • Put-that-There, MIT, 1979 • Put-that-there, MIT 1983 • Richard A. Bolt "Put-That-There":Voice and Gesture at the Graphics Interface (pdf) SIGGRAPH 8004 Kasım 2011 Cuma
  24. 24. Kinect Sensor IR Laser Projector IR Camera04 Kasım 2011 Cuma
  25. 25. Step 1: Depth Sensing Kinect’s IR-Grid Watch the Video Watch the Video 204 Kasım 2011 Cuma
  26. 26. 04 Kasım 2011 Cuma
  27. 27. Step 2: Tracking Algorithms Skeletal Tracking http://www.youtube.com/watch? v=ifEVIsdBQr4&feature=related04 Kasım 2011 Cuma
  28. 28. Alternative Devices Watch the Video04 Kasım 2011 Cuma
  29. 29. 04 Kasım 2011 Cuma
  30. 30. 4 Tracking Possibilities • Depth Filtering • User Tracking • Hand Tracking • Skeletal Tracking04 Kasım 2011 Cuma
  31. 31. Depth Filtering04 Kasım 2011 Cuma
  32. 32. Hand Tracking Watch the Video04 Kasım 2011 Cuma
  33. 33. User+Skeletal Tracking Watch the Video04 Kasım 2011 Cuma
  34. 34. 04 Kasım 2011 Cuma
  35. 35. Technical Specs of Kinect • Field of View Horizontal field of view: 57 degrees Vertical field of view: 43 degrees Physical tilt range: ± 27 degrees Depth sensor range: 1.2m – 3.5m (10m) • Skeletal Tracking System Tracks up to 6 people, including 2 active players Tracks 20 joints per active player • Works in complete darkness, problems in direct sunlight04 Kasım 2011 Cuma
  36. 36. 04 Kasım 2011 Cuma
  37. 37. IR Projector IR Camera04 Kasım 2011 Cuma
  38. 38. Examples http://nui.mlog.taik.fi/interesting-projects04 Kasım 2011 Cuma

×