CHALLENGES IN DESIGNING A                                   MENU SYSTEM WITH DEPTH                                  MOTION...
MENUS ARE FORGOTTEN ABOUT .                            OR, WHAT CAN DRIVE USERS CRAZY ?© Omek Interactive, Ltd. All rights...
© Omek Interactive, Ltd. All rights reserved.
MENU SYSTEM DESIGN GUIDELINES             • Confident users understand:                               – where they are    ...
3D DEPTH SENSOR                            OR, WHAT IS IT ?© Omek Interactive, Ltd. All rights reserved.
3D DEPTH SENSOR                            OR, WHAT IS IT ?© Omek Interactive, Ltd. All rights reserved.
TOUCH-BASED TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
TOUCH-LESS TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
TOUCH-LESS TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
DEPTH MOTION CAPTURE              Body tracking                    × Tangible feedback              Natural gestures    ...
CHALLENGES                            OR, HOW TECHNOLOGY INFLUENCE DESIGN ?© Omek Interactive, Ltd. All rights reserved.
CHALLENGES OF MOTION CAPTURE             • Feeling of a click :                  virtual tangibility             • Active ...
CHALLENGES OF MOTION CAPTURE             • Feeling of a click             • Active & inactive modes :                  dif...
CHALLENGES OF MOTION CAPTURE             • Feeling of a click             • Active & inactive modes             • Am I in ...
CHALLENGES OF MOTION CAPTURE             •             Feeling of a click             •             Active & inactive mode...
DEALING WITH CHALLENGES                            SELECTING AND SCROLLING ITEMS© Omek Interactive, Ltd. All rights reserv...
Selecting Items             • Click gesture :                  imitate mouse             • Hot-spot             • Grand ge...
© Omek Interactive, Ltd. All rights reserved.
Selecting Items             • Click gesture             • Hot-spot             • Grand gesture :                   grandio...
Selecting Items             • Click gesture             • Hot-spot             • Grand gesture             • Swipe in :   ...
Selecting Items             •             Click gesture             •             Hot-spot             •             Grand...
Selecting Items                                                                            Gran                           ...
Scrolling Items             • 2D cursor :                  imitate mouse             • 1D cursor             • Static acti...
Scrolling Items             • 2D cursor             • 1D cursor :                   scroll on one axis             • Stati...
Scrolling Items             • 2D cursor             • 1D cursor             • Static active spot :                  magnif...
Scrolling Items             • 2D cursor             • 1D cursor             • Static active spot             • Dynamic act...
Scrolling Items             •             2D cursor             •             1D cursor             •             Static a...
Scrolling Items                                                                                       Gran                ...
Pagination             • Just another button: hot-spot / click             • Grand gesture: Minority Report             • ...
User Real Location             •             Camera view: what the camera sees?             •             B&W: grey scale ...
Feeling of a Click© Omek Interactive, Ltd. All rights reserved.
CONCLUSIONS                            OR, WHAT DID WE LEARN ?© Omek Interactive, Ltd. All rights reserved.
Conclusions             • Tracking:                  essential constant feedback             • Gesture:                  t...
Conclusions             • Tracking:                  essential constant feedback             • Gesture:                  t...
User Expectations             • One hand based control scheme             • Grandiose gestures only once in a while       ...
Depth Sensor Menu Design Guidelines             •             Consistency             •             Simplicity            ...
Future Experiments             • Sound based feedback (for blind)             • Speech based / speech components© Omek Int...
References             •             Frosted Glass / CAMPAR             •             EyeSight             •             D...
Thank You For Listening ,         Shachar Oz        Game & UX Designer      Omek Interactive Studios     www.OmekInteracti...
Upcoming SlideShare
Loading in …5
×

Challenges in Designing a Menu System with Touch-less Technology

2,330 views
2,255 views

Published on

Lecture presented by Shachar Oz at UXI conference in February 2012 (http://uxi.org.il/pages/13265).
Shachar is a game and user experience designer at Omek Interactive Studios. For more information on Omek and its products, visit www.omekinteractive.com

Published in: Design, Technology, Art & Photos
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,330
On SlideShare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • I work at studios of omekOutside you see my day jobI’m a uxNot about the game
  • Menus are forgotten, Bad UI system can drive users crazy
  • In order for users not be mad…
  • In one look, at any time, the user understands:Where they areWhere they wereWhat can they doHow can they do it
  • 5 minWhat is it? Sensor gives data of the distance from the camera per pixel. Bat like view of the worldDemo: video tracking viewerhttp://youtu.be/UGGKh3cyc7w?t=1m00s
  • 5 minWhat is it? Sensor gives data of the distance from the camera per pixel. Bat like view of the worldDemo: video tracking viewerhttp://youtu.be/UGGKh3cyc7w?t=1m00s
  • Keyboard, push buttonsJoystickMouseTouch screen
  • Augmented realityVoice recognitionFace recognitionEye tracking (gaze)Brain-wave controlDepth motion capture
  • Augmented realityVoice recognitionFace recognitionEye tracking (gaze)Brain-wave controlDepth motion capture
  • Gives:Body tracking: 3D position and rotationNatural gestures: body languageFree from controllers: wet hands, accessibility Not give: Tangibility feedback100% stable & responsiveness (not yet)Small & accurate action square: 10cm X 10cm (Windows mouse)
  • 5 min
  • I can touch a button and still it will not do anything. I know when I am going to press a button. No errors.I can make the movement of clicking even without clicking. No errors.We want to create a no error environment for that tech as well.What is the amount of feedback we should give, to create the virtual tangibility?
  • User research shows 4 main types of actions with cursor: inactive, reading, action (click/ scroll ..), examineWith mouse you have accurate control of cursor. Again no errors.How can we know when the user wants to select or just to look at items?
  • Confidence How do we give the user the feeling that the application knows their identity and position?Where are they at? Does the app recognize them? Does it understand what the user is trying to do?
  • The app cannot predict what the user wants to do. It tracks movement.Therefore there is always a small delay between the user decision to make an action, and the gesture itself. We need to design the feedback to be as quick as possible or to be combined with the user’s motion.
  • Click : imitate the mouse, move a cursor across the screen and push hand forward to select. Remember position before a few framesProblem: Non detectableBig movement relative to the sourcehttp://youtu.be/VLzHtsRgEBY?t=1m 1:00-1:20
  • 2. Hot spot : stand and wait……..We don’t like waiting! Web development, ui design, etc. Problem: not differentiate between selection and scrolling. Kinect hub emphasized that difference by starting the selection process only after the cursor is above the item and not moving. This caused even more delay in the selection processKinect menu 0:39 http://vimeo.com/16779301
  • 3. Significant gesture : body as a keyboard. Problem:uncomfortable and unusable Need trainingExamples:Adrenaline Misfits / Konami : (gesture based menu) http://youtu.be/vA6xfK7bl94?t=2m10s http://youtu.be/UGGKh3cyc7w?t=2m45s
  • 4. Tracking based : swipe gesture with constant feedback for the process and option to cancel at any point.Problems:Differentiate between 2 modes.Needed to be taught.Examples:Fruit Ninja / Microsofthttp://www.youtube.com/watch?NR=1&feature=fvwp&v=yeu9qOEUJ-gPowerup Heroes / Ubisoft: (improved Dance Central’s UI)http://www.youtube.com/watch?v=3WWyLuAZKjUOption 2- http://youtu.be/ZBh_1cO9EeE?t=1m50s
  • 5. Grab gesture : relatively Intuitive. Grab gesture (drag n drop).Problems:Not that stable.Too accurate.requires high quality camera data, and currently to stand close to camera. Examples:http://www.youtube.com/watch?v=W6eSUJFSRaA/*http://www.youtube.com/watch?v=NKjkQLxwTekhttp://www.youtube.com/watch?v=itmfARsVx5w
  • More ways:Wave gestureCircle gestureRelease (pull hand back) Move to a special spot
  • Move cursor freely : imitating mouse. Normal guiProblem:unstable. Delay.Simple solutions:Big buttons. Relatively large movements.Stabling the cursor. Centering it to the item.Examples:HotSpot_item
  • Tracking based : cursor marked slightly (solve some navigation). Constant feedback. Constant active item.Leave menu from top or bottom.Examples:Powerup heroes
  • No cursor. Main active spot :Scroll on one axis.Problem:navigation issue (where am i?). Slow reactionExamples:Media center live
  • When click, stable cursorExample:Jinni live or video 0:40
  • No cursor, gesture basedExamples:Adrenaline misfitsPanasonic demoMedia Center / Canesta http://youtu.be/3pekrVJU_hg?t=2m40s
  • Simple button: hot-spot / clickSignificant gesture:Minority ReportDrag n’ drop
  • Cursor: what if camera doesn’t work? Or user not identified?Camera view: what the camera sees?B&W: grey scale colorsFreeze!: control over the characterTrapezoid: view playfield from upNote over character: main area of concern
  • Examples:http://youtu.be/jORsG8AG72I?t=1m50s
  • 4 min
  • Constant feedback about cursor, selection progress & physical locationTracking based methods  more stable, comfortable, usable  constant feedback Gesture based selection  faster, show serious timeShowing a cursor  not a must
  • Constant feedback about cursor, selection progress & physical locationTracking based methods  more stable, comfortable, usable  constant feedback Gesture based selection  faster, show serious timeShowing a cursor  not a must
  • User expectations:One hand based control schemeGrandiose gestures only once in a whileConstant feedback for each movement: physical, joint
  • Back to basics:Consistency:Do not change model of controlSimplicity:Do not use 2 models of controldo not use more than 1 handEasily understand the scale and the modelFeeling of control:Physically comfortable for the userEmotionally comfortable for the userOption to cancel & undoAble to wander around without false actions. Allow user to think and play without errorsFeedback:Make sure the user get the adequate feedback for the his actions
  • Challenges in Designing a Menu System with Touch-less Technology

    1. 1. CHALLENGES IN DESIGNING A MENU SYSTEM WITH DEPTH MOTION SENSOR TECHNOLOGY Shachar Oz Game & UX Designer Omek Interactive Studios© Omek Interactive, Ltd. All rights reserved.
    2. 2. MENUS ARE FORGOTTEN ABOUT . OR, WHAT CAN DRIVE USERS CRAZY ?© Omek Interactive, Ltd. All rights reserved.
    3. 3. © Omek Interactive, Ltd. All rights reserved.
    4. 4. MENU SYSTEM DESIGN GUIDELINES • Confident users understand: – where they are – where they were – what can they do – how can they do it© Omek Interactive, Ltd. All rights reserved.
    5. 5. 3D DEPTH SENSOR OR, WHAT IS IT ?© Omek Interactive, Ltd. All rights reserved.
    6. 6. 3D DEPTH SENSOR OR, WHAT IS IT ?© Omek Interactive, Ltd. All rights reserved.
    7. 7. TOUCH-BASED TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
    8. 8. TOUCH-LESS TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
    9. 9. TOUCH-LESS TECHNOLOGY© Omek Interactive, Ltd. All rights reserved.
    10. 10. DEPTH MOTION CAPTURE  Body tracking × Tangible feedback  Natural gestures × 100% stability  Free from controllers × Accuracy© Omek Interactive, Ltd. All rights reserved.
    11. 11. CHALLENGES OR, HOW TECHNOLOGY INFLUENCE DESIGN ?© Omek Interactive, Ltd. All rights reserved.
    12. 12. CHALLENGES OF MOTION CAPTURE • Feeling of a click : virtual tangibility • Active & inactive modes • Am I in control ? • Inner delay of the interface© Omek Interactive, Ltd. All rights reserved.
    13. 13. CHALLENGES OF MOTION CAPTURE • Feeling of a click • Active & inactive modes : differentiate between user behaviors • Am I in control ? • Inner delay of the interface© Omek Interactive, Ltd. All rights reserved.
    14. 14. CHALLENGES OF MOTION CAPTURE • Feeling of a click • Active & inactive modes • Am I in control ? feedback for the users • Inner delay of the interface© Omek Interactive, Ltd. All rights reserved.
    15. 15. CHALLENGES OF MOTION CAPTURE • Feeling of a click • Active & inactive modes • Am I in control ? • Inner delay of the interface : cannot predict the user© Omek Interactive, Ltd. All rights reserved.
    16. 16. DEALING WITH CHALLENGES SELECTING AND SCROLLING ITEMS© Omek Interactive, Ltd. All rights reserved.
    17. 17. Selecting Items • Click gesture : imitate mouse • Hot-spot • Grand gesture • Swipe in • Grab gesture© Omek Interactive, Ltd. All rights reserved.
    18. 18. © Omek Interactive, Ltd. All rights reserved.
    19. 19. Selecting Items • Click gesture • Hot-spot • Grand gesture : grandiose arm movement • Swipe in • Grab gesture© Omek Interactive, Ltd. All rights reserved.
    20. 20. Selecting Items • Click gesture • Hot-spot • Grand gesture • Swipe in : continuous feedback • Grab gesture© Omek Interactive, Ltd. All rights reserved.
    21. 21. Selecting Items • Click gesture • Hot-spot • Grand gesture • Swipe in • Grab gesture : pinch or fist© Omek Interactive, Ltd. All rights reserved.
    22. 22. Selecting Items Gran Grab Click d ge Hot- Swip gest g estu sture sp e in ure ot re Responsiveness v x v v v Active & v x v ½ ½ inactive Stablilty x v v v ½ Comfortable ½ v x v v Usability v v x ½ ½© Omek Interactive, Ltd. All rights reserved.
    23. 23. Scrolling Items • 2D cursor : imitate mouse • 1D cursor • Static active spot • Dynamic active spot • Grand gesture© Omek Interactive, Ltd. All rights reserved.
    24. 24. Scrolling Items • 2D cursor • 1D cursor : scroll on one axis • Static active spot • Dynamic active spot • Grand gesture© Omek Interactive, Ltd. All rights reserved.
    25. 25. Scrolling Items • 2D cursor • 1D cursor • Static active spot : magnifying glass • Dynamic active spot • Grand gesture© Omek Interactive, Ltd. All rights reserved.
    26. 26. Scrolling Items • 2D cursor • 1D cursor • Static active spot • Dynamic active spot : magnifying glass++ • Grand gesture© Omek Interactive, Ltd. All rights reserved.
    27. 27. Scrolling Items • 2D cursor • 1D cursor • Static active spot • Dynamic active spot • Grand gesture : grandiose arm movements© Omek Interactive, Ltd. All rights reserved.
    28. 28. Scrolling Items Gran Stati activ Dyna t 2D c 1D c d ge spot c act e spo sture urso urso mic ive r Responsiveness r v v ½ v v Active & v v x v v inactive Stablilty ½ v ½ v v Comfortable v ½ v ½ x Usability v v ½ v x© Omek Interactive, Ltd. All rights reserved.
    29. 29. Pagination • Just another button: hot-spot / click • Grand gesture: Minority Report • Drag n’ drop: scroll the entire list© Omek Interactive, Ltd. All rights reserved.
    30. 30. User Real Location • Camera view: what the camera sees? • B&W: grey scale colors • Freeze!: control over the character • Trapezoid: view playfield from up • Note over character: main area of concern© Omek Interactive, Ltd. All rights reserved.
    31. 31. Feeling of a Click© Omek Interactive, Ltd. All rights reserved.
    32. 32. CONCLUSIONS OR, WHAT DID WE LEARN ?© Omek Interactive, Ltd. All rights reserved.
    33. 33. Conclusions • Tracking: essential constant feedback • Gesture: toggle action modes© Omek Interactive, Ltd. All rights reserved.
    34. 34. Conclusions • Tracking: essential constant feedback • Gesture: toggle action modes© Omek Interactive, Ltd. All rights reserved.
    35. 35. User Expectations • One hand based control scheme • Grandiose gestures only once in a while • Constant feedback for each movement© Omek Interactive, Ltd. All rights reserved.
    36. 36. Depth Sensor Menu Design Guidelines • Consistency • Simplicity • Feeling of control • Feedback • Comfortable© Omek Interactive, Ltd. All rights reserved.
    37. 37. Future Experiments • Sound based feedback (for blind) • Speech based / speech components© Omek Interactive, Ltd. All rights reserved.
    38. 38. References • Frosted Glass / CAMPAR • EyeSight • Dance Central / Harmonix • Adrenaline Misfits / Konami • Omek Interactive Studios • Media Center / Primesense • PointGrab • Powerup Heroes / Ubisoft • Huang J., White R.W., Buscher G. (2012). User See, User Point: Gaze and Cursor Alignment in Web Search. CHI’12, May 5–10, 2012, Austin, Texas, USA© Omek Interactive, Ltd. All rights reserved.
    39. 39. Thank You For Listening , Shachar Oz Game & UX Designer Omek Interactive Studios www.OmekInteractive.com Shachar@OmekInteractive.com

    ×