Investigating	  Perceptible	  Affordances	  of	  Touch-­‐Based	  NUIs:	  New	  Designs	  Paradigms	  for	  Novel	  Interac...
Extreme Reality Ltd. XTR3D (Android)Phones with hover state (SONY XPERIA) and eye gaze interaction (SAMSUNG S4)
It’s not supposed to be like this…SIRI
Or like this...Gmail	  Motion,	  April	  2011	  
Is there a problem?
(…) How is anyone to know, first, that thismagical gesture exists, and second, in whichsettings it operates?Norman, D. Nie...
•Shift from mouse and keyboard towards touch-based devices:•No hover state, no close button, no right click, no keyboard s...
•Strange hybrids emerged...
•Unfamiliarity with novel visual metaphorswithin NUIs.•The ‘hidden menus and toolbars’ issue.•The ‘hidden gestures’ issue.
Windows 8
Josh	  Clark,	  Buttons	  are	  a	  Hack,	  2011:	  http://vimeo.com/21796111
BlackBerry Playbook, 2012
Some theory...
"Perceived or Perceptible Affordances" areactions you understand just by lookingat the object, before you start using it (...
+ WHAT IS IT?+ WHAT AM I SUPPOSED TO DO?+ WHAT’S GOING TO HAPPEN IF I DID?= GOOD PERCEPTIBLE AFFORDANCEWHAT"S GOING TO HAP...
PERCEPTIBLE AFFORDANCE
INPUTNUI [MODE OF INTERACTION LAYER]PERCEPTIBLE AFFORDANCEPost-WIMP GUI [INTERFACE LAYER]OUTPUTExecution CycleEvaluation C...
NUI [MODE OF INTERACTION LAYER]Post-WIMP GUI [INTERFACE LAYER]Execution CycleEvaluation CyclePERCEPTIBLE AFFORDANCEAmantʼs...
12th point systemObservation Phase• Perception1. Perception (Yes/No)• Identification2. What is it?3. What should you do?• ...
Post-WIMP GUI [INTERFACE LAYER]Evaluation Cycle
NUI [MODE OF INTERACTION LAYER]Execution Cycle
The Eye Tracking study
Eye Tracking study:order + focus + planning + expectation + bias = MENTAL MODEL
Issues to be investigated/Interactions to tested (08)• Hidden Menus and toolbarsUnveiling MenuBringing the picture to the ...
DESIGN 01: “SMUDGY”
DESIGN 02: “DOTS”
1. Unveiling MenuSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification...
2. Bringing the picture to the pageSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/N...
3. Press and hold for optionsSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Id...
4. Zoom In/OutSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2...
5. Flip pagesSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2....
6. Alternate applicationsSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identi...
7. Unveil task switcherSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identifi...
8. Minimize applicationSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identifi...
More questions thananswers...
How people are making sense of new technologies?How to create better designs adapted to NUI?When is the best moment to pre...
“The most profound technologies are those that disappear.They wave themselves into the fabric of everyday life untilthey a...
Thank you for your attention.Questions?Jacques Chuekejacques.chueke.1@city.ac.uk@jchueke
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London
Upcoming SlideShare
Loading in …5
×

Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London

1,560 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,560
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Investigating Perceptible Affordances of Natural User Interfaces - Jacques Chueke, City University London

  1. 1. Investigating  Perceptible  Affordances  of  Touch-­‐Based  NUIs:  New  Designs  Paradigms  for  Novel  InteractionsJacques ChuekeLondon, UK, May 2011George Buchanan(1st Supervisor)Lecturer, Centre for HCI DesignStephanie Wilson(2nd Supervisor)Lecturer, Centre for HCI DesignMaster in Design, PUC-Rio, RJ, BrazilPhD Researcher at the Centre for HCI DesignSchool of Informatics, City University London
  2. 2. Extreme Reality Ltd. XTR3D (Android)Phones with hover state (SONY XPERIA) and eye gaze interaction (SAMSUNG S4)
  3. 3. It’s not supposed to be like this…SIRI
  4. 4. Or like this...Gmail  Motion,  April  2011  
  5. 5. Is there a problem?
  6. 6. (…) How is anyone to know, first, that thismagical gesture exists, and second, in whichsettings it operates?Norman, D. Nielsen, J. (2010)One of the powers of modern computers isdiscoverability, you can explore, but with gesturesystems it’s a pain. It’s amazing how many thingspeople don’t know about the computers they useand there’s no way to find out.Norman, D. (2012)
  7. 7. •Shift from mouse and keyboard towards touch-based devices:•No hover state, no close button, no right click, no keyboard shortcuts...
  8. 8. •Strange hybrids emerged...
  9. 9. •Unfamiliarity with novel visual metaphorswithin NUIs.•The ‘hidden menus and toolbars’ issue.•The ‘hidden gestures’ issue.
  10. 10. Windows 8
  11. 11. Josh  Clark,  Buttons  are  a  Hack,  2011:  http://vimeo.com/21796111
  12. 12. BlackBerry Playbook, 2012
  13. 13. Some theory...
  14. 14. "Perceived or Perceptible Affordances" areactions you understand just by lookingat the object, before you start using it (orfeeling it, if its a physical device rather thanan on-screen UI element).Gaver (1991), Norman (1999 & 2008),Amant (1999), Nielsen (2008)
  15. 15. + WHAT IS IT?+ WHAT AM I SUPPOSED TO DO?+ WHAT’S GOING TO HAPPEN IF I DID?= GOOD PERCEPTIBLE AFFORDANCEWHAT"S GOING TO HAPPEN IF I DID IS BEING REGARDED NOWADAYS AS FEEDFORWARD, CHECKTHE WORK OF Wensveen & Djajadiningrat and Vermeulen
  16. 16. PERCEPTIBLE AFFORDANCE
  17. 17. INPUTNUI [MODE OF INTERACTION LAYER]PERCEPTIBLE AFFORDANCEPost-WIMP GUI [INTERFACE LAYER]OUTPUTExecution CycleEvaluation CycleInterpretationPerception EvaluationFormulation ofIntentionSpecification ofActions SequenceExecution*  Based  on  Norman’s  Theory  of  Action,  1986
  18. 18. NUI [MODE OF INTERACTION LAYER]Post-WIMP GUI [INTERFACE LAYER]Execution CycleEvaluation CyclePERCEPTIBLE AFFORDANCEAmantʼs Conceptual Framework on Perceptible Affordances (1999).Wigdor & Wixton (2011) review of Wuʼs theory of RCT.
  19. 19. 12th point systemObservation Phase• Perception1. Perception (Yes/No)• Identification2. What is it?3. What should you do?• Comprehension4. How many fingers?5. Touch/hold or swipe?6. In which direction?7. What will change?SCORE“0” for wrong assessment/inability to respond“0.5” for lack of confidence or right assessment/interaction on the 2nd and 3rd times“1” for correct response/interactionInteraction Phase• Registration8. Touch to confirm• Continuation9. Perform touch/hold or swipe10. Perform direction• Termination11. System status12. Restore status (restartingthe observation phase)
  20. 20. Post-WIMP GUI [INTERFACE LAYER]Evaluation Cycle
  21. 21. NUI [MODE OF INTERACTION LAYER]Execution Cycle
  22. 22. The Eye Tracking study
  23. 23. Eye Tracking study:order + focus + planning + expectation + bias = MENTAL MODEL
  24. 24. Issues to be investigated/Interactions to tested (08)• Hidden Menus and toolbarsUnveiling MenuBringing the picture to the page• Object ManipulationPress and hold for options• Workspace ManipulationFlip pages• Hidden gestures for iOs ManipulationZoom In/OutAlternate between running appsUnveil task switcherMinimize app
  25. 25. DESIGN 01: “SMUDGY”
  26. 26. DESIGN 02: “DOTS”
  27. 27. 1. Unveiling MenuSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  28. 28. 2. Bringing the picture to the pageSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  29. 29. 3. Press and hold for optionsSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  30. 30. 4. Zoom In/OutSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  31. 31. 5. Flip pagesSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  32. 32. 6. Alternate applicationsSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  33. 33. 7. Unveil task switcherSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  34. 34. 8. Minimize applicationSeries 02” “SMUDGY”Series 01: “DOTS”Observation Phase• Perception: 1. Perception (Yes/No)• Identification: 2. What is it? 3. What should you do?• Comprehension: 4. How many fingers? 5. Touch/hold or swipe? 6. In which direction? 7. What will change?Interaction Phase• Registration 8. Touch to confirm• Continuation 9. Perform touch/hold or swipe 10. Perform direction• Termination 11. System status 12. Restore status (restarting the observation phase)
  35. 35. More questions thananswers...
  36. 36. How people are making sense of new technologies?How to create better designs adapted to NUI?When is the best moment to present the visual cue? Inwhich context?Automatic at first runs? How many times? If unrequiredwould imply in loss of control? Will be obtrusive?Perhaps only if summoned/triggered by user action?Interactive tutorials only? Mandatory tutorials?Conflict of specific application mode x OS mode. How todesign something which is CLEARLY related to either?
  37. 37. “The most profound technologies are those that disappear.They wave themselves into the fabric of everyday life untilthey are indistinguishable from it.” Weiser (1991)The equipment fades into the background. This unspokenbackground against which our actions are played out is atthe heart of Heideggerʼs view of being-in-the-world.”Dourish (2005: 109)
  38. 38. Thank you for your attention.Questions?Jacques Chuekejacques.chueke.1@city.ac.uk@jchueke

×