Personal interaction via multi-touch gesturesPresentation Transcript
Personal interaction via multi-touch gestures A step toward NUI Alessio Malizia, Associate Professor Andrea Bellucci, Ph.D. candidate Dept. of Computer Science, Universidad Carlos III de Madrid, Spain
SynopsisResearch areasBackgroundProblem(s)Question(s)HypothesisContributionExpected result
Research areasHuman Computer InteractionNatural User InterfacesInteraction/UX Design
Human Factors1960s –CLI 1980s –GUI 2000s –NUICommand Line Interface Graphic User Interface Natural User InterfaceMemorization/Recall Recognition (recall = shortcuts) Recall? Gestures are like shortcuts
ExplorationDon’t touch me!Demo @ACM Int. Conference AVI 2010 TESIS: Turn Every Surface into an Interactive Surface Demo @ACM Int. Conference ITS 2011
Question(s)• How can I create an interface in which users may quickly gain expertise using actions that feel natural to them? (research question 1)• How can I be sure that I succeeded in creating an interface that feels natural to the user? (research question 2)
Hypothesis• Natural properties are related to universals• The concept of affordance is crucial for defining a widget-level architecture for NUI – Interface as a Service – Widgets expose their affordances• Personalization and adaptation/evolution as a step toward natural interaction
I’d like to convince you (thesis)• Gestures (as signs) are something alive, which change depending on cultural and human aspects, time and context. Therefore users’ personalization can be a step to address natural interaction.• Refined research question 1: to what extend personalization of multi-touch gestures can be valuable for enabling natural interaction?
Natural properties and Universals• Concept of perfect natural properties from metaphysics – Perfect natural properties are shared among all the elements of a set• For we humans, perfect natural properties are our perception faculties (hearing, feeling by touch, seeing, tasting, smelling) – There are also acquired skills...• Could there exist such a thing as a natural gesture? – Take into account users innate and learned abilities
Which one is natural? iPhone: pinch for zooming Samsung Galaxy s2: holding down on the screen withZooming in real life: two fingers andbring the newspaper closer moving your phone back and forth
Affordance• Why affordance is important for natural interaction? – Gibson (1977) defined affordances as all action possibilities latent in the environment – Affordances as perceived action possibilities (Norman, 1988) – Affordance is the visual clue to the function of the object (an opportunity for action)
Design without affordanceYou can move an object in single-pixelincrements by nudging it.Touch and hold the object with one finger,and then use another finger to swipe acrossthe page in the direction you want the objectto move. From iPad’s Pages gestures list
A good affordance
What happens if you attempt to makeusers interact in a way which is notnatural to them?
Beyond affordance languages Form factor Adaptation/Medium/High Evolution Personalization Small Display Personal/ Public/ Private Shared