Re Surface Designing Natural User Interfaces - Shawn Konopinsky


Published on

Re Surface Designing Natural User Interfaces
Mix Essentials 2009 - Microsoft

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Need better wording here...
  • AT&T is using Microsoft Surface to interact with customers in new ways and help them learn about mobile devices and applications. With Microsoft Surface, customers in AT&T stores can simply place a mobile device on a Microsoft Surface display, then immediately see the range of features and options on that device. Customers can also touch and move maps to view coverage plans in their area. In the future, AT&T customers will be able to “grab” content such as ringtones, graphics, video and more and drop it directly into their mobile device.Harrah’s Entertainment is a Microsoft Surface pioneer that is using Microsoft Surface to deliver a richer experience to guests. Harrah’s created a suite of applications for Microsoft Surface, ranging from sending photos across a lounge to other patrons, customizing drink orders, playing games, or taking a virtual tour of Harrah’s properties.Sheraton Hotels is installing Microsoft Surface to enrich customer experience. Customers at a Microsoft Surface can scroll through menu items, order food and drinks, learn about the wine they’re drinking because Microsoft Surface can identify “tagged” glasses and call up information about what’s in the glass. Microsoft Surface also can identify customers through loyalty cards, recalling their favorite items and automatically billing selected items to room.
  • ….. The three points above really tell a mini-story of their own.Engage- Pull customers over the lease line with a new and exciting device (BofA OR perhaps there is a lounge (Emirates) where people chilled before,.. OR have children engage because it has elements of play like a Nintendo Wii. and now they have this unobtrusive device that has nature affordances to draw people to engage.Discovery – Now they are engaged with device, people of all ages and computer literacy to further communicate the benefits of being a member, or where the other locations are…. Or just pure education. Collaboration – now people are getting information and we throw another person in the mix… now consequential communication starts to play OR even using the device as a medium for collaboration. Sales. Kids learns from one and another. Etc… Deliver a memorable experience that is educating and entertaining. Microsoft Surface is unlike any other type of computing experience. Its engaging, social user interface allows customers to create experiences that your competitors aren’t offering. Help customers intuitively discover your products and services by exploring and discovering similar products and associated services. For example, in the hospitality industry users can discover other properties and even take a virtual tour of related locations. Enable customers to interact with each other. Microsoft Surface encourages 360-degree interaction, information exchange, and collaboration among multiple users. Give ownership of the result by putting the end-user in the driver’s seat, and having them play a major role in discovering, learning, and making informed decisions about products and services.
  • Need better wording here...
  • Microsoft Surface is fundamentally different from any other computing platform or touch-screen device.First, it can accommodate multiple users, all working simultaneously on Microsoft Surface. No mouse-sharing, no sequential exchanges of data. As many people as can fit around a Microsoft Surface can work on a Microsoft Surface – all at the same time. That makes collaboration easy and productive in a way that a kiosk or standalone PC cannot.Second, Microsoft Surface is “multi-touch.” No need to use a single finger or mouse pointer. People can draw, manipulate shapes and move data with one finger, five fingers, or 10. In fact, Microsoft Surface can recognize up to 52 different touch points.Third, Microsoft Surface is completely natural. People move data around Microsoft Surface the same way they move a saucer across a table – with their hands. And because they touch Microsoft Surface, the interaction is more natural, without the need for another electronic gadget to act as intermediary.Finally, Microsoft Surface does more than recognize touch – it can see and identify objects with specific labels (barcode or similar tag) as well as digital wireless devices such as wireless phones, digital cameras, PDAs and more. It also can manage multiple devices at a time, and handle tasks such as transferring photos from a camera to a wireless phone.
  • Principals – creating great UI by mixing and matching the principals.Intuitive – because the objects have implicit affordances.Directed – driven highly by the user. Exploratory. File menu, list of choices, etc.. Contextual. Given some elements and derive the experience based on the context given to me.Possible paths vs. rate of interactions. Lots of paths but low interaction. … NUI. Few paths, but very frequent dialogue with the system.The last pattern is the perception of the system behaviour. Static – doesn’t move unless you count moving up each time you type a line. Responsive. As you interact with a GUI the system is changing and updating in response to the input. The NUI is also responsive but takes it a step further to be evocative. Help suggest use based on how it responds.
  • Performance Aesthetics:Principal of Direct of Manipulation. WYDWYGScaffolding. Just enough to understand the next level of complexity. Analogy is scaffold, once you have mastered the first level you can get to the second level. Game play. First learn to jump, then to shoot, then to jump and shoot. Contextual Environment. Principal of Super Real. Leverage existing knowledge of how people interact with everyday things. But we take that too literally, and we try to mimic the real world. What we can do is add to base experience to add things that wouldn’t be possible in real life, and that feeds into the aspect of performance aesthetics. Why bother?Interacting with NUI is new and magicalMechanics of the situation – really strong emotional connections with these NUI experiences. Very strong brand builder.Now we have behaviour – it is influential because it isn’t passive. It forges a connection.
  • Portal.
  • Local vs. Global Control.Thinking about how multiple people will interact with a system requires some thought on the separation between the application as a whole, the global aspect, versus what a single user may take control of, the local aspect.
  • Designing the application to be approached from any direction.
  • Social Design.Not only designing the application,But designing the social interactionsContext / Appropriate to jump in? Who drives?Occluding someone else's controls.
  • New technology. Expensive Looking device that isn’t theirs. Therein lies the challenges to draw users into the experience and keep them engaged.
  • Virtual keyboard has a huge error rate.Alternatives. THR Card ID system.
  • It is also important to remember that a NUI can understand a substantial amount of information about the placement and orientation of a hand. Leverage this to your advantage in a design. Think of how people actually view their cards when playing poker and incorporate that gesture. Don’t try and force an unnatural or unintuitive gesture into a context that doesn’t fit. Mime the action to understand its natural movement in the real world.
  • In GUI we had really no choice to control by proxy. Adjust a slider, or hit a button... With Surface, the content becomes the interface. Control directly if possible without the indirection.Who do people know that it is movable? Use the introduction animations as a clue. Have them start at the end position and slide back into the starting position, giving the user clues as to what it does.
  • Paper Prototype image from MS Concierge.They tested this against users... During the user testing the participants could write their thoughts on the laminated cards to involve them not only by testing, but in the design. They actually had paper on the side so people could create things that they expected to see but weren’t involved in the original design. Much stronger connection between you and the folks that will eventually using the products.This is ideal for NUI because when you are doing web 2.0 there is a different metaphor and it mouse.. In NUI it is easy because the paper is much closer to the actual objects, and the interaction is the gestures that folks actually take on the paper objects.
  • Re Surface Designing Natural User Interfaces - Shawn Konopinsky

    1. 1. re:Surface Designing Natural User Interfaces Shawn Konopinsky
    2. 2. Surface Video
    3. 3. Business Design Approach
    4. 4. Business Design
    5. 5. • Multi-User • Direct Interaction Multi-Touch Object Recognition
    6. 6. Principle Driven Design
    7. 7. Static Responsive Evocative Disconnected Indirect Direct High-Low Double Medium Fast Few Directed Exploratory Contextual Recall Recognition Intuition CLI GUI NUI Text Graphics Objects
    8. 8. Principle of Performance Aesthetics Evocative Principle of Direct Manipulation Direct Principle of Scaffolding Fast Few Principle of Contextual Environments Contextual Principle of the Super Real Intuition NUI Objects
    9. 9. Principle of Performance Aesthetics Joy of Doing. Pleasure from interaction over accomplishment.
    10. 10. Principle of Direct Manipulation WYSIWYG WY[DO]IWYG
    11. 11. Principle of Scaffolding
    12. 12. Principle of Contextual Environments
    13. 13. Principle of Super Real
    14. 14. Principal Driven Design
    15. 15. Spatial
    16. 16. Seamless
    17. 17. Social
    18. 18. Challenge: Multi-User
    19. 19. Challenge: Multi-User Collaboration
    20. 20. Challenge: Multi-User
    21. 21. Challenge: Multi-User Control
    22. 22. Challenge: Multi-User 360⁰ Interactions
    23. 23. Challenge: Multi-User Social Design
    24. 24. Challenge: Engaging Users
    25. 25. Challenge: Engaging Users Attract or Instruct Exploration Instruct
    26. 26. Challenge: Engaging Users Forgiving
    27. 27. Challenge: Engaging Users Visualizing Touch
    28. 28. Challenge: Input
    29. 29. Challenge: Input Avoid the Keyboard
    30. 30. Challenge: Input Avoid the Keyboard
    31. 31. Challenge: Input Touch Direction
    32. 32. Challenge: Input Content is the Interface
    33. 33. Business Design Approach
    34. 34. Concept Interaction Ideation Visual Design Production Polish Design
    35. 35. Development Sketch Evaluate Prototype Design Evaluate
    36. 36. Approach: Sketching
    37. 37. Approach: Prototyping
    38. 38. Approach: Team XAML Production Surface Design Visual Interaction Design Design
    39. 39. Shawn Konopinsky Twitter: @shawnkono