Dynamic UIs
James Haliburton, Concept Lead
CONTENTS


•Today & Tomorrow’s Mobile

•Future Visions

•Emerging Tech

•Dynamic UIs
   • Legacy of Mobile
   • What’s in ...
What is a Dynamic UI?




        A dynamic UI experience is about valuing the in-the-
        moment use of the mobile ph...
What is a Dynamic UI?




        4 Principles
        •Provide apt feedback
        •Invite playful, non-task based behav...
Philosophy about the future

At TAT we believe that it is necessary to examine and innovate the future of technology. We a...
TODAY’S MOBILE
TODAY’S MOBILE


   •Capacitive touchscreens   •Accelerometers
   •Multitouch                •Compasses
   •GPS           ...
TOMORROW’S MOBILE




                                        projected UI:s
                3D           Gestures
       ...
UI/UX in mobile moving forward



Changed Input Methods
We have recently moved from phones like the Sony Ericsson T610 wit...
FUTURE VISIONS
FUTURE VISIONS
FUTURE VISIONS
EMERGING TECH
PROJECTOR PHONES
PROJECTOR PHONES: FUTURE
PROJECTOR PHONES

A Radical Shift
•Not merely a matter of scaling up existing UIs and mobile content
•Offers a paradigm sh...
FLEXIBLE DISPLAYS
MULTIMODAL

 A multimodal user interface is a UI that involves more than one modality, or human sense

 A multimodal UI is...
MULTIMODAL
 Sounds can be added to augment the visual design, adding a new dimension to user
   immersion

 Compared to ha...
MULTIMODAL
 Sound can help users to better understand the effects of their interactions, especially
   when using touchscr...
MULTIMODAL
 As proven by film and computer games, sounds can be very expressive

 Sounds should be subtle and varied, to a...
MULTIMODAL: FUTURE




                     22
MULTIMODAL

 A UI ECOSYSTEM

 Mobile UIs are no longer just about button input and visual output
 New input paradigms requ...
AUGMENTED REALITY
GESTURES


What is a gesture?
•Essentially any movement can be a
gesture
•Rich and subtle
•Occurs over space and time
•Unt...
GESTURES


A model for understanding gestures   Symbolic
•Symbolic                            •3x3 Grid
•Iconic           ...
GESTURES


A model for understanding gestures   Iconic
•Symbolic                            •Uses “real world” gestures ma...
GESTURES


A model for understanding gestures   Metaphoric
•Symbolic                            •Uses a familiar metaphor
...
GESTURES


          SYMBOLIC                           ICONIC                       METAPHORIC
    •Pro-user             ...
3D & PHYSICS
3D & PHYSICS
3D & PHYSICS
3D & PHYSICS
                                                                      Focus           Declutter
Flexible Info...
3D & PHYSICS
QUALITIES FROM THIS NEW TECHNOLOGY


Integrating our devices with the real world
•We can use our devices as social tools
•...
TODAY’S MOBILE


   •Capacitive touchscreens   •Accelerometers
   •Multitouch                •Compasses
   •GPS           ...
WHAT’S THE RESULT OF ALL THIS TECH?

A nicer box with a longer feature list.
Minimal UI innovation.
WHAT’S THE RESULT OF ALL THIS TECH?

What about the UI & UX?

UIs are still based around static screen states.

This is le...
WHAT’S THE RESULT OF ALL THIS TECH?




             PC             Mobile Evolution So Far

 Windows                   St...
WHAT’S THE RESULT OF ALL THIS TECH?

Input innovation has been big (qwerty, touch,
accelerometer, predictive text)

•But o...
DYNAMIC UIs - Reacts to real world behaviour
DYNAMIC UIs - Persistency of elements
DYNAMIC UIs - Soft Values, Generative visuals
DYNAMIC UIs - Movable light sources
DYNAMIC UIs - Adaptive Elements
DYNAMIC UIs - Situated Change



                                                      Sony Ericsson S500i

              ...
DYNAMIC UIs - Personalized Space
DYNAMIC UIs - Personalized Behavior
DYNAMIC UIs - Multimodal Design




Dynamic sound can help user to better understand the effects of
their interactions, es...
DYNAMIC UIs - Analogue Manipulation, Physics, Transition Free
DYNAMIC UIs

TRANSITIONS ARE THE NEW STATE

Dynamic Paradigm
     UI
        UIs support real behaviour.Type
             ...
DYNAMIC UIs

TRANSITIONS ARE THE NEW STATE

Dynamic UIs support real behaviour.

This makes sense when you use
technology ...
DYNAMIC UIs

                Dynamic UI Qualities           Static State Based UI Qualities

  Adaptive                   ...
DYNAMIC UIs
                          Dynamic UI Patterns

              Persistency of UI Elements

              Generat...
DYNAMIC UIs

Design tools & processes

Clear mental / behavioural models
•The device has a persona
•Range of behaviour and...
Concept Sketching


          •wireframes                  Typical design production and documentation
v2   v2   •function...
Concept Sketching

Spatial metaphors can be used to reduce
cognitive load since you can work with
familiar principles.

Us...
Concept Sketching
Concept Sketching

Shape vs. Metaphor

•Metaphors have behaviour
•You can put your shape in an
environment

•Simple is goo...
Traditional work process


 Planned
Traditional work process


 Actual
Iterative work process with Cascades
DYNAMIC UIs

Patterns & Tools

3D, Touch, Gestures
Physics
Generative movement & animation
Multimodal design
Clear mental ...
DYNAMIC UIs

Patterns & Tools

3D, Touch, Gestures
Physics
Generative movement & animation
Multimodal design
Clear mental ...
Inspirational Seminars

•3D
•Android
•Multimodal
•Advanced Displays
•Physics
•Sketching & Prototyping
Creating Real Customer Value With 3D

  This seminar explores the use of 3D
  graphics in UIs on mobile devices

  • Inspi...
Android Seminar & Workshop

  TAT presents an inspiring look at the options
  with Android today and tomorrow.

  • Inspir...
Multimodal UIs - User Experience and WOW

  An exploration of a cohesive use of
  sound, haptics, & animation

  • Inspira...
ProCams and the Future of Mobile Displays

  Novel displays and the potentials of
  ProCams (projectors combined with
  ca...
Realism and Physics

  New levels of realism in UIs enabled by
  physics modeling

  • Inspirational examples
  • How can ...
Thinking Beyond Flash Prototyping

  How to capture creativity and design strong
  products through efficient prototyping
...
THANK YOU!
TAT Dynamic UIs 250609
Upcoming SlideShare
Loading in...5
×

TAT Dynamic UIs 250609

1,863

Published on

TAT Webinar on Dynamic UIs

Published in: Technology, Business
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,863
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
162
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

TAT Dynamic UIs 250609

  1. 1. Dynamic UIs James Haliburton, Concept Lead
  2. 2. CONTENTS •Today & Tomorrow’s Mobile •Future Visions •Emerging Tech •Dynamic UIs • Legacy of Mobile • What’s in a phone today • Patterns for Achieving Dynamic UIs • Transitions are the new state • Design tools & processes •Inspirational Seminars
  3. 3. What is a Dynamic UI? A dynamic UI experience is about valuing the in-the- moment use of the mobile phone, and not only the product’s overall usefulness. A dynamic UI creates a pleasurable experience.
  4. 4. What is a Dynamic UI? 4 Principles •Provide apt feedback •Invite playful, non-task based behavior •Respond to “incorrect” interaction •Design forms of interaction which support natural behavior
  5. 5. Philosophy about the future At TAT we believe that it is necessary to examine and innovate the future of technology. We are examining strong use cases which justify and direct the need for this development. We are designing for these scenarios now, as the future is arriving quickly.
  6. 6. TODAY’S MOBILE
  7. 7. TODAY’S MOBILE •Capacitive touchscreens •Accelerometers •Multitouch •Compasses •GPS •High res •Bluetooth •More processing power than it •Wifi took to put a man on the moon!
  8. 8. TOMORROW’S MOBILE projected UI:s 3D Gestures Gyros Augmented reality
  9. 9. UI/UX in mobile moving forward Changed Input Methods We have recently moved from phones like the Sony Ericsson T610 with indirect input (keys) and a color graphical output to the time of the iPhone. Changes in the UI are partially enabled by the change from indirect input to direct input (touch). But do our designs actually take into account that we have constructed new types of devices?
  10. 10. FUTURE VISIONS
  11. 11. FUTURE VISIONS
  12. 12. FUTURE VISIONS
  13. 13. EMERGING TECH
  14. 14. PROJECTOR PHONES
  15. 15. PROJECTOR PHONES: FUTURE
  16. 16. PROJECTOR PHONES A Radical Shift •Not merely a matter of scaling up existing UIs and mobile content •Offers a paradigm shift in the way we interact •Phones have a social presence in our lives more than other technology •They are not fixed to a location or specific task - they aim to be flexible and align to our behavior and to enhance our experiences •Our behavior and experiences are more often than not, situated amongst other people •Private device in a public setting causes a disparity and disconnect •The key difference with procam enabled UIs will be that we can begin to truly interact, work, and play together. We will not be forced to remove ourselves from a conversation to check a mapping application or record a phone number - these can be collaborative actions. •The UI can become a conversational tool.
  17. 17. FLEXIBLE DISPLAYS
  18. 18. MULTIMODAL A multimodal user interface is a UI that involves more than one modality, or human sense A multimodal UI is thus a UI that provides feedback to several human senses, e.g. visual, auditory and haptic (tactile) feedback 18
  19. 19. MULTIMODAL Sounds can be added to augment the visual design, adding a new dimension to user immersion Compared to haptics, sounds can convey information efficiently, e.g. through sound effects, music or even speech Globe Sejf Trum 19
  20. 20. MULTIMODAL Sound can help users to better understand the effects of their interactions, especially when using touchscreens Visual design themes can have corresponding sound themes, enhancing the metaphor of the GUI Water Fire Paper 20
  21. 21. MULTIMODAL As proven by film and computer games, sounds can be very expressive Sounds should be subtle and varied, to avoid annoying people around the person using the device Example: Asia cute UI with sound 21
  22. 22. MULTIMODAL: FUTURE 22
  23. 23. MULTIMODAL A UI ECOSYSTEM Mobile UIs are no longer just about button input and visual output New input paradigms require new approaches to output, for example: Touchscreen input -> usability need for haptic and/or sound feedback Gesture input (via touchscreen or motion sensor) -> expectation of visual output that follow gestures realistically, e.g. through physics modeling 3D aesthetics and realism -> increasing expectations that UIs should mimic the real world, e.g. by including sound and haptics In a well designed device, output is closely linked and matched to user input Changing one part of a UI will affect other aspects 23
  24. 24. AUGMENTED REALITY
  25. 25. GESTURES What is a gesture? •Essentially any movement can be a gesture •Rich and subtle •Occurs over space and time •Until recently we’ve been limited mostly to 1-dimensional button pressing •But touch brings in new possibilities
  26. 26. GESTURES A model for understanding gestures Symbolic •Symbolic •3x3 Grid •Iconic •Must learn what the gesture means •Metaphoric •“automagical” •Gesture is symbolic of the action occuring
  27. 27. GESTURES A model for understanding gestures Iconic •Symbolic •Uses “real world” gestures matched to actions on •Iconic the wcreen •Metaphoric •Flipping like you do your CD collection •Motion matches something we already know how to do
  28. 28. GESTURES A model for understanding gestures Metaphoric •Symbolic •Uses a familiar metaphor •Iconic •Doesn’t necessarily match real world interaction •Metaphoric •Needs support of sound, and visuals
  29. 29. GESTURES SYMBOLIC ICONIC METAPHORIC •Pro-user •Instantly understood •easily remembered •magic quality •slower to perform than •dependent on other UI •good for supporting surprise arbitrary gestures elements •can be seperate from the •context dependent •can be used to discover new graphic UI •highly satisfying functionality •‘But supported by feedback in •can find inspiration in industrial •gesture is enabler other UI elements design affordances •Can be difficult to remember •gesture is the meaning •Gesture is its own layer A model for understanding gestures
  30. 30. 3D & PHYSICS
  31. 31. 3D & PHYSICS
  32. 32. 3D & PHYSICS
  33. 33. 3D & PHYSICS Focus Declutter Flexible Information Visualization (FIV) Describes the elements of 3D which allow a UI to better display large amounts of data, like playlists, in Zoom & Overview FIV Realtionships Optimize interesting ways relevant to the user and to the application being used. Reduce Cognitive Load Show Options Naturalized Interaction (NI) Describes the elements of a 3D UI which minimize how much a user needs to think about how theyʼre using an application, and allows them to simply use it. NI Flatten Learning Curve Show Affordances WOW - Effect Visual Style and Feedback (VSF) Describes how 3D can add a “wow effect” to a product and better differentiate itself from competitors. Visual Immersion VSF Holistic & Seamless Feedback
  34. 34. 3D & PHYSICS
  35. 35. QUALITIES FROM THIS NEW TECHNOLOGY Integrating our devices with the real world •We can use our devices as social tools •They can be flexible to our needs •Encourage fun and playfulness •Designs can be rich and creative
  36. 36. TODAY’S MOBILE •Capacitive touchscreens •Accelerometers •Multitouch •Compasses •GPS •High res •Bluetooth •More processing power than it •Wifi took to put a man on the moon!
  37. 37. WHAT’S THE RESULT OF ALL THIS TECH? A nicer box with a longer feature list. Minimal UI innovation.
  38. 38. WHAT’S THE RESULT OF ALL THIS TECH? What about the UI & UX? UIs are still based around static screen states. This is legacy of WIMP (windows, icons, menus, pointing device), but we’ve been talking about post-WIMP for 15 years. This makes sense with simple task based operations. Screen states support consistency and simple flows.
  39. 39. WHAT’S THE RESULT OF ALL THIS TECH? PC Mobile Evolution So Far Windows Static screens Icons Icons, Widgets Menus Menus in soft keys Pointing Device (mouse, D-Pad, Trackball, Touch trackpad) pointing
  40. 40. WHAT’S THE RESULT OF ALL THIS TECH? Input innovation has been big (qwerty, touch, accelerometer, predictive text) •But output and presentation have been mostly "cosmetics" (if that) •iPhone did a big jump in creating a more natural interface (Realtime, direct manipulation) •There are more steps in making devices less like machines - interface with real world feel (dynamic)
  41. 41. DYNAMIC UIs - Reacts to real world behaviour
  42. 42. DYNAMIC UIs - Persistency of elements
  43. 43. DYNAMIC UIs - Soft Values, Generative visuals
  44. 44. DYNAMIC UIs - Movable light sources
  45. 45. DYNAMIC UIs - Adaptive Elements
  46. 46. DYNAMIC UIs - Situated Change Sony Ericsson S500i Theme changes appearance depending on time of day, date and special events. Appearance changes based on the time of day and special events.
  47. 47. DYNAMIC UIs - Personalized Space
  48. 48. DYNAMIC UIs - Personalized Behavior
  49. 49. DYNAMIC UIs - Multimodal Design Dynamic sound can help user to better understand the effects of their interactions, especially when using touchscreens. Water Fire Paper 50
  50. 50. DYNAMIC UIs - Analogue Manipulation, Physics, Transition Free
  51. 51. DYNAMIC UIs TRANSITIONS ARE THE NEW STATE Dynamic Paradigm UI UIs support real behaviour.Type Transition User Reaction Static Screen No Transition Where am I? This makes sense when youscreen transition Static Screen Full use Something happened, I’m technology that connects your device to somewhere else the real world. Static Screen Smart full screen transition I understand where I’ve arrived Dynamic UI Link flow transition I can do some new things now Dynamic UI Transformation of UI elements I can repurpose these UI elements for what I want Dynamic UI User controlled space I’m casually browsing, but have control to do what I like
  52. 52. DYNAMIC UIs TRANSITIONS ARE THE NEW STATE Dynamic UIs support real behaviour. This makes sense when you use technology that connects your device to the real world.
  53. 53. DYNAMIC UIs Dynamic UI Qualities Static State Based UI Qualities Adaptive Fixed User expectations are different PC based expectation Metaphoric, natural behavior Abstract behavior Empathetic Unforgiving Creates dialogue with user Directing Explorative Functional Playful Task based Holistic Fragmented Can communicate soft values Hard values
  54. 54. DYNAMIC UIs Dynamic UI Patterns Persistency of UI Elements Generative Visuals Apt Reactions to Physical Input Movable Light Sources Contextual Change Personalization of Space Adaptive UI Elements Synchronized Multimodal Design Analogue Manipulation Patterns Free Moving Elements With Physics Models
  55. 55. DYNAMIC UIs Design tools & processes Clear mental / behavioural models •The device has a persona •Range of behaviour and personality is the consistency which pulls a dynamic UI together - aligns to situations •Can derive both design patterns & tech requirements Agile Work Process •All stakeholders involved •Horizontal, not vertical process •Rapid iterations •Important to sketch on device!
  56. 56. Concept Sketching •wireframes Typical design production and documentation v2 v2 •functional specifications •paper prototypes •flash movies has difficulty communicating the qualities of a v1 •UI flows dynamic user experience. •etc. The tools we use for sketching also leave us short as designers in actually testing the experience of what the UI will be like. We should be able to experience and communicate the rich behavior and interactions of our UIs.
  57. 57. Concept Sketching Spatial metaphors can be used to reduce cognitive load since you can work with familiar principles. Using a metaphor can make it easier to achieve consistency throughout the UI. Communicating camera angles in 3D UIs It is the first step in creating a consistent behavioral model. Communicating 3D or behvioral UIs within teams can be troublesome, to have a spatial metaphor can make discussions more direct. A shape alone is not a metaphor. Sketching of 3D UIs using spatial metaphors However, don’t go overboard! It’s still a UI and you can break the rules!
  58. 58. Concept Sketching
  59. 59. Concept Sketching Shape vs. Metaphor •Metaphors have behaviour •You can put your shape in an environment •Simple is good •Spatial metaphors are all about being consistent •Like personas for interaction behaviour •You are testing how the UI behaves and feels
  60. 60. Traditional work process Planned
  61. 61. Traditional work process Actual
  62. 62. Iterative work process with Cascades
  63. 63. DYNAMIC UIs Patterns & Tools 3D, Touch, Gestures Physics Generative movement & animation Multimodal design Clear mental models, Agile work processes A dynamic UI experience is about valuing the in-the-moment use of the mobile phone, and not only the product’s overall usefulness. A dynamic UI creates a pleasurable experience.
  64. 64. DYNAMIC UIs Patterns & Tools 3D, Touch, Gestures Physics Generative movement & animation Multimodal design Clear mental models, Agile work processes There is no best practice. Our UI design patterns will come from practices of physical computing, industrial design, and good field research - not the desktop PC.
  65. 65. Inspirational Seminars •3D •Android •Multimodal •Advanced Displays •Physics •Sketching & Prototyping
  66. 66. Creating Real Customer Value With 3D This seminar explores the use of 3D graphics in UIs on mobile devices • Inspirational examples • Framework for analyzing and working with 3D • What’s the real customer value of 3D? • Provide a 3D experience from low-end to high-end devices • A 3D roadmap to the future • TAT designs & concepts for you Typically the palette of "3D techniques" has been comprised of lighting, shadows, focus, depth, camera angles and similar visual elements. At TAT, we believe that 3D is more than 3D graphics. We understand "3D" in user interfaces as a paradigm where the appearance of the interface displays three-dimensional or real-world qualities, while the technology behind the visualization is not necessarily built up as a real 3 dimensional simulation. To benefit from the full potential of 3D in mobile user interfaces we also need to consider the total user experience.
  67. 67. Android Seminar & Workshop TAT presents an inspiring look at the options with Android today and tomorrow. • Inspirational examples for you • Differentiating your UI on Android • Configurations for 3D, hardware acceleration, silo apps, more • Suggested roll-out and strategy for this new platform • How does the Cascades UI Framework enhance Android? • How to get better performance and enhance hardware efficiency • Hands-on demonstration of Android development The new kid on the block is making waves, but how do we harness all that Android has to offer? What is Android? What can it do for you today? What can it do for you tomorrow? By providing the interaction design and the visual look & feel of the G1, TAT has been part of Android right from the beginning. Already we have deep experience and insight into both the technical and UX opportunities available and can help you push the limits of 3D, unique transitions, work processes, prototyping, and more. Mixing pragmatism and inspiration TAT will help you address key issues for your organization with Android.
  68. 68. Multimodal UIs - User Experience and WOW An exploration of a cohesive use of sound, haptics, & animation • Inspirational examples • How can sound and haptics enhance UX? • A taxonomy for multimodal UIs • Usability factors & aesthetic concerns • Synchronizing animations with sound and haptics • TAT designs & concepts for you Like films and computer games, the mobile UI can become more engaging and immersive by including sounds and haptics. Sound and haptics can improve usability on touchscreen devices where visual feedback is not always sufficient. To achieve a true WOW factor, sound, haptics, and visuals must all be synchronized together.
  69. 69. ProCams and the Future of Mobile Displays Novel displays and the potentials of ProCams (projectors combined with cameras) • Inspirational examples • Overview of emerging display technologies • Opportunities and challenges of new displays • Different aspects of interaction with projected UIs • New interaction paradigms of flexible and transparent displays • TAT designs & concepts for you Projected graphics will enable mobiles to become far more social than today. Collaboration can be enabled by using the projection as input surface Since pressing buttons will interfere with projections, other ways of interaction, such as camera captured gestures or speech input, become necessary Flexible and transparent displays open up new interaction possibilities, such as “bend to zoom”
  70. 70. Realism and Physics New levels of realism in UIs enabled by physics modeling • Inspirational examples • How can physics and realism enhance UX? • To what parts of a UI can physics be applied? • How to design realistic output that corresponds to input gestures • Benefits and drawbacks of realism • Simulating physics on low end and high end devices • TAT designs & concepts for you Mobiles with touchscreen or motion sensor input can become more intuitive and impressive if graphics behave realistcally, modeling physics. UIs that model physics are responsive to subtle gestures, which makes them inviting and playful. When used right, realistic UIs can both enhance usability and add a considerable WOW-factor.
  71. 71. Thinking Beyond Flash Prototyping How to capture creativity and design strong products through efficient prototyping • Inspirational workflow examples • What is a sketch? What is a prototype? • An overview of prototyping tools and methods • Communicating ideas through a multitude of sketches • Useful skills for efficient prototyping • TAT process and tools for prototyping Introducing new prototyping methods can open up new opportunities by increasing product differentiation At TAT we believe that many important aspects of new products and services can be designed and evaluated before entering a resource-heavy prototyping phase. By introducing practices for improved sketching at your company, a more creative and flexible prototyping culture will emerge. TAT believes that artefacts radically improve communication and that all aspects of products, services, processes and business models can be visualized.
  72. 72. THANK YOU!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×