Presentation CE-4026 by David Brebner at the AMD Developer Summit (APU13) November 11-13, 2013

1. NEW INTERFACES
DAVID BREBNER

2. New Interfaces
Gestural & Cinematic experiences

3. GESTURAL AND CINEMATIC
The continuum of interaction
Content Creation & Productivity
Content Consumption and Entertainment
From precise to casual
3 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

4. GESTURAL AND CINEMATIC
Interaction with others
Individual
4 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Online
Together

5. GESTURAL AND CINEMATIC
Ergonomics
Un-answered questions…
How will people use large form factor systems?
How long do they use these devices for?
Do they stand or sit?
What angle is their arm, wrist, hand?
What % of interaction is touch vs. keyboard vs.
mouse?
• How do the ratios change depending on task?
•
•
•
•
•
Lots of publicly available research and
recommendations
5 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Very little publicly available research and no industry
guidelines

6. GESTURAL AND CINEMATIC
NEW INTERFACES
What does the App do?
Function
Gestural
6 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Minimal
Consistent
Obvious
Cinematic

7. GESTURAL AND CINEMATIC
NEW INTERFACES
Function
How does the App look?
Gestural
7 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Cinematic
Responsive
Animated
Explanatory
Revelatory
Delightful

8. GESTURAL AND CINEMATIC
NEW INTERFACES
Function
How do you interact?
Manipulate
Direct
Responsive
Reversible
Hinted
Consistent
Gestural
8 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Cinematic

9. GESTURAL AND CINEMATIC
NEW INTERFACES
Gestural
‒ Direct manipulation – not memorizing gestures
‒ Touch, voice, 3D and more
‒ Sweeping movements
‒ Allow for mastery
Cinematic
‒ Great visuals
‒ Show actions with animation and state
‒ Use lighting, particles, focus, depth of field to highlight elements
‒ Use warps and motion to transition between elements
‒ Use 3D content and environments but provide 2D workspaces
9 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

10. GESTURAL AND CINEMATIC
ACTIONS
Give feedback of action triggered
Provide a way to undo the action
Actions to be commutative (any order)
Physical ‘manipulation’ should be obvious and
reversible (move, rotate, resize etc)
Actions should be repeatable and reliable
(don’t rely on gestures which the sensor cannot repeat)
Reveal the gestures and their function (with hints)
10 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
Classic animation techniques like bounce, squash &
stretch, follow through are now being used (often on
abstract shapes) to illustrate actions

11. GESTURAL AND CINEMATIC
SPATIAL GESTURE
Using a 3D camera can have lighting issues
It’s NOT touch at a distance
Because there is no control surface it is hard
distinguishing between actions and movement
Support spatial control surfaces
Add speech for actions
We need to find new gestural rules
3D Gesture has tracking but no ‘action’
Fingers, hands and arms obscuring the view of
the screen
11 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

12. GESTURAL AND CINEMATIC
GESTURAL INPUT
Touch
Multi touch
Near touch
Touch + pressure
Stylus + touch
Stylus + pressure + angle
3D camera
Gaze + voice
Wearable's + GPS + Gyro
12 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

13. GESTURAL AND CINEMATIC
VOICE, PEN, NEAR TOUCH, TOUCH
Reach towards the screen and receive localised hints
Touch the screen to manipulate (move, resize, rotate,
drag, pick)
Reach towards the screen with pen and receive hints
Touch the screen to act (cut, draw, circle)
Also speak to act on any visible hints
13 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

14. GESTURAL AND CINEMATIC
TOUCH - MULTITOUCH - MULTIUSER
GUI point and click replaced with touch is
hard for users
New interfaces with touch, supporting fluid
dragging and picking is a big improvement
New interfaces with multi-touch, allowing
multi finger gestures like resizing and
rotation is better
Multi-user is a new challenge on larger
devices, and requires the designer to set
aside localised and orientated workspaces
for multiple users
14 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

15. NATURAL USER INTERFACE
IT’S A BROAD CONCEPT – NOT A DETAILED FRAMEWORK
NUI is not a fixed toolkit (like a GUI)
Don’t just mimic GUI or web apps
NUI is still abstract, it’s not real world
NUI should seem an extension of the body
NUI can leverage previously learned skills
Involves game aspects
‒ enjoyable interaction
‒ a path to skilled usage
‒ but NOT challenge
Direct manipulation
Aim for skilled and fluid actions
15 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

16. Technology stack is
growing & improving

17. HIGH SPEED ACCURATE MULTI TOUCH
17 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

18. FLEXIBLE PERFORMANT COMPUTE ARCHITECTURES
18 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

19. A GROWING RANGE OF GESTURE TECHNOLOGIES
19 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

20. A GROWING RANGE OF VOICE CAPABILITIES
Client side – cloud side – hybrid
20 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

21. ATTENTION TRACKING
Turned
away
Paying
attention
21 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

22. VR
22 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

23. GPU EFFECTS
Fluid
simulation
Paint, glass, chrome, plastic, environment, AO
23 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

24. BUT
needs hardcore development

25. UMAJIN APP BUILDER
25 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

26. UMAJIN APP BUILDER
ANYONE CAN BUILD NEW INTERFACES
26 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

27. NEW INTERFACES
MANY SCREENS
From 4”
27 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL
to 80”

28. NEW INTERFACES
TOUCH, VOICE, GESTURE
Demo…
28 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

29. DISCLAIMER & ATTRIBUTION
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.
The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap
changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software
changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD
reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of
such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY
INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE
LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION
CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices,
Inc. in the United States and/or other jurisdictions. SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Other names
are for informational purposes only and may be trademarks of their respective owners.
29 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL