Your SlideShare is downloading. ×
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Lessons Learned from the Trenches of NUI (Tim Huckaby)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Lessons Learned from the Trenches of NUI (Tim Huckaby)

112

Published on

Join Tim Huckaby in a discussion of the lessons learned in 7+ years of design, development and delivery of interactive software driven by NUI (touch, gesture, and voice). All the bad; all the good; …

Join Tim Huckaby in a discussion of the lessons learned in 7+ years of design, development and delivery of interactive software driven by NUI (touch, gesture, and voice). All the bad; all the good; and the tools, tips and tricks learned along the way that will help you in designing great natural interactive usability into your own software.

Way back in 2007, Microsoft shipped Windows Vista which was touch capable, but not touch usable. With Windows 7, Microsoft introduced NUI capabilities like Multi-touch native to the OS. Windows 8 is designed for touch. Hardware vendors are now shipping large form factor multi-touch capable HD screens with native Windows 7 & 8 drivers at a consumer price point. Couple that with other form factors including mobile form factors where multi-touch is the norm and the power of WPF and WinRT then you have compelling low cost high fidelity hardware and software solutions that are revolutionizing numerous vertical industries.

This session will also update you on the types of multi-touch capable devices available right now and those coming in the immediate future. And what is now capable with .NET and WinRT in the multi-touch kiosk and interactive digital signage space.

But, frequently touch capability is not possible or just doesn’t make sense. In many airports, for instance, you do not want to touch anything. It’s just not safe. Bacteria (and other pathogens) live on touch screens in public places and are a real threat. In sterile environments like a hospital touch is legally not possible either. That is where gesture and voice controlled software comes into play nicely. With Microsoft Kinect (and other 3D cameras) innovation solutions are not only possible, but ultra-cool…and come with their own myriad of usability issues.

This demo-heavy session was designed to show you a number of real NUI solutions and how they were built and all the usability problems they evolved through.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
112
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lessons from the Trenches of NUI: User Experience and User Interaction Design  Tim Huckaby  InterKnowlogy Chairman/Founder  Actus Interactive Software Founder & CEO E-mail: TimHuck@InterKnowlogy.com TimH@Actus-Software.com Phone: Office: 760-444-8640 Mobile:: 619 990 9200 Blog: http://team.interknowlogy.com/blogs/timhuckaby FaceBook: http://www.facebook.com/Tim.Huckaby.Carlsbad LinkedIn: http://www.linkedin.com/in/timhuckaby Twitter: @TimHuckaby
  • 2. 2 © DEVintersection. All rights reserved. http://www.DEVintersection.com A little bit about me and why I am here… Lessons from the Trenches of NUI: User Experience and Interaction Design
  • 3. Delivering the Art of Software interknowlogy.com 1525 Faraday Ave, Suite 250, Carlsbad CA 92008 curious@interknowlogy.com Natural User Interface (NUI). IK has been focused in NUI for the past 7 years. We ideate, design, build, and deploy NUI apps on all shapes of hardware, including phone, tablet, wall, and/or table. IK knows how to build apps where touch and gesture are the primary ways users interact with the experience. Enterprise Mobile Apps (EMA). IK has two focus areas in EMA – expanding your brand and service through massive consumer consumption, and increasing the productivity of your employees through mobile enablement of your enterprise software. User Interaction Design. IK’s expertise in User Interaction design, (IxD), as implemented in the Natural User Interface (NUI), is heavily focused on satisfying the needs and desires of the majority of people who will use our software products. Our main focus is on natural behaviors (touch, gesture and voice recognition) when using the software we build. Other disciplines, like software engineering, have a heavy focus on designing for technical stakeholders of a project. We believe in designing software for the actual users. Big DATA + Rich Data Visualization. IK visualizes data in context to the user, breaking the barriers of standard maps and charts for information viewing. By intelligently displaying data, rather than using broad and generic strokes, we enable users to glean insights not before possible. App Renovation. IK transforms existing apps with NUI technology. We can re-factor any app into a NUI experience. But our app renovation isn’t skin deep – we go beyond transforming your app for touch and bringing the well-travelled app pathways to the forefront. IK rebuilds the app with a solid app architecture foundation so that new features can be easily added, and the app can be more thoroughly tested in less time. InterKnowlogy (IK) is an end-to-end service delivery shop focused on bringing tech to the enterprise that is intuitive and easy to use. IK services include ideation to transform business challenges and vague concepts into amazing solutions, interaction design to create beautiful user experiences, to complex app development and deployment with support services. We build our natural user interface (NUI) solutions on XAML, WPF, WinRT, HTML5 and/or Windows Phone.
  • 4. Delivering the Art of Software Who we work for
  • 5. Interactive Kiosk & Active Digital Signage Software Actus Interactive Software www.Actus-Software.com The Content Defines the Experience •Agnostic to Industry and Vertical Market •Large Tiles / Shallow Navigation / Flexible Content The Natural User Interface (NUI): Touch, Gesture & Voice Controlled Attract Mode Engages Powerful CMS / Dynamic Synchronization of Content JayH@Actus-Software.com www.Actus-Software.com
  • 6. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Objectives And Agenda The Good, the Bad & the Ugly (The Success Stories and the Failures) from 8 years of RealWorld Experience in NUI which includes Lessons Learned the hard way Technology Roadmap, Speculations & Predictions WhereWe have Been andWhere we are Going Demos,Videos, Demos,Videos, Demos…. This is a Design Session; It is not a “Stare at C#” session. It is targeted at Technologists Generically Guidance in NUI:Touch, Gesture &Voice Driven User Experiences- User Engagement & User Interaction Design
  • 7. Delivering the Art of Software Converse World-Wide Shoe Launch Tim Huckaby  Chairman / Founder, InterKnowlogy & Actus Software
  • 8. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Multi-Touch Capable •Common Gestures of Touch •Physical Object Interaction •Virtual Object Interaction NUI: Gesture Capable •Motion Based Interface •Tracking of full-body movement, facial expression, and voice with precision Voice Capable •Multi-array Microphone •Uniquely Distinguish Audio & Voices Neural Capable •Conscious and non-conscience interfaces •Non Invasive The Natural User Interface
  • 9. Delivering the Art of Software November, 2007
  • 10. Delivering the Art of Software
  • 11. Breakout Session Delivering the Art of Software The “Cancer App” & The “AutoCAD App” Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 12. Delivering the Art of Software January 27, 2010
  • 13. Delivering the Art of Software
  • 14. Delivering the Art of Software The 7 Tenets of an Engaging UX 1. Use the Power of Faces  People love faces; We see faces where they don’t exist  We have DNA with the sole purpose of recognizing faces  Take advantage of this by putting faces on your App  Have the faces to look at important part of your App  People instinctively follow the eyes of the face. 2. Use Food, Sex and Danger  When someone sees something new, they place it into one of four categories: 1. Can I Eat it? 2. Can it Eat Me? 3. Can I Mate with it? 4. …and everything else.  If you want your App to be engaged and interesting, put it in one of the first three categories.
  • 15. Delivering the Art of Software The 7 Tenets of an Engaging UX 3. Tell a story  We learn through stories. It’s how we teach our children. It’s the Bible; It’s what we see in movies.  Stories are a big part of how information is conveyed to us.  If you have information to give to the user, put it in story form.  It doesn’t matter what medium you use (words, pictures, music), but using a story narrative will help your user understand and retain your information. 4. Build commitment over time  Let the user choose how they want to interact with you (RSS, Twitter, Facebook)  Make sure that you don’t take advantage of the trust that is implicit in that interaction. 5. Use Natural Interactions of Objects  Natural Movements  Fluidity  Animate / Exaggerate: Slow things down so the eye can follow
  • 16. Delivering the Art of Software 6. “Game-ification”  We Live in a Gaming Culture  The Digital Natives Expect a Contest  Even When Learning  It doesn’t fit all use cases, but if you can provide “game-fication” you’ll get engagement 7. Make it Intuitive and Easy to Use  Todays Generations, especially the Digital Natives, Don’t Read User Manuals.  Accessibility is expected in the Elderly Culture  The “Grandma Huckaby Theory”  If Grandma Huckaby walks up to a piece of software we have built and is effective we have succeeded. The 7 Tenets of an Engaging UX
  • 17. Breakout Session Delivering the Art of Software Cincinnati Children's Hospital - The Power of Faces Tim Huckaby ◦ Chairman / Founder, InterKnowlogy & Actus Software
  • 18. Breakout Session Delivering the Art of Software EMN8 – Fast Food Kiosk Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 19. 2 We have come a Long Way in User Interface… The “Green Screen” 20Lessons from the Trenches of NUI: User Experience and Interaction Design
  • 20. Delivering the Art of Software 2 Image Courtesy of Computer History Museum Lessons from the Trenches of NUI: User Experience and Interaction Design We have come a Long Way in User Interaction
  • 21. Breakout Session Delivering the Art of Software Touch Capable Computing Devices Are Not New Past 1972: PLATO IV Touch Screen Terminal Perspective PixelApple iPad Present Future (now)
  • 22. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Multi-Touch  Inexpensive Touch Capable, Non Proprietary Hardware  A Operating Systems designed for Multi- Touch  A Consistent Multi-Touch API for the major Hardware/Software Platforms  Developer Productivity in Touch Capable Apps  The Forecast of Multi-Touch Hardware Capability in Every New Computing Device in the Short Term What is Relatively New is: Gesture & Voice Recognition  Inexpensive 3D Cameras  Natural Movements / Natural Interactions  Fidelity: Reliable and Dependable  Safe: anti-bacterial  Becoming a norm of using computing systems
  • 23. Breakout Session Delivering the Art of Software Barriers to Good Interaction Design in Touch on the Windows Platform • As Microsoft Matured their App-Dev Touch platform, the Windows Team, Hardware OEMs and Software ISVs ignored it …While Apple & it’s App Builders Embraced it and Designed for it. • In WPF, touch is a first class citizen • But, it’s ignored by the software control vendors • Example: Originally, Google chrome was not built to the Windows touch specification • Ignored touch events
  • 24. 2 HTML 5 Touch Events Spec W3C Recommendation 10 October 2013 Source: http://www.w3.org/TR/touch-events/ Defines a set of low-level events that represent one or more points of contact with a touch-sensitive surface Changes of those points with respect to the surface and any DOM elements displayed upon it (e.g. for touch screens) or associated with it (e.g. for drawing tablets without displays). Addresses pen-tablet devices, such as drawing tablets, with consideration toward stylus capabilities. HTML5 Touch Events Extensions W3C Working Group Note 31 October 2013 https://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html Lessons from the Trenches of NUI: User Experience and Interaction Design
  • 25. Delivering the Art of Software “You are not the User.”
  • 26. Delivering the Art of Software
  • 27. Delivering the Art of Software
  • 28. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Wireframe vs. Comp / Design
  • 29. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software User Interaction Design (IxD) User Interaction Design: • Extends Beyond the Screen • Shapes Digital Things for People’s Use • Is the practice of designing interactive digital products, environments, systems, and services. • Has a Main focus on Human Behavior and Interaction Distinction: • User Interaction Design: heavily focused on satisfying the needs and desires of the majority of people who will use the product • Application Programming: heavily focused on designing for technical stakeholders of the product • User Experience Design: is typically artwork, look and feel
  • 30. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software User Interaction Design Sketch
  • 31. Delivering the Art of Software Good User Interaction Design: Fluid Layout, Naturally Behaving Objects  Large Form Factors demand natural behaviors  Physical & Virtual Objects:  Some Touch Capable Hardware has unique features  Software Physics Engines are readily available in open source  Interaction Design & The User Experience is “never done” …It can always be improved with User Interaction Design  Tablet Computing paved the way…
  • 32. Breakout Session Delivering the Art of Software Pixelsence Craps, LOC Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 33. Breakout Session Delivering the Art of Software Fluid Layout / Natural Data Bench Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 34. Breakout Session Delivering the Art of Software Good User Interaction Design: Software Designed for Touch  Large Controls  Intuitive  Smoothing; Fidelity Correction  Ubiquitous Touch Gestures like “pinch”
  • 35. Breakout Session Delivering the Art of Software NASA Marsbound – Designed for Touch Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 36. Breakout Session Delivering the Art of Software 3D Modeler – Designed for Touch Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 37. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Developer Pacification – “I can’t handle it unless I can stare at code.” Check out the method level support for Rotation This is why I should have paid attention in high school calculus..
  • 38. The Engaging User Experience & the Natural User InterfaceDelivering the Art of Software Kinect for Windows  $250  http://www.microsoft.com/en-us/kinectforwindows/Multi- Platform  .NET; awesome SDK and samples Leap Motion  $70  https://leapmotion.com  Multi-Platform SoftKinetic  http://www.softkinetic.com  $249  Distance and Near Mode  Multiple Products; In the space for over a decade Panasonic  Works outside in direct sunlight -$2,600 3D Cameras – Major Players
  • 39. Breakout Session Delivering the Art of Software User Interaction Design with Gesture: From Bad to Good  Using your hands as the mouse  Intuitive  Smoothing; Fidelity Correction
  • 40. Delivering the Art of Software • Kinect for Windows Human Interface Guidelines v1.8.0 • • • roadmap to building exciting human-computer interaction solutions you once thought were impossible. • “Living Document”- regularly updated it to reflect both ongoing findings and the evolving capabilities of Kinect for Windows. Human Interface Guidelines Guidance Published by the K4W Team
  • 41. Breakout Session Delivering the Art of Software Before & After: IK Physical Therapy, IK Touch-less Operating Room Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
  • 42. Delivering the Art of Software The Future of Interaction Design with Gesture & Voice: Facial Recognition, Identity, Demographics Tim Huckaby ◦ Chairman / Founder, InterKnowlogy & Actus Software
  • 43. User Interface Innovation: What’s on the Horizon?Delivering the Art of Software Brand new device • Not an upgrade • Replaces the prime sense device purchased by Apple • 2 to 3 times better functionality across the board • Specs that are unmatched by any current 3D camera • Fidelity so good it’s capable of detecting human pulse Kinect for Windows v2 All-new: • Active IR camera enables it to see in the dark. And by using advanced three- dimensional geometry, it can even tell if you’re standing off balance. • Multi-microphone array, the advanced noise isolation capability lets the sensor know who to listen to, even in a crowded space. Officially: “Kinect for Windows v2 worldwide launch next summer.” Unofficially: “…we’ll ship in spring.”
  • 44. Delivering the Art of Software interknowlogy.com 1525 Faraday Ave, Suite 250, Carlsbad CA 92008 curious@interknowlogy.com NUI provides a number of challenges to good User Interaction Design: • Large and Intuitive Controls • Large Form Factors • Naturalizing Interactions withVirtual and Physical Objects • Fidelity and environment weaknesses of 3D Cameras The hardware, tools, plumbing and platform for good user Interaction Design in Applications are powerful and available now. …and will just continue to get better, cheaper, powerful and more available The concepts of engagement and Good User Interaction Design change the way we design and build software in many Use Cases. …when the goal is better user experiences Summary
  • 45. Delivering the Art of Software More info on InterKnowlogy: ◦www.InterKnowlogy.com ◦www.vimeo.com/InterKnowlogy More info on Actus Software: ◦www.Actus-Software.com About Tim Huckaby… Founder/Chairman, InterKnowlogy Founder/Chairman, Actus Software Microsoft® Regional Director – Southern California Microsoft® .NET Partner Advisory Council Founder / Member Microsoft® MVP – Kinect for Windows INETA Speaker – International .NET Users Group Association Board Member: Association for Competitive Technology Board Member: DevIntersection Board Member: Microsoft Healthcare Partner Advisory Board Author / Speaker www.TimHuckaby.com Contact Tim: E-mail: TimHuck@InterKnowlogy.com TimH@Actus-Software.com Phone: • Office: 760-444-8640 • Mobile:: 619 990 9200 Blog: http://team.interknowlogy.com/blogs/timhuckaby FaceBook: http://www.facebook.com/Tim.Huckaby.Carlsbad LinkedIn: http://www.linkedin.com/in/timhuckaby Twitter: @TimHuckaby Tim Huckaby

×