Developing Multi-touch Applications<br />	Reed Townsend<br />	Program Manager<br />	Microsoft Corporation<br />	Anson Ts...
Multi-touch Is Here!<br />Hardware<br />Multi-touch capable machines in market today, multi-touch appears in a broad set o...
Multi-touch in Action<br />	Reed Townsend<br />	Program Manager<br />	Windows<br />demo <br />
Agenda<br />Introduction<br />Touch scenarios, “Good, Better, Best” model, and developer roadmap<br />Platform details:<br...
Touch Scenarios And Windows 7<br />That led to four areas of investment:<br />Developer Platform: At the root is the touch...
A Structured Approach To Adding Touch Support To Your Application<br />Which application scenarios get the biggest benefit...
Touch Development Roadmap<br />Windows 7 Release<br />NET 4.0 / Surface 2.0 Release<br />Native<br />Win32<br />Applicatio...
Touch Platform Overview<br />
Windows 7 Gestures & WM_GESTURECOMMAND<br />	Reed Townsend<br />	Program Manager<br />	Windows<br />demo <br />
WM_GESTURECOMMAND<br />WM_GESTURECOMMAND<br />Better: Good gesture support is key<br />Window message conveying gesture no...
WM_GESTURECOMMAND<br />BOOL WINAPI GetGestureCommandInfo( <br />	__in UINT uMsg, <br />	__in WPARAM wParam, <br />	__in LP...
WM_GESTURECOMMAND<br />Interpreting gestures:<br />
WM_GESTUREFuture Windows 7 Build<br />Minor API surface changes:<br />Naming: WM_GESTURE, GESTUREINFO, GetGestureInfo(), e...
Single-finger Panning Future Windows 7 Build<br />Single-finger panning (SFP) provides an even easier way to pan where it ...
WM_TOUCH<br />	Reed Townsend<br />	Program Manager<br />	Windows<br />demo <br />
WM_TOUCH<br />WM_TOUCH* family of messages<br />Best: WM_TOUCH enables touch-optimized experiences<br />WM_TOUCHDOWN, WM_T...
WM_TOUCH<br />BOOL WINAPI RegisterTouchWindow(<br />    __in HWND hwnd,<br />    __in ULONG ulFlags);<br />BOOL WINAPI Get...
WM_TOUCH Future Windows 7 Build<br />WM_TOUCH consolidated to a single message<br />
Manipulations & Inertia<br />	Reed Townsend<br />	Program Manager<br />	Windows<br />demo <br />
Manipulations<br />Best: Manipulations are a great foundation for touch-optimized experiences<br />Examples of higher leve...
Inertia<br />Provides basic physics<br />Works hand in hand with Manipulations<br />Interfaces:<br />IInertiaProcessor<br ...
Multi-touch In .Net Framework<br />For WPF:<br />3.5 update near Windows 7 launch <br />Multi-touch, gesture APIs<br />4.0...
Multi-Touch In WPF<br />UIElement and UIElement3D changes<br />Gesture events (tracking)<br />Touch system gesture events ...
Multi-touch WPF Application<br />	Anson Tsao<br />	Senior Program Manager<br />	WPF<br />demo <br />
Gestures In WPF<br />Gesture events on UIElement & UIElement3D<br />TouchGestureStarted, TouchGestureChanged, TouchGesture...
Gestures in WPF<br />	Anson Tsao<br />	Senior Program Manager<br />	WPF<br />demo <br />
Touch Events In WPF<br />Touch Events on UIElement and UIElement3D<br />TouchDown, TouchMove, TouchUp<br />TouchEnter, Tou...
Touch Events In WPF<br />	Anson Tsao<br />	Senior Program Manager<br />	WPF<br />demo <br />
Tablet Platform Extensions<br />Real-time Stylus (RTS)<br />RTS is an excellent raw touch data provider<br />IRealTimeStyl...
UX Considerations<br />General guidelines:<br />Big targets<br />Whitespace<br />Avoid hover<br />See UX guidelines<br />B...
Now Is The Time!<br />ISVs are enthusiastic about multi-touch!<br />We are working with several ISVs today on multi-touch ...
Call To Action<br />Decide which application scenarios make sense for touch<br />Decide which tier of investment is approp...
Getting Started<br />What do you need to start coding?<br />Hardware:<br />Multi-touch beta drivers for in-market devices:...
Additional Resources<br />Related PDC Sessions:<br />PC46 – WPF Roadmap<br />PC17 – Developing for Microsoft Surface<br />...
Evals & Recordings<br />Please fill out your evaluation for this session at:<br />This session will be available as a reco...
Please use the microphones provided<br />Q&A<br />
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be...
Backup Slides<br />
Now Is The Time!<br />Multi-touch offers clear differentiation with a high “WOW” factor for consumers<br />Multi-touch enh...
Now Is The Time!ISVs are investing in multi-touch<br />ISVs are enthusiastic about multi-touch!<br />We are working with s...
Multi-Touch Capable PCs Are Available Today<br />HP TouchSmart<br />All-in-one PC with 2-touch digitizer from NextWindow<b...
Multi-touch & Windows 7<br />Multi-touch is a major investment area and strong reason for customers to upgrade to a Window...
WM_GESTUREFuture Windows 7 Build<br />typedefstructtagGESTUREINFO {<br />    UINT cbSize;                    // size, in b...
WM_GESTURENOTIFYFuture Windows 7 Build<br />typedefstructtagGESTURECONFIG {<br />    DWORD dwID;<br />    DWORD dwWant; <b...
SFP & SMB_GETSCROLLINFO<br />To help ensure SFP can control the scrollbars: <br />SBM_GETSCROLLINFO, SBM_SETSCROLLINFO, SB...
Tell Me More!<br />What is Multi-touch? <br />Multi-touch is two or more independently trackable contacts<br />It is not g...
Pc03
Upcoming SlideShare
Loading in …5
×

Pc03

1,151 views
1,076 views

Published on

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,151
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
24
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Pc03

  1. 1. Developing Multi-touch Applications<br /> Reed Townsend<br /> Program Manager<br /> Microsoft Corporation<br /> Anson Tsao<br /> Program Manager<br /> Microsoft Corporation<br />PC03<br />
  2. 2. Multi-touch Is Here!<br />Hardware<br />Multi-touch capable machines in market today, multi-touch appears in a broad set of form factors<br />Software<br />Windows 7, PDC partner demonstrations & Touch platform announcement, .Net 4.0<br />Consumers<br />New scenarios, next wave of user experience, high “WOW” factor<br />
  3. 3. Multi-touch in Action<br /> Reed Townsend<br /> Program Manager<br /> Windows<br />demo <br />
  4. 4. Agenda<br />Introduction<br />Touch scenarios, “Good, Better, Best” model, and developer roadmap<br />Platform details:<br />Native Win32 gesture and touch support<br />Manipulations and Inertia<br />WPF support<br />UX guidelines for touch applications<br />Where to get started - SDK, drivers, etc<br />Wrap up<br />
  5. 5. Touch Scenarios And Windows 7<br />That led to four areas of investment:<br />Developer Platform: At the root is the touch developer platform that exposes touch APIs for any application <br />UI Enhancements: Focusing on the core scenarios, many parts of the core UI have been optimized for touch experiences. <br />Gestures: Multi-touch gestures have been added to enable consistent panning and zooming in most applications.<br />Applications: A set of multi-touch focused applications that demonstrate the power of touch. These will ship separate from Win7.<br />We’ve identified several key scenarios for multi-touch:<br />Navigating and consuming the web<br />Reading & sorting email<br />Viewing photos<br />Playing casual games<br />Consuming music and video<br />Navigating files & arranging windows<br />Using Office applications<br />All focused on consumption<br />
  6. 6. A Structured Approach To Adding Touch Support To Your Application<br />Which application scenarios get the biggest benefit from touch?<br />Content consumption, mobile, media and entertainment<br />Decide which tier of investment is appropriate:<br />Good – Free support, UX tweaks<br />Better – Gesture support, touch-friendly UX<br />Best – Touch-optimized experience<br />
  7. 7. Touch Development Roadmap<br />Windows 7 Release<br />NET 4.0 / Surface 2.0 Release<br />Native<br />Win32<br />Application<br />WinFormsApplication<br />WPF Application<br />Surface Application<br />Surface SDK<br />1.0<br />Surface SDK 2.0<br />Multi-Touch Controls<br />Surface <br />Multi-Touch Controls & API<br />Multi-Touch API<br />WPF 3.5 SP1<br />WPF 4.0<br />WPF 3.5<br />Managed Wrapper and Interop<br />Managed Wrapper and Interop<br />Multi-Touch API and Controls<br />Surface Hardware<br />Windows 7<br />Surface Hardware<br />Windows Vista<br />Windows 7<br />Multi-Touch API<br />
  8. 8. Touch Platform Overview<br />
  9. 9. Windows 7 Gestures & WM_GESTURECOMMAND<br /> Reed Townsend<br /> Program Manager<br /> Windows<br />demo <br />
  10. 10. WM_GESTURECOMMAND<br />WM_GESTURECOMMAND<br />Better: Good gesture support is key<br />Window message conveying gesture notifications and information to applications<br />Why use it? Richer than legacy notifications, smoother more responsive scrolling and zooming<br />Compound gestures<br />Coalescing<br />Message bubbling and capture<br />Legacy handling & SFP<br />
  11. 11. WM_GESTURECOMMAND<br />BOOL WINAPI GetGestureCommandInfo( <br /> __in UINT uMsg, <br /> __in WPARAM wParam, <br /> __in LPARAM lParam, <br /> __in LPARAM lExtraInfo, <br /> __inout PGESTUREINFO pGestureInfo<br />);<br />typedefstruct _GESTURECOMMANDINFO { UINT cbSize; DWORD dwFlags; DWORD dwCommand; DWORD dwArguments; POINTS ptsLocation;<br />} GESTURECOMMANDINFO, *PGESTURECOMMANDINFO;<br />
  12. 12. WM_GESTURECOMMAND<br />Interpreting gestures:<br />
  13. 13. WM_GESTUREFuture Windows 7 Build<br />Minor API surface changes:<br />Naming: WM_GESTURE, GESTUREINFO, GetGestureInfo(), etc<br />Additional fields in GESTUREINFO<br />Added gesture configuration support:<br />WM_GESTURENOTIFY<br />GESTURECONFIG struct<br />Set/GetGestureConfig()<br />
  14. 14. Single-finger Panning Future Windows 7 Build<br />Single-finger panning (SFP) provides an even easier way to pan where it is natural<br />How to control SFP in your application:<br />SetGestureConfig() for GC_PAN<br />Specify GC_PAN_WITH_SINGLE_FINGER_VERTICALLY and/or GC_PAN_WITH_SINGLE_FINGER_HORIZONTALLY<br />If you aren’t getting the SFP default:<br />Consider standard scrollbars<br />SBM_GETSCROLLINFO (remarks here are helpful)<br />Implement WM_GESTURE support<br />
  15. 15. WM_TOUCH<br /> Reed Townsend<br /> Program Manager<br /> Windows<br />demo <br />
  16. 16. WM_TOUCH<br />WM_TOUCH* family of messages<br />Best: WM_TOUCH enables touch-optimized experiences<br />WM_TOUCHDOWN, WM_TOUCHMOVE, WM_TOUCHUP<br />Semantically similar to mouse messages<br />Conveys raw touch data to Win32 apps<br />Scenario examples:<br />Finger painting, custom gestures, feeding higher level controls, etc<br />Capture<br />Coalescing<br />
  17. 17. WM_TOUCH<br />BOOL WINAPI RegisterTouchWindow(<br /> __in HWND hwnd,<br /> __in ULONG ulFlags);<br />BOOL WINAPI GetTouchInputInfo(<br /> __in HANDLE hTouchInput, // input event handle<br /> __in UINT cInputs, // number of elts in the array<br /> __out_ecount(cInputs) PTOUCHINPUT pInputs, // array of touch inputs<br /> __in intcbSize); // sizeof(TOUCHINPUT)<br />typedefstructtagTOUCHINPUT {<br /> LONG x;<br /> LONG y;<br /> HANDLE hSource;<br /> DWORD dwID;<br /> DWORD dwFlags;<br /> DWORD dwMask;<br /> DWORD dwTime;<br /> ULONG_PTR dwExtraInfo;<br /> DWORD cxContact;<br /> DWORD cyContact;<br />} TOUCHINPUT, *PTOUCHINPUT;<br />typedef TOUCHINPUT const * PCTOUCHINPUT;<br />BOOL WINAPI CloseTouchInputHandle(<br /> __in HANDLE hTouchInput); // input event handle<br />
  18. 18. WM_TOUCH Future Windows 7 Build<br />WM_TOUCH consolidated to a single message<br />
  19. 19. Manipulations & Inertia<br /> Reed Townsend<br /> Program Manager<br /> Windows<br />demo <br />
  20. 20. Manipulations<br />Best: Manipulations are a great foundation for touch-optimized experiences<br />Examples of higher level APIs/controls<br />Manipulations are:<br />2D affine transformations (translate, scale, rotate)<br />Superset of supported gestures<br />Supports multiple concurrent manipulations<br />Need a source of raw data: WM_TOUCH or RTS<br />Similar to Surface APIs<br />Interfaces:<br />IManipulationProcessor<br />IManipulationEvents<br />
  21. 21. Inertia<br />Provides basic physics<br />Works hand in hand with Manipulations<br />Interfaces:<br />IInertiaProcessor<br />IManipulationEvents – Same event interface as Manipulations<br />
  22. 22. Multi-touch In .Net Framework<br />For WPF:<br />3.5 update near Windows 7 launch <br />Multi-touch, gesture APIs<br />4.0 release<br />Multi-touch specific new controls<br />For WinForms (Windows 7 launch):<br />Interop to native Win32 APIs – see sample<br />Use managed Manipulations/Inertia wrappers<br />
  23. 23. Multi-Touch In WPF<br />UIElement and UIElement3D changes<br />Gesture events (tracking)<br />Touch system gesture events (single)<br />Raw touch events<br />Touch and touch digitizer devices<br />Multi-touch support in controls:<br />ScrollViewer update to accept pan gestures<br />Base controls updated to be multi-touch aware<br />Multi-capture support<br />New multi-touch specific controls<br />Compatible with Surface SDK 2.0<br />
  24. 24. Multi-touch WPF Application<br /> Anson Tsao<br /> Senior Program Manager<br /> WPF<br />demo <br />
  25. 25. Gestures In WPF<br />Gesture events on UIElement & UIElement3D<br />TouchGestureStarted, TouchGestureChanged, TouchGestureEnded<br />NOT routed, to enable multi-capture scenarios<br />Gesture in WPF based on Manipulations<br />Enable Inertia at TouchGestureStarted<br />Inertial movements as TouchGestureChanged Events<br />Touch system gestures<br />
  26. 26. Gestures in WPF<br /> Anson Tsao<br /> Senior Program Manager<br /> WPF<br />demo <br />
  27. 27. Touch Events In WPF<br />Touch Events on UIElement and UIElement3D<br />TouchDown, TouchMove, TouchUp<br />TouchEnter, TouchLeave<br />Preview variants<br />Promotion: Touch -&gt; Stylus -&gt; Mouse<br />Multi-Capture<br />CaptureTouchDevice(TouchDevice)<br />ReleaseTouchDeviceCapture(TouchDevice)<br />ReleaseAllTouchDeviceCaptures()<br />TouchDeviceCollectionTouchDevicesCaptured { get; }<br />TouchDeviceCollectionTouchDevicesCapturedWithin { get; }<br />IsAnyTouchDeviceCaptured();<br />IsAnyTouchDeviceCapturedWithin();<br />
  28. 28. Touch Events In WPF<br /> Anson Tsao<br /> Senior Program Manager<br /> WPF<br />demo <br />
  29. 29. Tablet Platform Extensions<br />Real-time Stylus (RTS)<br />RTS is an excellent raw touch data provider<br />IRealTimeStylus3::MultiTouchEnabled()<br />Getting multi-touch data via InkCollector is a little trickier. E.g.:<br />ATOM m_atom = ::GlobalAddAtom(MICROSOFT_TABLETPENSERVICE_PROPERTY);<br />m_dwProperty = TABLET_ENABLE_MULTITOUCHDATA;<br />::SetProp(m_hwnd, (LPTSTR)m_atomPenService, (HANDLE)m_dwProperty);<br />
  30. 30. UX Considerations<br />General guidelines:<br />Big targets<br />Whitespace<br />Avoid hover<br />See UX guidelines<br />Be aware of hardware:<br />Form factor<br />Avoid on-hover UI<br />Edges, jitter, etc<br />Gesture guidelines:<br />Use common gestures<br />Gestures need to be intuitive and natural<br />Moving away from shortcut style gestures to manipulations and direct manipulation<br />
  31. 31. Now Is The Time!<br />ISVs are enthusiastic about multi-touch!<br />We are working with several ISVs today on multi-touch applications, including:<br />AutoDesk<br />Avoco Secure<br />Corel<br />Cyberlink<br />Fuel Industries<br />Identity Mine<br />Sonic<br />
  32. 32. Call To Action<br />Decide which application scenarios make sense for touch<br />Decide which tier of investment is appropriate<br />Build something amazing!<br />
  33. 33. Getting Started<br />What do you need to start coding?<br />Hardware:<br />Multi-touch beta drivers for in-market devices:<br />HP TouchSmart 2Dell Latitude XT<br />Software:<br />Window 7 M3 build – On the drive<br />Touch Platform SDK and samples – On the drive<br />Not in the Windows Platform SDK you received earlier!<br />WPF Multi-touch<br />.NET 4.0 Preview Releases in 2009<br />.NET 3.5 update to be announced<br />
  34. 34. Additional Resources<br />Related PDC Sessions:<br />PC46 – WPF Roadmap<br />PC17 – Developing for Microsoft Surface<br />Hands on Lab:<br />PCHOL18: Developing Multi-touch Applications in Windows 7<br />PDC Whitepaper – Windows 7: Reach Out and Touch It!<br />Product Pavilion – In the Windows booth<br />Ask the Experts & Presentation and Client track lounge<br />Contact us:<br />wintouch@microsoft.com<br />
  35. 35. Evals & Recordings<br />Please fill out your evaluation for this session at:<br />This session will be available as a recording at:<br />www.microsoftpdc.com<br />
  36. 36. Please use the microphones provided<br />Q&A<br />
  37. 37. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.<br />The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.<br />
  38. 38. Backup Slides<br />
  39. 39. Now Is The Time!<br />Multi-touch offers clear differentiation with a high “WOW” factor for consumers<br />Multi-touch enhances on-the-go use and opens up new scenarios for business<br />90% of people in focus groups claimed they would pay a 20-30% premium for a PC with multi-touch<br />Seeing is believing: Consumer purchase intent doubles after hands-on experience<br />
  40. 40. Now Is The Time!ISVs are investing in multi-touch<br />ISVs are enthusiastic about multi-touch!<br />We are working with several ISVs today on multi-touch applications, including:<br />AutoDesk<br />Avoco Secure<br />Corel<br />Cyberlink<br />Fuel Industries<br />Identity Mine<br />Sonic<br />
  41. 41. Multi-Touch Capable PCs Are Available Today<br />HP TouchSmart<br />All-in-one PC with 2-touch digitizer from NextWindow<br />Dell Latitude XT<br />Laptop with multi-touch and pen digitizer from N-Trig<br />
  42. 42. Multi-touch & Windows 7<br />Multi-touch is a major investment area and strong reason for customers to upgrade to a Windows 7 PC<br />MT scored very favorably in Concept Value testing<br />Consumer awareness and interest in touch is high<br />Customers perceive touch as the next stage of computing<br />Touch offers a sense of personal connection and freedom<br />Touch would help users be more efficient and be fun to use<br />Media response to MT disclosure has been very positive: <br />&gt;130 unique articles, 1M video views in 72hrs, continued monthly coverage<br />
  43. 43. WM_GESTUREFuture Windows 7 Build<br />typedefstructtagGESTUREINFO {<br /> UINT cbSize; // size, in bytes, of this structure // (including variable length Args field)<br /> DWORD dwFlags; // see GF_* flags<br /> DWORD dwID; // gesture ID, see GID_* defines<br /> HWND hwndTarget; // handle to window targeted by this gesture<br /> POINTS ptsLocation; // current location of this gesture<br /> DWORD dwInstanceID; // internally used<br /> DWORD dwSequenceID; // internally used<br /> ULONGLONG ullArguments; // arguments for gestures whose arguments fit // in 8 BYTES<br /> UINT cbExtraArgs; // size, in bytes, of extra arguments, if // any, that accompany this gesture<br />} GESTUREINFO, *PGESTUREINFO;<br />BOOL WINAPI GetGestureInfo(<br /> __in HGESTUREINFO hGestureInfo,<br /> __out PGESTUREINFO pGestureInfo);<br />
  44. 44. WM_GESTURENOTIFYFuture Windows 7 Build<br />typedefstructtagGESTURECONFIG {<br /> DWORD dwID;<br /> DWORD dwWant; <br /> DWORD dwBlock; <br />} GESTURECONFIG, *PGESTURECONFIG;<br />BOOL WINAPI SetGestureConfig(<br /> __in HWND hwnd, <br /> __in DWORD dwReserved, <br /> __in UINT cIDs, <br /> __in_ecount(cIDs) PGESTURECONFIG pGestureConfig,<br /> __in UINT cbSize);<br />BOOL WINAPI GetGestureConfig(<br /> __in HWND hwnd, <br /> __in DWORD dwReserved, <br /> __in DWORD dwFlags, <br /> __in PUINT pcIDs, <br /> __inout_ecount(*pcIDs) PGESTURECONFIG pGestureConfig,<br />__in UINT cbSize);<br />
  45. 45. SFP & SMB_GETSCROLLINFO<br />To help ensure SFP can control the scrollbars: <br />SBM_GETSCROLLINFO, SBM_SETSCROLLINFO, SBM_SETPOS and SBM_GETPOS<br /> They also only have to do this if their scrollbar is not a system scrollbar (WS_VSCROLL and WS_HSCROLL styles I believe).  The APIs we actually use would be the following which internally send these messages to the application: GetScrollInfo, GetScrollPos, GetScrollbarInfo, SetScrollInfo (may not need to use this one), SetScrollPos.<br /> See the remarks section of this topic to see what developers need to do to support these messages.  http://msdn.microsoft.com/en-us/library/bb787583(VS.85).aspx<br /> We typically detect these types of scrollbars today, it’s just we don’t control them since they don’t pay attention to the WM_VSCROLL and WM_HSCROLL message params since these messages only support 16 bit scroll values.  So they don’t need to do anything else for us to recognize their scrollbars (like set WS_VSCROLL).  This won’t help windowless controls though since we cannot target portions of a window with this.  Those apps would have to implement WM_GC to get their support.<br />
  46. 46. Tell Me More!<br />What is Multi-touch? <br />Multi-touch is two or more independently trackable contacts<br />It is not gesture support grafted on top of a touch or pointing device nor pen and touch together<br />Choose the right input for the task at hand<br />Touch and multi-touch are good for quick input tasks, content consumption vs. creation, transactional input<br />

×