4. Using touch in a desktop application
• Touch and gestures
– Tap/Double tap – Zoom
– Panning with inertia – Rotate
– Selection/Drag – Two-finger tap
– Press and tap – Flicks
– Press and hold
• Do’s and Don’t of gesture interfaces
• Demo: Use touch and gestures to select, move and
manipulate three images
• The Windows Touch API
• Using touch with XMAL
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
4
5. Adding touch to a desktop application
• Touch is not a mouse
• Make touch targets large enough
• Support standard gestures and behaviors
• Portrait vs. Landscape
• Do not use touch just for touch’s sake
• Touch should be forgiving
• Microsoft User Experience Guidelines
– msdn.microsoft.com/en-
us/library/windows/desktop/cc872774.aspx
– blogs.msdn.com/b/ie/archive/2012/04/20/guidelines-for-
building-touch-friendly-sites.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
5
6. Designing for Touch User Experience
• Design UI to include traditional laptop style and tablet
mode usages
• Space and size controls to register fuzzy touch input
– Common Controls: 23X23 pixels
– Command Controls: 40X40 pixels
• Use multiple form tabs for touch efficiency
• Consider the target size while designing your
application UI
• Use natural and intuitive controls
• Ultrabook Optimal Resolution: 1366X768
Source: MSDN
http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
6
7. Designing for Touch User Experience
• Use common controls as much as possible
• Choose custom controls that support touch
• Prefer constrained controls to unconstrained
controls
• Provide default values and auto text-
completion
• Use check boxes instead of multiple selection
lists
Source: MSDN
http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
7
8. Designing for Touch User Experience
• Place controls in areas better utilized for touch
• Command controls should be more easily
accessible
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
8
9. Using Windows Touch API
• Touch and gesture events are delivered via Windows
Touch messages
– Windows must register for Windows Touch input
• WM_TOUCH reports action, position and identifier
• WM_GESTURE describes the gesture via GESTUREINFO
structure
• Special interfaces to help process gesture messages
– IManipulationProcessor
– IInertiaProcessor
• API reference at msdn.microsoft.com/en-
us/library/windows/desktop/dd371406%28v=vs.85%2
9.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
9
10. Sensors and Windows Native
• Sensor manager controls sensors
– Use sensor manager to get to sensors
– Notifies when a sensor connects
• Sensors report data, changes in state and
disconnection
• Access both via COM interfaces
– Use API to communicate to sensors
– Sensor events handled using callbacks
– API reference at msdn.microsoft.com/en-
us/library/windows/desktop/dd318953%28v=vs.85%2
9.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
10
11. WinNative: Receive gesture message
LRESULT CALLBACK
WndProc(…)
{
…
switch (message)
{
• Check for case WM_GESTURE:
// Call code to
WM_GESTURE message // interpret the
in WndProc // gesture
return
DecodeGesture(…);
…
}
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
11
12. WinNative: Process gesture
LRESULT DecodeGesture(…)
{
• Retrieve the additional …
BOOL bResult = GetGestureInfo(…);
gesture information from BOOL bHandled = FALSE;
the GESTUREINFO if (bResult)
structure {
// now interpret the gesture
switch (gi.dwID){
case GID_ZOOM :
// Code for zooming goes here
bHandled = TRUE;
• Handle each of the break;
case GID_PAN :
possible gestures …
case GID_ROTATE :
…
case GID_TWOFINGERTAP:
…
case GID_PRESSANDTAP:
…
default:
// A gesture was not recognized
break;
} … (Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
12
14. Using touch and XAML/.NET
• XAML provides access to WPF touch-enabled
UI components
– WPF touchs event are available in both Windows* 7 and
Windows* 8.
• Typical gesture events:
– ManipulationStarting
– ManipulationDelta
– ManipulationInertiaStarting
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
14
15. XAML: TouchControl.xaml
<UserControl x:Class="Win8Demo.TouchControl"
• Three images placed in …>
a Grid inside of a <Grid x:Name="cont" ClipToBounds="True" >
<!-- Set the IsManipulationEnabled to
TouchControl true -->
<Image x:Name="photo_1"
Source="/Assets/ultrabook1.png“
IsManipulationEnabled="True"
Width="500" />
• Each image has <Image x:Name="photo_2“
Source="/Assets/ultrabook2.png“
IsManipulationEnabled IsManipulationEnabled="True“
Width="500" Margin="125,-79,25,79"/>
<Image x:Name="photo_3“
set to true Source="/Assets/ultrabook3.png“
IsManipulationEnabled="True“
Width="500" Margin="0,-59,150,59" />
</Grid>
</UserControl>
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
15
19. XAML: Manipulation starting event
handler
• ManipulationStarting event void
handler is called at start of m_rect_ManipulationStarting
touch event (…)
{
…
• By setting Manipulation- e.ManipulationContainer =
Container to the touched this;
image all subsequent
manipulation events will be }
relative to that element
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
19
21. XAML: Manipulation delta event
handler
• ManipulationDelta void m_rect_ManipulationDelta(…)
{
event happens as user …
//Get the manipulation’s delta
moves fingers var delta = e.DeltaManipulation;
//Compute the transformed center point
Point rectCenter = new Point(rect.ActualWidth *
0.5, rect.ActualHeight * 0.5);
• Compute new image rectCenter = matrix.Transform(rectCenter);
center point, scale, and //Adjust the element’s scale, rotation and
translation
rotation using event matrix.ScaleAt(delta.Scale.X, delta.Scale.Y,
rectCenter.X, rectCenter.Y);
data matrix.RotateAt(delta.Rotation, rectCenter.X,
rectCenter.Y);
matrix.Translate(delta.Translation.X,
delta.Translation.Y);
• Set Handled to true so //Update the element’s render transformation
rect.RenderTransform =
other handlers don’t new MatrixTransform(matrix);
process same event e.Handled = true;
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
21
23. XAML: Inertia starting event handler
• Inertia actions are handled by the void m_rect_ManipulationInertiaStarting(…)
{
ManipulationInertiaStarting event //Set the manipulations inertia values
handler e.TranslationBehavior = new
InertiaTranslationBehavior()
• Define the behavior of the inertia {
by specifying InitialVelocity =
e.InitialVelocities.LinearVelocity,
• Linear velocity DesiredDeceleration =…
• Deceleration };
• Expansion velocity (used in e.ExpansionBehavior = new
InertiaExpansionBehavior()
pinch or spread) {
• Angular velocity (used in InitialVelocity =
e.InitialVelocities.ExpansionVelocity,
rotation) DesiredDeceleration = …
};
e.RotationBehavior = new
InertiaRotationBehavior()
{
InitialVelocity =
• Again, set Handled to true so other e.InitialVelocities.AngularVelocity,
DesiredDeceleration = …
handlers don’t process same event };
e.Handled = true;…
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
23
25. XAML: Handle inertia manipulation
if (e.IsInertial)
• Check if inertia is moving the {
image across the grid //Get the containing element’s size
Rect containingRect = …
(e.ManipulationContainer).RenderSize);
//Get the transformed element’s new
//bounds
Rect shapeBounds =
rect.RenderTransform.TransformBounds
(…);
//If element falls out of bounds
• If so, make sure image does not if
(!containingRect.Contains(shapeBounds))
travel outside of the grid by {
//Report boundary feedback
calling the Complete method to e.ReportBoundaryFeedback(…);
end the inertia //Stop any after inertia
e.Complete();
}
}
…
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
25