POINTING, SELECTING &
DIRECT MANIPULATION
Old models applied to new technologies…
Presentation for the class: HCI Desktop to Mobile
Lecturer: Dennis Krannich
Place & Time: Uni Bremen, 14. Jan. 2009
Presenters: Mohammad Shahin & Jan Smeddinck
http://www.io.com/~hcexres/power_tools/teams/glove_small.gif
http://www.freewebs.com/sonypsfree/nintendo_ds_lite.jpg
http://scr3.golem.de/screenshots/0807/iPhone_3G/xx_iphone3g_pair.jpg

DIRECT MANIPULATION – REDUCE COMPLEXITY
14 January 2009
2

DOMAIN - TERRY WINOGRAD
14 January 2009
 From direct manipulation to “being there”:
 We interact with the world around us in three main
ways; manipulation, locomotion, and conversation.
 manipulation: move things around with hands
 locomotion: move oneself from place to place
 conversation: say something and another person
says something back
3
http://www.stanford.edu/group/dschool/images/left/people_terry_winograd_top.jpg

ABOUT FACE 3.0: THE ESSENTIALS OF
INTERACTION DESIGN
14 January 2009
 by Alan Cooper, Robert Reimann, Dave Cronin
 A “how-to” book
 How to design the user experience based on first
identifying user goals and then building and utilizing
personas to build user-centered interaction
 Not all that scientific, but useful
 First edition (1995), now: 3.0 (2007)
 Today: Chapter 19
 www.cooper.com
4

DIRECT MANIPULATION
14 January 2009
Definition:
A human-computer interaction style
which involves continuous
representation of objects of interest,
and rapid, reversible, incremental
actions and feedback
5

EXAMPLE
14 January 2009
 Showing direct manipulation in Windows Explorer
6

COINED BY SHNEIDERMAN, 1974
14 January 2009
1. Visual representation of manipulated objects
2. Physical actions instead of text entry
3. Immediately visible impact of the operation 7

7 VARIETIES OF DIRECT MANIPULATION
14 January 2009
1. Pointing
2. Selection
3. Drag and drop
4. Control manipulation
5. Palette tools
6. Object manipulation
7. Object connection
 Program manipulation vs. content manipulation
8
http://www.allhatnocattle.net/swiss-knife.jpg

POINTING DEVICES
14 January 2009
 The mouse
 Alternatives:
 Light Pens / CRTs
 Trackball
 Tablets
 Touchpads
 Touchscreens
9

USING THE MOUSE
14 January 2009
 Mouse buttons
 The left mouse button
 The right mouse button
 The middle mouse button
 The scroll wheel
 Meta-keys
 Pointing and clicking
 Pointing
 Clicking
 Clicking and dragging
 Double-clicking
 Chord-clicking
10
 Double-clicking and dragging

14 January 2009
11
POINTING

14 January 2009
12
CLICKING

CLICKING & DRAGGING
14 January 2009
13

DOUBLE-CLICKING
14 January 2009
14

EVENTS: MOUSE-UP & MOUSE-DOWN
14 January 2009
 Mouse-down over an object or data should select
the object or data
 Mouse-down over controls means propose action;
mouse-up means commit to action.
15

POINTING AND THE CURSOR
14 January 2009
 Pliancy and Hinting
 Object hinting
 Cursor hinting
 Wait cursor hinting
16

SELECTION
14 January 2009
 The act of choosing an object or
control is referred to as selection
 Command ordering and selection
 verb-object and object-verb order
 Discrete and contiguous selection
 Mutual exclusion
 Additive selection
 Group Selection
 Insertion and replacement
 Visual indication of selection
17

PLAYTIME
14 January 2009
 http://www.dontclick.it/
18

DRAG-N-DROP
14 January 2009
 Very powerful idiom!
 Efficient: Connects 2 locations with a
function (in 1 move)
 source -> target
 Drag to new position / window /
function icon (trash)
 Interior vs. external
 External difficult:
 interface standards, target capabilities,
formats, DnD data vs. functions, no standard
 Long time little use online. That
changes…
19
 e.g. http://www.bbc.co.uk/

DRAG-N-DROP - INTERACTIONS
14 January 2009
 Forgiving interaction: Encourage users to try out DM
 Indicate drop targets
 not by cursor-hinting!
 No drop-target = no action
 Hinting at drag pliancy difficult
 Assume: Everything is dragable
 Show visual indication while dragging
 Use insertion targets, when necessary
 Indicate completion
20

DRAG-N-DROP - ISSUES
14 January 2009
 Auto-scroll! (where to begin,
how fast?)
 Not automatic scrollbar
insertion, but automatic
scroll-movement
 Selecting vs. dragging
 need drag-threshold after
click
 Fine scrolling
 Exact positioning of dragged
data not always easy
 Use meta-keys for fine
resolution
21
 Or arrow-keys for positioning

DRAG-N-DROP – MOBILE DEVICES
14 January 2009
 One hand problem!
 Past: Only with stylus…
 Currently: Revival with touch screens / multi touch
 Future: Important!
22
http://assets.gearlive.com/blogimages/gallery/iphone-113-preview/002-iphone-113-drag-calc_medium.jpg

CONTROL MANIPULATION
14 January 2009
 Menus used to be difficult click-n-drag
motion
 Beginners use menus often, but they are
difficult!
 Menus now often “click-based”
 Palette Tools:
 Modal
 Not too many!
 Charged cursor
 Not for functions!
 Good for objects
 With mobile devices:
 Not yet that important
 Will get more important with more tools 23
available
http://z.about.com/d/graphicssoft/1/0/5/P/1/cs2-workspace-2menus.gif

OBJECT MANIPULATION
14 January 2009
 Repositioning
 Click-n-drag in certain area, constrain drags
with meta-keys, snapping
 Resizing & reshaping
 Where to grab, vertex handles, constrains,
 3D object manipulation
 Display, input, movement, indication,
shadow, etc., problems
 With mobile devices:
 Very difficult…
 Not often used. Difficult motion. High
24
potential.

OBJECT MANIPULATION 3D
14 January 2009
25
http://www.free3dtutorials.com/userimages/dorian/11)-wireframe_view-fig.jpg
http://www.radoxist.com/download/my_works/worht_enough_shade_wire.jpg

OBJECT CONNECTION
14 January 2009
 Click-n-drag to connect objects
 rubberbanding
 Find valid target locations
 Indicate with handles
 Useful to indicate connection /
flow, but no insertion / function
execution
 With mobile devices:
 Until now hardly ever used…
26

EFFICIENCY STATS
14 January 2009
27
Interaction Design for Handheld
Computers
Jesper Kjeldskov and Nikolaj Kolbe

CURRENT TRENDS
14 January 2009
 Direct Manipulation Web-
Applications (AJAX)
 Rich visual feedback
 New unsuspected uses
 Video Browsing by Direct
Manipulation
 http://www.aviz.fr/dimp/dimp-
chi08.pdf
 Virtual physical direct
manipulation
 BumpTop
http://www.youtube.com/watch?v=M0ODskdEPnQ
 Devices: Motion, alignment,
28
position

FUTURE – PHYSICAL DIRECT MANIPULATION
14 January 2009
http://cache.gizmodo.com/assets/resources/2007/08/405587346_ba45
30bdbd.jpg
29

3D FUTURE & PERSPECTIVE PRESENCE
14 January 2009
http://www.cs.cmu.edu/~johnny/projects/wii/
http://medien.informatik.uni- 30
bremen.de/research/hci/mrt/

FUTURE – DM VS. INTERFACE AGENTS
14 January 2009
 Direct Manipulation will stay
important
 Become more intuitive
 Faster with better feedback
 Limits with huge data sets
 Agents can help handling that
 or: SoundTorch
 http://vimeo.com/groups/digmed/videos/2446867
 Be polite to users and give
them controls that let them
31
focus on their jobs
http://farm1.static.flickr.com/8/7157243_a5cf2853bc.jpg?v=0

14 January 2009
32
QUESTIONS