This document discusses informal perceptual user interfaces (PUIs) that do not require recognition. It suggests that immediate recognition can interfere with creativity and communication. PUIs leverage innate abilities like speech, sketching, and handwriting for mobile and home use. Informal interfaces encourage creativity by communicating unfinished work and allowing faster creation. The document advocates for minimizing or deferring recognition in creative or communication-oriented tasks and applying informal interfaces to sketching, speech, and handwriting inputs.
2. Mar 25 2001 2
WIMPy User Interfaces
• Windows, Icons, Menus, Pointer
Hitting limitations of this kind of interface
• WIMP designed for limited audience
Full use of eyes
Literate
Full use of hands
• WIMP designed for limited situations
Fixed location (often offices)
Sitting down
Single person
3. Mar 25 2001 3
Perceptual User Interfaces
• PUIs leverage our innate perceptual,
motor, and cognitive abilities
Speech recognition, computer vision,
sketching, handwriting
• Sketching is one part of this bigger push
• PUIs more useful for different audiences
and wider range of situations
Mobile workers or Home
Speech => Handsfree
Location Tracking => Physical motion
4. Mar 25 2001 4
PUIs and Recognition
• Key question:
When should perceptual input be recognized?
• Immediate recognition can interfere with
creativity and communications
5. Mar 25 2001 5
Communication and Creativity
• Informal visual
representation
communicates “unfinished”
encourages creativity
faster to create
higher level comments
• Formal visual representation
communicates “finished”
inhibits creativity (detailed)
slower to create
lowerlevel comments
6. Mar 25 2001 6
Informal User Interfaces
• Historic bias towards formal user interfaces
Computation over creativity and communication
Structured input to simplify computation
• Position:
Recognition interferes with creativity and
communication due to errors and perception
Recognition should be minimized or deferred
for these kinds of apps
7. Mar 25 2001 7
Informal User Interfaces
• Sketching
• Speech
• Handwriting
8. Mar 25 2001 8
Informal User Interfaces
• Sketching
• Speech
• Handwriting
13. Mar 25 2001 13
Informal User Interfaces
• Sketching
• Speech
• Handwriting
14. Mar 25 2001 14
Slide
Stifelman, Arons, Schmandt (CHI2001)
15. Mar 25 2001 15
SUEDE:
Informal Prototyping for Speech-based UIs
• Supports design practice
- example scripts
- Wizard of Oz
- error simulation
- iterative design
• Informal user interface
- no speech recognition or
synthesis
- need not be programming expert
- fast & fluid design
Read my
important
email
18. Mar 25 2001 18
Informal User Interfaces
• Sketching
• Speech
• Handwriting
19. Mar 25 2001 19
NotePals:
Informal Handwriting Capture
Davis et al (CHI99)
20. Mar 25 2001 20
NotePals:
Informal Handwriting Capture
21. Mar 25 2001 21
Takeaway Ideas
• Perceptual user interfaces useful for more
audiences and more places
• Informal User Interfaces valuable for PUIs
Minimize or defer recognition
Creative or communicationsoriented tasks
• Informal User Interfaces can be applied to a
range of perceptual input
Sketching
Speech
Handwriting
24. Mar 25 2001 24
Backup Slides
Igarashi, Edwards, LaMarca, and Mynatt (AVI2000)
Editor's Notes
WIMP - Windows Icon Menu Pointer
1997 report from National Research Council, 5% of individuals over age 15 in the US have vision problems
1992 report from US Department of Education, 21% of Americans over age 16 have limited literacy skills
Neglected audiences include: Mobile workers, homes, groups of people
Saw a video game in Hong Kong where you had boxing gloves, and had sensors on top of the video game, letting you physically dodge
Palm Pilot example
Position is that Informal User Interfaces seem better for documenting than for transforming
Suggests a malleability of form (useful for creativity)
Recognition errors interfere
Position is that Informal User Interfaces also better for communicating
Unrecognized input keeps the nuances of expression
Recognition errors interfere
Take you through some apps
Precision makes it easy for the computer to represent the objects in the system.
Precision is also important for mechanical drawings
But this is very different from how we do things without computers
Sketching is very valuable for creative design tasks
Explore more ideas
Don't have to worry about details such as color, font, and alignment (until necessary)
SILK, PhD dissertation by Landay, is an example of a tool that supports sketching for GUI design
Does immediate recognition of objects but keeps strokes the same
Still has some problems due to immediate recognition and feedback
Freehand drawing for conceptual design
Also does immediate recognition of objects (tree, house)
Little built-in recognition
Groups things that might be words together
Arrows
Otherwise no recognition
Show part of DENIM video here
Speech
Most speech recognition systems try to recognize human speech and translate it immediately into a form that machines can deal with
Often causes people to enter a dialog with the system to correct mistakes as they occur
Immediate recognition is useful in many cases, but gets in the way in writing, designing, and brainstorming
Informal Speech
Audio Notebook
Freeform ink and speech, notes and speech are un-interpreted
Synchronizes the notes and speech, letting you explore either and link to the other