Types of Interaction 5

1. Context-Sensitive Interaction
Computer Vision, Barcodes, and Devices “Seeing”
Multimodal Interaction and Interface Combinations
Types of Interaction 5
Yonggeun Kim

2. Announcement

3. Announcement
1. Teaching Evaluation Trim 1, 2018/2019 (Term ID 1811)
a. https://cms.mmu.edu.my/psp/csprd/?cmd=login&languageCd=ENG&
2. Quiz late submission (closed)
a. 0 marks ⇢ 1121118293, 1141128388, 1122702918
3. Assignment (overdue) 🔗
a. Due; 23rd September 2018
4. Barring List

4. Checklist
Assessment Mark Status
Quiz 10% Closed
Assignment 20% Open (due; 23rd September)
Project 30% Open (Guideline or MMLS)
Final Exam 40%
Overall 100%

6. https://www.wikihow.com/Email-a-Professor or visit MMLS

7. Revision

8. Types of Interaction 4
Gestural, Tangible, and
Tactile Interaction

9. 1. Gestural Interaction
▸ Input
■ Is great
➜ … for video games. Gestural control
makes games more immersive and
physically challenging.
➜ … for short interactions. The “gorilla
arm” becomes a problem with lengthy
use. If gestural interaction is kept
short this is less of an issue.
➜ … when it is obvious what gestures
are possible. Users need to know or
recall what gestures are possible, or
have a guide.
▸ Input
■ Is less appropriate…
➜ … when precision and lengthy
interaction is required. Giving precise
input is hard with gestural interfaces.
Fatigue and muscle pains too.
➜ … when false positive recognition can
have serious consequences.
recognition is still unreliable and false
positives can occur.
➜ … when there is no time to learn.

10. 2. Tangible and tactile interaction
▸ Tangible user interfaces
■ Are great
➜ … for digitally enabled experiences
that don’t feel like interacting with a
computer. For example, museum
exhibits that let visitors explore and
experiment, or musical instruments
like the reactable that lets users
interact with sounds and music.
➜ … for educational products.
▸ Tangible user interfaces
■ Are inappropriate
➜ … when keeping all parts together is
critical. If lost parts mean lost data or
lost access, the drawbacks of
tangible user interfaces might
outweigh their benefits.
➜ … when there is no time to learn. If
the application doesn’t allow for a
learning phase to let users
understand how to use the interface,
other interface types might be more
appropriate.

11. 3. Tactile output: Vibration, force feedback, and shape shifting
▸ Tactile output
■ Are inappropriate
➜ … for creating interfaces that demand
less attention. Tactile output can
reduce cognitive load. Instead of
interpreting visual output, it can be
faster (but less precise) to feel the
information.
■ Is less appropriate
➜ … (currently) for affordable, reliable
systems. Mechanical systems involve
friction, wear and tear, and they have
a limited lifespan.
https://help.apple.com/watch/en.lproj/static.html
https://www.theverge.com/2013/1/2/3827752/hyundai-kia-bringing-google-maps-to-car-navigation-systems

12. TOC
1. Type of Interactions
a. Basics
b. Uncommon Modalities 1; Taste
c. Uncommon Modalities 2; Smell
d. Types of Interaction
i. Physical Controls
ii. Audio and Voice Interface
iii. Visual and Screen Interfaces
iv. Gestural, Tangible, and Tactile Interaction
v. Context-sensitive Interaction
vi. COMPUTER VISION, BARCODES, AND DEVICES “SEEING”
vii. MULTIMODAL INTERACTION AND INTERFACE COMBINATIONS

13. Reading
Rowland, Claire, et al., editors. Designing Connected Products: UX for the
Consumer Internet of Things. O’Reilly, 2015. P. 290-294 🔗

14. Context-sensitive Interaction
Types of Interaction 5

15. 1. Context-sensitive Interaction
1. Definition
a. “the circumstances that form the setting for an event, statement, or idea, and in terms of
which it can be fully understood”
2. Necessity
a. To make computers sense and act on inputs such as time of day, the weather, the current
location, or the identity of the user.
i. Apps or websites
1. Device location, user identity, social network data, calendar entries, etc.
ii. Smart Devices
1. Proximity sensor, GPS (outdoor only)
2. motion, environment, motion, input/output capabilities

18. An opportunity for
context-sensitive
interfaces is to
demand less of the
user’s attention and
require less
interaction.

19. 1. Context-sensitive Interaction
3. Requirements
a. A very good understanding of your users. ⇢ persona
i. Designers can only ever assume the user’s context through a proxy that a device can
sense, like location or proximity . It’s the designer that needs to establish the correlation
between such data points and user intentions or desire .
Room E Walkthrough@youtube

20. COMPUTER VISION, BARCODES,
AND DEVICES “SEEING”
Types of Interaction 5

21. COMPUTER VISION, BARCODES, AND DEVICES “SEEING”
▸ Vision technology-based enabler (not
interface)
■ it’s faster and easier (regardless of security)
■ to input complex data quickly
▸ Are great…
■ … for replacing cumbersome input.
Computer vision and barcodes can replace
manual input like typing in wiFi credentials,
UrLs, or book titles.
▸ Are less appropriate…
■ … when the interaction is more complex than
its alternatives. If additional software (like a
Qr code reader) is required, users may view
the alternative interaction as simpler.

22. COMPUTER VISION, BARCODES, AND DEVICES “SEEING”
▸ Vision technology-based enabler (not
interface)
■ it’s faster and easier (regardless of security)
■ to input complex data quickly
▸ Are great…
■ … for replacing cumbersome input.
Computer vision and barcodes can replace
manual input like typing in wiFi credentials,
UrLs, or book titles.
▸ Are less appropriate…
■ … when the interaction is more complex than
its alternatives. If additional software (like a
Qr code reader) is required, users may view
the alternative interaction as simpler.
https://indexprints.net/products/details/234/MCA-ID-Badge-2

23. MULTIMODAL INTERACTION
AND INTERFACE
COMBINATIONS
Types of Interaction 5

24. MULTIMODAL INTERACTION AND INTERFACE
COMBINATIONS
▸ In a similar way, interaction with a
connected device can combine different
types of interfaces into one . This is called
multimodal interaction.
▸ By combining interface types, designers
can create interactions that feel more like
human-to-human interaction.
▸ Friendly, kind, and thoughtful.