See, hear, speak… No evil! - Carmel Hassan

1. See, hear, speak… No evil!
Designing better experiences through multimodal interaction

2. ● Computer Engineer, since 2005
● Interaction Designer, since 2007
● Founder of Yes We Tech, since 2015
Carmel Hassan
Who’s talking
https://carmel.es - @KARMEL

3. 🙈🙉🙊
the three wise monkeys
Mizaru Kikazaru Iwazaru
See, hear, speak… No evil! @KARMEL

4. 🙈🙉🙊
the three wise monkeys
Mizaru Kikazaru Iwazaru
See, hear, speak… No evil! @KARMEL
acknowledge your surroundings to avoid evil

5. 🙈🙉🙊
see hear speak
the three wise interactions
See, hear, speak… No evil! @KARMEL

6. 🙈🙉🙊
see hear speak
the three wise interactions
See, hear, speak… No evil! @KARMEL
acknowledge people’s need to avoid frustration

7. See, hear, speak… No evil!
not all interfaces allow people to
experience technology with freedom
and control in every context
@KARMEL

8. See, hear, speak… No evil!
not all interfaces allow people to
experience technology with freedom
and control in every context
@KARMEL
what about all at once?

9. See, hear, speak… No evil!
unimodal interactions
what’s wrong with

10. See, hear, speak… No evil!
unimodal interactions
what’s wrong with
on graphical and voice interfaces

11. See, hear, speak… No evil!
keyboard
mouse

12. See, hear, speak… No evil!
touch
hear
🙈

13. See, hear, speak… No evil!
understanding
discovery
🙈 🙉 attention

14. See, hear, speak… No evil!
👹is real
🙊
🙈 🙉

16. See, hear, speak… No evil!
let’s make it multimodal
@KARMEL

17. See, hear, speak… No evil!
let’s make it multimodal
@KARMEL
“Sustain all objection but face the truth”
- Dr. Zira, The Planet of Apes
Multimodality is “the missing link”

18. See, hear, speak… No evil! @KARMEL - 18
“Any exchange between a
device and a human being
where multiple input or output
modalities may be used
simultaneously or sequentially
depending upon context and
preference.”
- Design beyond devices, Cheryl Platz, 2020
what’s multimodality

19. See, hear, speak… No evil!
put that there
1982, MIT Media Lab
first multimodal demonstration
@KARMEL

20. “Normal human function doesn’t exist”
- Emily Ladau, Demystifying Disability
accesible

21. “Normal human function doesn’t exist”
- Emily Ladau, Demystifying Disability
multimodal interfaces accommodate the interface
to individual differences
accesible

22. See, hear, speak… No evil! @KARMEL - 22
efficient

24. design for multimodality
how can we

25. a day in a life of…

26. check the weather

27. order some food

28. workout

29. drive to…

30. cooking

31. sleep routine

32. don’t compete for attention

33. understand how people talk

34. screen and voice in sync

35. voice is aesthetic

36. take the best of each mode

37. design for privacy and security

39. 🙈🙉🙊
see hear speak
See, hear, speak… No evil! @KARMEL
don’t turn blind eye on bad design

40. 🙈🙉🙊
see hear speak
See, hear, speak… No evil! @KARMEL
don’t turn blind eye on bad design
acknowledge people’s need to avoid frustration

41. No evil!
Use multimodal interaction to design better experiences
https://carmel.es - @KARMEL