The document discusses natural human-machine communication and its components. It begins with an introduction of the speaker and their background in developing personal assistants, navigation apps, and technologies for smart homes, robots, cars and wearables. It then outlines that the agenda will cover what natural communication is, why it is interesting, and provide an overview of the components with examples. The document discusses technologies that can't support conventional input and the era of conversational user interfaces. It provides an overview of the key building blocks including hardware, recognition modules for audio, text, computer vision, objects and gestures. It also discusses intent extraction, world knowledge, user knowledge, and how knowledge is used with examples of semantic fusion.

1. Natural Human to
Machine
Communication
By Alon Eirew

2. Who Am I
• Software Architect at Intel Multimodal Language
Understanding Group
• Smart Home, Robotics, Smart Car, Mobile & Wearables
• Lead Developer at Intel New Devices Group
• Personal Assistant for Mobile and Wearables
• Lead Developer Telmap (Navigation App)
• Navigation Assistant for Mobile
https://il.linkedin.com/in/aloneirew

3. 3
Agenda
 What is Natural Communication?
 Why is it interesting?
 Overview the components and see some
examples

4. 4
Human to Human

5. Natural Communication Feature

7. Change the way we are communicating with our smart devices

8. 8
Technologies that can’t support conventional input sources

9. The Era Of Conversational UI
How can I help
you?

10. 10
Building Blocks
Hardware
Recognition
Understanding
1
2
3

11. 11
Hardware and Recognition Modules
Automatic Sound
Recognition
Speech To Text
Computer Vision
Object Recognition
Person Recognition
Gesture Recognition

12. Intent Extraction (NLU)
12
Please turn
off the lights
in the living
room at
4PM
Location: living room
Time: 4PM
Object: lights
Action: turn off
Subject: (you)

13. World Knowledge
13

14. User Knowledge
14

15. 15
Knowledge Use Cases

16. Semantic Fusion
Object [1] [N-Best]
Label Prob Location
Chair 0.7
x1, y1, z1
Stool 0.4
Table 0.2
Label Prob Location
Table 0.8 x4, y4, z4
Stool 0.3 x4, y4, z4
Object 0.2 x4, y4, z4
Object [2] [N-Best]
Gesture List
Object Rec 1
Object Rec 2
End Point
Clean
next to
this ????
????
Word Prob
School Desk 0.7
Ship Deck 0.5
Sport Deko 0.2
ASR [N-Best]
Object [3] [N-Best]
Label Prob Location
Plant 0.7
x7, y7, z7
Tree 0.4
Object 0.2

17. Dialog Features
17

18. Thank You!