Slides from my 2017 Innovation day talk on voice conversational interfaces. The presentation starts of by extrapolating and identifying future potential and halfway the presentation it turn really practical to show how you can start developing for your company. I am always lecture or provide extra information on the topic.

1. Conversational user interfaces. 1
CONFIDENTIAL Template Innovation Day 2017CONFIDENTIAL
Conversational user interfaces
Ir. Jochem Grietens
Systems engineer & AI enthusiast.
jochem.Grietens@verhaert.com
TRACK 1 - TRANSFORMATIONAL TECHNOLOGIES

2. Conversational user interfaces. 2
CONFIDENTIAL
Yours truly
Ir. Jochem Grietens
System engineer @ Verhaert
Product development .
NOT a software engineer
Big enthusiast of software as a tool for innovation.

3. Conversational user interfaces. 3
CONFIDENTIAL
What are we talking about
We will be focusing on spoken language, voice control rather than chatbots or
text based VUI.
KITT

4. Conversational user interfaces. 4
CONFIDENTIAL
The future of voice interfaces

5. Conversational user interfaces. 5
CONFIDENTIAL
Cars are more than carriages without a horse.
Smartphones have become more than telephones without a cord.
Voice interfaces will be more than computer without keyboards & displays.
Voice interface, a transformational technology ?

6. Conversational user interfaces. 6
CONFIDENTIAL
"I think there is a world market for maybe five computers." --
Thomas Watson, chairman of IBM, 1943.
"Nuclear-powered vacuum cleaners will probably be a reality
within ten years.“ -- Alex Lewyt, president of Lewyt vacuum company,
1955
About predicting the future…

7. Conversational user interfaces. 7
CONFIDENTIAL
Why Voice ?

8. Conversational user interfaces. 8
CONFIDENTIAL
Processors obey Moore's law, can interfaces follow ?
Bandwidth of communication
Quad Core mobile:
6,4 Giga operations/s
Extreme multicore computer @ 100 Hz
Enourmous ammount operations/cycle.
= 1016 operation/s
3-4 symbols/sec2-3 symbols/sec
20 symbols/sec
UNINTERUPTED

9. Conversational user interfaces. 9
CONFIDENTIAL
Natural mode of communication
Speech is the most natural mode of communication. =
Low effort.
“For all the times you are yelling at your devices...
Just image them actually listening... “

10. Conversational user interfaces. 10
CONFIDENTIAL
Natural mode of communication
Speech is the most natural mode of communication. =
Low effort.
“For all the times you are yelling at your devices...
Just image them actually listening... “

11. Conversational user interfaces. 11
CONFIDENTIAL
Accessibility
Interface with devices where previously impossible.
• Hands-free
• Control machines in manufacturing
• Hand labor
• More complex applications in cars.
• No room
• Clean rooms
• Kicthen

12. Conversational user interfaces. 12
CONFIDENTIAL
Data capture
• Gather information for advertisement
• Capture content from customers directly.
Speech to text
NLP extracts: dishes, amounts, …
{Course : fried chicken} {sides : fries , salad}
{Course : Ceaser salad} {exclusions : tomatoes } {sides : bread }
Local network
Queuing system & preprocessing
Speech alert, dishes ready
Next meal to cook
Confirmation

13. Conversational user interfaces. 13
CONFIDENTIAL
Lowering the threshold
Lower the threshold to data and computing

14. Conversational user interfaces. 14
CONFIDENTIAL
Factory Talks - analytics for devices
Rockwell automation.
• Shelby = data scientist in a box
• A production line Chabot -> gives production line managers status
updates and diagnostics platform.
• Already had a text based conversational interface or chatbot.
• A voice interface was created based on Microsofts Cortana.

15. Conversational user interfaces. 15
CONFIDENTIAL
Human replacement
Call centers/ telephone services & support.
More interactive Information
Reception systems

16. Conversational user interfaces. 16
CONFIDENTIAL
Augmenting human expierence
Invisible interface
Extend the power of computing to all people
Your personal translator
VISUALS NODIG
Google pixel earbuds

17. Conversational user interfaces. 17
CONFIDENTIAL
Are people ready for this interface.
85 % of consumers say:
“Voice interface experiences are great when they work but
irritating when they don’t work properly .”
-Creative Strategies.
This raises the question? How do we create a good user
interface.

18. Conversational user interfaces. 18
CONFIDENTIAL
Voice centered design

19. Conversational user interfaces. 19
CONFIDENTIAL
How to engage users
1. The skill makes a task faster and easier with voice.
2. The skill has an intuitive and memorable name.
3. The skill sets clear expectations on what it can do.
4. The skill minimizes friction.
5. The skill surprises and delights customers.
6. The skill delivers fresh content each time.
7. The skill is consistently reliable.
8. Sufficient guiding information is available to know the options.
9. Guiding information is adapted to the specific user and changing in time.

20. Conversational user interfaces. 20
CONFIDENTIAL
So what’s under the hood ?

21. Conversational user interfaces. 21
CONFIDENTIAL
Speech capture
•Hardware
•Microphone
•Basic signal
processing and
conditioning.
Speech
Recognition
•Speech to text.
Natural language
processing
•Recognize
intents.
•Recognize
similar
sentences
•Extracting
words and
classify.
Voice interaction
Model
•Model that
links user
utterances to
actions. (
Actions in the
software).
Software program
•Code that is
linked to voice
interaction
model.
•Is called by
voice
interaction
model and
executed
Actions
•Speech to text.
•Electronics
being
controlled.(I.e.
Lights turned
on)
• Audio player
started.
•External
software call (
i.e. Google
calander entry)
Components

22. Conversational user interfaces. 22
CONFIDENTIAL
Acoustic modeling
Speech
Waveform
generation
Grapheme-to-
phenome
conversion
Text She has twenty dollars In her pocket

23. Conversational user interfaces. 23
CONFIDENTIAL
Language model & pronunciation model
Speech
Waveform
generation
Grapheme-to-
phenome
conversion
Text
ˈrɛkɪɡˌ naɪz ˈspiːtʃ | ˈrɛk ə ˈnaɪsˈbiːtʃ
Recognize speech | Wreck a nice beach
P(Recognize speech) > P(Wreck a nice beach)

24. Conversational user interfaces. 24
CONFIDENTIAL
Deep Learning In Speech Recognition
Conclusions on how we got here:
• Available data +
• Computational capacity +
• A few algorithmic inventions enabled deep
networks +

25. Conversational user interfaces. 25
CONFIDENTIAL
Technology Conclusion
• Voice conversational UI requires sophisticated infrastructure.
• A lot of data and expertise needed to train complex models.
Focus in our Mission :
“Building engaging and powerful products and services.”

26. Conversational user interfaces. 26
CONFIDENTIAL
Alexa , Google home, … Device & software
Platform
Using existing frameworks
25,000 skills
Alexa Skills portal

27. Conversational user interfaces. 27
CONFIDENTIAL
Let’s get specific
“Any sufficiently advanced technology is indistinguishable from
magic.”
- Arthur C. Clarke

28. Conversational user interfaces. 28
CONFIDENTIAL
Input Sentence
Alexa, tell Benjamin I’m available
Components of that sentence:
Wake word : Activates the alexa to listen.
Launch command : Launches the application that we aim on using.
Invocation name : The name I gave the application
Utterance : The command you want give to the application.
Status Slot:
• Busy
• Meeting
• AvailableA
B
M

29. Conversational user interfaces. 29
CONFIDENTIAL
Dataflow overview
Skill: Benjamin
Utterance: “I’m available”

30. Conversational user interfaces. 30
CONFIDENTIAL
Data Flow - ASK
1. Define Intents
2. Provide a finite number of
sample utterances for each
intent.
3. ASK automatically builds
and deploys machine
learning models.
1
2
3

31. Conversational user interfaces. 31
CONFIDENTIAL
Data Flow
Request:
IntentRequest
requestId: EdwRequestId.a6c89145-279e-4e46-86bd-7833d1b90ddd
intent: SetStatusIntent
slot: Status = Available.
Response:
OutputSpeech:
ssml: <speak> ok </speak>,
shouldEndSession : true

32. Conversational user interfaces. 32
CONFIDENTIAL
Data Flow – IOT platform
OR

33. Conversational user interfaces. 33
CONFIDENTIAL
Data Flow – Endpoints

34. Conversational user interfaces. 34
CONFIDENTIAL
Data Flow – Improvements
XMOS VocalFusion 4-Mic
Dev Kit
Conexant 4-Mic Development Kit

35. Conversational user interfaces. 35
CONFIDENTIAL
Data Flow – Improvements

36. Conversational user interfaces. 36
CONFIDENTIAL
Data Flow – Improvements

37. Conversational user interfaces. 37
CONFIDENTIAL
The end
Any questions for me or my assistant ?

38. Conversational user interfaces. 38
CONFIDENTIAL
Complex voice interactions.
State : [Start]
• SetStatus intent
• Start Intent
• Stop Intent
• Help Intent
State : [Meeting
Continuation]
• No intent
• Yes intent
Utterance:
Benjamin,
I’m off to a meeting
Response:
Would you like
to tell me when you’ll be back?
Context variable:
State = Meeting continuation
State : […]
• ….
• ….
Additional question:
…
Context variable:
State = …
Perform logic
Response:
What time would
that be?

39. Conversational user interfaces. 39
CONFIDENTIAL
Innovation Day is an initiative of Masters in Innovation,
the umbrella brand of the Verhaert Group which aims
to connect, train and accelerate professional innovators.
Kruibeke
Belgium
Hogenakkerhoekstraat 21
B-9150 Kruibeke
T +32 3 250 19 00
E info@verhaert.com
www.verhaert.com
Nivelles
Belgium
Noordwijk
Netherlands
Av. Robert Schuman 102
B-1400 Nivelles
T +32 67 47 57 10
E info@lambda-x.com
www.lambda-x.com
Kapteynstraat 1
2201 BB Noordwijk
T +31 71 760 05 50
E info@verhaert.com
connect.verhaert.com
INDUSTRY
TECHXFER
MEDICAL
AEROSPACE
TECHXFER
FMCGCONNECT
TECHXFER
FMCGCONNECT
MEDICAL
Aveiro
Portugal
Av. Dr. Lourenço
Peixinho 96D 4o
3800-159 Aveiro
T +351 234 604 088
E info@load-interactive.com
www.load-interactive.com
CONNECT
Gentbrugge
Belgium
Bruiloftstraat 55-57
B-9050 Gentbrugge
T +32 9 330 27 90
E info@moebiusdesign.be
www.moebiusdesign.com
ON SITE CONSULTANCY