The document discusses speech recognition as a user interface. It begins with an introduction of the speaker and an overview of what will be covered, including a definition of speech recognition, examples of what it can be used for, and a demonstration. The document then discusses how speech recognition works from an technical perspective involving acoustic signal processing and various machine learning techniques. It covers aspects like speaker dependence and independence as well as continuous speech recognition. The document discusses moving beyond basic dictation to understand a user's intent in order to take appropriate actions. It provides examples of speech recognition systems and APIs from companies like Google, Apple, Amazon and Microsoft. It concludes with code examples and contact information for the speaker.

1. PAGE1
© 2015 Apio Systems, Inc. Confidential 1
Jared Sheehan @ Driversiti
Speech Recognition as a User Interface

2. PAGE2
© 2015 Apio Systems, Inc. Confidential 2
Who am I
Glass explorer, speech recognition enthusiast and big android nerd
Android Lead @Driversiti - driving safety for the mobile generation
Speech Recognition application for the Amazon Fire Phone
Suite of applications - AIM Android, Engadget Android, Distro Android, TechCrunch
Android, AOL HD, AIM Blackberry
Meetup evangelist – “DC Android Meetup Group” – Join today!

3. PAGE3
© 2015 Apio Systems, Inc. Confidential 3
Overview
What is voice/speech recognition?
What awesome stuff you can do with it?
How it works…
Demo!
Question and Answer

4. PAGE4
© 2015 Apio Systems, Inc. Confidential 4
Hello Computer…

5. PAGE5
© 2015 Apio Systems, Inc. Confidential 5
Definition

6. PAGE6
© 2015 Apio Systems, Inc. Confidential 6
What can you do with SR?
Technology that allows spoken input into software systems.
You speak to your computer, tablet, phone or device and it uses what you said as input to
trigger some sort of action.
Replace other methods of input like clicking, swiping, typing or selecting in other ways.
It is a means to make devices and software more user-friendly and to increase productivity.
It is used extensively as a form of accessibility assistance.

7. PAGE7
© 2015 Apio Systems, Inc. Confidential 7
ASR - Dictation
Automatic speech recognition (ASR) also called Dictation
Translates speech input into words, sentences and punctuation.
Audio is input through a microphone and streamed somewhere
The result is usually returned as a string with a confidence level
Very easy integration with Android – 2 ways to do it.

8. PAGE8
© 2015 Apio Systems, Inc. Confidential 8
How does it work?
A user speaks into a recording device of some sort
Speech recognition begins with the digital sampling of speech and then acoustic signal
processing of the audio.
Several processes including DTW (Dynamic time warping), HMM (Hidden Markov models)
and NN’s (Neural Networks) can achieve the desired results
Most systems use language specific knowledge to tune the models.
Next is the actual recognition of phonemes, groups of phonemes and words

9. PAGE9
© 2015 Apio Systems, Inc. Confidential 9
Speech Recognition system architecture

10. PAGE10
© 2015 Apio Systems, Inc. Confidential 10
Into the weeds
Speaker dependence
Speaker independence
Continuous Speech
How good is your system? Hint: Word Error Rate
Isolated word
Is that all it does??

11. PAGE11
© 2015 Apio Systems, Inc. Confidential 11
Dictation is cool, but not that cool
Next step is understanding what the user wants to do
Then act on it
Generally, the ASR results are passed into an Intent recognition system with additional
information
Contextual information can be, where the utterance is coming from (mobile phone,
computer), what app they are using, location etc.
That information is used to determine the user’s intent and execute the request.

12. PAGE12
© 2015 Apio Systems, Inc. Confidential 12
Intent recognition
Recognizing speech is only part of the process. How does Google Now know that I want to
send an SMS message to a friend? How does Siri know when I want to know how tall
Kobe Bryant is?
ASR is only the first step in true Speech as a user interface. To successfully help users
perform useful actions we must understand their intent. How to do this?
Three systems; ASR, Intent Recognition and a Dialog Engine
The Dialog engine takes the output from the IR system and sends responses and
actionable information to the caller.

13. PAGE13
© 2015 Apio Systems, Inc. Confidential 13
Android Speech APIs

14. PAGE14
© 2015 Apio Systems, Inc. Confidential 14
Android Speech APIs
http://developer.android.com/reference/android/speech/package-summary.html
Relatively easy implementation
<uses-permission android:name="android.permission.RECORD_AUDIO" />
A UI and no UI API
InputMethodServices use the no UI version - Keyboards

15. PAGE15
© 2015 Apio Systems, Inc. Confidential 15
Recognizer Intent
UI is supplied for you
Fire the intent and get a result
Again very easy to use

16. PAGE16
© 2015 Apio Systems, Inc. Confidential 16
SpeechRecognizer
UI is not supplied for you
Results are streamed directly to the EditText
Still “fairly” easy to use

17. PAGE17
© 2015 Apio Systems, Inc. Confidential 17
Google Now – Onto Intent recognition systems…

18. PAGE18
© 2015 Apio Systems, Inc. Confidential 18
Google Now – On tap

19. PAGE19
© 2015 Apio Systems, Inc. Confidential 19
Apple – Siri

20. PAGE20
© 2015 Apio Systems, Inc. Confidential 20
Amazon – Fire phone, Fire Tv and Echo

21. PAGE21
© 2015 Apio Systems, Inc. Confidential 21
Microsoft – Cortana

22. PAGE22
© 2015 Apio Systems, Inc. Confidential 22
Speech providers – Google, Nuance, IBM Watson

23. PAGE23
© 2015 Apio Systems, Inc. Confidential 23
Google Voice Interaction API

24. PAGE24
© 2015 Apio Systems, Inc. Confidential 24
Nuance Speech SDK
Dragon Mobile – SDK – Free up to 20k transactions per/month
Upload custom vocabularies
Developer: Uploads a new song and music vocabulary
Utterance: “Eminem” higher probability then “M&M”

25. PAGE25
© 2015 Apio Systems, Inc. Confidential 25
User Interface examples - Google Glass

26. PAGE26
© 2015 Apio Systems, Inc. Confidential 26
User Interface examples - Google Glass continued…

27. PAGE27
© 2015 Apio Systems, Inc. Confidential 27
User Interface examples - Google Glass continued…

28. PAGE28
© 2015 Apio Systems, Inc. Confidential
Enough talk!

29. PAGE29
© 2015 Apio Systems, Inc. Confidential
Show me code!

30. PAGE30
© 2015 Apio Systems, Inc. Confidential
jared.sheehan@driversiti.com
http://www.meetup.com/DCAndroid/
Tweet: @jayroo5245
THANK YOU