This talk covers the current state-of-the-art open source projects related to AI & machine learning applied to realtime communications. Includes practical examples and containers that allow for utilizing them today.

1. Machine Learning
&
A.I.

2. What we are: a phone assistant
What we do: assist phones.
Who Am I

3. What Is Machine Learning
A form of artificial intelligence (AI) that provides systems the
ability to automatically learn and improve from experience
without being explicitly programmed

4. Open Source Projects
TensorFlow Keras
Torch

5. Machine Learning Applied
GoogleTranslate Google Speech API
Amazon Polly Amazon Lex Wit.ai

6. +

7. Latest Algorithms & Projects

8. Natural Language
Processing

9. A suite a tools that allows for NLU / NLP tasks
Word Vectors King Queen Man Woman
Hunting
Royalty
Elizabeth
Princess
0.99
0.01
0.06
0.76
0.99
0.08
0.88
0.14
0.05
0.03
0.03
0.82
0.06
0.93
0.92
0.31
A way to represent the ‘meaning’ of a word.

10. A suite a tools that allows for NLU / NLP tasks
Word Vectors
"Woman - Man + King = ?"
Queen!
King
Woman
Man (0.5,.0.4)
(0.3, 0.8)
(0.3,0.8) - (0.5,0.4) + King = ?
(-0.2,0.4) + King = ?
(-0.2,0.4) + (0.5,0.8) = ?
(0.3,1.2)(0.5,0.8)
(0.3,1.2)

11. A suite a tools that allows for NLU / NLP tasks
Word Vectors
0.99
0.01
0.06
0.76
0.99
0.08
0.88
0.14
0.05
0.03
0.03
0.82
0.06
0.93
0.92
0.31
King Queen Man Woman
Hunting
Royalty
Elizabeth
Princess
A way to represent the ‘meaning’ of a word.
Pairs of words acquire vector offsets that are
shared by words with similar relationships
“King – Man + Woman = ?” Queen!
Xtrees - Xtree ≈ Xchairs - Xchar ≈ Xcities - Xcity

12. Sentence Tokenization - Part-of-Speech Tagging - Named Entity Recognition
curl -s localhost:8080/ent -d '{"text":"A place like Digium is full of amazing people. And Matt Jordan.", "model":"en"}'
[
{
"end": 19,
"start": 13,
"text": "Digium",
"type": "GPE"
},
{
"end": 62,
"start": 51,
"text": "Matt Jordan",
"type": "PERSON"
}
]%

13. https://github.com/RasaHQ/rasa_nlu
## intent:affirm
- yes
- yep
- yeah
- indeed
- that's right
- ok
- great
- right, thank you
- correct
- great choice
- sounds really good
## intent:goodbye
- bye
- goodbye
- good bye
- stop
- end
- farewell
- Bye bye
- have a good one
## intent:greet
- hey
- howdy
- hey there
- hello
- hi
- good morning
- good evening
- dear sir
## intent:restaurant_search
- i'm looking for a place to eat
- I want to grab lunch
- I am searching for a dinner spot
- i'm looking for a place in the [north](location) of town
- show me [chinese](cuisine) restaurants
- show me [chines](cuisine:chinese) restaurants
- show me a [mexican](cuisine) place in the [centre](location)
- i am looking for an [indian](cuisine) spot called olaolaolaolaolaola
- search for restaurants
- anywhere in the [west](location)
- anywhere near [18328](location)
- I am looking for [asian fusion](cuisine) food
- I am looking a restaurant in [29432](location)
- I am looking for [mexican indian fusion](cuisine)
- [central](location) [indian](cuisine) restaurant
curl -XPOST localhost:5000/parse -d '{"q":"I want something delicious",
"project":"astricon_model"}'
{
"entities": [],
"intent":
{
"confidence": 0.45734276497763227,
"name": "restaurant_search"
},
"text": "I want something delicious",
"intent_ranking":
[{
"confidence": 0.45734276497763227,
"name": "restaurant_search"
},
{
"confidence": 0.32659239812473556,
"name": "affirm"
},
{
"confidence": 0.11959261779323696,
"name": "goodbye"
},
{
"confidence": 0.09647221910439494,
"name": "greet"
}]
}

14. Sentiment Analysis

15. Sentiment Analysis
AFINN-165 - list of English words rated between -5 -> +5
curl --data "text=Astricon%20is%20such%20an%20amazing%20time" http://localhost:8888
{"score":4,"comparative":0.6666666666666666,"tokens":["astricon","is","such","an","amazing","time"],"words":
["amazing"],"positive":["amazing"],"negative":[]}%
curl --data "text=👍 %20🍻 " http://localhost:8888
{"score":4,"comparative":2,"tokens":["👍 ","🍻 "],"words":["🍻 ","👍 "],"positive":["🍻 ","👍 "],"negative":[]}%
Emoji Sentiment Ranking

16. Speech Recognition

17. • Implementation of the Baidu Deep Speech research paper
• UtilizesTensorflow as the underlying implementation engine
• Based on a deep learning neural network model
Mozilla Deep Speech

18. Deep Learning Neural Network
Input Layer
Hidden Layer Hidden Layer
Output Layer
Cost Function
Backpropogation updates
weights throughout
network
Backpropagation Simple Example

19. Gradient Descent

20. Mozilla Deep Speech
Word Error Rate of 13% with best available corpora
10,000 hours of recordings for release in Q4
https://voice.mozilla.org
• Implementation of the Baidu Deep Speech research paper
• UtilizesTensorflow as the underlying implementation engine
• Based on a deep learning neural network model

21. Speech & Music Synthesis

22. WaveNet
Concatenative
Parametric
WaveNet
MOH
Neural Network Based

23. Tacotron
“The buses aren't the problem, they actually provide a solution.”
“The buses aren't the PROBLEM, they actually provide a SOLUTION.”
Google research paper: https://arxiv.org/pdf/1703.10135.pdf
A Fully End-to-End Text-To-Speech Synthesis Model,
• Uses <text, audio> pairs to generate complete voices directly from text
• Much faster to train than WaveNet while retaining high MOS scores
• Can handle complex & interesting variation in sentence structure
"Basilar membrane and otolaryngology are not auto-correlations"
Stress
Complexity

24. An Intro Into Partial
Differential Equations
Just Kidding.

25. Questions?

26. Evan McGee
@startledmarmot
evan@hifelix.io