Discussion about product definition for Machine Learning and Artificial Intelligence technologies, describing in details the specific solution for Google Home, in particular Neural Adaptive Beam forming, LAS architecture and BERT learning transfer

1. Engineering
Machine Learning
Technologies
From ideas to products with ML/AI
Andres L. Martinez a.k.a almo
Google Cloud DevRel
@davilagrau

2. Why is this time different?Photo by Ross Findon on Unsplash

3. Photo by Quino Al on Unsplash
Innovation

4. Photo by Finn Hackshaw on Unsplash

5. Fast{er}
Photo by Josh Calabrese on Unsplash

6. Focus on the users

7. the user

8. Let’s start
with the
basics
Photo by Hello I'm Nik on Unsplash

9. Don’t be afraid to launch a product
without machine learning

10. Don’t be Evil 2018
AI for Good

11. Objectives for AI applications
● Be socially beneficial.
● Avoid creating or reinforcing unfair
bias.
● Be built and tested for safety.
● Be accountable to people.
● Incorporate privacy design principles.
● Uphold high standards of scientific
excellence.
● Be made available for uses that
accord with these principles.
AI applications we will not
pursue:
● Technologies that cause or are likely to
cause overall harm.
● Weapons or other technologies whose
principal purpose or implementation is to
cause or directly facilitate injury to people.
● Technologies that gather or use information
for surveillance violating internationally
accepted norms.
● Technologies whose purpose contravenes
widely accepted principles of international
law and human rights.
AI Principles

12. Productification
Photo by AbsolutVision on Unsplash
Photo by Fancycrave on Unsplash

13. First, design and implement metrics

14. Product
Design

17. What does it make it so special?
● Filtering Definition + Signal Reconstruction
● Speech Recognition
● Natural Language Processing

18. Product Pipeline
Acoustic
Model
Pronunciation
Model
Language
Model

19. References
Control
Engineering
Neural Network
Adaptive Beamforming
for Robust Multichannel
Speech Recognition
(2016)
http://bit.ly/2zQWp08
Speech
Recognition
State of the Art Speech
Recognition With
Sequence to Sequence
Model (2018)
http://bit.ly/2Bb8JIo
Natural Language
Processing
BERT: Pre-training of
Deep Bidirectional
Transformers for
Language
Understanding (2018)
http://bit.ly/2Qt3Vsn
http://bit.ly/2rt7hfW

20. Neural {Adaptive} Beamforming
Neural Network Adaptive Beamforming for Robust
Multichannel Speech Recognition (2016)

21. LSTM
LSTM LSTM
LinearLinear
X1
(k)[t] X2
(k)[t]
FP
h1
(k)[t] h2
(k)[t]
FS
y(k)[t]
Acoustic
Model
Gated Feedback
Output Targets
LSTM
DNN
DNN
tConv
pool
+
nonlin
Linear
LSTM
LSTM
LSTM
DNN
Linear
Multitask
Learning
Improvements

22. Experimental Setup: simulated
Training Setup
● 2000 hours noisy training data
● 3 millions English utterances
● Room Simulator
○ Noise
○ Reverberation
● Noise
○ YouTube
○ Daily Noise
● SNR 0-20 dB (12 dB average)
Evaluation Setup
● 200 hours
● 30k English Utterances
● Different target speaker
● Different noise position
● Similar mic array geometry

23. Speech Recognition
State of the Art Speech Recognition With Sequence to
Sequence Model (2018)

24. L{isten} A{ttend} S{pell}
Encoder Attention Decoder
Acoustic Model Pronunciation Model Language Model
Queries
Queries
Queries
Queries
Project Source
Project Source
Project Source
Project Source
Project Source
Word piece
Model
Multi-Headed Attention
Improvements

25. Experimental Setup: simulated
Training Setup
● 12500 hours noisy training data
● 15 millions English utterances
● Room Simulator
○ Noise
○ Reverberation
● Noise
○ YouTube
○ Daily Noise
● SNR 0-30 dB (15 dB average)
Evaluation Setup
● 14800 English Utterances
● 15700 Longer sentences

26. Natural Language Processing
BERT: Pre-training of Deep Bidirectional Transformers for Language
Understanding (2018)

27. T1
Tm
Tm
E1
T2
Tm
Tm
E2
TN
Tm
Tm
EN
BERT
(Open Source)
Fine- Tune
Improvements

28. Experimental Setup and benchmark
Training Setup
● BooksCorpus 800M
● Wikipedia 2500M
Benchmark
● GLUE Dataset
● MNLI
● QQP
● QNLI
● SST-2
● CoLa
● STS-B
● MRPC
● RTE
● WNLI
● SQuAD v1.1
● SWAG
(Adversarial)

29. Summary
● Modular Approach
○ Beamforming / Speech Recognition /NLP
● Moral Risk insignificant or reduced
○ AI Principles easy to apply and verify
● Availability of training data
○ Simulated
● Transfer Learning
● Short/Incremental productification cycle

30. Don’t forget to check this
out!
Best Practices for ML Engineering
(Martin Zinkevich)
http://bit.ly/2EDsLza

31. … and this!
AI Principles
http://bit.ly/2EDsLza

32. Progress of AI and Machine Learning
https://goo.gl/FWWBv2
… and this!!!

33. Thank you!
Questions?
Andres L. Martinez a.k.a almo
@davilagrau