The document discusses Google Voice transcriptions and introduces recurrent neural networks (RNNs) and long short-term memory (LSTM) networks, emphasizing their architecture and learning process through backpropagation. It mentions various applications of RNNs and LSTMs, including time series analysis, natural language processing, and automatic cars. The document also suggests further reading materials for those interested in deepening their understanding of these neural networks.

1. Google voice transcriptions demystified:
Introduction to recurrent neural networks & LSTM
Lieva Anastasia @lievAnastazia
Human Talks Montpellier

2. Human Talks Montpellier @lievAnastazia
Google voice transcriptions

3. Human Talks Montpellier @lievAnastazia
Long Short-Term Memory LSTM

4. Human Talks Montpellier @lievAnastazia
Long Short-Term Memory LSTM
- Neural networks
- Recurrent neural networks (RNN)
- LSTM

5. Human Talks Montpellier @lievAnastazia
Neural networks

6. Human Talks Montpellier @lievAnastazia
Category
Neural networks

7. Human Talks Montpellier @lievAnastazia
Neural networks

8. Human Talks Montpellier @lievAnastazia
Neural networks

9. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Expected
Learning by Backpropagation
Ground truth

10. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Learning by Backpropagation
First iteration truth

11. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Learning by Backpropagation

12. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Update weights in every layer w/ an optimization algorithm
Learning by Backpropagation

13. Human Talks Montpellier @lievAnastazia
L
A
Y
E
R
1
L
A
Y
E
R
2
L
A
Y
E
R
3
L
A
Y
E
R
4
L
A
Y
E
R
5
Input
Data
Prediction
Error
Update weights in every layer w/ an optimization algorithm
Retry prediction with updated weights
LeaLearning by Backpropagation
ning by Backpropagation

14. Human Talks Montpellier @lievAnastazia
Recurrent neural networks

15. Human Talks Montpellier @lievAnastazia
“Découvrez Montpellier une belle ville en Occitanie”

16. Human Talks Montpellier @lievAnastazia
Recurrent neural networks

17. Human Talks Montpellier @lievAnastazia
Je vais passer les vacances en Occitanie.
Je veux bien apprendre à parler Occitan

18. Human Talks Montpellier @lievAnastazia
Je vais passer les vacances en Occitanie
Je veux bien apprendre à parler ????

19. Human Talks Montpellier @lievAnastazia
- Long Short-Term Memory networks

20. Human Talks Montpellier @lievAnastazia

21. Human Talks Montpellier @lievAnastazia
Je vais passer les vacances en Occitanie.
Je veux bien apprendre à parler Occitan

22. Human Talks Montpellier @lievAnastazia
Use cases
- Time series
- Video processing
- Natural language processing
- Audio processing
- Sequence data in bio-informatics
- Automatic cars
- ...

23. Human Talks Montpellier @lievAnastazia
Variations of RNN & LSTM
- Bi-directional RNNs
- Hierarchical RNNS
- Depth Gated RNN
- GRU
- LSTM with peephole connections
- ...

24. Human Talks Montpellier @lievAnastazia
To go further ( or deeper :P )
- Andrej Karpathy blog
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
- Colah blog
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- Books and courses on stanford.edu
- Articles of Jürgen Schmidhuber
- Recent Advances in Recurrent Neural Networks
https://arxiv.org/abs/1801.01078

25. Google voice transcriptions demystified:
Introduction to recurrent neural networks & LSTM
Lieva Anastasia @lievAnastazia
Human Talks Montpellier