This document provides an overview of recurrent neural networks (RNNs) and some of their applications. It discusses the basic structure of RNNs and how they can be used for tasks like sequence prediction and language modeling. It also introduces more advanced RNN architectures like LSTMs, GRUs, and encoder-decoder models that address some limitations of traditional RNNs. Examples of applications discussed include sequence-to-sequence learning, language translation, and recursive neural networks.

1. : 0
1
hirono

2. • 10.1
• 10.2 (RNN)
• 10.3 RNN
• 10.4 Encoder-Decoder Seq2seq
• 10.5
• 10.6

3. • 10.7
• 10.8
• 10.9 Leaky
• 10.10 LSTM RNN
• 10.11
• 10.12

4. RNN
• (recurrent neural
network)
•
recurrent neural network
recursive neural network
recurrent neural network?

5. t = 3
t
•

6. •
t t+1 1

7. •
t t+1 1
h

8. •
•

9. RNN :
•
• !!

10. RNN :
( )
(L x y )
→ 10.2.2

11. RNN :
•
• o à

12. RNN :
•
• o à

13. (teacher forcing)
• t + 1 y^t
y^t o^t

14. RNN :
•
• …

15. RNN
• (BPTT)
i t t
t + 1 i

16. RNN
• (BPTT)

17. RNN
•
FB
p.55

18. •
• h
v
v
RNN

19. v
RNN
•
• t
RNN
t
t + 1
t
t

20. RNN
•
•
•

21. • x
• x RNN
w x
x y

22. x
x h
•
x
…

23. •

24. RNN
• 2 RNN
•
o
t
t

25. Encoder-Decoder Seq2Seq
•
…
• 2014 Encoder-decoder Sequence-to-sequence
• RNN
)
Good morningRNN

26. Encoder-Decoder Seq2Seq
C…
• C
• C
( 12.4.5.1)
2 RNN

27. RNN
• RNN
: U
: W
: V
3

28. • Recursive neural network
•
•
RNN Recursive NN

29. •
o

30. • /
RNN
1 →0
1 →

31. (ESN)
• h(t-1) h(t) x(t) h(t)
•
• /
:

32. (ESN)
• RNN
→ 1
( )
1 n …!
v λ J
…!

33. Leaky
• ( )
• Leaky

34. LSTM( )
•
•
•

35. LSTM( )
0 ~ 1

36. LSTM( )
0 ~ 1

37. LSTM( )
0 ~ 1

38. LSTM( )

39. LSTM RNN
• GRU(Gated Recurrent Unit(GRU))
•
•

40. •
RNN
…

41. •
1
• →
•
LSTM

42. • :
NN
…
:

43. •
•
→12.4.5.1