Module 11a - Recurrent Neural Networks theory

Table of Contents

Theory of RNNs

0:00 Recap
0:52 Introduction to RNNs
1:17 1D convolutional networks for sequences
2:16 Various tasks for RNNs
5:15 Theory of RNN
7:59 Backprop for RNN
10:30 A binary classification problem for sequences
17:17 Elman network
21:02 Training RNN
22:51 Results for Elman network
24:22 Gating for RNN
28:10 Gated RNN in PyTorch
29:27 Results for gated RNN
30:12 LSTM and GRU
34:11 Equations for GRU
37:23 Equations for LSTM
40:31 LSTM in PyTorch
42:44 Results for LSTM
43:43 Empirical results for LSTM and GRU

Slides

• slides

References

• Understanding LSTM Networks by Christopher Olah