Module 11b - Recurrent Neural Networks practice

Table of Contents

Theory of RNNs

0:00 Generating the dataset for binary classification of parentheses
4:56 Elman network
11:25 RNN with gating
14:06 LSTM
18:33 Be careful with errors given on the training set!

Notebook

• notebook in colab

Practicals

• notebook (or opened in colab) for predicting engine failure with RNN.

References

RNNs can generate bounded hierarchical languages with optimal memory (2020) John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning arXiv:2010.07515

Self-Attention Networks Can Process Bounded Hierarchical Languages (2021) Shunyu Yao, Binghui Peng, Christos Papadimitriou, Karthik Narasimhan arXiv:2105.11115