Module 11b - Recurrent Neural Networks practice

Table of Contents

Theory of RNNs

0:00 Generating the dataset for binary classification of parentheses
4:56 Elman network
11:25 RNN with gating
14:06 LSTM
18:33 Be careful with errors given on the training set!




RNNs can generate bounded hierarchical languages with optimal memory (2020) John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning arXiv:2010.07515

Self-Attention Networks Can Process Bounded Hierarchical Languages (2021) Shunyu Yao, Binghui Peng, Christos Papadimitriou, Karthik Narasimhan arXiv:2105.11115