Module 11b - Recurrent Neural Networks practice

Table of Contents

Theory of RNNs

0:00 Generating the dataset for binary classification of parentheses
4:56 Elman network
11:25 RNN with gating
14:06 LSTM
18:33 Be careful with errors given on the training set!




Hewitt, J., Hahn, M., Ganguli, S., Liang, P., & Manning, C. D. (2020). RNNs can generate bounded hierarchical languages with optimal memory.arXiv:2010.07515

Machine translation

Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).

Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014, October). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1724-1734).

SketchRNN: RNN and VAE

Ha, D., & Eck, D. (2018, February). A Neural Representation of Sketch Drawings. In International Conference on Learning Representations.

Generating LaTeX code from handwritten maths

Zhang, J., Du, J., & Dai, L. (2017, November). A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) (Vol. 1, pp. 902-907). IEEE.

Solving mathematical expressions

Lample, G., & Charton, F. (2019, September). Deep Learning For Symbolic Mathematics. In International Conference on Learning Representations.