This site collects resources to learn Deep Learning in the form of Modules available through the sidebar on the left. As a student, you can walk through the modules at your own pace and interact with others thanks to the associated Discord server. You don’t need any special hardware or software.
The main goal of the course is to allow students to understand papers, blog posts and codes available online and to adapt them to their projects as soon as possible. In particular, we avoid the use of any high-level neural networks API and focus on the PyTorch library in Python.
The course is divided into sessions (containing possibly several modules), each session requiring a significant amount of coding. At the end of this course, students were able to read very recent papers and reproduce (or even ameliorate) their experiments.
All the code used in this course is available on the GitHub repository dataflowr/notebooks. You will find the solutions to the practicals on this repo! You can fork the repo if you want to run the code locally: GitHub Docs about fork then follow the steps in Module 0. Most of the code will not require a GPU.
⚠ When a GPU is required , you can launch the code on colab by following the corresponding link given in the module (see for example Module 1).
Pre-requisites:
Mathematics: basics of linear algebra, probability, differential calculus and optimization
Programming: Python. Test your proficiency: quiz
Start right away and train a deep neural network on a GPU with Module 1 - Introduction & General Overview
Be sure to build your own classifier with more dogs and cats in the practicals. Things to remember
you do not need to understand everything to run a deep learning model! But the main goal of this course will be to come back to each step done today and understand them...
to use the dataloader from Pytorch, you need to follow the API (i.e. for classification store your dataset in folders)
using a pretrained model and modifying it to adapt it to a similar task is easy.
if you do not understand why we take this loss, that's fine, we'll cover that in Module 3.
even with a GPU, avoid unnecessary computations!
Module 2b - Automatic differentiation + Practicals
MLP from scratch start of HW1
Pytorch tensors = Numpy on GPU + gradients!
in deep learning, broadcasting is used everywhere. The rules are the same as for Numpy.
Automatic differentiation is not only the chain rule! Backpropagation algorithm (or dual numbers) is a clever algorithm to implement automatic differentiation...
Module 5 - Stacking layers and overfitting a MLP on CIFAR10
how to regularize with dropout and uncertainty estimation with MC Dropout: Module 15 - Dropout
Loss vs Accuracy. Know your loss for a classification task!
know your optimizer (Module 4)
know how to build a neural net with torch.nn.module (Module 5)
know how to use convolution and pooling layers (kernel, stride, padding)
know how to use dropout
Module 8b - Collaborative filtering and build your own recommender system: 08_collaborative_filtering_empty.ipynb (on a larger dataset 08_collaborative_filtering_1M.ipynb)
Module 8c - Word2vec and build your own word embedding 08_Word2vec_pytorch_empty.ipynb
Module 16 - Batchnorm and check your understanding with 16_simple_batchnorm_eval.ipynb and more 16_batchnorm_simple.ipynb
start of Homework 2: Class Activation Map and adversarial examples
know how to use dataloader
to deal with categorical variables in deep learning, use embeddings
in the case of word embedding, starting in an unsupervised setting, we built a supervised task (i.e. predicting central / context words in a window) and learned the representation thanks to negative sampling
know your batchnorm
architectures with skip connections allows deeper models
Module 9a: Autoencoders and code your noisy autoencoder 09_AE_NoisyAE.ipynb
Module 10: Generative Adversarial Networks and code your GAN, Conditional GAN and InfoGAN 10_GAN_double_moon.ipynb
start of Homework 3: VAE for MNIST clustering and generation
Module 11b - Recurrent Neural Networks practice and predict engine failure with 11_predicitions_RNN_empty.ipynb
Correcting the PyTorch tutorial on attention in seq2seq: 12_seq2seq_attention.ipynb
Build your own microGPT: GPT_hist.ipynb
Build your own Real NVP: Normalizing_flows_empty.ipynb
Train your own DDPM on MNIST: ddpm_nano_empty.ipynb
Finetuning on CIFAR10: ddpm_micro_sol.ipynb
For more updates: 
and check the
Marc Lelarge, Andrei Bursuc with Jill-Jênn Vie
Super fast track to learn the basics of deep learning from scratch:
Have a look at the slides of Module 1: Introduction & General Overview
Run the notebook (or in colab) of Module 2a: Pytorch Tensors
Run the notebook (or in colab) of Module 2b: Automatic Differentiation
Check the Minimal working examples of Module 3: Loss functions for classification. If you do not understand, have a look at the slides.
Have a look at the slides of Module 4: Optimization for Deep Learning
Try playback speed 1.5 for the video from Module 5: Stacking layers.
Run the notebook (or in colab) of Module 6: Convolutional Neural Network
Try playback speed 2 for the video from Module 7: Dataloading
Have a look at the slides of Module 8a: Embedding layers
Well done! Now you have time to enjoy deep learning!
Join the GitHub repo dataflowr and make a pull request. What are pull requests?
Thanks to Daniel Huynh, Eric Daoud, Simon Coste
Materials from this site is used for courses at ENS and X.
Last modified: September 15, 2023. Website built with Franklin.jl and the Julia programming language. 
