Deep Learning Do It Yourself!

This site collects resources to learn Deep Learning in the form of Modules available through the sidebar on the left. As a student, you can walk through the modules at your own pace and interact with others thanks to the associated Discord server. You don’t need any special hardware or software.

Practical deep learning course

The main goal of the course is to allow students to understand papers, blog posts and codes available online and to adapt them to their projects as soon as possible. In particular, we avoid the use of any high-level neural networks API and focus on the PyTorch library in Python.

The course is divided into sessions (containing possibly several modules), each session requiring a significant amount of coding. At the end of this course, students were able to read very recent papers and reproduce (or even ameliorate) their experiments.

All the code used in this course is available on the GitHub repository dataflowr/notebooks. You will find the solutions to the practicals on this repo! You can fork the repo if you want to run the code locally: GitHub Docs about fork then follow the steps in Module 0. Most of the code will not require a GPU.

⚠ When a GPU is required , you can launch the code on colab by following the corresponding link given in the module (see for example Module 1).


🌻 Session 1 - Finetuning VGG

Start right away and train a deep neural network on a GPU with Module 1 - Introduction & General Overview

Be sure to build your own classifier with more dogs and cats in the practicals.

Things to remember

  • you do not need to understand everything to run a deep learning model! But the main goal of this course will be to come back to each step done today and understand them...

  • to use the dataloader from Pytorch, you need to follow the API (i.e. for classification store your dataset in folders)

  • using a pretrained model and modifying it to adapt it to a similar task is easy.

  • if you do not understand why we take this loss, that's fine, we'll cover that in Module 3.

  • even with a GPU, avoid unnecessary computations!

🌻 Session 2 - PyTorch tensors and Autodiff

Things to remember
  • Pytorch tensors = Numpy on GPU + gradients!

  • in deep learning, broadcasting is used everywhere. The rules are the same as for Numpy.

  • Automatic differentiation is not only the chain rule! Backpropagation algorithm (or dual numbers) is a clever algorithm to implement automatic differentiation...

🌻 Session 3

Things to remember
  • Loss vs Accuracy. Know your loss for a classification task!

  • know your optimizer (Module 4)

  • know how to build a neural net with torch.nn.module (Module 5)

  • know how to use convolution and pooling layers (kernel, stride, padding)

  • know how to use dropout

🌻 Session 4

Things to remember
  • know how to use dataloader

  • to deal with categorical variables in deep learning, use embeddings

  • in the case of word embedding, starting in an unsupervised setting, we built a supervised task (i.e. predicting central / context words in a window) and learned the representation thanks to negative sampling

  • know your batchnorm

  • architectures with skip connections allows deeper models

🌻 Session 5

🌻 Session 6

🌻 Session 7



Marc Lelarge, Andrei Bursuc with Jill-Jênn Vie

Course in a hurry

Super fast track to learn the basics of deep learning from scratch:

For contributors

Join the GitHub repo dataflowr and make a pull request. What are pull requests?

Thanks to Daniel Huynh, Eric Daoud, Simon Coste

Materials from this site is used for courses at ENS and X.