Module 3 - Loss functions for classification

Table of Contents

Loss functions for classification

0:00 Recap
2:25 How to choose your loss?
3:18 A probabilistic model for linear regression
7:50 Gradient descent, learning rate, SGD
11:30 Pytorch code for gradient descent
15:15 A probabilistic model for logistic regression
17:27 Notations (information theory)
20:58 Likelihood for logistic regression
22:43 BCELoss
23:41 BCEWithLogitsLoss
25:37 Beware of the reduction parameter
27:27 Softmax regression
30:52 NLLLoss
34:48 Classification in pytorch
36:36 Why maximizing accuracy directly is hard?
38:24 Classification in deep learning
40:50 Regression without knowing the underlying model
42:58 Overfitting in polynomial regression
45:20 Validation set
48:55 Notion of risk and hypothesis space
54:40 estimation error and approximation error

Slides and Notebook

• slides

• notebook in colab An explanation of underfitting and overfitting with polynomial regression.

Frequently asked questions

Why using the sigmoid as link function for classification?

Because its gradient has simple form: $\sigma' = \sigma(1 - \sigma)$.

Edit this page on Last modified: January 12, 2021. Website built with Franklin.jl and the Julia programming language.