Overview

Flexibility and the bias-variance tradeoff

Pages

Polynomial regression	Making a linear model more flexible
k nearest neighbours	A method with no parameters to fit
Bias-variance decomposition	What the expected error is made of
Regularization	Ridge and lasso, making a model less flexible
Exercises

Goals

The goal of this week is to

1. see a regression and a classification example where linear methods are not flexible enough,
2. understand and learn to run polynomial regression and classification,
3. understand and learn to run k nearest neighbour regression and classification,
4. understand the error decomposition in regression,
5. understand the bias-variance decomposition in regression,
6. understand ridge regression (L2 regularization), and
7. understand lasso regression (L1 regularization).