Skip to main content

Exercises

Exercises

Conceptual

Exercise 1 Conceptual

Given an input volume of width \(n\), height \(n\) and depth \(c\), and a convolutional layer with \(k\) filters of size \(f \times f \times c\), stride \(s\) and padding \(p\).

  1. Give the width, the height and the depth of the output volume.
  2. Give the number of parameters of the layer.
  3. Give the number of multiplications needed for one forward pass.

Exercise 2 Conceptual

We want to classify RGB images of 100 by 100 pixels into 20 classes. Compute the number of parameters for each architecture.

  1. Flatten, then one dense layer of 200 neurons, then the output layer.
  2. Two convolutional layers with 32 filters of size 3 by 3, each followed by 2 by 2 max pooling, then flatten, then the output layer.
  3. The same as 2 but with 64 filters in the second layer.

Exercise 3 Conceptual

Here is a 5 by 5 image with padding 1 already applied, and two 3 by 3 filters. Compute the output of a convolutional layer with stride 1 and a relu non-linearity, by hand.

Use the image with 1 on the diagonal and 0 elsewhere, and the filters

\[ F_1 = \begin{pmatrix} 1 & 0 & -1 \\ 1 & 0 & -1 \\ 1 & 0 & -1\end{pmatrix}, \qquad F_2 = \begin{pmatrix} 1 & 1 & 1 \\ 0 & 0 & 0 \\ -1 & -1 & -1\end{pmatrix}. \]

What does each filter respond to.

Exercise 4 Conceptual

Explain in two sentences each.

  1. Why does a convolutional layer have far fewer parameters than a dense layer on the same input.
  2. Why does max pooling make the representation slightly shift invariant.
  3. Why can a recurrent network handle sequences of different lengths while a convolutional network cannot.

Applied

Exercise 5 Applied

Train a convolutional network on MNIST.

  1. Build the network from the example page and train it for five epochs.
  2. Report the test accuracy and compare to the fully connected network from week 7.
  3. Plot the filters of the first layer. Can you see what they respond to.
  4. Remove the pooling layers and train again. What happens to the number of parameters and to the accuracy.

Exercise 6 Applied

Transfer learning.

  1. Take a network pretrained on ImageNet, for example resnet18 from torchvision.
  2. Replace the output layer by one with the number of classes you need.
  3. Freeze all other parameters and train only the new layer, on a few hundred images.
  4. Compare to training the same architecture from scratch on the same data.