Exercises

Exercises

Exercise 1

Given an input volume of width \(n\), height \(n\) and depth \(c\), and a convolutional layer with \(k\) filters of size \(f \times f \times c\), stride \(s\) and padding \(p\).

1. Give the width, the height and the depth of the output volume.
2. Give the number of parameters of the layer.
3. Give the number of multiplications needed for one forward pass.

Exercise 2

Here is a 5 by 5 image with padding 1 already applied, and two 3 by 3 filters. Compute the output of a convolutional layer with stride 1 and a relu non-linearity, by hand.

Use the image with 1 on the diagonal and 0 elsewhere, and the filters

\[ F_1 = \begin{pmatrix} 1 & 0 & -1 \\ 1 & 0 & -1 \\ 1 & 0 & -1\end{pmatrix}, \qquad F_2 = \begin{pmatrix} 1 & 1 & 1 \\ 0 & 0 & 0 \\ -1 & -1 & -1\end{pmatrix}. \]

What does each filter respond to.

Exercise 3

Explain in two sentences each.

1. Why does a convolutional layer have far fewer parameters than a dense layer on the same input.
2. Why does max pooling make the representation slightly shift invariant.
3. Why can a recurrent network handle sequences of different lengths while a convolutional network cannot.

Exercise 4

Train a convolutional network on MNIST.

1. Build the network from the example page and train it for five epochs.
2. Report the test accuracy and compare to the fully connected network from week 7.
3. Plot the filters of the first layer. Can you see what they respond to.
4. Remove the pooling layers and train again. What happens to the number of parameters and to the accuracy.