Skip to main content

Exercises

Exercises

Conceptual

Exercise 1 Conceptual

Let \(X\) be centred with \(n\) rows and \(p\) columns, and let \(X = U S V^\top\).

  1. Show that \(\Phi = V\) diagonalizes \(\hat\Sigma = \frac{1}{n-1}X^\top X\), and give the eigenvalues in terms of the singular values.
  2. Show that \(Z = US\).
  3. Show that the reconstruction from all components, \(Z\Phi^\top\), gives back \(X\) exactly.

Exercise 2 Conceptual

Answer with a short justification.

  1. The loadings \(\phi_1\) and \(-\phi_1\) describe the same component. Why.
  2. Two data sets differ only in the units of one column. Do they have the same principal components.
  3. We add a constant to every entry of one column. Do the principal components change.
  4. Can the proportion of variance explained by the first component be smaller than \(1/p\).

Exercise 3 Conceptual

On principal component regression.

  1. Explain why PCR can help when the predictors are strongly correlated.
  2. Explain why PCR can fail badly even when it explains 99 percent of the variance in \(X\).
  3. What is the relationship between PCR and ridge regression, in one sentence.

Exercise 4 Conceptual

A colleague shows you a t-SNE plot and says three things. For each one, say whether it is justified.

  1. These two clusters are far apart, so the two cell types are very different.
  2. This cluster is larger, so there are more cells of that type.
  3. These points are next to each other, so they are similar.

Applied

Exercise 5 Applied

PCA on the wine data.

  1. Load it and standardize the 13 measurements.
  2. Make a scree plot and decide how many components to keep.
  3. Make a biplot of the first two components. Which variables are correlated.
  4. Colour the points by the true class. Do the classes separate.
  5. Repeat everything without standardizing. What changes, and why.

Exercise 6 Applied

Compression and denoising on the digits data.

  1. Reconstruct the images from 2, 5, 10 and 30 components and plot them.
  2. Plot the reconstruction error against the number of components.
  3. Add normal noise with standard deviation 4 and repeat. At which number of components is the reconstruction closest to the clean image.
  4. Explain why that number is smaller than the number needed without noise.

Exercise 7 Applied · optional

Compare PCA and t-SNE on MNIST.

  1. Take 5000 images. Plot the first two principal components, coloured by digit.
  2. Run t-SNE on the same 5000 images and plot the result.
  3. Run t-SNE three times with different seeds. How much does the picture change.
  4. Run PCA to 50 components first and then t-SNE on those. Compare the result and the running time.