Tips and Tricks for Cracking ML's Favorite Image Dataset

View the Project on GitHub fsix/mnist

A Guide to MNIST

Authored by Dibya Ghosh and Alvin Wan


This guide is written for coders just beginning with MNIST; MNIST is a dataset of handwritten digits published in the 1990s, MNIST is perhaps one of the most iconic exercises for beginning machine learning - a milestone in using computers to structurally analyse images. This guide will serve as an exploration of the MNIST dataset, looking at several techniques and their effects on training.

Preprocessing Techniques


Deskewing images involves offsetting and skewing images so their center of masses coincide with the center of the image and such that the x and y covariance of the pixel intensities is 0. By reversing the proposed affine transformation, we essentially "standardize" the images, making it much easier for the ML algorithm to distinguish between images.

Check out the Jupyter Notebook here

Optimizing Hyperparameters

Line/Grid Search

Bayesian Optimization

We'll comment on this later.

Adverserial Training


Perturbation is the technique of generating "jittered" images from our original training dataset. In particular, we found that adding random noise, random rotations, skews, and elastic distortions help generalize our dataset against adversarial images.

This technique is a specialized form of training for "adversarial images", when the testing metric is designed to challenge and perform the worst on overfit and nongeneralized models.

Check out the Jupyter Notebook here

Guiding Approximations

Momentum and Annealing for SGD