A Guide to MNIST

Authored by Dibya Ghosh and Alvin Wan

Introduction

This guide is written for coders just beginning with MNIST; MNIST is a dataset of handwritten digits published in the 1990s, MNIST is perhaps one of the most iconic exercises for beginning machine learning - a milestone in using computers to structurally analyse images. This guide will serve as an exploration of the MNIST dataset, looking at several techniques and their effects on training.

Preprocessing Techniques

Deskewing

Deskewing images involves offsetting and skewing images so their center of masses coincide with the center of the image and such that the x and y covariance of the pixel intensities is 0. By reversing the proposed affine transformation, we essentially "standardize" the images, making it much easier for the ML algorithm to distinguish between images.

Check out the Jupyter Notebook here

Optimizing Hyperparameters

Line/Grid Search

Bayesian Optimization

We'll comment on this later.

Adverserial Training

Perturbing

Perturbation is the technique of generating "jittered" images from our original training dataset. In particular, we found that adding random noise, random rotations, skews, and elastic distortions help generalize our dataset against adversarial images.

This technique is a specialized form of training for "adversarial images", when the testing metric is designed to challenge and perform the worst on overfit and nongeneralized models.