Digit Recognition System Digit recognition system is the working of a machine to train itself or recognizing the digits from different sources like emails, bank cheque, papers, images, etc. and in different real-world scenarios for online handwriting recognition on computer tablets or system, recognize number plates of vehicles, processing bank cheque amounts, numeric entries in forms filled up by hand (say — tax forms).
Samples provided from MNIST (Modified National Institute of Standards and Technology) dataset includes handwritten digits total of 70,000 images consisting of 60,000 examples in training set and 10,000 examples in testing set, both with labeled images from 10 digits (0 to 9). This is a small segment form the wide set from NIST where size was normalized to fit a 2020 pixel box and not altering the aspect ratio. Handwritten digits are images in the form of 2828 gray scale intensities of images representing an image along with the first column to be a label (0 to 9) for every image. The same has opted for the case of the testing set as 10,000 images with a label of 0 to 9. Yann Lecun, Corinna Cortes, and Christopher Burges developed this MNIST dataset for evaluating and improving machine learning models on the handwritten digit classification problem. The MNIST dataset was developed from the special dataset from NIST with special database 3 (United States Census Bureau employees) and special database 1 (high school students) which consist with the binary images of handwritten digits. Earlier SD-3 (special database -3) was considered as training and SD-1 (special database -1) as testing set with easier recognizing level of SD-3. Therefore to keep it challenging, disjoint and fair among different learning classifiers, NIST dataset was mixed up. Division of the MNIST took place by 30,000 samples from SD-3 and 30,000 samples from SD- 1 with 250 writers approx. and 5,000 samples from SD-3 and remaining 5,000 samples from SD-1 to form a different test set. Images of digits were taken from various scanned digits, normalized in size and justify as centered. This makes it an excellent dataset for evaluating models and allowing the machine learning aspirant to focus on deep learning and machine learning with very little data cleaning. Talking about the newer or more modified version which is similar to the standard MNIST, an EMNIST or Extended MNIST have been emerged out in the year 2017 with the samples of 2, 40,000 images in training set along with increment to 40,000 images in the testing set consisting of handwritten digits.
The MNIST dataset is provided in the format of IDX. This IDX file format is a simple format which comes handy when operating with vectors and high dimensional matrices of different numerical types. Starting with the magic number in the description column available in the file format. We can define a magic number as an integral value (say MSB first), where the first 2 bytes are always seen to be zero. This gives us the following information: The 0000 (2 bytes) informing the beginning of the file. 08 tells us that third byte is of unsigned byte type. The fourth byte, 03 tells us that the matrix has three dimensions and 01 informing with just one dimension. The third byte represents whether the data is an integer, float, short, long or unsigned type. The fourth byte tells the dimension of the vector or matrix i.e. the number of rows and columns. If it is equal to 1, then it’s a vector else it is a matrix. The number of items variable is also read as MSB first. Changing from IDX to simpler CSV As our dataset is available in IDX format [Yann LeCun, the MNIST database of Handwritten Digits], we can change our dataset into CSV formats by algorithm [Joseph Chet Redmon, Algorithm to change idx into csv] and we can achieve MNIST dataset in CSV format. To better understand the CSV: The first column or value is the “label”, that is, the actual true digit that the handwriting is supposed to classify, such as a “7” or “9”. It is the correct solution to which the classifier is aspiring to classify. The remaining values or all comma separated values, are the pixel values intensities of the handwritten digit, varying from 0 to 255. The size of an image is 28 by 28, so there are 784 (28*28) values for the label. [Joseph Chet Redmon, Algorithm to change idx into csv].