Tag Archives: Machine Learning

Hand writing recognition

K-nearest neighbors


I have recently been very interested towards learning machine learning and have picked up a book to learn more about this topic.

The language used is Python, and NumPy is used for scientific calculations. I find this great, as I am pretty familiar with Python, and this means that I do not really have to spend a lot of time learning a new language.


Machine learning problems can be sub-divided into 2 categories:

  1. Supervised Learning – Given a dataset, predict/foretell certain values

    1. Classification: Classify a problem into 1 of a fixed number of classes/types

    2. Regression: Output a value in a given range

  1. Unsupervised Learning – There is no dataset involved here

    1. Clustering: Adjust your data into groups

    2. Density Estimation: Get the probability of your data belonging to a group

 The algorithm used for the OCR code is the k-nearest neighbors algorithm.

The steps involved are:

  1. Get distances of your required value from dataSet members

  2. Sort the distances in ascending order and choose the k smallest distances & corresponding dataset entries

  3. Classify the dataset entries and return the class that contains the majority of the dataset entries.


Given below is a look into an application of the k-Nearest Neighbors classification algorithm – recognition of hand-written digits (0 – 9)

I was able to achieve an error rate of 0.0124 which is pretty good for a 1st machine learning algo.


The script attached does the following:

  1. Hand-written images are stores in arrays of size 32×32. The array has 0s and 1s as contents

  2. Our classifier accepts an input vector of size 1024. The image is converted into this

  3. Images from the training dataset are taken and the dataSet matrix is created and passed

  4. Images from the test data set are used for testing.


The script is uploaded on Bitbucket




Tagged ,