Recognition of Pashto Handwritten Characters Based on Deep Learning

Abstract

Handwritten character recognition is increasingly important in a variety of automation fields, for example, authentication of bank signatures, identification of ZIP codes on letter addresses, and forensic evidence. Despite improved object recognition technologies, Pashto's hand-written character recognition (PHCR) remains largely unsolved due to the presence of many enigmatic hand-written characters, enormously cursive Pashto characters, and lack of research attention. We propose a convolutional neural network (CNN) model for recognition of Pashto hand-written characters for the first time in an unrestricted environment. Firstly, a novel Pashto handwritten character data set, "Poha", for 44 characters is constructed. For preprocessing, deep fusion image processing techniques and noise reduction for text optimization are applied. A CNN model optimized in the number of convolutional layers and their parameters outperformed common deep models in terms of accuracy. Moreover, a set of benchmark popular CNN models applied to Poha is evaluated and compared with the proposed model. The obtained experimental results show that the proposed model is superior to other models with test accuracy of 99.64 percent for PHCR. The results indicate that our model may be a strong candidate for handwritten character recognition and automated PHCR applications.

Keywords: Pashto handwritten character recognition; computer vision; convolutional neural networks; deep features fusion; deep learning.

Figure A1

A confusion matrix for 44 classes of Pashto handwritten characters.

Figure A2

Confusion matrix for the Urdu dataset that contains 40 classes of characters [42].

Figure A3

The confusion matrix for Devanagarithat contains 36 classes of characters [54].

Figure 1

Block diagram of our proposed methodology.

Figure 2

Form distributed to authors and used to compile the dataset: (a) The form completed by native Pashto speakers in Khyber Pukhtunkhwa, KPK Pakistan; (b) this form completed by non-native speakers in South Korea; (c) the individual letter “Khin” written by one user.

Figure 3

Image dataset denoising steps: (a) scanner-induced noise; (b) denoised image; (c) application of Gaussian blur filter.

Figure 4

Graph depicting the accuracy of the proposed model, ResNet18, and Resnet34 on the Poha dataset: (a) training and validation accuracy; (b) training and validation loss; (c) ResNet 18 training and validation accuracy; (d) ResNet 18 training and validation loss; (e) ResNet 34 training and validation accuracy; (f) ResNet 34 training and validation loss.

Figure 5

A sample of input images for the proposed model from the Urdu dataset [42].

Figure 6

Correct classification of Poha dataset characters by the proposed model.

Figure 7

Observed errors in recognition of the Poha dataset by the proposed model.