language: en

Comparison of Statistical and Neural Classifiers and Their Applications to Optical Character Recognition and Speech Classification

Ethem Alpaydn, Fikret Gurgen
Department of Computer Engineering Bogazi~ci University
TR-80815
_ Istanbul Turkey
falpaydin,gurgeng@boun.edu.tr

[Neural Network Systems Techniques and Applications (in print) C. T. Leondes (Ed.), c flACADEMIC Press]

October 24, 1996

Abstract We give a review of basic statistical and neural techniques for classification. Statistical techniques are based on the idea of estimating class-conditional likelihoods and using Bayes rule to convert these to posterior class probabilities whereas neural techniques estimate directly the posteriors. Statistical techniques include (i) Parametric (Gaussian) Bayes classifiers, (ii) Non-parametric kernel-based density estimators like k-nearest neighbor and Parzen windows, and (iii) mixtures of (Gaussian) densities (a special case of which is the Learning Vector Quantization). As neural classifiers, we include simple perceptrons and multilayer perceptrons with sigmoid and Gaussian hidden units. The neural and statistical techniques are quite similar in many respects and many approaches have been discovered independently twice, once in 1960s by statisticians and once in 1980s by the neural network researchers. One of the aims of this article is to make this link more apparent. We also discuss two, most popular, pattern recognition applications: Optical character recognition and speech recognition. Though they seem different, in many respects, the two applications are similar and in the past, almost the same techniques have been applied for their implementation. We implement the well known statistical and neural classification techniques for two datasets of these applications and compare them in terms of generalization accuracy, memory requirement and learning time. We especially advise to take into account statistics of the sample even if a neural classifier is to be used. The similarity between statistical and neural techniques is greater than generally agreed and simple statistical methods like k-NN perform generally quite well and much of the functionality of neural networks like distributed parallel computation can be obtained by such methods without requiring complicated computation and precise error minimization procedures.

Keywords| Statistical pattern recognition, artificial neural networks, optical character recog nition, speech recognition, Bayes decision theory, nonparametric estimation.