Unsupervised information theory-based training algorithms for multilayer neural networks

This paper describes a novel learning algorithm for multilayer neural networks. The trained neural networks are used as vector quantizer (VQ) in a hidden Markov model (HMM) based speech recognition system. The new approach offers the following innovations: (1) It represents an unsupervised learning algorithm for multilayer neural networks, i.e. the neurons of the output layer do not receive any specific target values or supervisor signal during training, but instead the output is learned during training using principles of self-organization. Usually, multilayer neural networks are only trained in supervised mode. (2) Information theory principles are used as learning criteria for the neural networks. (3) The neural networks are not trained using the standard backpropagation algorithm, but using instead a new developed unsupervised learning procedure. The aim of this research work is the development of improved methods for the combination of neural network and information theory algorithms for speech recognition. The use of a neural network as vector quantizer trained with the new algorithm in combination with a Hidden Markov model based speech recognition system results in a 25% error reduction compared to the same HMM system using a standard k-means vector quantizer.