A new unsupervised learning algorithm for multilayer perceptrons based on information theory principles

The author describes a novel learning algorithm for multilayer perceptrons (MLPs). The trained MLPs are used as the vector quantizer (VQ) in a hidden Markov model (HMM) based speech recognition system. This approach represents an unsupervised learning algorithm for multilayer perceptrons, i.e., the neurons of the output layer do not receive any specific target values during training, but instead the output is learned during training using principles of self-organization. Information theory principles are used as learning criteria for the MLP. When using VQ in a HMM-based speech recognition system, multiple features such as cepstral parameters, differential cepstral parameters, and energy can be used as joint input into the same VQ, thus avoiding the use of multiple codebooks. In this case, the principle of 'sensor fusion' can be transferred to the speech recognition area with same intention, namely using neural networks for merging the output of different information sources in order to obtain an improved feature extractor for more robust pattern recognition.