language: en

Rule Revision with Recurrent Neural Networks

Christian W. Omlin a;b and C.L. Giles a;c
a NEC Research Institute,
4 Independence Way, Princeton, New Jersey
b Computer Science Department, Rensselaer Polytechnic Institute,
Troy, New York
c Institute for Advanced Computer Studies, University of Maryland,
College Park, Maryland

Abstract Recurrent neural networks readily process, recognize and generate temporal sequences. By encoding grammatical strings as temporal sequences, recurrent neural networks can be trained to behave like deterministic sequential finite-state automata. Algorithms have been developed for extracting grammatical rules from trained networks. Using a simple method for inserting prior knowledge (or rules) into recurrent neural networks, we show that recurrent neural networks are able to perform rule revision. Rule revision is performed by comparing the inserted rules with the rules in the finite-state automata extracted from trained networks. The results from training a recurrent neural network to recognize a known non-trivial, randomly generated regular grammar show that not only do the networks preserve correct rules but that they are able to correct through training inserted rules which were initially incorrect. (By incorrect, we mean that the rules were not the ones in the randomly generated grammar.)

Index Terms: Deterministic Finite-State Automata, Genuine and Incorrect Rules, Knowledge Insertion and Extraction, Recurrent Neural Networks, Regular Languages, Rule Revision.

[Published in IEEE Trans. on Knowledge and Data Engineering, vol. 8, no. 1, p. 183, 1996. Copyright IEEE.]