language: en

[In: Proceedings of EURASIP Workshop, February 15-17, 1990, Portugal]

INVERSION IN TIME

Sebastian Thrun Alexander Linden
Gesellschaft fur Mathematik und Datenverarbeitung mbH
D-5205 St. Augustin, West Germany

Abstract Inversion of multilayer synchronous networks is a method which tries to answer questions like "What kind of input will give a desired output?" or "Is it possible to get a desired output (under special input/output constraints)?". We will describe two methods of inverting a connectionist network. Firstly, we extend inversion via backpropagation (Linden/Kindermann [4], Williams [11]) to recurrent (El-man [1], Jordan [3], Mozer [5], Williams/Zipser [10]), time-delayed (Waibel at al. [9]) and discrete versions of continuous networks (Pineda [7], Pearlmutter [6]). The result of inversion is an input vector. The corresponding output vector is equal to the target vector except a small remainder. The knowledge of those attractors may help to understand the function and the generalization qualities of connectionist systems of this kind. Secondly, we introduce a new inversion method for proving the non-existence of an input combination under special constraints, e.g. in a subspace of the input space. This method works by iterative exclusion of invalid activation values. It might be a helpful way to judge the properties of a trained network. We conclude with simulation results of three different tasks: XOR, morse signal decoding and handwritten digit recognition.

Keywords: connectionist systems, backpropagation, inversion, recurrent neural networks, digit recognition

1 Introduction