language: en

Multilayer perceptrons may learn simple rules quickly

R. Urbanczik
Institut fur theoretische Physik Universitat Wurzburg
Am Hubland D-97074 Wurzburg Germany
November 27, 1997

Abstract Zero temperature Gibbs learning is considered for a connected committee machine with K hidden units. For large K, the scale of the learning curve strongly depends on the target rule. When learning a perceptron, the sample size P needed for optimal generalization scales so that N t P t KN, where N is the dimension of the input. This even holds for a noisy perceptron rule if a new input is classified by the majority vote of all students in the version space. When learning a committee machine with M hidden units, 1 t M t K, optimal generalization requires p