language: en

[To appear in Jordan, MI, Kearns MJ, and Solla, SA Advances in Neural Information Processing Systems 10. MIT Press: Cambridge, MA, 1998.]

Hierarchical Non-linear Factor Analysis and Topographic Maps

Zoubin Ghahramani and Geoffrey E. Hinton
Dept. of Computer Science, University of Toronto
Toronto, Ontario, M5S 3H5, Canada
fzoubin,hintong@cs.toronto.edu

Abstract We first describe a hierarchical, generative model that can be viewed as a non-linear generalisation of factor analysis and can be implemented in a neural network. The model performs perceptual inference in a probabilistically consistent manner by using top-down, bottom-up and lateral connections. These connections can be learned using simple rules that require only locally available information. We then show how to incorporate lateral connections into the generative model. The model extracts a sparse, distributed, hierarchical representation of depth from simplified random-dot stereograms and the localised disparity detectors in the first hidden layer form a topographic map. When presented with image patches from natural scenes, the model develops topo graphically organised local feature detectors.

1 Introduction