language: en

[In M. C. Mozer. M. I. Jordan and T. Petsche (editors), Advances in Neural Information Processing Systems 9, MIT Press, Cambridge MA, 1997]

Continuous sigmoidal belief networks trained using slice sampling

Brendan J. Frey
Department of Computer Science, University of Toronto
6 King's College Road, Toronto, Canada M5S 1A4

Abstract Real-valued random hidden variables can be useful for modelling latent structure that explains correlations among observed variables. I propose a simple unit that adds zero-mean Gaussian noise to its input before passing it through a sigmoidal squashing function. Such units can produce a variety of useful behaviors, ranging from deterministic to binary stochastic to continuous stochastic. I show how "slice sampling" (Neal 1996) can be used for inference and learning in top-down networks of these units and demonstrate learning on two simple problems.

1 Introduction