Topographic Product Models Applied to Natural Scene Statistics
Simon Osindero, Max Welling, Geoffrey Hinton
icon We present an energy-based model that uses a product of generalised Student-t distributions to capture the statistical structure in datasets. This model is inspired by and particularly applicable to natural datasets such as images. We begin by providing the mathematical framework, where we discuss complete and overcomplete models, and provide algorithms for training these models from data. Using patches of natural scenes we demonstrate that our approach represents a viable alternative to independent components analysis as an interpretive model of biological visual systems. Although the two approaches are similar in flavor there are also important differences, particularly when the representations are overcomplete. By constraining the interactions within our model we are also able to study the topographic organization of Gabor-like receptive fields that are learned by our model. Finally, we discuss the relation of our new approach to previous work in particular Gaussian Scale Mixture models, and variants of independent components analysis.

Download: pdf

Text Reference

Simon Osindero, Max Welling, and Geoffrey E. Hinton. Topographic product models applied to natural scene statistics. Neural Comput., 18(2):381–414, 2006. doi:

BibTeX Reference

    author = "Osindero, Simon and Welling, Max and Hinton, Geoffrey E.",
    TAG = "ecological_statistics",
    title = "Topographic Product Models Applied to Natural Scene Statistics",
    journal = "Neural Comput.",
    volume = "18",
    number = "2",
    year = "2006",
    issn = "0899-7667",
    pages = "381--414",
    doi = "",
    publisher = "MIT Press",
    address = "Cambridge, MA, USA"