Over the past decade, significant progress has been made in the understanding of cortical organization within the context of extracting information from images and comprehending their content. In those areas of striate cortex in which retinotopic relationships and spatial frequency information is preserved, the organization of receptive fields and relationships of structure and function are beginning to be revealed. At the other end of the chain of visual cortical information processing, inferior temporal (IT) cortex cells have been shown to respond selectively to highly complex stimuli, including specific faces. The cortical information processing paradigm presented here suggests how relatively local features derived from the visual scene by the striate cortex could be transformed into composite features of great specificity that are identifiable in IT cortex. Biophysical properties of dendrites and spines, in particular mechanisms of synaptic modifiability and memory storage in conjunction with often ignored but critically important statistical properties of sensory inputs, facilitates a self-organization of neuronal connectivity for information processing that allow for increasing specificity in successive processing layers.