Skip to content. | Skip to navigation

Personal tools
You are here: Home Projects soft WTA models

soft Winner-Take-All networks as models of cortical computation

by Giacomo Indiveri last modified Jun 10, 2011 04:47 PM

The cerebral cortex is a remarkable computational system [DM07]. It uses slow, unreliable and inhomogeneous computing elements, and yet it outperforms the most powerful computers in relatively routine functions such as, for example, vision. The disparity between the effectiveness of computation in cortical circuits systems and in a computer, in such types of functions, is primarily attributable to the way the elementary devices are used in the system, and to the kind of computational primitives they implement [M90]. Rather than using Boolean logic, precise digital representations, and clocked operations, cortical circuits carry out robust and reliable computation using hybrid analog/digital components; they emphasize distributed, event-driven, collective, and massively parallel mechanisms, and make extensive use of adaptation, self-organization and learning. Understanding these principles, and how they can lead to artificial systems that exhibit cortical computation properties is one of the major challenges of modern science. Mechanisms based on cortical inhibition, recurrent excitation, and soft Winner-Take-All (WTA) networks have been suggested to underlie these computational principles in the cortex. These mechanisms however are described in the current state-of-the-art using approximate functional models and are implemented using abstract artificial neural networks. It is still not clear how these functions are realized by the actual networks of neocortex [DM04], how these networks are interconnected locally, and how perceptual and cognitive computations can be supported by them.

In this project we will investigate the computational properties of networks of spiking neurons, using conductance-based as well as integrate-and-fire (I&F) neuron models, to evaluate the computational properties of cortical networks with detailed anatomical connectivity patterns [BDM04]. In addition to the role of the precise network structures, we will evaluate the role of synaptic dynamics (short term depression and/or facilitation), as well as long-term synaptic plasticity (spike-driven learning), to determine under which condition the soft-WTA properties listed below emerge.

soft WTA behaviors

Soft Winner-Take-All linear and non-linear behaviors

The results of these initial theoretical/modeling studies will be applied to event-based neuromorphic VLSI systems that implement soft-WTA networks with hybrid analog/digital circuits. These architectures comprise biologically plausible neural networks with generalized I&F neuron circuits [LI09] and populations of excitatory and inhibitory synapses with short-term and long-term plasticity mechanisms [BI07]. They comprise hardwired synaptic connections for implementing both cooperative and competitive aspects of soft-WTA networks [C06], [CID07]. The project involves using the neuromorphic VLSI chips interfaced to a Linux workstation via USB interfaces and Python scripts. Chip bias values, corresponding to neural network parameters, can be read and set in software directly from the workstation. Analog voltage traces measured from the neuron and synapse circuits can be acquired in an automated way using a network-connected oscilloscope. Specific aims of the project include the evaluation the effect of the soft-WTA networks and their different parameters settings on the spike-based learning properties of the neuromorphic circuits, as well as the comparative analysis of properties of these SW/HW architectures with those observed in planned in-vivo experiments on real cortical circuits. Ideally the experience and insight gained on physical realizations of soft-WTA networks will provide suggestions for directing or influencing the planned neuro-physiology experiments, and/or for setting up new experiments with the VLSI system, in order to compare artificial soft-WTA networks, with real cortical ones.

[BI07]C. Bartolozzi and G. Indiveri. Synaptic dynamics in analog VLSI. Neural Computation, 19(10):2581–2603, Oct 2007.
[BDM04]T. Binzegger, R. J. Douglas, and K. Martin. A quantitative map of the circuit of cat primary visual cortex. Journal of Neuroscience, 24(39):8441–53, 2004.
[C06]E. Chicca. A Neuromorphic VLSI System for Modeling Spike–Based Cooperative Competitive Neural Networks. PhD thesis, ETH Zürich, Zürich, Switzerland, April 2006.
[CID07]E. Chicca, G. Indiveri, and R.J. Douglas. Context dependent amplification of both rate and event-correlation in a VLSI network of spiking neurons. In B. Schölkopf, J.C. Platt, and T. Hofmann, editors, Advances in Neural Information Processing Systems 19, pages 257–264, Cambridge, MA, Dec 2007. Neural Information Processing Systems Foundation, MIT Press.
[DM07]R. Douglas and K. Martin. Recurrent neuronal circuits in the neocortex. Current Biology, 17(13):R496–R500, 2007.
[DM04]R.J. Douglas and K.A.C. Martin. Neural circuits of the neocortex. Annual Review of Neuroscience, 27:419–51, 2004.
[LI09]P. Livi and G Indiveri. A current-mode conductance-based silicon neuron for address-event neuromorphic systems. In IEEE International Symposium on Circuits and Systems, ISCAS 2009, pages 2898–2901. IEEE, May 2009.
[M90]C. Mead. Neuromorphic electronic systems. Proceedings of the IEEE, 78(10):1629–36, 1990.