From sensing to perception systems
The SENSEMAKER project designed electronic architectures that integrate diverse sensory information into a complete perceptual model of the environment. By mimicking the biological principles of sensory receptor and nervous system function the environmental space is represented at a higher cognitive level. Similar to the brain, the SenseMaker system can extract correlated information from sensory representations that are simultaneously refined with the aid of different sensory modalities. Based on a predefined library, the system opts for a minimal set of sensory modalities that are combined to allow a reliable object/environment distinction and identification. With the aid of neuro-mimetic modelling, the perceptual system was extensively explored and finally implemented using programmable mixed analogue/digital Application-Specific Integrated Circuits (ASICs). Given an unpredicted change in the environment or, in the case of a partial impairment of some sensors, the system is capable of self-reconfiguration through formation of supplementary cross-connections. Spiking neurons and adapting synapses were implemented using a mixed-signal neural network ASIC. The spiking neural network imitates neural behaviour which is explained using membrane models. Hence, spikes can be generated under certain threshold membrane voltage values while the conductance-related synapses offer a realistic basis for their reversal potentials. Due to variations in the manufacture process each one of the transistors is different from the rest just like neurons. Aiming to control these fluctuations in order to develop neural microcircuits with a known statistical distribution of their parameters, each electronic neuron features several individually tuneable parameters. Furthermore, a better insight on the details of brain's adaptation to the environment is provided through plasticity. Thereby, associative plasticity rules were borrowed from the biological counterpart using spike-timing dependent plasticity algorithms derived from mammalian neo-cortex studies and used to store long-term memories. In the spiking neural network chip each synapse measures the correlation between pre- and postsynaptic signal. Life-like perception systems mimic the sophisticated interaction of a living system with its environment. Merging sensory information, being vision, hearing and/or touch, with cognition, control and response, these systems can potentially extend the capabilities of machines and augment human senses.
