Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Perception and Neurorobotics Multimodal sensory integration in human neocortex Baran Çürüklü, Senior lecturer, <[email protected]> Computational Perception Laboratory Intelligent Sensor Systems, Innovation and Product Realization School of Innovation, Design and Engineering Mälardalen University Contents • Introduction • Organization of the nervous system • Evaluation of models using robots What does the neocortex? It is involved in higher functions such as sensory perception, generation of motor commands, spatial reasoning, conscious thought, and in humans, language. Multimodal sensory integration • Senses in the multimodal sensory integration • Vision, hearing, taste, smell, somatic, … • Vision: Shape, color, movement • Just think about multimodal input • Keyboard and mouse or other modalities such as, speech, pen, touch, manual gestures, gaze and head and body movements Computational neuroscience Computational neuroscience is an interdisciplinary field which draws on neuroscience, computer science and applied mathematics. It most often uses mathematical and computational techniques such as computer simulations and mathematical methods to understand the function of the nervous system. The brain White and gray matter Gray matter Neurons communicate by spiking axon from a presynaptic neuron dendrites axon soma Classical Results by E.D. Adrian, 1926 • Spike coding is universal: Sensory neurons produce stereotyped spikes just like the motor neurons. • Spike-rate coding: Larger static stimulus generates higher spike rate. • Adaptation: Spike rate decline over the time for a static stimulus. Somatic Sensory System Receptor density on the skin. Receptors: Touch, vibration, pressure, deep pressure, pain, temperature, muscle length & tension The auditory system Early stages of vision Representation of the distorted retinotopic map within the LGN (left hemisphere) Organization of the neocortex • Modular – Hypercolums (microcircuits) – Minicolumns – Neurons • Laminar (6 layers) – Long-range connections through the white matter – Horizontal longrange connections – Local connections Modular organization of the neocortex Primary visual cortex Orientation minicolumns Hypercolumn Modular organization of the neocortex Primary visual cortex (V1, area 17) Layer 2/3 patchy long-range horizontal (L-R) connections within tree shrew primary visual cortex Orientation maps, ocular dominance and spatial frequency in V1 Orientation maps and spatial frequency Orientation maps and ocular dominance Modular organization of the neocortex Higher visual areas • Laminar organization Primary visual cortex Differences between layers: • Pyramidal cell sizes – increase in size along layer 2 → layer 3 – large pyramidal cells in layer 5B • Density of the inhibitory cells – 20% on average – 50% of the inhibitory cells are within layer 4 ↔ layer 3lower Laminar organization Primary visual cortex Layout of the horizontal connections • Layer 2/3 excitatory connections, incl. L-R – – – – Patchy (biased towards the iso-orientation domains) Patches are found up to ~2.5 mm (in cat) Patches vary in size and shape Elongated along the orientation axis • Layer 4 excitatory connections – Up to ~1.5 mm (in cat) – Equally distributed and isotropic • Inhibitory connections – Large basket cells ~1 mm. Small basket, chandelier and bipolar cells <100 µm. – Equally distributed and isotropic Hierarchical organization of the areas Communication through burst and spike synchronization • How: Burst or near-zero phase lag spike • Where: – Early stages of vision • Retina LGN Layer 4 – Within a patch, such as the visual cortex • Layer 4 layer 2/3 • Within layer 2/3 and between the hemispheres (mediated by the reciprocal L-R connections) – Between two (or more) locations in neocortex • Each location might represent a specific modality/feature. • Why: – Signal enhancement, e.g. in low-contrast conditions – Figure-ground separation L-R connections account for spike and burst synchronization • Robust burst synchronization • Zero-lag spike synchronization despite the delays – Tighter between neurons within a minicolumn Population activity Cross-correlation between minicolumns 1 and 6 -0.2 0.2 A generic network model Long-range connections A theoretical framework for multimodal information processing… … for robots! Phase 1: 2006 2007 Phase 2: 2008 2009 Current status • Multimodality – Vision, speech, gestures System overview The video http://www.youtube.com/watch?v=la2GOTef-1k