* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download PSY105 Neural Networks 2/5
Development of the nervous system wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Neurophilosophy wikipedia , lookup
Perceptual learning wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Psychoneuroimmunology wikipedia , lookup
Caridoid escape reaction wikipedia , lookup
Biology and consumer behaviour wikipedia , lookup
Emotion and memory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Convolutional neural network wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Artificial neural network wikipedia , lookup
Synaptic gating wikipedia , lookup
Neural modeling fields wikipedia , lookup
Neuroethology wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Visual extinction wikipedia , lookup
Evoked potential wikipedia , lookup
Metastability in the brain wikipedia , lookup
Response priming wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Negative priming wikipedia , lookup
Time perception wikipedia , lookup
Perception of infrasound wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Recurrent neural network wikipedia , lookup
C1 and P1 (neuroscience) wikipedia , lookup
Conditioned place preference wikipedia , lookup
Biological neuron model wikipedia , lookup
Neural coding wikipedia , lookup
Nervous system network models wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Operant conditioning wikipedia , lookup
Classical conditioning wikipedia , lookup
PSY105 Neural Networks 5/5 5. “Function – Computation Mechanism” Lecture 1 recap • We can describe patterns at one level of description that emerge due to rules followed at a lower level of description. • Neural network modellers hope that we can understand behaviour by creating models of networks of artificial neurons. Lecture 2 recap • Simple model neurons – Transmit a signal of (or between) 0 and 1 – Receive information from other neurons – Weight this information • Can be used to perform any computation Lecture 3 recap • Classical conditioning is a simple form of learning which can be understood as an increase in the weight (‘associative strength’) between two stimuli (one of which is associated with an ‘unconditioned response’) Lecture 4 recap • A Hebb Rule for weight change between two neurons is: – Δ weight = activity 1 x activity 2 x learning rate constant • In order to use this rule to associate two stimuli which are separated in time we need neuron activity associated with stimuli to persist in time. – This can be implemented as an ‘eligibility trace’ • I present you with a robot which uses a simple neural network to acquire classically conditioned responses. It can, for example, learn to associate a warning stimulus with an upcoming wall and hence turn around before it reaches the wall. Describe an experiment which you would do to test the details of how the robot learns. Say what you would do, and what aspect(s) of the robot's learning the results would inform you of, and why. The problem of continuous time Stimulus 1 Stimulus 2 Traces Stimulus 1 Stimulus 2 Consequences of this implementation • • • • • • Intensity of CS stimulus Duration of CS stimulus Intensity of UCS stimulus Duration of UCS stimulus Separation in time of CS and UCS The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model) Association We have designed an information processing system that learns associations Sutton, R.S., Barto, A.G. (1990). Timederivative models of pavlovian reinforcement. In Learning and Computational Neuroscience: Foundations of Adaptive Networks, M. Gabriel and J. Moore, Eds., pp. 497-537. MIT Press. Trials Stop pairing Association Without continued pairing Trials Stop pairing Association Without continued pairing -> extinction Trials Analysis of information processing systems • Function (‘computational level’) • Computation (‘algorithmic level’) • Mechanism (‘implementational level’) Marr, David (1982) Vision. San Francisco: W.H. Freeman Marrian analysis of classical condtioning • Function: learn to predict events based on past experience • Computation: Stimuli evoke ‘eligibility traces’. Hebb Rule governs changes in weights [+ other additional assumptions which are always needed when you try and make a computational recipe] • Mechanism: At least one response neuron, one unconditioned stimulus neuron and one neuron for each conditioned stimulus Kim, J. J., & Thompson, R. F. (1997). Cerebellar circuits and synaptic mechanisms involved in classical eyeblink conditioning. Trends in Neurosciences, 20(4), 177-181. Marrian analysis: a simple example • Function • Computation • Mechanism Theory < - > Experiments Synthesis < - > Analysis Our classical conditioning networks S-S link Stimuli CS2 CS1 UCS Responses Internal representation of the conditioned stimulus The lab : using Marrian analysis to make predictions Function • What is the purpose of learning for an animal? – Does our model behave in a sensible (‘adaptive’) way when it follows our rule? – Is the rule sufficient to explain animal learning? • Test: think of a way you would want the model/robot to behave, test if it does Computation • • • • • • Intensity of CS stimulus Duration of CS stimulus Intensity of UCS stimulus Duration of UCS stimulus Separation in time of CS and UCS The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model) • The learning rate • The rate of decay of the trace • The frequency of pairing Mechanism S-S link Stimuli CS2 CS1 UCS Responses • What is your prediction? What will you do to the rule or the environment? • How will you know if it has been confirmed or falsified? π ψ Ω