* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
The Connectionist Approach (Neural Networks) • The brain should be modeled with neuronlike structures • Focus on parallel processing (even if you have to fake it with a serial computer) • It can be applied to cognitive processes if the process is heavily rule-based (language) Fuzzy Nature of Neurons • Q: How can perceptions be consistent if neurons aren’t? • A: Perceptions arise from many active neurons and the responses (opinions) are averaged. – No neuron has the final say – No neuron is indispensable Access by Content • Connectionist networks allow access by content, rather than address. – Phone book: Access by address • You can find a number if you know name • You can’t find a name if you know a number • You cant find a number if you know an address – Access by content allows you to work either way. Any bit of info can lead to all other bits Distributed Information Storage • The “benign” qualities of a tumor live in the same set of weights and connections as the malignant qualities Access by Content • Each piece of information is linked to the other pieces and, therefore, can activate them (bring them into awareness). • Access is fault-tolerant, so small errors in input can still lead to a correct solution. A distributed system will look for a “best-fit.” – An erroneous spelling will not lead to a correct phone number What’s so “neural” about it? 1. The basic computational operation in the brain is one neuron passing a information related to the sum of it’s input to other neurons What’s so “neural” about it? 2. Learning changes the strength of the connections between neurons and thus the influence that one has on another What’s so “neural” about it? 3. Cognitive processes (Thinking: memory, speech, problem solving etc.) involve the basic computations being performed in parallel by large numbers of neurons Not all Cognitive Models are Neurally Based • This memory model just explains the hypothetical links between different types of memory. • It’s useful in some cases, but makes no attempt to link memory with the physiology of the brain Five Assumptions About the Brain on which Connectionist Models are Based 1. Neurons Integrate Information – Collect information from one set of neurons and pass the combined message on to another set Five Assumptions About the Brain on which Connectionist Models are Based 2. Neurons Pass Information about the level of their input. – The information being sent to the axon is proportional to the activity at the cell body Five Assumptions About the Brain on which Connectionist Models are Based 3. Brain structure is layered Hidden Units Five Assumptions About the Brain on which Connectionist Models are Based 4. 5. The influence of one neuron on another depends on the strength of the connection between them. Learning is achieved by changing the strengths of connections between neurons. History • McCulloch and Pitts - Logical operations with neuron-like components (1943) Hot Sensor Cold Sensor “I feel Hot” “I feel Cold” • A static model - Like your calculator The Connectionist Approach (Neural Networks) • Donald Hebb (1949) and synaptic change – We knew synapses had to change – Hebb showed us how (in some cases) Rosenblatt’s Perceptron (1958) • A simple neural network inspired by the human retina Connectionist Terms • Although both neurons A and B have excitatory synapses with C, Activity in cell A will have a bigger effect on C than the same amount of activity in cell B. A C Symbols and Equations • Activity (ai) The activity that is passed on (through the weights) to the other units – If the Netinput to a unit does not exceed the threshold, activity is zero ai Symbols and Equations • Weight (wij) Unit j influences unit I by passing information about its activity level. aj wij ai Symbols and Equations • Input: The input from unit j to unit i is the product of the activity of unit j (aj) and the weight of the connection between them (wij). • inputij=ajwij aj wij ai Symbols and Equations • NetInput: The total input to unit i (netinput) is found by summing the inputs from all the units which send input to it. • netinputi= Σ ajwij aj wij ai Symbols and Equations • Activation Function: How is netinputi converted to activityi? Linear Sigmoid netinputi netinputi netinputi Activity (ai) Binary Rosenblatt’s Perceptron (1958) • Used Hebb’s principles and a form of teaching to create a learning network Rosenblatt’s Perceptron (1958) Stimulus Activity in Inputs The Rules of Learning No is no. No is always No. If they say, “No” it means a thousand times “No!” Rosenblatt’s Perceptron (1958) Stimulus Activity in Inputs Mistakes will result in changes in weights Perceptron Learning Rule • “Weight gain and loss” • Δ wij = [ai(desired)-ai(obtained)]ajε – No change if the outcome matches the desired result – More activity leads to greater change – Change can be accelerated ε aj wij ai Rosenblatt’s Perceptron (1958) • A one-layer neural network that learned the alphabet with simple “Yes” or “No” feedback. PDP as a Model of Cognition • • • • Immune to minor damage Work when input is noisy or incomplete Knowledge is distributed Computations take place in parallel Conclusions are based on consensus! Human vs. Network Network Performance Human Response Time Exception 570 HAVE NAVE GAVE 6 4 2 530 High Exception Regular Regular 550 8 SUAVE Mean Squared Error Mean Naming Latency 590 Low High Low PDP vs. Traditional Computer • Processing is parallel, not serial – In a serial system, one broken link stops everything • Information is distributed, not local Connectionist Models are far from Perfect • We don’t completely understand how the brain is wired – Billions of neurons, trillions of synapses • We don’t completely understand how and where all neurotransmitters are used (Inhibition/Excitation). • Connectionist models are simplified. However, they are computational and easily tested. The output is quantitative and can be directly compared to the quantitative results from behavioral studies – if one doesn’t work, it should be modified or discarded.