* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download No Slide Title - Computer Science Home
Mirror neuron wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
State-dependent memory wikipedia , lookup
Molecular neuroscience wikipedia , lookup
Neuroanatomy wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Artificial intelligence wikipedia , lookup
Single-unit recording wikipedia , lookup
Cognitive neuroscience wikipedia , lookup
Neurocomputational speech processing wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Neuroethology wikipedia , lookup
Neuroeconomics wikipedia , lookup
Neuroesthetics wikipedia , lookup
Cortical cooling wikipedia , lookup
Time perception wikipedia , lookup
Neural oscillation wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Optogenetics wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Neural modeling fields wikipedia , lookup
Central pattern generator wikipedia , lookup
Neural coding wikipedia , lookup
Psychophysics wikipedia , lookup
Sparse distributed memory wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Synaptic gating wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Biological neuron model wikipedia , lookup
Metastability in the brain wikipedia , lookup
Neural binding wikipedia , lookup
Artificial neural network wikipedia , lookup
Convolutional neural network wikipedia , lookup
Nervous system network models wikipedia , lookup
Neural engineering wikipedia , lookup
Development of the nervous system wikipedia , lookup
Introduction to Neural Net CS480 Computer Science Seminar Fall 2002 Characteristics not present in von Neumann or modern parallel computers Typical tasks a neural net is capable of performing • • • • • • • Pattern classification Clustering/categorization Function approximation prediction/forecasting Optimization Content-addressable memory control Kohonen’s Self-Organizing Map Neural network definition • A machine that is designed to model the way in which the brain performs a particular task or function of interest. • It is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles brain in two respects: 1. Knowledge is acquired through learning process, 2. Interneuron connection strengths known as synaptic weights are used to stored the knowledge. Some interesting facts • • • • Silicon gate speed: 10-9 sec Neural event: 10-3 sec # gates on Pentium III: < 107 # neurons in human brain: 1010 with 60x1012 synapses or connections • Power consumption: for human brain---10-16 joules per operations per second; for today's best computer---10-6 joules per operations per second Some interesting facts (cont’d) • The brain is a highly complex, nonlinear, and parallel processor. • Brain is superior in performing pattern recognition, perception, and motor control), e.g., it takes a brain 100-200 msec to recognize a familiar face embedded in an unfamiliar scene (will take days for the computer to do the similar tasks) Some interesting facts (cont’d) • Brain has the ability to build up its own rules through “experience”over the years with most dramatic development in the first two years from birth (about 106 synapses formed per second) Neural net architecture • Feed-forward neural net – Single layer – Multi-layer • Recurrent neural net – There are closed loops (feedback) paths. Training the neural net • Training: setting the weights – Supervised – Unsupervised Supervised learning • Supervised training: presenting a sequence of training vectors or patterns, each is associated with an associated target output vector. The weight is then adjusted according to a learning algorithm. • Examples – Pattern classification: the output is either 1 or –1 (belong or not belong to the category) – Patten association: to associate a set of input vectors with a corresponding set of output vectors (the neural net is called associative memory). After training, the the neural net can identify input vectors that are sufficiently close to (but not necessary the same as the input vectors). History of neural net • 1940s: McCulloch-Pitts neurons (perform logic/combinational functions); introduced the idea of “threshold”. • 1949: Hebb learning algorithm---if two neurons are active, the connection strength (weight) should be increased. • 1950s and 1960s: the first golden age of neural net – Perceptrons (Rosenblatt) for pattern recognition – Adaline (adaptive linear neurons). • 1970s: the quiet years (research funding stopped) – Kohonen, Anderson, Grossberg, and Carpenter History of neural net (cont’d) • 1980s: – Backpropagation (of errors) learning algorithm – Hopfield nets: capable of solving constraint satisfaction problems such as traveling salesman problem – Neocognitron: a self-organizing neural for pattern recognition (position or rotation-distorted characters) – Boltzmann machine: non-deterministic neural net (weights or activations are changed on the basis of a probabilistic density function) – Hardware implementation: digital and analog implementations using using VLSI technology Neural net architecture • Feed-forward neural net – Single layer – Multi-layer • Recurrent neural net – There are closed loops (feedback) paths. Training the neural net • Training: setting the weights – Supervised – Unsupervised Supervised learning • Supervised training: presenting a sequence of training vectors or patterns, each is associated with an associated target output vector. The weight is then adjusted according to a learning algorithm. • Examples – Pattern classification: the output is either 1 or –1 (belong or not belong to the category) – Patten association: to associate a set of input vectors with a corresponding set of output vectors (the neural net is called associative memory). After training, the net is the neural net can identify input vectors that are sufficiently close to (but not necessary the same as the input vecotrs) A simple example: McCulloch-Pitts neurons that perform logic operations • Presumption: – Binary activation: neuron either fires or does not fire – Neurons are connected by directed, weighted paths – A connection path is excitatory if the weight on the path is positive; otherwise, its inhibitory – Each neuron has a fixed threshold such that if the net input into a neuron is greater than the threshold, the neuron fires. – The threshold is set so that inhibition is absolute, i.e., any nonzero inhibitory input will prevent the neuron from firing – It takes one time step for a signal to pass over one connection link McCulloch-Pitts Neuron: AND function The threshold on y is 2 McCulloch-Pitts Neuron: OR function McCulloch-Pitts Neuron: OR function The threshold on y is 2 McCulloch-Pitts Neuron: ANDNOT function McCulloch-Pitts Neuron: ANDNOT function The threshold on y is 2 McCulloch-Pitts Neural net: XOR function The threshold on y is 2 McCulloch-Pitts Neural net: XOR function The threshold on y is 2 McCulloch-Pitts Neural net application: modeling the perception of heat and cold • Interesting physiological phenomenon: if cold stimulus is applied to a person’s skin for a very short period of time, the person will perceive heat. If the same stimulus is applied for a longer period, the person will perceive cold. McCulloch-Pitts Neural net: modeling the perception of heat and cold (time step functions are included in the modeling) A cold stimulus applied for one time step A cold stimulus applied for two time steps Another example: character recognition Noisy input Possible topics for further investigation • • • • Current development and applications Architectures Training algorithms Fuzzy logic and fuzzy-neural systems Supervised learning • Supervised training: presenting a sequence of training vectors or patterns, each is associated with an associated target output vector. The weight is then adjusted according to a learning algorithm. • Examples – Pattern classification: the output is either 1 or –1 (belong or not belong to the category) – Patten association: to associate a set of input vectors with a corresponding set of output vectors (the neural net is called associative memory). After training, the net is the neural net can identify input vectors that are sufficiently close to (but not necessary the same as the input vecotrs) McCulloch-Pitts Neuron: AND function McCulloch-Pitts Neuron: AND function The threshold on y is 2 McCulloch-Pitts Neuron: OR function McCulloch-Pitts Neuron: OR function The threshold on y is 2 McCulloch-Pitts Neuron: ANDNOT function McCulloch-Pitts Neuron: ANDNOT function The threshold on y is 2 McCulloch-Pitts Neural net: XOR function The threshold on y is 2 McCulloch-Pitts Neural net: XOR function The threshold on y is 2 McCulloch-Pitts Neural net application: modeling the perception of heat and cold • Interesting physiological phenomenon: if cold stimulus is applied to a person’s skin for a very short period of time, the person will perceive heat. If the same stimulus is applied for a longer period, the person will perceive cold. McCulloch-Pitts Neural net: modeling the perception of heat and cold (time step functions are included in the modeling) A cold stimulus applied for one time step A cold stimulus applied for two time steps Another example: character recognition Noisy input Possible topics for further investigation • • • • Current development and applications Architectures Training algorithms Fuzzy logic and fuzzy-neural systems