* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Artificial Intelligence Connectionist Models Inspired by the brain
Cognitive epidemiology wikipedia , lookup
Intelligence quotient wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neural oscillation wikipedia , lookup
Human intelligence wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Neural coding wikipedia , lookup
Evolution of human intelligence wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Neural modeling fields wikipedia , lookup
Optogenetics wikipedia , lookup
Environment and intelligence wikipedia , lookup
Neuroscience and intelligence wikipedia , lookup
Central pattern generator wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Biological neuron model wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Intelligence wikipedia , lookup
Philosophy of artificial intelligence wikipedia , lookup
Synaptic gating wikipedia , lookup
Neural engineering wikipedia , lookup
Development of the nervous system wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
Intelligence explosion wikipedia , lookup
Metastability in the brain wikipedia , lookup
Catastrophic interference wikipedia , lookup
Artificial neural network wikipedia , lookup
Artificial intelligence wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Convolutional neural network wikipedia , lookup
Nervous system network models wikipedia , lookup
Connectionist Models In contrast to symbolic models ● Based on the brain paradigm or brain metaphor ● Have enjoyed much success in recent years ● Synonyms ● Artificial Intelligence Introduction to Neural Networks – – – – CS 4633/6633 Artificial Intelligence CS 4633/6633 Artificial Intelligence Inspired by the brain ● ● ● ● ● ● ● ● ● neural computing neural networks connectionism parallel distributed processing Large number of simple processing units (neurons) Highly connected Highly parallel, distributed control Neurons are slow devices compared to digital computers Can perform complex tasks in a short period of time Graceful degradation when neurons fail Handles fuzzy situations very well. Information accessed on the basis of content Learns from experience Brief History ● ● ● ● ● ● ● ● CS 4633/6633 Artificial Intelligence 1943: Neurophysiologist Warren McCullock and logician Walter Pitts propose MuCulloch-Pitts model of a neuron 1949: Neurophysiologist Donald Hebb proposes model of dynamic memory and simple learning rule 1960’s: Frank Rossenblatt advocates perceptron model and proves convergence of learning algorithm for it 1969: Marvin Minsky and Seymour Papert prove that the perceptron has severe computational limits. Many abandon field. 1982: Nobel prize winner John Hopfield uses concepts from physics to analyze neural networks and applies them to new problems 1985: Backpropagation training rule for muli-layer neural networks is rediscovered 1987: First IEEE conference on neural networks. Over 2000 attend. The revival is underway! CS 4633/6633 Artificial Intelligence A Simple “Neuron” Receives information from other neurons, performs simple processing, and sends result to one or more neurons a1 W1 a2 W2 a3 W3 . . an Activations ● Input = ΣW jaj Output = g(input) Σ g Input Activation Function Function Wn CS 4633/6633 Artificial Intelligence ● The activation of a neuron is a numerical value that represents its “state of excitement” – May be discrete {-1, 0, 1} or {0 , 1} – May be continuous [0,1] Determined by a cell’s activation function 1, if x ≥ t stept ( x) = î 0, if x < t sigmoid ( x) = 1 1 + e− x CS 4633/6633 Artificial Intelligence Threshold (bias) The minimum total weighted input necessary to cause a neuron to fire. The t in step function is a threshold. Often implemented as additional weight, as follows. 1 I1 w1,j w0,j n in j = ∑ w i , j ∗ input i w2,j I2 i=0 uj I3 Neural Network Neurons are organized into networks. ● Connections are one or two-way communication links between neurons ● Weights are the strength of connections. A weight wij is a real number than indicates the influence neuron ui has on neuron uj ● w3,j CS 4633/6633 Artificial Intelligence CS 4633/6633 Artificial Intelligence Neural Network: Simple Example I1 Network Structures Feed-forward networks – no cycles – compute a function from input to output – two kinds: ● H1 O1 I2 » perceptron » multi-layer H2 Input Layer Recurrent networks – cycles are allowed--activation is fed back to the unit that caused it – more complex and difficult to train ● I3 Hidden Layer Output Layer CS 4633/6633 Artificial Intelligence CS 4633/6633 Artificial Intelligence Representational Power A feed-forward network computes a function from the inputs to the outputs ● A feed-forward network with one hidden layer can approximate any continuous function of the inputs ● A feed-forward network with two hidden layers can approximate any function at all ● However, the hidden layers may need lots of neurons Learning ● CS 4633/6633 Artificial Intelligence A neural net can be trained to represent any function by adjusting its weights ● A feed-forward network is usually trained by supervised learning ● We will discuss learning algorithms for neural networks in the next two classes ● CS 4633/6633 Artificial Intelligence