* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Introduction to AI
Survey
Document related concepts
Neural modeling fields wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Technological singularity wikipedia , lookup
Pattern recognition wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Philosophy of artificial intelligence wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
Convolutional neural network wikipedia , lookup
Intelligence explosion wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Transcript
Neural Networks Multilayer Perceptron (MLP) Oscar Herrera Alcántara [email protected] Outline Neuron Artificial neural networks Activation functions Perceptrons Multilayer perceptrons Backpropagation Generalization Introduction to Artificial Intelligence - APSU Neuron A neuron is a cell in the brain collection, processing, and dissemination of electrical signals 1011 neurons of > 20 types, 1014 synapses, 1ms-10ms cycle time brain’s information processing relies on networks of such neurons Introduction to Artificial Intelligence - APSU Biological Motivation dendrites: nerve fibres carrying electrical signals to the cell cell body: computes a non-linear function of its inputs axon: single long fiber that carries the electrical signal from the cell body to other neurons synapse: the point of contact between the axon of one cell and the dendrite of another, regulating a chemical connection whose strength affects the input to the cell. Introduction to Artificial Intelligence - APSU Artificial neural networks A mathematical model of the neuron is McCulloch-Pitts unit Neural networks consists of nodes (units) connected by directed links 1 x1 b :Bias wi1 x2 m y j (v) (wi , j b) j 1 Neuron i S v j y x3 xm Wim Synaptic Induced local field Activation Inputs Weights Activation potential function Output A bias weight Wi,0 connected to a fixed input xi,0 = +1 Introduction to Artificial Intelligence - APSU Activation functions 1 if 0 j (v) 0 if 0 j (v ) 1 1 e av a) Step function or Threshold function b) Sigmoid function c) Hyperbolic tangent function j (v) a tanh( b j (n)) a, b 0 Introduction to Artificial Intelligence - APSU Perceptron learning Learn by adjusting weights to reduce error on training set Error correction learning rule e( n ) d ( n ) y ( n ) 1 2 E e 2 w ji (n 1) w ji (n) [d (n) y (n)] xi (n) η learning rate parameter Gradient g(n) -x(n)e(n) Perform optimization search by gradient descent Introduction to Artificial Intelligence - APSU Implementing logic functions McCulloch-Pitts unit can implement any Boolean function X1 X2 X1 y X1 y B v w0 1 w1 X 1 w2 X 2 1 if v 0 y 0 otherwise Introduction to Artificial Intelligence - APSU y Expressiveness of perceptrons A perceptron can represent AND, OR, NOT can represent a linear separator (function) in input space: Introduction to Artificial Intelligence - APSU Multilayer Perceptron (MLP): Architecture Bias Input Hidden Layers Layer j j x1 Inputs x2 1 j j j y1 Outputs 1 j x3 Output Layer 1 wij j j j wjk j wkl Introduction to Artificial Intelligence - APSU y2 Solve XOR problem using MLPs A two-layer network with two nodes in the hidden layer The hidden layer maps the points from non linear separable space to linear separable space. The output layer finds a decision line -1 w01 A w11 g1 y1 w13 w21 w12 B w22 g2 y2 w23 g3 w03 -1 w02 -1 j (v) Introduction to Artificial Intelligence - APSU y Back-propagation Algorithm 1. Initialization. Weights are initialized with random values whose mean is zero 2. Presentations of training examples 3. Forward computation 4.-Backward computation for the neuron j of the hidden layer l j (n) j ' (v j (n)) k (n) wkj (n) l l 1 l k for the neuron j of the output layer L j (n) j ' (v j (n))e j l l l 1 w ji (n 1) w ji a[ w ji (n 1)] j (n) yi (n) l l l l 5.- Iteration. Repeat step 2 to 4 until E< desired error a the momentum parameter is ajusted the learning-rate parameter is ajusted Introduction to Artificial Intelligence - APSU L l 1 MLP Training i Left j k Forward Pass • Fix wji(n) • Compute yj(n) x Right y Backward Pass • Calculate j(n) • Update weights wji(n+1) Left i j Introduction to Artificial Intelligence - APSU k Right Generalization Total Data are divided in two parts: Data Training (80%) MLP is trained with Data Training Data Test (20%) MLP is tested with Data Test Generalization MLP is used with inputs which have never been presented in order to predict the outputs Introduction to Artificial Intelligence - APSU