Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Gene expression programming wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Neural modeling fields wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Machine learning wikipedia , lookup
Pattern recognition wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Backpropagation wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Introduction to Neural Networks Freek Stulp Overview Biological Background Artificial Neuron Classes of Neural Networks 1. 2. 3. Perceptrons Multi-Layered Feed-Forward Networks Recurrent Networks Conclusion 2 Biological Background Neuron consists of: Cell body Dendrites Axon Synapses Neural activation : Throught dendrites/axon Synapses have different strengths 3 Artificial Neuron Input links (dendrites) aj Unit (cell body) Output links (axon) Wji ini = ai = SajWji g(ini) ai 4 Class I: Perceptron Ij Wj -1 W0 W1 a1 W2 a2 O in = a= SajWj g(in) a = g(-W0 + W1a1 + W2a2) a { 0, in<0 g(in) = 1, in>0 5 Learning in Perceptrons Perceptrons can learn mappings from inputs I to outputs O by changing weights W Training set D: Inputs: I0, I1 ... In Targets: T0, T1 ...Tn Example: boolean OR D: d I T 0 00 0 1 01 1 2 10 1 3 11 1 Output O of network is not necessary equal to T! 6 Learning in Perceptrons Error often defined as: E(W) = 1/2SdD(td-od)2 Go towards the minimum error! Update rules: wi = wi + Dwi Dwi = -hdE/dwi dE/dwi = d/dwi 1/2SdD(td-od)2 = SdD(td-od)iid i This is called gradient descent 7 Class II: Multi-layer Feed-forward Networks Multiple layers: hidden layer(s) Input Hidden Output Feed-forward: Output links only connected to input links in the next layer Complex non-linear functions can be represented 8 Learning in MLFF Networks For output layer, weight updating similar to perceptrons. Problem: What are the errors in the hidden layer? Backpropagation Algorithm For each hidden layer (from output to input): For each unit in the layer determine how much it contributed to the errors in the previous layer. Adapt the weight according to this contribution This is also gradient descent 9 Class III: Recurrent Networks No restrictions on connections Input Hidden Output Behaviour more difficult to predict/ understand 10 Conclusion Inspiration from biology, though artificial brains are still very far away. Perceptrons too simple for most problems. MLFF Networks good as function approximators. Many of your articles use these networks! Recurrent networks complex but useful too. 11 Literature Artificial Intelligence: A Modern Approach Stuart Russel and Peter Norvig Machine Learning Tom M. Mitchell 12