Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Neural Networks Universiteit Leiden Neural Networks • Book: Introduction to the theory of Neural Computation by Hertz, Krogh, Palmer • Website: http://www.liacs.nl/~mvwezel/nn2010/ • Additional Material: Powerpoint Sheets, Journal Articles, practical exercises. Neural Networks • Other recommended books: Haykin, Bishop • Excellent book on statistical learning: The elements of Statistical Learning. Hastie, Tibshirani, Friedman. Downloadable for free !!!!! here. (Book is more advanced and has wider scope.) The Von Neumann architecture The Hungarian-born mathematician, John von Neumann (1903-1957) The biological architecture Biological computers Five distinguishing properties: • Highly parallel • Robust and fault tolerant • Adaptive • Deals with fuzzy, noisy information • Small, compact Graceful Degradation performance damage Neurons Brain consists of 100000000000 (1011) neurons Neural activity out in Artificial Neuron (Called McCulloch-Pitts neuron if 0/1 output.) Input-output function 1 • nonlinear function: f(x) = 1 + e -x/a a0 f(e) a e Artificial Connections (Synapses) • wAB – The weight of the connection from neuron A to neuron B A wAB B Supervised Networks Example Real Neurons • Nonlinear Summation • Sequences of pulses • No fixed time-delay History • • • • • • • 1943: McCulloch and Pitts: artificial neuron 1960: Rosenblatt: perceptrons 1969: Minsky and Papert: XOR problem 197?: Associative content-addressable memory 198?: Hopfield Networks, Boltzmann Networks 1985: Backpropagation learning rule 2000+ : Spiking networks, Support Vector Machines Hopfield Minsky McCulloch Boltzmann Pitts Papert Hebb Issues • Neurocomputing vs. Neuroscience • Types of Learning: – Supervised – Unsupervised – Reinforcement Research Questions • • • • Design: what is best architecture? Learning: find good algorithms Analysis: what is the power of networks? Implementation: how should the network be implemented?