Download PowerPoint-presentatie

Neural Networks
Universiteit Leiden
Neural Networks
• Book: Introduction to the theory of Neural
Computation by Hertz, Krogh, Palmer
• Website:
http://www.liacs.nl/~mvwezel/nn2010/
• Additional Material: Powerpoint Sheets,
Journal Articles, practical exercises.
Neural Networks
• Other recommended books: Haykin, Bishop
• Excellent book on statistical learning: The
elements of Statistical Learning. Hastie,
Tibshirani, Friedman. Downloadable for
free !!!!! here. (Book is more advanced and
has wider scope.)
The Von Neumann architecture
The Hungarian-born
mathematician, John von
Neumann (1903-1957)
The biological architecture
Biological computers
Five distinguishing properties:
• Highly parallel
• Robust and fault tolerant
• Adaptive
• Deals with fuzzy, noisy information
• Small, compact
Graceful Degradation
performance
damage
Neurons
Brain consists of
100000000000
(1011) neurons
Neural activity
out
in
Artificial Neuron
(Called McCulloch-Pitts neuron if 0/1 output.)
Input-output function
1
• nonlinear function:
f(x) =
1 + e -x/a
a0
f(e)
a
e
Artificial Connections
(Synapses)
• wAB
– The weight of the connection from neuron
A to neuron B
A
wAB
B
Supervised Networks
Example
Real Neurons
• Nonlinear Summation
• Sequences of pulses
• No fixed time-delay
History
•
•
•
•
•
•
•
1943: McCulloch and Pitts: artificial neuron
1960: Rosenblatt: perceptrons
1969: Minsky and Papert: XOR problem
197?: Associative content-addressable memory
198?: Hopfield Networks, Boltzmann Networks
1985: Backpropagation learning rule
2000+ : Spiking networks, Support Vector
Machines
Hopfield
Minsky
McCulloch
Boltzmann
Pitts
Papert
Hebb
Issues
• Neurocomputing vs. Neuroscience
• Types of Learning:
– Supervised
– Unsupervised
– Reinforcement
Research Questions
•
•
•
•
Design: what is best architecture?
Learning: find good algorithms
Analysis: what is the power of networks?
Implementation: how should the network
be implemented?