Download Neural Networks

Neural Networks
Teacher:
Elena Marchiori
R4.47
[email protected]
Neural Networks
Assistant:
Kees Jong
S2.22
[email protected]
NN 1
1
Course Outline
Basics of neural network theory and practice for
supervised and unsupervised learning.
Most popular Neural Network models:
• architectures
• learning algorithms
• applications
Neural Networks
NN 1
2
Course Outline
Rules: - 4 s.p
- Final mark is based on two assignments, which will
be available at the end of the course.
- one assignment is on theory (to do alone).
- one assignment is on practice (to do in couples).
- Programming in Matlab 5.3.
- Registration: send email to [email protected]
Neural Networks
NN 1
3
Course Organization
• There is no text book.
• Course schedule, slides and exercises will be
available at
http://www.cs.vu.nl/~elena/nn.html
Neural Networks
NN 1
4
Neural Networks
• A NN is a machine learning approach inspired by the
way in which the brain performs a particular learning
task:
– Knowledge about the learning task is given in the form of
examples.
– Inter neuron connection strengths (weights) are used to
store the acquired information (the training examples).
– During the learning process the weights are modified in
order to model the particular learning task correctly on the
training examples.
Neural Networks
NN 1
5
Learning
• Supervised Learning
– Recognizing hand-written digits, pattern recognition,
regression.
– Labeled examples
(input , desired output)
– Neural Network models: perceptron, feed-forward, radial
basis function, support vector machine.
• Unsupervised Learning
– Find similar groups of documents in the web, content
addressable memory, clustering.
– Unlabeled examples
(different realizations of the input alone)
– Neural Network models: self organizing maps, Hopfield
networks.
Neural Networks
NN 1
6
Network architectures
• Three different classes of network architectures
– single-layer feed-forward
– multi-layer feed-forward
– recurrent
neurons are organized
in acyclic layers
• The architecture of a neural network is linked with the
learning algorithm used to train
Neural Networks
NN 1
7
Single Layer Feed-forward
Input layer
of
source nodes
Neural Networks
Output layer
of
neurons
NN 1
8
Multi layer feed-forward
3-4-2 Network
Output
layer
Input
layer
Hidden Layer
Neural Networks
NN 1
9
Recurrent network
Recurrent Network with hidden neuron(s): unit
delay operator z-1 implies dynamic system
z-1
input
hidden
output
z-1
z-1
Neural Networks
NN 1
10
Neural Network Architectures
Neural Networks
NN 1
11
The Neuron
• The neuron is the basic information processing unit of
a NN. It consists of:
1 A set of synapses or connecting links, each link
characterized by a weight:
W1, W2, …, Wm
2 An adder function (linear combiner) which
m
computes the weighted sum of
the inputs:
j 1
u   wjxj
3 Activation function (squashing function)  for
limiting the amplitude of the
output of the neuron.
y   (u  b)
Neural Networks
NN 1
12
The Neuron
Bias
b
x1
Input
signal
w1
x2

w2

xm

Local
Field
v
Activation
function
 ()
Output
y
Summing
function
wm
Synaptic
weights
Neural Networks
NN 1
13
Bias of a Neuron
• Bias b has the effect of applying an affine
transformation to u
v=u+b
• v is the induced field of the neuron
v
u
m
u   wjxj
j 1
Neural Networks
NN 1
14
Bias as extra input
• Bias is an external parameter of the neuron. Can be
m
modeled by adding an extra input.
x0 = +1
x1
Input
signal
v   wj xj
w0
j 0
w0  b
w1
x2

w2

xm
Neural Networks
Local
Field
v
Activation
function
 ()
Output
y
Summing
function

wm Synaptic
weights
NN 1
15
Dimensions of a Neural
Network
• Various types of neurons
• Various network architectures
• Various learning algorithms
• Various applications
Neural Networks
NN 1
16
Face Recognition
90% accurate learning head pose, and recognizing 1-of-20 faces
Neural Networks
NN 1
17
Handwritten digit recognition
Neural Networks
NN 1
18