Download Artificial Neural Networks - Introduction -

Artificial Neural Networks
- Introduction -
Overview
1. Biological inspiration
2. Artificial neurons and neural networks
3. Application
Biological Neuron
Animals are able to react adaptively to changes in their
external and internal environment, and they use their nervous
system to perform these behaviours.
An appropriate model/simulation of the nervous system
should be able to produce similar responses and behaviours in
artificial systems.
Biological Neuron
The information transmission happens at the synapses.
Artificial neurons Neuron
Artificial neurons
x1
w1
x2
Inputs
xn-1
xn
z   wi xi ; y  H ( z )
w2
x3
…
n
i 1
..
w3
.
wn-1
wn
one possible model
Output
y
Artificial neurons
Nonlinear generalization of neuron:
y  f ( x, w)
y is the neuron’s output, x is the vector of inputs, and w
is the vector of synaptic weights.
Examples:
y
1
1 e
ye
w xa
T
|| x  w||2

2a 2
sigmoidal neuron
Gaussian neuron
Other Model
Hopfield
Retropropagation
From Logical Neurons to Finite Automata
1
AND
1.5
1
1
OR
0.5
1
NOT
0
-1
Artificial neural networks
Inputs
Output
An artificial neural network is composed of many artificial
neurons that are linked together according to a specific
network architecture. The objective of the neural network
is to transform the inputs into meaningful outputs.
Artificial neural networks
Tasks to be solved by artificial neural networks:
• controlling the movements of a robot based on selfperception and other information (e.g., visual
information);
• deciding the category of potential food items (e.g.,
edible or non-edible) in an artificial world;
• recognizing a visual object (e.g., a familiar face);
• predicting where a moving object goes, when a robot
wants to catch it.
Neural network mathematics
Inputs
Output
 y11  2
 1  y1  f ( y1 , w12 )
 y32 
 2
2
3
y 12  f ( x 2 , w12 ) y1   y 2  2
2
1
2
y

f
(
y
,
w
y

y


y 2  f ( y , w2 )
Out
1)
3

1
 2 
y 31  f ( x3 , w31 )
 y3  2
1
2
y3 
1  y3  f ( y , w3 )


 y4 
y 14  f ( x 4 , w14 )
y11  f ( x1 , w11 )
Neural network mathematics
Neural network: input / output transformation
yout  F ( x,W )
W is the matrix of all weight vectors.
Learning principle for
artificial neural networks
ENERGY MINIMIZATION
We need an appropriate definition of energy for artificial
neural networks, and having that we can use
mathematical optimisation techniques to find how to
change the weights of the synaptic connections between
neurons.
ENERGY = measure of task performance error
Perceptron application
+
++
+ +
+
+
+ + + ++ +
+
+ +
+
+
+
++ +
+ + + + ++
+ +
+
+
+
+
y  1
++
y  sign (v)
c0
1
 v  c0  c1 x1  c2 x2
c1
x1
c2
x2
y  1
c0  c1 x1  c2 x2  0
Multi-Layer Perceptron
• One or more hidden
layers
• Sigmoid activations
functions
Output layer
2nd hidden
layer
1st hidden
layer
Input data
Multi-Layer Perceptron Application
Structure
Single-Layer
Two-Layer
Three-Layer
Types of
Decision Regions
Result
Half Plane
Bounded By
Hyperplane
A
B
B
A
Convex Open
Or
Closed Regions
A
B
B
A
A
B
B
A
Abitrary
(Complexity
Limited by No.
of Nodes)
Conclusion
NN have some desadvantages such as:
1. Preprocessing
2. Results interpretation by high dimension
3. Learning phase/Supervised/Non
Supervised
References
1. http://neuron.eng.wayne.edu/software.html
Many useful example.
2. http://ieee.uow.edu.au/~daniel/software/libn
eural/BPN_tutorial/BPN_English/BPN_Engli
sh/BPN_English.html
3. http://www.ai-junkie.com/
4. http://diwww.epfl.ch/mantra/tutorial/english/
Demo: OCR