Download Materialy/06/Lecture12- ICM Neuronal Nets 1

Slovak University of Technology
Faculty of Material Science and Technology in Trnava
Intelligent Control
Methods
Lecture 13: Neuronal Nets (Part 1)
History:






1921: First attempt of McCulloch to model a brain
1943: First McCulloch’s publication of model of
neuron
1947: McCulloch and Pitt described a behaviour of
connected neurons
1949: Hebb designed a net with memory
1958: Rosenblatt described learning (“back
propagation”)
1962: first neurocomputer
2
Charakteristics:

inspired by brain
= 40 – 100 mld. neurons, in artificial nets only
tens till hundreds, it is enough for simulating of some
functions
 brain





distributed parallel information processing (the
whole net in the same time)
resistant to mistakes (failure of 1 element
influences the whole system only slightly)
knowledge is represented by connections
between neurons
they are able to learn
they solve no-algorithmic tasks – they need
training set instead of algorithm
3
Biological neuron:
Dendrits
(Input channels)
Body (soma)
Axón (output channel)
synapsis
(connection with next dendrit)
4
Formal (artificial) neuron
(McCulloch 1943)
Inputs
Weights
x1
w1
Local
memory
x2
w2
Summer
f
(xiwi)
potential
xn
y
transfer function
y = f((xiwi))
wn
5
Transfer functions:
y
physical restriction
n
y  k *  ( xiwi )
Linear:
i 1
0
 xiwi
y
Linear with threshold:
physical restriction
n
y  k * ( ( xiwi)   )
i 1
0

 xiwi
6
Nonlinear transfer functions:
Unit jump
(Unit jump with
threshold)
n
y  f ( ( xiwi ), { })
y
1
i 1
= 1 if xiwi  0 
= 0 if xiwi  0 
0 {}
 xiwi
7
Nonlinear transfer functions:
1 y
Signoidal function
0,5
(S. f. with threshold)
n
y  f ( ( xiwi ),{ }, T ) 
i 1
0 {}
1

{ }
1 e
0
 wixi
T
xiwi
T gives output steepness.
Wide scope,
sharp sensitivity around 0 {}.
8
Nonlinear transfer functions:
1 y
Hyperbolic tangent
(H. t. with threshold)
xiwi

1 e
y
 K  xiwi
1 e
K
0
-1
0 {}
xiwi
Wide scope,
sharp sensitivity around 0 {}.
9
Neuronal net:


Net of neurons
Represented by graph
– neurons
 arcs – synaptic connections
 rates of arcs – synaptic weights
 nodes
10
Neuronal net:
x1
x2
y1
x3
y2
x4
y3
.
.
.
ym
xn
inputs
input
layer
hidden
layer(s)
output
layer
outputs
11
Neuronal net:

Inputs:
 qualitative
(binary) – they express the existence of
property
 quantitative (they evaluate the property)



numeric values of variables
fuzzy values of linguistic variables
Outputs:
 usually
of the same type as inputs
12
Neuronal nets topologies:

according to number of layers:
 single-layer
(input layer = output layer)
 multi-layer (without and with hidden layers;
commercial software usually estimates the number of hidden
layers automatically

according to feed-back:
 direct (outputs lead only to inputs of next layer)
 with
feed-back (outputs
lead to inputs of previous layers,
too)
13
Neuronal net modes:
decision (solution, active dynamics)
 learning (adaptive dynamics)

14