Download Neural Networks

Neural Networks
An Introduction
A Neuron
Dendrites
Axon
Cell Body
Synapse
An Introduction to Neural Networks
Computer Representation
p
w

n
f
a
b
1
Output= a(n)
= a(pw+b)
An Introduction to Neural Networks
A Single Neuron with Multiple Inputs
p1
p2
w1,1
w
. 1,2

.
.
pr
n
f
a
b
w1,r
1
An Introduction to Neural Networks
Single Layer Neural Network
with Multiple Neurons
p1
w1,1

f
a1
b1
p2
1
p3

.
.
.
pr
n1
n2
f
a2
b2
.
.
.
1
ws,r

ns
f
.
.
.
as
bs
1
An Introduction to Neural Networks
Multiple Layer Neural Network
First Layer
Inputs
p1
w1,1

n1
f
a1
w1,1

b1
p2
1
p3

.
.
.
pr
Second Layer
f
a1
w1,1


n2
a2
f

ns
f
.
.
.
n2
ws,r

bs
a1
a2
f

n2
f
a2
b2
.
.
.
1
as
f
1
b2
.
.
.
n1
b1
1
1
1
n1
b1
b2
ws,r
Third Layer
ns
f
..
.
..
.
.
.
1
as
ws,r

bs
1
An Introduction to Neural Networks
bs
1
ns
f
.
.
.
as
Activation Functions



Hard Limit
 a = 0
n<0
 a = 1
n >= 0
Symmetrical Hard Limit
 a = -1
n<0
 a = +1 n >= 0
Saturating Linear
 a = 0
n<0
 a = n
0 <= n <= 1
 a = 1
n>1
An Introduction to Neural Networks
Activation Functions


Linear
 a = n
Symmetric Saturating Linear




a = -1
a=n
a=1
n < -1
-1 <= n <= 1
n>1
Log-Sigmoid
 a =
1
1+ e-n
An Introduction to Neural Networks
Activation Functions



Hyperbolic Tangent Sigmoid
 a =
en - e-n
en + e-n
Positive Linear
 a = 0
n<0
 a = n
n >= 0
Competitive


a=1
a=0
neuron with max n
all other neurons
An Introduction to Neural Networks
The History of Development
of Neural Networks





The Beginning of Neural Networks (1940's)

McCulloch Pitts Neuron

Hebb Learning
The First Golden Age of Neural Networks (1950's and 1960's)

Perceptrons

Adaline
The Quiet Years: 1970's

Kohonen

Anderson

Grossberg

Carpenter
Renewed Enthusiasm: 1980's

Backpropagation

Hopfield nets

Neocognitron

Boltzman machine
Hardware Implementation
An Introduction to Neural Networks
Developing a Neural Network
System
Choose a neural network architecture
Train the neural network using
a training set
Apply the neural network to identify patterns.
This involves implementing the application algorithm
An Introduction to Neural Networks
Choosing a Neural Network
Architecture

Identify the number of inputs


Identify the number of outputs


Number of network inputs = number of problem
inputs.
Number of neurons in output layer = number of
problem outputs.
The output layer transfer function is partly
determined by problem specification of the
outputs.
An Introduction to Neural Networks