Download Artificial Intelligence Methods

Artificial Intelligence
Methods
Neural Networks
Lecture 1
Rakesh K. Bissoondeeal
([email protected])
Biological Neural Networks
Biological Neuron

Synapses
- Gap between adjacent neurons across which
chemicals are transmitted: input



Dendrites
- Receive synaptic contacts from other neurons
Cell body/Soma
- Metabolic centre of the neuron: processing
Axon
- produces the output
Artificial Neuron

Artificial neurons are the building blocks of
Artificial Neural Networks
Artificial Neurons

Artificial neurons simulate the four basic
functions of natural neurons
- Signals are passed between neurons over
connection links
- Each connection link has an associated weight
which multiplies the signal transmitted
- Each neuron applies an activation function to is
net input (sum of weighted input signals) to
produce an output signal
Why study Artificial Neural
Networks




Desire to understand the brain and to imitate
some of its strength
Traditional computers implement a sequence of
logical and arithmetic operations but don’t have
the ability to adapt their structure or learn
Learn from examples, Generalisation
Used to solved task where it is beneficial to use a
machine but impossible to program all possible
outcomes
Applications






List of applications mentioned in the
literature
Aerospace -high performance aircraft
autopilot
Banking –check and other document
reading
Defence – weapon steering
Financial –financial analysis
Speech – speech recognition
Brief History of ANNs



1943 W.S. McCulloch and W. Pitts
- Original idea published
1949 D. Hebb
- Publishes ideas on learning in
biological neurons
1958 F. Rosenblatt
- First practical working networks
called perceptrons
Brief History of ANNs



1969 . M Minsky and S. Papert
- Rubbish ANNs
- Most research on ANNs stop
1970s Widrow, Parker and others
- Low level of activity
- Backpropagation invented
1980s Rumelhart and others
- Rediscovery of Backpropagation
- Revival of interest in ANNs
McCulloch-Pitts Neuron


First mathematical model of the
biological neuron
- Mc Culloch and Pitts (1943)
Most models used today are
descended from McCulloch and Pitts
neuron
McCulloch-Pitts Neuron
 The output of a neuron is binary. That is,
the neuron either fires (output of one) or
does not fire (output of zero).
X1
2
X2
2
Y
-1
X3
McCulloch-Pitts Neuron
Neurons in a McCulloch-Pitts network are connected
by directed, weighted paths
 A connection path is excitatory if the weight on the
path is positive; otherwise it is inhibitory
X1
2
X2
2
Y
-1
X3
McCulloch-Pitts Neuron

Each neuron has a fixed threshold (θ). If the net
input to the neuron is greater than the threshold,
the neuron fires
X1
2


If net input >= θ, output=1
X2
If net input < θ, output = 0
2
Y
-1
X3
Example 1





Logic Functions: AND
x1
x2
AND
True=1, False=0
If both inputs true, output true
Else, output false
Threshold(Y)=2
0
0
0
1
0
0
1
1
0
1
0
1
X1
1
Y
X2
1
AND Function
Example 2





Logic Functions: OR
x1
x2
OR
True=1, False=0
If either of inputs true, output true
Else, output false
Threshold(Y)=2
0
0
0
1
0
1
1
1
0
1
1
1
X1
2
Y
X2
2
OR Function
McCulloch-Pitts Neuron

Structure does not change
- Fixed system that takes inputs to produce
output


Has no concept of learning
However, McCulloch-Pitts Neuron forms
the foundation of modern ANNs
- Changes made to allow learning
Recommended Reading



Fundamentals of neural networks;
Architectures, Algorithms and
Applications, L. Fausett, 1994.
Artificial Intelligence: A Modern
Approach, S. Russel and P. Norvig,
1995.
An Introduction to Neural Networks.
2nd Edition, Morton, IM.