Download Neural network: information processing paradigm inspired by

INTRODUCTION TO NEURAL
NETWORKS
A new sort of computer
• What are (everyday) computer systems
good at... and not so good at?
Good at..
Rule-based systems: doing
what the programmer wants
them to do
Not so good at..
Dealing with noisy data
Dealing with unknown
environment data
Massive parallelism
Fault tolerance
Adapting to circumstances
2
Neural networks to the rescue…
• Neural network: information processing
paradigm inspired by biological nervous
systems, such as our brain
• Structure:
large
number
of
highly
interconnected
processing
elements
(neurons) working together
• Like people, they learn from experience (by
example)
3
What is NN?
“Data processing system consisting of a
large
number
of
simple,
highly
interconnected
processing
elements
(artificial neurons) in an architecture
inspired by the structure of the cerebral
cortex of the brain”
(Tsoukalas & Uhrig, 1997).
4
Inspiration from Neurobiology
Human Biological Neuron
5
Inspiration from Neurobiology
Signal Processing
• A neuron: many-inputs /
one-output unit
• output can be excited or
not excited
• incoming
signals
from
other neurons determine if
the neuron shall excite
("fire")
• Output
subject
to
attenuation
in
the
synapses,
which
are
junction parts of the neuron
6
Inspiration from Neurobiology
Artificial Neuron
Four basic components of a human biological
neuron
The components of a basic artificial neuron
7
Inspiration from Neurobiology
• Neural networks are configured for a
specific application, such as pattern
recognition or data classification,
through a learning process
• In a biological system, learning involves
adjustments to the synaptic connections
between neurons
 same for artificial neural networks
(ANNs)
8
Inspiration from Neurobiology
NN General Architecture
• NN deals with training samples belonging to
known classes and finding a generalized
classifier to predict the class for any new
samples.
Input layer
Hidden layer
Output layer
Attribute1
Attribute2
Attribute3
NN general architecture
9
Where can neural network systems help…
• when we can't formulate an
algorithmic solution.
• when we can get lots of examples of
the behavior we require.
‘learning from experience’
• when we need to pick out the
structure from existing data.
10
History
•
•
•
•
1943 McCulloch-Pitts neurons
1949 Hebb’s law
1958 Perceptron (Rosenblatt)
1960 Adaline, better learning rule
(Widrow, Huff)
• 1969 Limitations (Minsky, Papert)
• 1972 Kohonen nets, associative
memory
11
History
• 1977 Brain State in a Box (Anderson)
• 1982 Hopfield net, constraint
satisfaction
• 1985 ART (Carpenter, Grossfield)
• 1986 Backpropagation (Rumelhart,
Hinton, McClelland)
• 1988 Neocognitron, character
recognition (Fukushima)
12
Characterizations
• Architecture – a pattern of connections
between neurons
• Learning Algorithm – a method of
determining the connection weights
• Activation Function
13
Problem Domains
•
•
•
•
•
Storing and recalling patterns
Classifying patterns
Mapping inputs onto outputs
Grouping similar patterns
Finding solutions to constrained
optimization problems
14
Problem Domains
Coronary
Disease
STOP
10
01
Neural
Net
11
10
00
11
00
00
11
Input patterns
Input layer
Output layer
00
01
00
00
15
10
11
10
11
11
Sorted
.
patterns
Problem Domains
00
11
10
10
11
00
00
01
16
11
Features
• Neurons can generalize novel input
stimuli
• Neurons are fault tolerant and can
sustain damage
17
Who is interested?...
• Electrical Engineers – signal
processing, control theory
• Computer Engineers – robotics
• Computer Scientists – artificial
intelligence, pattern recognition
• Mathematicians – modelling tool when
explicit relationships are unknown
18
ANN Applications
• Signal processing
• Pattern recognition, e.g. handwritten
characters or face identification.
• Diagnosis or mapping symptoms to a
medical case.
• Speech recognition
• Human Emotion Detection
• Educational Loan Forecasting
19
ANN Applications
Abdominal Pain Prediction
20
10
Ulcer
Pain
Cholecystitis
Duodenal Non-specific
Perforated
0
AppendicitisDiverticulitis
37
1
0
1
WBC
20
Obstruction Pancreatitis
Small Bowel
0
Temp
Age
0
Male
Intensity Duration
Pain
Pain
adjustable
1
1
weights
0
0
ANN Applications
Voice Recognition
21
ANN Applications
Educational Loan Forecasting System
22