Download Artificial Intelligence, Expert Systems, and DSS

Artificial Neural Networks
An Introduction
What is a Neural Network?




A human Brain
A porpoise brain
The brain in a living creature
A computer program


Simulates (at a very rudimentary level)
a biological brain
Limited connections
Artificial Neural Networks


Artificial neural networks are information technology
inspired by studies of the brain and nervous system
ANNs are used to simulate the massively parallel
processes that are effectively used in the brain for
learning, and storing information and knowledge
Biological Neuron







Dendrites
Axon
Soma
Membrane
Synapse
Neurotransmitter
Spikes
Dendrites
Axon
Synapse
Simple Neuron Configuration
Inputs
X1
X2
X3
Weights
W1
W2
W3
W4
X4
Summation
(weighted)
Transfer
Output (Y)
Threshold Logic Units


Outputs are 0 or 1
If the activation
(accumulated weighted
input) is larger than
threshold the unit
generates a signal
1
0
Sigmoidal Transfer function
Outputs are in the range
from 0 to 1
y=1/(1+exp(-a))
Is differentiable
Neural Network Architecture



In feedforward NN, neurons are grouped into layers
The neurons on each layer are the same type
There are different types of layers



Input layer: receive input from external sources
Output layer: communicate to user
Hidden layer(s): neurons communicate only with
other layers
Sample Network Configuration
Input
layer
Hidden
layer
Output
layer
Some Characteristics of ANN







Tolerance to noise;
Reliability;
Two layer networks are restricted to linearly
separable problems;
Additional layers can solve more complicated
problems;
“Black Box”. Why? Non-linearity;
Logic hidden in weights;
Universal approximators.
Learning Methods

Supervised



Unsupervised



Error Backpropagation
Counter-Propagation
Hebb’s rule
Competitive Learning
Reinforcement
Error Backpropagation Algorithm





Generalized Delta Rule;
Allowed training multi-layer ANN;
Revived interest in ANN;
Error terms are propagated back through
the network;
The weight coefficients are updated
iteratively;
Error Backpropagation Algorithm:
Drawbacks





Local Minima;
Biologically implausible;
Possibility of “network paralysis”;
Slowness;
Oscillations.
Problems solved by ANN






Classification
Cluster Analysis
Approximation
Forecasting
Association
Data compression
Benefits of ANN



Parallelism
Learning
Generalization


NN can learn the characteristics of a general category
of objects on specific examples from that category
Robustness (reliability)


Tolerance to noise
Performance does not degrade appreciably if some of
its neurons or interconnections are lost (Distributed
memory)
Limitations of ANN




Two-layer NN limited to linearly separable problems
Local minima & oscillations
Number of hidden layers/units hard to determine
Lack of transparency (perspicuity)
Sample of Applications

Business







Credit scoring
Bankruptcy prediction
Bond rating
Security trading
Technological processes
Robotics
Consumer electronics