Download Neural Computing

Week 5
Artificial Neural Networks (ANNs)
1
Case Scenario
ABC Enterprise, as a multinational company has
invested in many country sectors for revenues
generation. One of the potential revenue for the
company comes from the investment in the BSKL
shares. The CEO of the company, Mr. Ridzuan intends
to invest in selected potential shares available in the
current market. He is aware that such investment is a
risky business. Unfortunately, he does not have
experienced staff to advice on the potential shares
investment. Thus, Mr. Ridzuan is thinking whether
there is any system that can predict short and long
term future of current shares trading in the market
with accurately. The system must be able to analyze
history of the current shares trading and predicts
when the best time to buy the shares is.
2
Human Brain


50 to 150 billion neurons in brain (estimation)
Neurons grouped into networks


Axons send outputs to cells
Received by dendrites, across synapses
3
Artificial Neural Networks (ANNs)




A model that emulates a biological neural
network.
Software simulations of the massively parallel
processes that involve processing elements
interconnected in a network architecture.
Originally proposed as a model of the human
brain’s activities.
The human brain is much more complex.
4
Processing Information
in an Artificial Neuron
Inputs
Weights
x1
w1j
x2
w2j
Neuron j
Σ wij xi
Summations
xi
ƒ
Output
Yj
Transfer function
wij
5
Processing



Processing elements are neurons
Allows for parallel processing
Each input is single attribute

Connection weight


Summation function



Adjustable mathematical value of input
Weighted sum of input elements
Internal stimulation
Transfer function

Relation between internal activation and output



Sigmoid/transfer function
Threshold value
Outputs are problem solution
6
Learning: Three Tasks
The neurons are connected by links, and each
link has a numerical weight associated with it.
Weights are the basic means of long term
memory in ANN. They express the strength or
importance of each neuron input. ANN learns
through repeated adjustments of these
weights. In summary, learning in ANN
involves three tasks:
1. Compute Outputs
2. Compare Outputs with Desired Targets
3. Adjust Weights and Repeat the Process
7
Learning Algorithms


Learning is a fundamental characteristic
of ANNs.
Two Basic Learning Categories


Supervised Learning
Unsupervised Learning
8
Supervised Learning




For a set of inputs with known (desired) outputs.
Connection weights derived from known cases.
Useful in pattern recognition (character, voice,
object etc.)
Examples


Backpropagation network
Hopfield network
9
Supervised Learning : Character Recognition
Demonstration of a neural network used within an optical character recognition (OCR) application. The
original document is scanned into the computer and saved as an image. The OCR software breaks the
image into sub-images, each containing a single character. The sub-images are then translated from an
image format into a binary format, where each 0 and 1 represents an individual pixel of the sub-image.
The binary data is then fed into a neural network that has been trained to make the association between
the character image data and a numeric value that corresponds to the character. The output from the
neural network is then translated into ASCII text and saved as a file.
10
Unsupervised Learning




Only input stimuli shown to the network.
Humans assign meanings and determine
usefulness.
Useful in clustering (objects) and knowledge
discovery.
Examples


Adaptive Resonance Theory (ART)
Kohonen Self-organizing Feature Maps
11
Unsupervised Learning :
Data Classification
Classification of raw data into two subgroups.
12
Development of Systems

Collect data




Separate data into training set to adjust weights
Divide into test sets for network validation
Select network topology




The more, the better
Determine input, output, and hidden nodes, and hidden
layers
Select learning algorithm and connection weights
Iterative training until network achieves preset error
level
Black box testing to verify inputs produce appropriate
outputs

Contains routine and problematic cases
13
Neural Network Software

Program in:



Tools (shells) incorporate: (MATLAB)



Programming language (C++, Java, VB)
Neural network package or NN programming tool
Training algorithms
Transfer and summation functions
May still need to:



Program the layout of the database
Partition the data (test data, training data)
Transfer the data to files suitable for input to an ANN
tool
14
Advantages of ANNs





Pattern recognition, classification,
generalization, interpretation of incomplete
and noisy inputs.
Character, speech and visual recognition.
Can tackle new kinds of problems.
Robust, flexible and easy to maintain.
Powerful hybrid systems.
15
Limitations of ANNs





Do not do well at tasks that are not done well by
people
Lack explanation capabilities
Limitations and expense of hardware technology
restrict most applications to software simulations
Training time can be excessive and tedious
Usually requires large amounts of training and test
data
16
ANN Examples

NeuroXL Classifier (add-in for Ms Excell)


N-OCR (character recognition)


http://www.neuroxl.com/index.htm
http://www10.brinkster.com/geniusportal/neural/nocr.html
Neural Network Toolbox (MathWorks)

http://www.mathworks.com/products/neuralnet/
17
Intelligence Density Dimension




Accuracy
Flexibility
Embeddedability
Independence from experts
18