Download Neural Computing

Neural Networks
& Cases
By
Jinhwa Kim
Neural Computing: The Basics

2
Neural Computing is a problem solving
methodology that attempts to mimic
how human brain function

Artificial Neural Networks (ANN)

Machine Learning
Neural Computing

Computing technology that mimic certain
processing capabilities of the human brain
Knowledge representations based on
 Massive parallel processing
 Fast retrieval of large amounts of information
 The ability to recognize patterns based on
historical cases
Neural Computing = Artificial Neural Networks (ANNs)

Purpose of ANN is to simulate the thought process
of human brain

Inspired by the studies of human brain and the
nervous system

3
The Biology Analogy

Neurons: brain cells





4
Nucleus (at the center)
Dendrites provide inputs
Axons send outputs
Synapses increase or
decrease connection
strength and cause
excitation or inhibition of
subsequent neurons
Figure 15.1
Artificial Neural Networks (ANN)




5
A model that emulates a biological neural
network
Software simulations of the massively
parallel processes that involve processing
elements interconnected in a network
architecture
Originally proposed as a model of the human
brain’s activities
The human brain is much more complex
Artificial Neural Networks (ANN)
Three Interconnected Artificial Neurons
Biological
Soma
Dendrites
Axon
Synapse
Slow speed
Many neurons
(Billions)
6
Artificial
Node
Input
Output
Weight
Fast speed
Few neurons
(Dozens)
ANN Fundamentals

Components and Structure


“A network is composed of a number of processing elements
organized in different ways to form the network structure”
Processing Elements (PEs) – Neurons
Network


Structure of the Network

Figure 15.3
7
Collection of neurons (PEs) grouped in layers
Topologies / architectures – different ways to interconnect PEs
ANN Fundamentals

8
Figure 15.4
ANN Fundamentals

Processing Information by the Network

Inputs
Outputs
Weights
Summation Function

Figure 15.5



9
ANN Fundamentals

Transformation (Transfer) Function




10
Computes the activation level of the neuron
Based on this, the neuron may or may not produce an output
Most common: Sigmoid (logical activation) function
AIS 15.3
Learning in ANN
1.
2.
3.

11
Compute outputs
Compare outputs with
desired targets
Adjust the weights and
repeat the process
Figure 15.6
Neural Network
Application Development

Preliminary steps




ANN Application Development Process
1.
2.
3.
4.
5.
6.
7.
8.
9.
12
Requirement determination
Feasibility study
Top management champion
Collect Data
Separate into Training and Test Sets
Define a Network Structure
Select a Learning Algorithm
Set Parameters, Values, Initialize
Weights
Transform Data to Network Inputs
Start Training, and Determine and
Revise Weights
Stop and Test
Implementation: Use the Network with
New Cases
Data Collection and Preparations

Collect data and separate it into





13
Training set (60%)
Testing set (40%)
Make sure that all three sets represent
the population: true random sampling
Use training and cross validation cases
to adjust the weights
Use test cases to validate the trained
network
Neural Network Architecture


14
There are several ANN architectures
Figure 15.9
Neural Network Architecture

Feed forward Neural Network

15
Multi Layer Perceptron, - Two, Three, sometimes
Four or Five Layers
How a Network Learns


Step function evaluates the summation of
input values
Calculating outputs
Measure the error (delta) between outputs and
desired values
 Update weights, reinforcing correct results
At any step in the process for a neuron, j, we get
Delta = Zj - Yj
where Z and Y are the desired and actual outputs,
respectively

16
How a Network Learns

Updated Weights are
Wi (final) = Wi (initial) + alpha × delta × X1
where alpha is the learning rate parameter




17
Weights are initially random
The learning rate parameter, alpha, is set low
Delta is used to derive the final weights, which then
become the initial weights in the next iteration (row)
Threshold value parameter: sets Y to 1 in the next
row if the weighted sum of inputs is greater than 0.5;
otherwise, to 0
How a Network Learns
18
Backpropagation






Backpropagation (back-error propagation)
Most widely used learning
Relatively easy to implement
Requires training data for conditioning the
network before using it for processing other
data
Network includes one or more hidden layers
Network is considered a feedforward
approach
Continue
19
Backpropagation
1.
2.
3.
4.
5.
Initialize the weights
Read the input
vector
Generate the output
Compute the error
Error = Out - Desired
Change the weights

Drawbacks:


20
A large network can take a very long time to train
May not converge
Testing







Test the network after training
Examine network performance: measure the
network’s classification ability
Black box testing
Do the inputs produce the appropriate outputs?
Not necessarily 100% accurate
But may be better than human decision makers
Test plan should include



21
Routine cases
Potentially problematic situations
May have to retrain
ANN Development Tools










22
NeuroSolutions
Statistica Neural Network Toolkit
Braincel (Excel Add-in)
NeuralWorks
Brainmaker
PathFinder
Trajan Neural Network Simulator
NeuroShell Easy
SPSS Neural Connector
NeuroWare
Benefits of ANN








23
Pattern recognition, learning, classification,
generalization and abstraction, and interpretation of
incomplete and noisy inputs
Character, speech and visual recognition
Can provide some human problem-solving
characteristics
Can tackle new kinds of problems
Robust
Fast
Flexible and easy to maintain
Powerful hybrid systems
Limitations of ANN




24
Lack explanation capabilities
Limitations and expense of hardware
technology restrict most applications to
software simulations
Training time can be excessive and
tedious
Usually requires large amounts of
training and test data
ANN Demonstration

www.roselladb.com
NeuroSolutions
http://www.nd.com/neurosolutions/products/ns/nnandnsvideo.html

by NeuroDimentions, Inc.

www.nd.com




DMWizard
By Knowledge Based Systems, Inc.
Funded by US Army
www.roselladb.com
25
Business ANN Applications

Accounting



Finance










26
Identify tax fraud
Enhance auditing by finding irregularities
Signatures and bank note verifications
Foreign exchange rate forecasting
Bankruptcy prediction
Customer credit scoring
Credit card approval and fraud detection*
Stock and commodity selection and trading
Forecasting economic turning points
Pricing initial public offerings*
Loan approvals
…
Business ANN Applications

Human Resources



Management





Consumer spending pattern classification
Sales forecasts
Targeted marketing, …
Operations


27
Corporate merger prediction
Country risk rating
Marketing


Predicting employees’ performance and behavior
Determining personnel resource requirements
Vehicle routing
Production/job scheduling, …
Bankruptcy Prediction with ANN

Based on a paper



ANN Architecture



Published in Decision Support Systems, 1994
By Rick Wilson and Ramesh Sharda
Three-layer (input-hidden-output) MLP
Backpropagation (supervised) learning network
Training data


Small set of well-known financial ratios
Data available on bankruptcy outcomes

28
Moody’s industrial manual (between 1975 and 1982)
Bankruptcy Prediction with ANN

Application Design Specifics


Five Input Nodes
X1: Working capital/total assets
X2: Retained earnings/total assets
X3: Earnings before interest and taxes/total assets
X4: Market value of equity/total debt
X5: Sales/total assets
Single Output Node: Final classification for each firm



29
Bankruptcy or
Nonbankruptcy
Development Tool: NeuroShell
Bankruptcy Prediction with ANN
30
Bankruptcy Prediction with ANN

Training



Data Set: 129 firms
Training Set: 74 firms; 38 bankrupt, 36 not
Ratios computed and stored in input files for:



Parameters



Number of PEs
Learning rate and Momentum
Testing

Two Ways


31
The neural network
A conventional discriminant analysis program
Test data set: 27 bankrupt firms, 28 nonbankrupt firms
Comparison with discriminant analysis
Bankruptcy Prediction with ANN

Results

The neural network correctly predicted:



ANN did better




32
predicting 22 out of the 27 cases
discriminant analysis predicted only 16 correctly
Error Analysis


81.5 percent bankrupt cases
82.1 percent nonbankrupt cases
Five bankrupt firms misclassified by both methods
Similar for nonbankrupt firms
Accuracy of about 80 percent is usually
acceptable for this problem domain