Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Neural Networks & Cases By Jinhwa Kim Neural Computing: The Basics 2 Neural Computing is a problem solving methodology that attempts to mimic how human brain function Artificial Neural Networks (ANN) Machine Learning Neural Computing Computing technology that mimic certain processing capabilities of the human brain Knowledge representations based on Massive parallel processing Fast retrieval of large amounts of information The ability to recognize patterns based on historical cases Neural Computing = Artificial Neural Networks (ANNs) Purpose of ANN is to simulate the thought process of human brain Inspired by the studies of human brain and the nervous system 3 The Biology Analogy Neurons: brain cells 4 Nucleus (at the center) Dendrites provide inputs Axons send outputs Synapses increase or decrease connection strength and cause excitation or inhibition of subsequent neurons Figure 15.1 Artificial Neural Networks (ANN) 5 A model that emulates a biological neural network Software simulations of the massively parallel processes that involve processing elements interconnected in a network architecture Originally proposed as a model of the human brain’s activities The human brain is much more complex Artificial Neural Networks (ANN) Three Interconnected Artificial Neurons Biological Soma Dendrites Axon Synapse Slow speed Many neurons (Billions) 6 Artificial Node Input Output Weight Fast speed Few neurons (Dozens) ANN Fundamentals Components and Structure “A network is composed of a number of processing elements organized in different ways to form the network structure” Processing Elements (PEs) – Neurons Network Structure of the Network Figure 15.3 7 Collection of neurons (PEs) grouped in layers Topologies / architectures – different ways to interconnect PEs ANN Fundamentals 8 Figure 15.4 ANN Fundamentals Processing Information by the Network Inputs Outputs Weights Summation Function Figure 15.5 9 ANN Fundamentals Transformation (Transfer) Function 10 Computes the activation level of the neuron Based on this, the neuron may or may not produce an output Most common: Sigmoid (logical activation) function AIS 15.3 Learning in ANN 1. 2. 3. 11 Compute outputs Compare outputs with desired targets Adjust the weights and repeat the process Figure 15.6 Neural Network Application Development Preliminary steps ANN Application Development Process 1. 2. 3. 4. 5. 6. 7. 8. 9. 12 Requirement determination Feasibility study Top management champion Collect Data Separate into Training and Test Sets Define a Network Structure Select a Learning Algorithm Set Parameters, Values, Initialize Weights Transform Data to Network Inputs Start Training, and Determine and Revise Weights Stop and Test Implementation: Use the Network with New Cases Data Collection and Preparations Collect data and separate it into 13 Training set (60%) Testing set (40%) Make sure that all three sets represent the population: true random sampling Use training and cross validation cases to adjust the weights Use test cases to validate the trained network Neural Network Architecture 14 There are several ANN architectures Figure 15.9 Neural Network Architecture Feed forward Neural Network 15 Multi Layer Perceptron, - Two, Three, sometimes Four or Five Layers How a Network Learns Step function evaluates the summation of input values Calculating outputs Measure the error (delta) between outputs and desired values Update weights, reinforcing correct results At any step in the process for a neuron, j, we get Delta = Zj - Yj where Z and Y are the desired and actual outputs, respectively 16 How a Network Learns Updated Weights are Wi (final) = Wi (initial) + alpha × delta × X1 where alpha is the learning rate parameter 17 Weights are initially random The learning rate parameter, alpha, is set low Delta is used to derive the final weights, which then become the initial weights in the next iteration (row) Threshold value parameter: sets Y to 1 in the next row if the weighted sum of inputs is greater than 0.5; otherwise, to 0 How a Network Learns 18 Backpropagation Backpropagation (back-error propagation) Most widely used learning Relatively easy to implement Requires training data for conditioning the network before using it for processing other data Network includes one or more hidden layers Network is considered a feedforward approach Continue 19 Backpropagation 1. 2. 3. 4. 5. Initialize the weights Read the input vector Generate the output Compute the error Error = Out - Desired Change the weights Drawbacks: 20 A large network can take a very long time to train May not converge Testing Test the network after training Examine network performance: measure the network’s classification ability Black box testing Do the inputs produce the appropriate outputs? Not necessarily 100% accurate But may be better than human decision makers Test plan should include 21 Routine cases Potentially problematic situations May have to retrain ANN Development Tools 22 NeuroSolutions Statistica Neural Network Toolkit Braincel (Excel Add-in) NeuralWorks Brainmaker PathFinder Trajan Neural Network Simulator NeuroShell Easy SPSS Neural Connector NeuroWare Benefits of ANN 23 Pattern recognition, learning, classification, generalization and abstraction, and interpretation of incomplete and noisy inputs Character, speech and visual recognition Can provide some human problem-solving characteristics Can tackle new kinds of problems Robust Fast Flexible and easy to maintain Powerful hybrid systems Limitations of ANN 24 Lack explanation capabilities Limitations and expense of hardware technology restrict most applications to software simulations Training time can be excessive and tedious Usually requires large amounts of training and test data ANN Demonstration www.roselladb.com NeuroSolutions http://www.nd.com/neurosolutions/products/ns/nnandnsvideo.html by NeuroDimentions, Inc. www.nd.com DMWizard By Knowledge Based Systems, Inc. Funded by US Army www.roselladb.com 25 Business ANN Applications Accounting Finance 26 Identify tax fraud Enhance auditing by finding irregularities Signatures and bank note verifications Foreign exchange rate forecasting Bankruptcy prediction Customer credit scoring Credit card approval and fraud detection* Stock and commodity selection and trading Forecasting economic turning points Pricing initial public offerings* Loan approvals … Business ANN Applications Human Resources Management Consumer spending pattern classification Sales forecasts Targeted marketing, … Operations 27 Corporate merger prediction Country risk rating Marketing Predicting employees’ performance and behavior Determining personnel resource requirements Vehicle routing Production/job scheduling, … Bankruptcy Prediction with ANN Based on a paper ANN Architecture Published in Decision Support Systems, 1994 By Rick Wilson and Ramesh Sharda Three-layer (input-hidden-output) MLP Backpropagation (supervised) learning network Training data Small set of well-known financial ratios Data available on bankruptcy outcomes 28 Moody’s industrial manual (between 1975 and 1982) Bankruptcy Prediction with ANN Application Design Specifics Five Input Nodes X1: Working capital/total assets X2: Retained earnings/total assets X3: Earnings before interest and taxes/total assets X4: Market value of equity/total debt X5: Sales/total assets Single Output Node: Final classification for each firm 29 Bankruptcy or Nonbankruptcy Development Tool: NeuroShell Bankruptcy Prediction with ANN 30 Bankruptcy Prediction with ANN Training Data Set: 129 firms Training Set: 74 firms; 38 bankrupt, 36 not Ratios computed and stored in input files for: Parameters Number of PEs Learning rate and Momentum Testing Two Ways 31 The neural network A conventional discriminant analysis program Test data set: 27 bankrupt firms, 28 nonbankrupt firms Comparison with discriminant analysis Bankruptcy Prediction with ANN Results The neural network correctly predicted: ANN did better 32 predicting 22 out of the 27 cases discriminant analysis predicted only 16 correctly Error Analysis 81.5 percent bankrupt cases 82.1 percent nonbankrupt cases Five bankrupt firms misclassified by both methods Similar for nonbankrupt firms Accuracy of about 80 percent is usually acceptable for this problem domain