* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download SOLARcief2003
Caridoid escape reaction wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Single-unit recording wikipedia , lookup
Development of the nervous system wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Neural modeling fields wikipedia , lookup
Artificial neural network wikipedia , lookup
Biological neuron model wikipedia , lookup
Neuroeconomics wikipedia , lookup
Optogenetics wikipedia , lookup
Process tracing wikipedia , lookup
Neural coding wikipedia , lookup
Central pattern generator wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Catastrophic interference wikipedia , lookup
Convolutional neural network wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Efficient coding hypothesis wikipedia , lookup
Synaptic gating wikipedia , lookup
Recurrent neural network wikipedia , lookup
SELF-ORGANIZING LEARNING ARRAY (SOLAR) AND ITS APPLICATION TO ECONOMIC AND FINANCIAL PROBLEMS by Janusz Starzyk, Zhen Zhu, Haibo He and Zhineng Zhu School of EECS Ohio University, Athens, OH 3rd International Workshop on Computational Intelligence in Economics and Finance CIEF'2003 Cary, NC, September 30th, 2003 OUTLINE – Motivation – SOLAR and its organization – Self-organizing principle – Evolution of SOLAR structure – Data processing – Examples of economic and financial applications – Future work and conclusion CIEF'2003 SELF-ORGANIZING LEARNING ARRAY What is SOLAR? New Biologically Inspired Network Organization Basic Fabric: A fixed lattice of distributed, parallel processing units (neurons) Self-organization Algorithm: Interconnections among neurons are dynamically CIEF'2003 refined. Neurons are dynamically re-configured. Number of neurons used is decided by problem complexity. Why do we need SOLAR? Needed a general purpose learning network Network that learns without algorithm Network that runs without software Network that is data driven Network that self-organizes Network that learns through associations Network that acts with self awareness Network that scales to a very large system Network that is fault tolerant Network that is modular CIEF'2003 SOLAR-Organization Neurons organized in a cell array Sparse randomized connections Local self-organization Data driven Entropy based learning Regular structure Suitable for large scale circuit implementation CIEF'2003 Structure of a single neuron RPU: reconfigurable processing unit CU: control unit DPE: dynamic probability estimator EBE: entropy based evaluator DSRU: dynamic self-reconfiguration memory. NI/NO: Data input/output CI/CO: Control input/output CIEF'2003 Self-organizing connections Each neuron is pseudorandomly connected to other neurons or primary inputs. O: processing unit : data : control CIEF'2003 Learned Structure Using a entropy-based metric, some of the connections would be found carrying more information and thus saved while other cut out. CIEF'2003 Self-organizing Principle Information deficiency Information index Es I 1 1 s c Emax Psc log( Psc ) Ps log( Ps ) s P log( P ) c c c Es s Emax p sc log( psc ) ps log( ps ) sc p c log( pc ) c If the space was divided, then the information index and information deficiencies are related as 1 I s s When subsequent space partitions take place, the information deficiency is expressed as a product of information deficiencies in each subsequent partition. p s p CIEF'2003 Neuron and its I/O Ports A priori information – class probabilities and input information deficiencies Transformation functions with thresholds, cut input space into two sub-spaces. Neuron self organize to maximize information deficiency reduction: I * in Learning stops when: in CIEF'2003 Neuron’s action Through learning, one pair of function and threshold will be chosen and solidified. This will provide a cut in the input space to separate different classes. CIEF'2003 Neuron’s action Other neurons can provide other cuts. Most confident neurons vote on final classification. CIEF'2003 SOLAR Behavior Behavior of a single neuron: Calculates its data and control outputs and provides them as inputs to others. Computes statistical information (for example, entropy based information deficiency) in its subspaces. Makes associations with other neurons. Behavior of the network: Clusters of neurons solve the problem. Network connections are active elements of learning. The system gathers information from all neurons and makes final decision based on well trained neurons CIEF'2003 SOLAR-Data Processing SOLAR receives input data as a 2-D matrix of feature samples. This matrix is pre-processed to make sure: – Missing parts are recovered – Symbolic data is represented in a proper way Data into each neuron is re-scaled for full resolution CIEF'2003 SOLAR-Data Processing Pre-processing input data Initialize SOLAR Neurons work in parallel System collects information and makes final decision CIEF'2003 Training: Each neuron selects most efficient Testing: inputs, function, threshold and outputs and pass Neurons calculate Testing: solidify them. them to others. Neurons calculate outputs and pass F1=f1(x,y) them on. F2=f2(x,y) F3=f3(x,y) F4=f4(x,y) Decision Making F5=f5(x,y) SOLAR-Data Processing We may use an ensemble of SOLAR networks to vote on the same target: CIEF'2003 Biologically Inspired NN NN CNN SOLAR Y Y Y Y Y Y Y Y Y Local Interconnect Y Y Complexity Driven Hardware Use Y Y Massively Parallel Local Decision Making Data Driven Self Organization CIEF'2003 Y SOLAR-Examples Bankruptcy Prediction – Amir F.Atiya, “Bankruptcy Prediction for Credit Risk Using Neural Networks: A survey and New Results,” IEEE Trans. on Neural Networks, Vol. 12, No. 4, July, 2001. Credit Card Approval Decision Loan Decision-Australian Adult Income Classification – D. Michie, D. J. Spiegelhalter, and C. C. Taylor, “Machine Learning, Neural and Statistical Classification” London, U. K. Ellis Horwood Ltd. 1994. – ftp at cs.uci.edu (128.195.1.1) CIEF'2003 SOLAR-Example The Bankruptcy Dataset: – 716 solvent US corporations – 195 defaulted ones (within 1 to 36 months) – Expanded into 1160 points by taking different instances before the default – Over 60 available indicators (features) Financial Ratio and Equity-based Indicator System: – Based on financial ratios and prices – Proved superior to traditional indicators – Optimal indicator pool based on traditional ANNs CIEF'2003 SOLAR-Example Bankruptcy prediction using SOLAR Time to default correct rate % of SOLAR 6 month or less Reported correct rate % 86.15 85.11 correct rate % of SOLAR using all 87.23 6 to 12 months 81.48 84.09 86.36 12 to 18 months 74.60 76.19 90.24 18 to 24 months 78.13 55.17 72.24 more than 24 months 66.67 64.29 75.00 total defaulted 78.13 75.13 83.96 solvent 90.07 92.74 93.42 total 85.50 85.80 90.04 CIEF'2003 SOLAR-Example Using the same “Financial Ratio and Equitybased Indicator System”, SOLAR shows equal performance to traditional ANNS. SOLAR is capable of handling all indicators and gives even better performance. CIEF'2003 SOLAR-Example prewired structure CIEF'2003 SOLAR-Example learned connections CIEF'2003 SOLAR-Example Detail of prewired structure Detail of learned connections CIEF'2003 SOLAR-Examples Credit Card Approval Decision and Loan Decision – Benchmark problems used to compare performance various methods in literature – learning algorithms, neural networks, statistical methods and SOLAR are compared CIEF'2003 SOLAR-Example Performance Comparison on Credit Card Approval Decision Algorithm Mis-prob. Algorithm Mis-prob. CAL5 0.131 CART 0.145 RBF 0.145 SOLAR 0.135 Itule 0.137 CASTLE 0.148 DIPOL92 0.141 Naivebay 0.151 Logdisc 0.141 IndCART 0.152 Discrim 0.141 Bprop 0.154 CIEF'2003 SOLAR-Example Performance Comparison on Loan Decision Algorithm Mis-prob. Algorithm Mis-prob. FSS Naive Bayes 0.1405 CN2 0.1600 NBTrees 0.1410 Naive-Bayes 0.1612 C4.5-auto 0.1446 Voted ID3 (0.8) 0.1647 IDTM(Decision table) 0.1446 T2 0.1687 HOODG/SOLAR 0.1482 1R 0.1954 C4.5 rules 0.1494 0.2035 OC1 0.1504 C4.5 0.1554 Nearest-Neighbor (3) Nearest-Neighbor (1) Pebls Voted ID3 (0.6) 0.1564 CIEF'2003 0.2142 Crashed Conclusion SOLAR is a new biologically inspired learning network organization. Efficient, general purpose, capable of large data sets, suitable for hardware implementation Used to solve financial and economic problems, prediction, identification, and decision making. CIEF'2003 Future work Applications to other fields Hardware implementation in FPGA – real time applications – modular and expandable structures Associative learning Temporal learning Stability issues Autonomous, goal driven behavior CIEF'2003 Questions ? CIEF'2003