Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Psychoneuroimmunology wikipedia , lookup
Central pattern generator wikipedia , lookup
Nervous system network models wikipedia , lookup
Catastrophic interference wikipedia , lookup
Adaptive collaborative control wikipedia , lookup
Convolutional neural network wikipedia , lookup
Radial Basis Networks: An Implementation of Adaptive Centers Nivas Durairaj ECE539 Final Project Brief Description of RBF Networks • Consists of 3 layers (input, hidden, output) • Input layer made up of nodes that connect network to environment • At input of each neuron (hidden layer), distance between neuron center & input vector is calculated • Apply RBF (Gaussian bell function) to form output of the neurons. • Output layer is linear and supplies response of network to activation function. Project Overview Purpose: Develop a Radial Basis Network with a supervised selection of centers A RBF network with multiple outputs Question: Are there any disadvantages or advantages between a fixed center RBF network and an adaptive RBF network? Adaptation Formulas RBF with supervised selection of centers require the following formulas: 1. Linear Weights (output layer) E (n) wi (n 1) wi (n) 1 wi (n) 2. Positions of centers (hidden layer) t i (n 1) t i (n) 2 E (n) t i (n) 3. Spreads of centers (hidden layer) i1 (n 1) i1 (n) 3 E (n) i1 (n) W: 1x1 T: 1xm vector i1 : mxm matrix M is the feature dimension Programming • Used Matlab to implement RBF Network with Adaptive Centers • Sample code for calculation of linear weights given below: wi (n 1) wi (n) 1 E (n) wi (n) %Calculation of linear weights weightdiff=0; for j=1:n g=exp(-0.5((x(j,:)-t(i,:)))*covinv(:,:,i)*((x(j,:)-t(i,:))')); weightdiff = weightdiff + e(j)*g; end w(i)=w(i) - (eta1*weightdiff); Testing & Comparison • Tested Adaptive Center RBF against Fixed Center RBF. • Used data for three functions, namely sinusoidal, piecewise-linear, and polynomial functions. • Made use of the cost function given below analyze differences between two networks Cost Function 1N 2 E e j 2 j 1 e j d j F * ( x j ) where M d j wi G ( x j t i i 1 Ci ) Sinusoidal Function Testing For fewer radial basis functions, adaptive center RBF network seems to perform a bit better. However, after number of RBFs increase, results in cost function are negligible. Sinosoid Function Data RBF with Adaptive Centers 1 test samples approximated curve train samples radial basis 0.6 0.5 Cost Function Output 0.5 0 -0.5 0.4 Fixed Center RBF Network 0.3 Adaptive Center RBF Network 0.2 0.1 -1 0 2 -1.5 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 3 4 5 6 No. of Radial Basis Functions 7 Piecewise Linear Function Testing Adaptive center RBF network performed better till the number of radial basis functions reached 6. I found that at higher numbers of radial basis functions (9 and above), both RBF networks were providing similar approximations of piecewise-linear function. Piecewise-Linear Function Data Chart RBF with Adaptive Centers 1.5 0.0045 1 0.004 0.5 Cost Function Output 0.0035 0 -0.5 -1 -1.5 test samples approximated curve train samples radial basis 0.003 0.0025 Fixed Center RBF Network Adaptive Center RBF Network 0.002 0.0015 0.001 0.0005 0 -2 2 -2.5 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 3 4 5 6 7 8 9 No. Of Radial Basis Functions 10 Polynomial Function Testing The adaptive center RBF network was clearly the winner in the approximation of the polynomial function. Differences in cost function for higher numbers of RBFs were too small for Excel to plot. Polynomial Function Data Chart RBF with Adaptive Centers 0.1 8.00E-04 0.08 7.00E-04 Cost Function Outputs 0.06 0.04 0.02 0 test samples approximated curve train samples radial basis -0.02 -0.04 5.00E-04 Fixed Center RBF Network 4.00E-04 Adaptive Center RBF Network 3.00E-04 2.00E-04 1.00E-04 0.00E+00 -0.06 -0.08 -0.5 6.00E-04 2 3 4 5 No. of Radial Basis Functions -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 6 Conclusion • Results show RBF network with adaptive centers performs slightly better than fixed-center RBF. • Advantage of Adaptive RBF: Performs better with fewer RBFs • Disadvantage of Adaptive RBF: Takes longer to run. • Unless situation is known, one cannot say with certainty that one model is better than other.