Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Perceptual control theory wikipedia, lookup

Convolutional neural network wikipedia, lookup

Gene expression programming wikipedia, lookup

Pattern recognition wikipedia, lookup

Neural modeling fields wikipedia, lookup

Catastrophic interference wikipedia, lookup

Hierarchical temporal memory wikipedia, lookup

Transcript

Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) X1 X2 X3 Y Input 1 1 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 1 1 0 0 1 0 X1 Black box Output X2 X3 Output Y is 1 if at least two of the three inputs are equal to 1. Y Artificial Neural Networks (ANN) X1 X2 X3 Y 1 1 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 1 1 0 0 1 0 Input nodes Black box X1 X2 X3 Output node 0.3 0.3 0.3 t=0.4 Y I (0.3 X 1 0.3 X 2 0.3 X 3 0.4 0) 1 where I ( z ) 0 if z is true otherwise Y Artificial Neural Networks (ANN) • Model is an assembly of inter-connected nodes and weighted links • Output node sums up each of its input value according to the weights of its links • Compare output node against some threshold t Input nodes Black box X1 Output node w1 w2 X2 Y w3 X3 t Perceptron Model Y I ( wi X i t ) i or Y sign ( wi X i t ) i General Structure of ANN x1 x2 x3 Input Layer x4 x5 Input I1 I2 Hidden Layer I3 Neuron i Output wi1 wi2 wi3 Si Activation function g(Si ) Oi threshold, t Output Layer Training ANN means learning the weights of the neurons y Oi Algorithm for learning ANN • Initialize the weights (w0, w1, …, wk) • Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples 2 – Objective function: E Yi f ( wi , X i ) i – Find the weights wi’s that minimize the above objective function • e.g., backpropagation algorithm Classification by Backpropagation • Backpropagation: A neural network learning algorithm • Started by psychologists and neurobiologists to develop and test computational analogues of neurons • A neural network: A set of connected input/output units where each connection has a weight associated with it • During the learning phase, the network learns by adjusting the weights so as to be able to predict the correct class label of the input tuples • Also referred to as connectionist learning due to the connections between units Neural Network as a Classifier • Weakness – Long training time – Require a number of parameters typically best determined empirically, e.g., the network topology or ``structure." – Poor interpretability: Difficult to interpret the symbolic meaning behind the learned weights and of ``hidden units" in the network • Strength – – – – – – High tolerance to noisy data Ability to classify untrained patterns Well-suited for continuous-valued inputs and outputs Successful on a wide array of real-world data Algorithms are inherently parallel Techniques have recently been developed for the extraction of rules from trained neural networks Backpropagation • Iteratively process a set of training tuples & compare the network's prediction with the actual known target value • For each training tuple, the weights are modified to minimize the mean squared error between the network's prediction and the actual target value • Modifications are made in the “backwards” direction: from the output layer, through each hidden layer down to the first hidden layer, hence “backpropagation” • Steps – – – – Initialize weights (to small random #s) and biases in the network Propagate the inputs forward (by applying activation function) Backpropagate the error (by updating weights and biases) Terminating condition (when error is very small, etc.) Software • • • • • Excel + VBA SPSS Clementine SQL Server Programming Other Statistic Package …