Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Perceptual control theory wikipedia , lookup
Machine learning wikipedia , lookup
Neural modeling fields wikipedia , lookup
Gene expression programming wikipedia , lookup
Pattern recognition wikipedia , lookup
Backpropagation wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) X1 X2 X3 Y Input 1 1 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 1 1 0 0 1 0 X1 Black box Output X2 X3 Output Y is 1 if at least two of the three inputs are equal to 1. Y Artificial Neural Networks (ANN) X1 X2 X3 Y 1 1 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 1 1 0 0 1 0 Input nodes Black box X1 X2 X3 Output node 0.3 0.3 0.3 t=0.4 Y I (0.3 X 1 0.3 X 2 0.3 X 3 0.4 0) 1 where I ( z ) 0 if z is true otherwise Y Artificial Neural Networks (ANN) • Model is an assembly of inter-connected nodes and weighted links • Output node sums up each of its input value according to the weights of its links • Compare output node against some threshold t Input nodes Black box X1 Output node w1 w2 X2 Y w3 X3 t Perceptron Model Y I ( wi X i t ) i or Y sign ( wi X i t ) i General Structure of ANN x1 x2 x3 Input Layer x4 x5 Input I1 I2 Hidden Layer I3 Neuron i Output wi1 wi2 wi3 Si Activation function g(Si ) Oi threshold, t Output Layer Training ANN means learning the weights of the neurons y Oi Algorithm for learning ANN • Initialize the weights (w0, w1, …, wk) • Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples 2 – Objective function: E Yi f ( wi , X i ) i – Find the weights wi’s that minimize the above objective function • e.g., backpropagation algorithm Classification by Backpropagation • Backpropagation: A neural network learning algorithm • Started by psychologists and neurobiologists to develop and test computational analogues of neurons • A neural network: A set of connected input/output units where each connection has a weight associated with it • During the learning phase, the network learns by adjusting the weights so as to be able to predict the correct class label of the input tuples • Also referred to as connectionist learning due to the connections between units Neural Network as a Classifier • Weakness – Long training time – Require a number of parameters typically best determined empirically, e.g., the network topology or ``structure." – Poor interpretability: Difficult to interpret the symbolic meaning behind the learned weights and of ``hidden units" in the network • Strength – – – – – – High tolerance to noisy data Ability to classify untrained patterns Well-suited for continuous-valued inputs and outputs Successful on a wide array of real-world data Algorithms are inherently parallel Techniques have recently been developed for the extraction of rules from trained neural networks Backpropagation • Iteratively process a set of training tuples & compare the network's prediction with the actual known target value • For each training tuple, the weights are modified to minimize the mean squared error between the network's prediction and the actual target value • Modifications are made in the “backwards” direction: from the output layer, through each hidden layer down to the first hidden layer, hence “backpropagation” • Steps – – – – Initialize weights (to small random #s) and biases in the network Propagate the inputs forward (by applying activation function) Backpropagate the error (by updating weights and biases) Terminating condition (when error is very small, etc.) Software • • • • • Excel + VBA SPSS Clementine SQL Server Programming Other Statistic Package …