Download Artificial Neural Networks (ANN)

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Perceptual control theory wikipedia, lookup

Convolutional neural network wikipedia, lookup

Gene expression programming wikipedia, lookup

Pattern recognition wikipedia, lookup

Neural modeling fields wikipedia, lookup

Catastrophic interference wikipedia, lookup

Hierarchical temporal memory wikipedia, lookup

Machine learning wikipedia, lookup

Backpropagation wikipedia, lookup

Transcript
Artificial Neural Networks
(ANN)
Artificial Neural Networks (ANN)
X1
X2
X3
Y
Input
1
1
1
1
0
0
0
0
0
0
1
1
0
1
1
0
0
1
0
1
1
0
1
0
0
1
1
1
0
0
1
0
X1
Black box
Output
X2
X3
Output Y is 1 if at least two of the three inputs are equal to 1.
Y
Artificial Neural Networks (ANN)
X1
X2
X3
Y
1
1
1
1
0
0
0
0
0
0
1
1
0
1
1
0
0
1
0
1
1
0
1
0
0
1
1
1
0
0
1
0
Input
nodes
Black box
X1
X2
X3
Output
node
0.3
0.3
0.3

t=0.4
Y  I (0.3 X 1  0.3 X 2  0.3 X 3  0.4  0)
1
where I ( z )  
0
if z is true
otherwise
Y
Artificial Neural Networks (ANN)
• Model is an assembly of
inter-connected nodes
and weighted links
• Output node sums up
each of its input value
according to the weights
of its links
• Compare output node
against some threshold t
Input
nodes
Black box
X1
Output
node
w1
w2
X2

Y
w3
X3
t
Perceptron Model
Y  I (  wi X i  t )
i
or
Y  sign (  wi X i  t )
i
General Structure of ANN
x1
x2
x3
Input
Layer
x4
x5
Input
I1
I2
Hidden
Layer
I3
Neuron i
Output
wi1
wi2
wi3
Si
Activation
function
g(Si )
Oi
threshold, t
Output
Layer
Training ANN means learning
the weights of the neurons
y
Oi
Algorithm for learning ANN
• Initialize the weights (w0, w1, …, wk)
• Adjust the weights in such a way that the
output of ANN is consistent with class
labels of training examples
2
– Objective function: E  Yi  f ( wi , X i )
i
– Find the weights wi’s that minimize the above
objective function
• e.g., backpropagation algorithm
Classification by
Backpropagation
• Backpropagation: A neural network learning algorithm
• Started by psychologists and neurobiologists to develop
and test computational analogues of neurons
• A neural network: A set of connected input/output units
where each connection has a weight associated with it
• During the learning phase, the network learns by
adjusting the weights so as to be able to predict the
correct class label of the input tuples
• Also referred to as connectionist learning due to the
connections between units
Neural Network as a Classifier
• Weakness
– Long training time
– Require a number of parameters typically best determined
empirically, e.g., the network topology or ``structure."
– Poor interpretability: Difficult to interpret the symbolic meaning
behind the learned weights and of ``hidden units" in the network
• Strength
–
–
–
–
–
–
High tolerance to noisy data
Ability to classify untrained patterns
Well-suited for continuous-valued inputs and outputs
Successful on a wide array of real-world data
Algorithms are inherently parallel
Techniques have recently been developed for the extraction of
rules from trained neural networks
Backpropagation
• Iteratively process a set of training tuples & compare the
network's prediction with the actual known target value
• For each training tuple, the weights are modified to
minimize the mean squared error between the network's
prediction and the actual target value
• Modifications are made in the “backwards” direction: from
the output layer, through each hidden layer down to the
first hidden layer, hence “backpropagation”
• Steps
–
–
–
–
Initialize weights (to small random #s) and biases in the network
Propagate the inputs forward (by applying activation function)
Backpropagate the error (by updating weights and biases)
Terminating condition (when error is very small, etc.)
Software
•
•
•
•
•
Excel + VBA
SPSS Clementine
SQL Server
Programming
Other Statistic Package …