Download Artificial Neural Networks

yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Neuropsychopharmacology wikipedia, lookup

Synaptic gating wikipedia, lookup

Nervous system network models wikipedia, lookup

Optogenetics wikipedia, lookup

Biological neuron model wikipedia, lookup

Neural coding wikipedia, lookup

Central pattern generator wikipedia, lookup

Neural oscillation wikipedia, lookup

Development of the nervous system wikipedia, lookup

Metastability in the brain wikipedia, lookup

Artificial general intelligence wikipedia, lookup

Convolutional neural network wikipedia, lookup

Pattern recognition wikipedia, lookup

Neural modeling fields wikipedia, lookup

Catastrophic interference wikipedia, lookup

Machine learning wikipedia, lookup

Holonomic brain theory wikipedia, lookup

Neural engineering wikipedia, lookup

Types of artificial neural networks wikipedia, lookup

Recurrent neural network wikipedia, lookup

Artificial neural network wikipedia, lookup

Neurocomputational speech processing wikipedia, lookup

Artificial intelligence wikipedia, lookup

Backpropagation wikipedia, lookup

Artificial Neural Networks
Rados Jovanovic
Biological Model of Neural Networks
Mathematical Model of Neural Networks
Building an Artificial Neural Network
Theoretical Properties
Applications of Artificial Neural Networks
Biological Model
• The brain is responsible for all processing and
• The brain consists of a complex network of cells
called neurons.
• Neurons communicate by transimitting
electrochemical signals throughout the network.
• Each input signal to a neuron can inhibit or excite
the neuron. When the neuron is excited enough,
it will fire its own electrochemical signal.
Biological Model (Cont.)
A neuron:
Mathematical Model
• An Artificial Neural Network is a network of
interconnected artificial neurons.
• Like in a biological neural network, artificial
neurons communicate by sending signals to
one another.
• Each input to an artificial neuron can either
inhibit or excite the artificial neuron.
Mathematical Model (Cont.)
An artificial neuron:
Building an Artificial Neural Network
• Topology of the network
• Learning type
• Learning algorithm
– Backpropagation
• Summary of backpropagation
Topology of the Network (Cont.)
There are many topologies, but the main
distinction can be maid between:
• Feed-Forward Neural Networks
• Recurrent Neural Networks
Feed-Forward Neural Networks
Feed-Forward Neural Networks have no
connections that loop.
Recurrent Neural Networks
Recurrent neural networks do contain looping
Building an Artificial Neural Network
• Topology of the network
• Learning type
• Learning algorithm
– Backpropagation
• Summary of backpropagation
Learning Type
• Supervised Learning
Requires a set of pairs of inputs and outputs to
train the artificial neural network on.
• Unsupervised Learning
Only requires inputs. Through time an ANN learns
to organize and cluster data by itself.
• Reinforcement Learning
An ANN from the given input produces some
output, and the ANN is rewarded or punished
based on the output it created.
Building an Artificial Neural Network
• Topology of the network
• Learning type
• Learning algorithm
– Backpropagation
• Summary of backpropagation
Learning Algorithm
Learning is adjustment of the weights of the
connections between neurons, according to
some modification rule.
• One of the more common algorithms for
supervised learning is Backpropagation.
• The term is an abbreviation for “backwards
propagation of errors"
• Backpropagation is most useful for feedforward networks.
Backpropagation (Cont.)
• An input is fed into the network and the
output is being calculated.
• We compare the output of the network with
the target output, and we get the error.
• We want to minimize the error, so we greedily
adjust the weights such that error for this
particular input will go towards zero.
• We do so using the delta rule.
Delta Rule
• The delta rule is a gradient descent learning rule.
• For a given neuron j and a weight i the delta rule
for its weight wji is:
tj is the target output
yj is the actual output
xi is the ith input
a is a small constant called the learning rate
Delta Rule (Cont.)
Delta Rule (Cont.)
Backpropagation (Cont.)
• The whole process is repeated for each of the
training cases, then back to the first case
• The cycle is repeated until the overall error
value drops below some pre-determined
• Backpropagation usually allows quick
convergence on satisfactory local minima for
Summary of Backpropagation
-While error over all training samples > threshold
-For each training sample:
-Present a training sample to the neural network.
-Compute the output of the network.
-Calculate the error for each neuron in the last layer by
comparing it to the target output.
-For each layer l starting from the last output layer until the
first layer
-Adjust the weights for each neuron in layer l to
decrease the local error.
-Propagate the error to each neuron one layer back
-end for
-end for
Theoretical Properties
Computational power
Theoretical Properties
• Computational power: The Cybenko theorem
proved single hidden layer, feed forward
neural network is capable of approximating
any continuous, multivariate function to any
desired degree of accuracy.
Theoretical Properties (Cont.)
• Capacity: It roughly corresponds to the neural
network’s ability to model any given function.
It is related to the amount of information that
can be stored in the network.
Theoretical Properties (Cont.)
• Generalisation: In applications where the goal
is to create a system that generalises well in
unseen examples, the problem of overtraining
has emerged.
• To lessen the overtraining cross validation is
Theoretical Properties (Cont.)
• Convergence: Not much can be said about
convergence since it depends on many factors,
such as the existence of local minima, choice
of optimization method, etc.
Application areas include:
system identification and control (vehicle control,
process control), quantum chemistry, gameplaying and decision making (backgammon,
chess, racing), pattern recognition (radar systems,
face identification, object recognition...),
sequence recognition (gesture, speech,
handwritten text recognition), medical diagnosis,
financial applications (automated trading
systems), data mining, visualization, e-mail spam