Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download ANNs - WordPress.com
Document related concepts
Gene expression programming wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Concept learning wikipedia , lookup
Machine learning wikipedia , lookup
Pattern recognition wikipedia , lookup
Catastrophic interference wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Transcript
CLASSIFICATION: Artificial Neural Networks (ANNs) Inspired by neuroscience of the brain Neurons linked together by axons (strands of fiber) Axons transmit nerve impulses between neurons Dendrites connect neurons to axons of other neurons at synapses Learning happens through changes in synaptic connection strength CLASSIFICATION: Artificial Neural Networks (ANNs) CLASSIFICATION: Artificial Neural Networks (ANNs) Perceptron Invented at Cornell Aeronautical Laboratory in 1957 by Frank Rosenblatt Single layer feed-forward neural network Initially promising but ultimately disappointing – only able to learn linearly separable patterns Minsky and Papert extended to multi-layer perceptrons CLASSIFICATION: Artificial Neural Networks (ANNs) Perceptron Algorithm for learning binary classifier Function that maps input x to output f(x) given by ì1, f ( x) = í î0, if w × x + b > 0 else CLASSIFICATION: Artificial Neural Networks (ANNs) Basic types of ANNs Feedforward No directed cycles Multilayer perceptron Recurrent Directed cycles Often used for handwriting recognition CLASSIFICATION: Artificial Neural Networks (ANNs) Recurrent ANN examples Fully recurrent Long short term memory (Jürgen Schmidhuber) CLASSIFICATION: Artificial Neural Networks (ANNs) Recurrent ANN examples Hopfield (ECANs) Symmetric connections John Hopfield, 1982 Attractor network: dynamics guaranteed to converge Can function as associative memory CLASSIFICATION: Artificial Neural Networks (ANNs) Deep learning architectures Hierarchical temporal memory (Jeff Hawkins and Dileep George) Deep belief networks (George Hinton) Convolutional networks (Yann Lecun, Yoshua Bengio) Deep Spatiotemporal Inference Networks (Itamar Arel) Google Deepmind CLASSIFICATION: Artificial Neural Networks (ANNs) Basic learning mechanisms Supervised learning Infer mapping implied by the training data Gradient descent/Backpropagation CLASSIFICATION: Artificial Neural Networks (ANNs) Basic learning mechanisms Unsupervised learning Minimize some given cost/energy function Reinforcement learning Data generated by agent’s interactions with environment Agent observes accumulated costs and adjust actions accordingly CLASSIFICATION: Artificial Neural Networks (ANNs) Characteristics of ANNs Choice of model Depends upon application Complex models generally more difficult to learn Learning algorithm May require considerable experimentation to determine appropriate cost function and parameters CLASSIFICATION: Artificial Neural Networks (ANNs) Characteristics of ANNs Choice of threshold function ANNs can be robust Easily implemented in parallel Neuromorphic computing
