* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Introduction to Neural Networks
Molecular neuroscience wikipedia , lookup
Caridoid escape reaction wikipedia , lookup
Synaptogenesis wikipedia , lookup
Neuroesthetics wikipedia , lookup
Cortical cooling wikipedia , lookup
Mirror neuron wikipedia , lookup
Binding problem wikipedia , lookup
Artificial intelligence wikipedia , lookup
Multielectrode array wikipedia , lookup
Neurocomputational speech processing wikipedia , lookup
Neuroinformatics wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Neuroethology wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Neuroeconomics wikipedia , lookup
Pattern recognition wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
Mind uploading wikipedia , lookup
Biological neuron model wikipedia , lookup
Circumventricular organs wikipedia , lookup
Pre-Bötzinger complex wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neural coding wikipedia , lookup
Neural oscillation wikipedia , lookup
Catastrophic interference wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Neural modeling fields wikipedia , lookup
Synaptic gating wikipedia , lookup
Central pattern generator wikipedia , lookup
Optogenetics wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Metastability in the brain wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Convolutional neural network wikipedia , lookup
Neural binding wikipedia , lookup
Artificial neural network wikipedia , lookup
Neural engineering wikipedia , lookup
Development of the nervous system wikipedia , lookup
Nervous system network models wikipedia , lookup
IE 585 Introduction to Neural Networks Modeling Continuum High Cost / Low Error Low Cost / High Error Unarticulated Wisdom Articulated Qualitative Models Empirical Categorical Models Empirical Continuous Models Theoretic (First Principles) Models 2 Rise of Empirical Models • • • • Sensoring - lots of data Fast computing Computing available on site More complicated systems - do not adhere to simple models • Easy to use software 3 Typical Empirical Models • • • • linear regression splines nearest neighbor clustering neural networks 4 What is a Neural Net? • An NN is a network of many simple processors (“units, neurons”), each possibly having a small amount of local memory. The units are connected by communication channels (“connections”) which usually carry numeric data, encoded by any of various means. The units operate only on their local data and on the inputs they receive via the connections. Usenet newsgroup comp.ai.neural-nets 5 What is a Neural Net? • An NN is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: 1. Knowledge is acquired by the network through a learning process. 2. Interneuron connection strengths known as synaptic weights are used to store the knowledge. Haykin (1994) 6 Objectives of Neural Nets • • • • High Computing Speed Large Memory Capacity Adaptive Learning Fault Tolerance 7 Neural Network Predictive Models - Advantages • Can accommodate non-linear relationships with interactions among variables • Generalize well even for noisy and imprecise data • No assumption of analytical function or theoretic relation needed • User friendly software available • Computationally very fast, once built 8 Neural Network Predictive Models - Disadvantages • Strongly data dependent • No statistical interpretation of significance or confidence • Difficult to build and validate properly - too many choices, too little general guidance, misleading validation results 9 How Do Neural Networks Work? • Inspired by the biological brain • Consist of small, but numerous, parallel elements - weighted connections (synapses) and summing nodes (neurons) • “Learn” relationships through repeated calculations called “training” • Remain fixed after training to be applied to new data 10 Biological Neuron 11 How are Signals Transmitted? 12 Elements of Neural Networks x1 w1 w2 x2 . . . xn y y = f ( wixi ) i wn y Simple Summing Node (Neuron) 1 0.5 0 wi xi Non-linear Transfer Function 13 Typical Neural Network Error Feedback Weighted Synapses During Training I N P U Output Layer Neural Network Output T S Hidden Layer Input Layer 14 Terminology • Neurons / nodes / units / cells / processing elements (PEs) • Transfer / activation function • Connections / links / synapses • Weights / bias (fixed input of 1) • Feedforward / feedback • Input / output vectors / patterns • Self organizing (unsupervised) / supervised • Training / testing data sets 15 Biological vs Artificial Neural Networks • Biological neurons are all excitatory (positive) or inhibitory (negative) - ANN neurons can be mixed • Biological neurons operate asynchronously - ANN neurons usually synchronize by layer • Biological neurons transmit signals at varying rates but ANN use a single rate 16 Biological vs Artificial Neural Networks • There are many specialized biological neurons - ANN neurons tend to be generic • Biological neurons work through chemical / electrical transmission (“wet ware”) • Biological neurons are much slower but there are many, many more of them (~ 1011 neurons with 104 synapses per neuron!) 17 Types of Neural Nets • • • • Supervised Unsupervised Associate Optimization 18 Common Neural Net Applications • Pattern classification / recall – medical – defense – manufacturing quality – machine vision / postal – speech recognition – security detection – noise removal 19 Common Neural Net Applications • Clustering / compression – data mining – signal processing – space exploration applications – speech recognition 20 Common Neural Net Applications • Prediction / simulation – financial / stock market – music composition – utility usage – fault / degradation detection – sunspots 21 Common Neural Net Applications • Control - real time / on line – robots – vehicles – manufacturing • Control - off line – batch manufacturing – process optimization 22 Common Neural Net Applications • Optimization – traveling salesman – routing – scheduling – facility location 23 Cool Neural Net Web Sites http://www.csse.monash.edu.au/~app/CSE5301/index.html Detailed class notes and some matlab code. http://www.geocities.com/CapeCanaveral/1624/ C source code for lots of neural nets. 24