* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Dr. Abeer Mahmoud - PNU-CS-AI
Artificial consciousness wikipedia , lookup
Neurocomputational speech processing wikipedia , lookup
Neuroeconomics wikipedia , lookup
Multielectrode array wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Neural oscillation wikipedia , lookup
Synaptogenesis wikipedia , lookup
Mirror neuron wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Neuroanatomy wikipedia , lookup
Molecular neuroscience wikipedia , lookup
Optogenetics wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Neurotransmitter wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Neural coding wikipedia , lookup
Chemical synapse wikipedia , lookup
Nonsynaptic plasticity wikipedia , lookup
Stimulus (physiology) wikipedia , lookup
Single-unit recording wikipedia , lookup
Central pattern generator wikipedia , lookup
Artificial intelligence wikipedia , lookup
Neural modeling fields wikipedia , lookup
Sparse distributed memory wikipedia , lookup
Pattern recognition wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Metastability in the brain wikipedia , lookup
Neural engineering wikipedia , lookup
Catastrophic interference wikipedia , lookup
Development of the nervous system wikipedia , lookup
Synaptic gating wikipedia , lookup
Artificial neural network wikipedia , lookup
Biological neuron model wikipedia , lookup
Convolutional neural network wikipedia , lookup
Nervous system network models wikipedia , lookup
Princess Nora University Faculty of Computer & Information Systems ARTIFICIAL INTELLIGENCE (CS 461D) Dr. Abeer Mahmoud Computer science Department Dr.Abeer Mahmoud (CHAPTER-18) LEARNING FROM EXAMPLES NEURAL NETWORK Dr.Abeer Mahmoud 3 Outlines 1. Biological Inspiration. 2. Artificial Neuron. 3. Artificial Neural Networks. 4. Applications of Artificial Neural Networks. 5. Artificial Neural Network Architectures. 6. Learning Processes. 7. Artificial Neural Networks Capabilities & Limitations Dr.Abeer Mahmoud 4 Biological Inspiration Dr.Abeer Mahmoud 5 Biological Inspiration Some numbers… The human brain contains about 10 billion nerve cells (neurons). Each neuron is connected to the others through 10000 synapses. Properties of the brain: It can learn, reorganize itself from experience. It adapts to the environment. Dr.Abeer Mahmoud 6 Biological Inspiration Animals are able to react adaptively to changes in their external and internal environment, and they use their nervous system to perform these behaviours. An appropriate model/simulation of the nervous system should be able to produce similar responses and behaviours in artificial systems. The nervous system is build by relatively simple units, the neurons, so copying their behaviour and functionality should be the solution. Dr.Abeer Mahmoud 7 The Neuron in Real Life The information transmission happens at the synapses. The neuron receives nerve impulses through its dendrites. It then sends the nerve impulses through its axon to the terminals where neurotransmitters are released to stimulate other neurons. Dr.Abeer Mahmoud 8 The Neuron in Real Life The unique components are: Cell body or Soma which contains the Nucleus. The Dendrites. The Axon. The Synapses. A neuron has: A branching input (dendrites). A branching output (the axon).. Dr.Abeer Mahmoud 9 The Neuron in Real Life The Dendrites: are short fibers (surrounding the cell body) that receive messages and carry signals from the synapses to the soma. The Axon: is a long extension from the soma that transmits messages, each neuron has only one axon. The axon carries action potentials from the soma to the synapses. The Synapses: are the connections made by an axon to another neuron. They are tiny gaps between axons and dendrites (with chemical bridges) that transmit messages. Dr.Abeer Mahmoud 10 Artificial Neuron Dr.Abeer Mahmoud 11 Artificial Neuron • What is an Artificial Neuron? • Definition: Neuron is the basic information processing unit of the Neural Networks (NN). It is a non linear, parameterized function with restricted output range. Dr.Abeer Mahmoud 12 Artificial Neural Networks Dr.Abeer Mahmoud 13 Artificial Neural Networks Artificial Neural Network (ANN): is a machine learning approach that models human brain and consists of a number of artificial neurons that are linked together according to a specific network architecture. Neuron in ANNs tend to have fewer connections than biological neurons. each neuron in ANN receives a number of inputs. An activation function is applied to these inputs which results in activation level of neuron (output value of the neuron). Knowledge about the learning task is given in the form of examples called training examples. Dr.Abeer Mahmoud 14 Applications of ANN Dr.Abeer Mahmoud 15 Applications of Artificial Neural Networks Some tasks to be solved by Artificial Neural Networks: Classification: Linear, non-linear. Recognition: Spoken words, Handwriting. Also recognizing a visual object: Face recognition. Controlling: Movements of a robot based on self perception and other information. Predicting: Where a moving object goes, when a robot wants to catch it. Optimization: Find the shortest path for the TSP. Dr.Abeer Mahmoud 16 Artificial Neural Networks Dr.Abeer Mahmoud 17 Artificial Neural Networks • Before using ANN, we have to define: 1. Artificial Neuron Model. 2. ANN Architecture. 3. Learning Mode. Dr.Abeer Mahmoud 18 Computing with Neural Units • Incoming signals to a unit are presented as inputs. • How do we generate outputs? • One idea: Summed Weighted Inputs. • Input: (3, 1, 0, -2) • Processing 3(0.3) + 1(-0.1) + 0(2.1) + -2(-1.1) = 0.9 + (-0.1) + 0 + 2.2 • Output: 3 Dr.Abeer Mahmoud 19 Activation Functions • Usually, do not just use weighted sum directly. • Apply some function to the weighted sum before it is used (e.g., as output). • Call this the activation function. Dr.Abeer Mahmoud 20 Activation Functions The choice of activation function determines the Neuron Model. Dr.Abeer Mahmoud 21 Bias of a Neuron • The bias b has the effect of applying a transformation to the weighted sum u v=u+b • The bias is an external parameter of the neuron. It can be modeled by adding an extra input. • v is called induced field of the neuron: Dr.Abeer Mahmoud 22 Example (1): Step Function Dr.Abeer Mahmoud 23 Example (2): Another Step Function Dr.Abeer Mahmoud 24 Example (3): Sigmoid Function The math of some neural nets requires that the activation function be continuously differentiable. A sigmoidal function often used to approximate the step function. Dr.Abeer Mahmoud 25 Dr.Abeer Mahmoud 26 Example (3): Sigmoid Function Dr.Abeer Mahmoud 27 Example • Calculate the output from the neuron below assuming a threshold of 0.5: Sum = (0.1 x 0.5) + (0.5 x 0.2) + (0.3 x 0.1) = 0.05 + 0.1 + 0.03 = 0.18 Since 0.18 is less than the threshold, the Output = 0 Repeat the above calculation assuming that the neuron has a sigmoid output function: Dr.Abeer Mahmoud 28 recognizing a simple pattern. • The total sum of the neuron is (1 x 1) + (0 x 0) + (1 x 1) + (0 x 0) = 2. So the neuron’s output is 1 and it recognizes the picture. • Even if there were some “interference” to the basic picture (some of the white pixels weren’t quite 0 and some of the shaded ones weren’t quite 1) the neuron would still recognize it, as long as the total sum was greater than 1.5. So the neuron is “Noise Tolerant” or able to “Generalize” which means it will still successfully recognize the picture, even if it isn’t perfect - this is one of the most important attributes of Neural Networks. Dr.Abeer Mahmoud 29 Example (cont) • Now the sum is (0 x 1) + (1 x 0) + (0 x 1) + (1 x 0) = 0 - the neuron won’t fire and doesn’t recognize this picture. Dr.Abeer Mahmoud 30 Show whether the network shown in the previous two slides would recognize this pattern: What advantage would using the sigmoid function give in this case.?? Dr.Abeer Mahmoud 31 Network Architecture Dr.Abeer Mahmoud 32 Network Architecture • The Architecture of a neural network is linked with the learning algorithm used to train. • There are different classes of network architecture: o Single-Layer Neural Networks. o Multi-Layer Neural Networks. o The number of layers and neurons depend on the specific task. Dr.Abeer Mahmoud 33 Single Layer Neural Network Dr.Abeer Mahmoud 34 Multi Layer Neural Network More general network architecture, where there are hidden layers between input and output layers. Hidden nodes do not directly receive inputs nor send outputs to the external environment. Multi Layer NN overcome the limitation of Single-Layer NN, they can handle non-linearly separable learning tasks. Dr.Abeer Mahmoud 35 Learning Dr.Abeer Mahmoud 36 Learning • Learning: the procedure that consists in estimating the parameters of neurons so that the whole network can perform a specific task. • Two types of learning: 1. The supervised learning. 2. The unsupervised learning. • Dr.Abeer Mahmoud 37 Supervised Learning The desired response of the neural network in function of particular inputs is well known. A “Teacher” may provide examples and teach the neural network how to fulfill a certain task. Teacher presents a number of inputs and their corresponding outputs to the ANN, See how closely the actual outputs match the desired ones. Modify the parameters to better approximate the desired outputs. ANN weights adjusted according to error. Dr.Abeer Mahmoud 38 Unsupervised Learning Main Idea: group typical input data in function of resemblance criteria, un-known a priori. Data Clustering. The network finds itself the correlations between the data. No need of a “Teacher”. Dr.Abeer Mahmoud 39 Learning If the output is correct then do nothing. If the output is too high, but should be low then decrease the weights attached to high inputs If the output is too low, but should be high then increase the weights attached to high inputs Dr.Abeer Mahmoud 40 ANN Capabilities & Limitations Dr.Abeer Mahmoud 41 Limitation Dr.Abeer Mahmoud 42 It is networks rather than single neurons which are used today and they are capable of recognizing many complex patterns. Dr.Abeer Mahmoud 43 Example of multilayer ANN • Calculate the output from this network assuming a Sigmoid Squashing Function. Dr.Abeer Mahmoud 44 • Try calculating the output of this network yourself. Dr.Abeer Mahmoud 45 Thank you End of Chapter 18 Dr.Abeer Mahmoud