Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Philosophy of artificial intelligence wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Perceptual control theory wikipedia , lookup
Machine learning wikipedia , lookup
Pattern recognition wikipedia , lookup
Neural modeling fields wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Introduction to Intelligent Systems and Control Prof Kang Li Email: [email protected] Prof. K. Li Introduction 1 / 31 Topics Introduction to AI, IC, and artificial neural networks (1) Neural model and single-layer perceptrons (1) BP and Multilayer perceptron (MLP) (2) Radial basis function network (RBF) (1) Genetic algorithms (1) Meta-heuristic optimization methods (2) (1)= one lecture (2)=two lectures (3)=three lectures Prof. K. Li Introduction 2 / 31 Today’s Topics Getting to know about AI, CI and IC Biological neuron networks and artificial neural networks Representations of artificial neurons and artificial neural networks Prof. K. Li Introduction 3 / 31 What is Intelligence? The capacity to acquire and apply knowledge. The faculty of thought and reason. Superior powers of mind. Three Questions on AI 1. Can the operations of the brain be simulated on a digital computer? - yes, if the operation can be described clearly. 2. Is the mind a computer program? - No. Syntactical via semantic 3. Is the brain a digital computer? - ill-defined question Prof. K. Li Introduction 4 / 31 Computational Intelligence - Problem solving methods and approaches that mimic biologically intelligent behaviour As a successor of AI, computational intelligence combines elements of learning, adaptation, evolution and fuzzy logic (rough sets) to create programs that are, in some sense, intelligent. Properties - Flexible to changing environments and goals Learns from experience Make appropriate choices given perceptual limitations and finite computation Prof. K. Li Introduction 5 / 31 Taxonomy of Computational Intelligence Neural Networks Evolutionary Algorithms Genetic Algorithms Fuzzy Systems Genetic Programming AI parts for IC Prof. K. Li Introduction 6 / 31 Intelligent Control AI IC AC Op AI Prof. K. Li IC Fu, 1970 K.S. Fu, Learning Control Systems--Review and Outlook, IEEE Transactions on Automatic Control, 15, 210-221, April 1970. Saradis, 1979 AC Introduction Saridis, G. (1979). "Toward the Realization of Intelligent Controls", Proceedings of the IEEE, Vol. 67, No. 8, August 7 / 31 Intelligent control - the discipline where control methods are developed that attempt to emulate important characteristics of human intelligence: adaptation and learning, planning under large uncertainty, coping with large amounts of data. Intelligent system - to act appropriately in an uncertain environment, where an appropriate action is that which increases the probability of success. In order for a man-made intelligent system to act appropriately, it may emulate functions of living creatures and ultimately human mental faculties. Prof. K. Li Introduction 8 / 31 Classical control design Reference (Goal) Disturbances u Controller Process Modelling “System Identification” Controller Design and Synthesis Mathematical Model Prof. K. Li y Introduction Mathematical beauty 9 / 31 An example of Intelligent Control Reference (Goal) Disturbances Manual control/ Human operator u Human-in-loop y Process Knowledge acquisition If-then rules Reference (Goal) Design and Synthesis Prof. K. Li .Neuro /Fuzzy controller Introduction Disturbances u Process y 10 / 31 Biological neuron networks Motivation from humans brains: Are able to process complex task efficiently (perception, pattern recognition, reasoning etc.) Can learn from examples and generalise Adapt to new situations Robust and fault tolerant (neurons die) Prof. K. Li Introduction 11 / 25 Biological neuron networks Unit nerve cells called neurons; many different types and extremely complex; around 1011 neurons in the brain. Interaction signal conveyed by action potentials, interactions could be chemical (release or receive ions) or electrical. Each neuron contacts with around 103 other neurons. Structure feed-forward, feedback and self-activation recurrent Prof. K. Li Introduction 12 / 25 Nerve cells A neuron has three parts: Dendrites - receive information from another cell and transmit the message to the cell body. Cell body - contains the nucleus, mitochondria and other organelles typical of eukaryotic cells. Axon- conducts messages away from the cell body. Synapse: The junction between a nerve cell and another cell is called a synapse. Messages travel within the neuron as an electrical action potential. The space between two cells is known as the synaptic cleft. To cross the synaptic cleft requires the actions of neurotransmitters. Neurotransmitters are stored in small synaptic vessicles clustered at the tip of the axon. Prof. K. Li Introduction 13 / 25 The artificial neural networks A network with interactions, attempting to mimic the brain Unit: artificial neuron (linear or nonlinear input-output unit), much smaller numbers compared with brain. Interaction: strength of interaction between artificial neurons are determined by weights. Structure: Feed-forward, feedback or recurrent Prof. K. Li Introduction 14 / 25 Artificial Neural Networks (cont.) ANNs are obtained by connecting artificial neurons together in a layered structure. Networks can be either single or multilayered and be feedforward or recurrent. Hidden Layers Output Layer Inputs O u tpu ts Input Layer Weighted links Neuron Single layer perceptrons Multilayer feedforward network Prof. K. Li Introduction 15 / 25 Feedforward Neural Networks Feedforward neural networks consist of an input layer, one or more intermediate layers and an output layer. The inputs to each neuron, in a given layer, are the outputs of all the neurons in the previous layer. The network is termed feedforward because there are no intra-layer connections, and inter-layer connections are unidirectional. The input layer consists of dummy neurons which perform no function other than to distribute the network inputs to the next layer. The intermediate layers are referred to as hidden layers because they are not connected directly to either the inputs or outputs. Prof. K. Li Introduction 16 / 25 Recurrent Neural Networks (ANNs) Recurrent networks are characterised by having at least one delayed feedback loop. This leads to nonlinear dynamical behaviour D D Single layer recurrent network (D = unit delay block) = input nodes (dummy neurons) Prof. K. Li Introduction 17 / 25 Neural network learning Learning (training from data set, adaptation) change weights or interaction between neurons according to examples, previous knowledge ... The purpose of learning is to minimize Training errors on learning data - Training performance Prediction errors on new, unseen data Generalisation performance Prof. K. Li Introduction 18 / 25 Learning Methods 1. Supervised learning- Has a teacher, telling you how to learn. Technically each example categorized, or alternatively you receive feedback after each decision 2. Unsupervised Learning- Learn by itself. No feedback. The goal is to group data into similar groups. 3. Other Approaches: Reinforcement learning: have a critics, wrong or correct Prof. K. Li Introduction 19 / 25 Issues in using neural networks 1. Choose a neural network architecture – Feed-forward network – Recurrent network – Each network has its own characteristics Simple networks may not be able to solve a complex problem For example, perceptron can only solve linearly separable problems Prof. K. Li Introduction 20 / 25 Using Neural Networks – cont. 2. Specify the network architecture – How many layers – How many neurons at each layer – A specific connection pattern 3. Choose a learning algorithm – Specify the parameters of the learning algorithm – Decide how to train the network Prof. K. Li Introduction 21 / 25 Using Neural Networks – cont. 4. Testing – A trained network will be normally tested on data that are not used during training – A network’s ability to process outside training is called generalization Generalization is critical for practical applications Generalization will be discussed later in detail Prof. K. Li Introduction 22 / 25 Artificial neuron model (typical representations) Activation potential u 1 u1 un 1 u n T Synapse x w b w1 wn 1 wn T Dendrite Dendritic representation Synapse 1 u u 1 un Input layer Prof. K. Li y Summing node b w1 x y Signal flow graph wn Output layer Introduction 23 / 25 u n W w1 x y Block-diagram w u 1 1 T y ( w u b ) , w , u w u n n Single neuron have 5 components Input Prof. K. Li Weight Bias Activation function Introduction Output 24 / 25 Some typical activation functions Activation function can generate either unipolar or bipolar signals Prof. K. Li Introduction 25 / 25 Summary Getting to know about AI, CI and IC Biological neuron networks and artificial neural networks Representations of artificial neurons and artificial neural networks Prof. K. Li Introduction 31 / 25