* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download CS-485: Capstone in Computer Science
Human multitasking wikipedia , lookup
Feature detection (nervous system) wikipedia , lookup
State-dependent memory wikipedia , lookup
Cognitive neuroscience wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Brain Rules wikipedia , lookup
Neurophilosophy wikipedia , lookup
Biological neuron model wikipedia , lookup
Neural coding wikipedia , lookup
Neural modeling fields wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Computer vision wikipedia , lookup
Neuroesthetics wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Neural oscillation wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Optogenetics wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Neuroinformatics wikipedia , lookup
Machine learning wikipedia , lookup
Synaptic gating wikipedia , lookup
Neuroeconomics wikipedia , lookup
Mind uploading wikipedia , lookup
Neuroanatomy wikipedia , lookup
Central pattern generator wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Pattern recognition wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Artificial intelligence wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Neural engineering wikipedia , lookup
Convolutional neural network wikipedia , lookup
Neural binding wikipedia , lookup
Development of the nervous system wikipedia , lookup
Artificial neural network wikipedia , lookup
Metastability in the brain wikipedia , lookup
Nervous system network models wikipedia , lookup
CS-485: Capstone in Computer Science Artificial Neural Networks and their application in Intelligent Image Processing Spring 2010 1 Organizational Details • Class Meeting: 12:25-3:45pm Tuesday, SCIT213 • Class webpage http://www.eagle.tamut.edu/faculty/igor/CS-485.htm • Instructor: Dr. Igor Aizenberg • Office: Science and Technology Building, 104C • Phone (903 334 6654) • e-mail: [email protected] • Office hours: • Monday, Thursday 10-30 – 6-30 • Tuesday, Wednesday 4-30 – 6-30 2 Text Book 1) I. Aizenberg, “Advances in Neural Networks”, University of Dortmund, 2005, Class notes (available from the class webpage) 2) Additional materials will also be available from the class webpage 3 Introduction to Neural Networks Artificial Intellect: Who is stronger and why? Applied Problems: •Image, Sound, and Pattern recognition •Decision making Knowledge discovery Context-Dependent Analysis … NEUROINFORMATICS - modern theory about principles and new mathematical models of information processing, which based on the biological prototypes and mechanisms of human brain activities 4 Applied Problems Natural language understanding (Translation of the texts) Decision Making Recognition of Images Learning and Adaptation Reasoning and Prediction Knowledge Discovery Cognitive analysis Team behavior Fuzzy Logic 5 The History of Neuroscience Renaissance of connectionism from the papers by Hopfield, and popularizing the back-propagation algorithm for multiplayer feedforward networks End of Perceptron era: Work “Perceptron” by Minsky and Papert Frank Rosenblatt invented the modern “perceptron” style of NN, composed of trainable threshold units 1949 1948 1943 1969 1957 1952 1951 1982 Ashby puts the idea that intelligence could be created by the use of “homeostatic” devices which learn through a kind of exhaustive search Minsky’s builts the first actual neural network learning system Hebb hypothesis that human and animal long-term memory is mediated by permanent alterations in the synapses. Notion of Wiener about key role of connectionism and feedback loops as a model for learning in neural networks McCulloch and Pitts introduced the fundamental ideas of analyzing neural activity via thresholds and weighted sums 6 ANN as a Brain-Like Computer NN as an model of brain-like Computer Brain The human brain is still not well understood and indeed its behavior is very complex! There are about 10 billion neurons in the human cortex and 60 trillion synapses of connections The brain is a highly complex, nonlinear and parallel computer (information-processing system) An artificial neural network (ANN) is a massively parallel distributed processor that has a natural propensity for storing experimental knowledge and making it available for use. It means that: Knowledge is acquired by the network through a learning (training) process; The strength of the interconnections between neurons is implemented by means of the synaptic weights used to store the knowledge. The learning process is a procedure of the adapting the weights with a learning algorithm in order to capture the knowledge. On more mathematically, the aim of the learning process is to map a given relation between inputs and output (outputs) of the 7 network. Intelligent Data Analysis in Engineering Experiment Interpretation and Decision Making Data Analysis Data Acquisition Signals & parameters Data Acquisition Rules & Knowledge Productions Characteristics & Estimations Adaptive Machine Learning via Neural Network Data Analysis Decision Making Knowledge Base 8 Mathematical Interpretation of Classification in Decision Making 1. Quantization of pattern space into p decision classes xi n f : n p F f t yi p m1 m2 mp m3 2. Mathematical model of quantization: Input Patterns x11 1 x xi 2 1 xn “Learning by Examples” Response: y11 1 y yi 2 1 yn 9 Learning via Self-Organization Principle Self-organization – basic principle of learning: Structure reconstruction Responce Input Images The learning involves change of structure Neuroprocessor Learning Rule Teacher 10 Applications of Artificial Neural Networks Advance Robotics Machine Vision Intelligent Control Technical Diagnistics Intelligent Data Analysis and Signal Processing Artificial Intellect with Neural Networks Image & Pattern Recognition Intelligentl Medicine Devices Intelligent Security Systems Intelligent Expert Systems 11 What we will learn and do? Artificial Neural Networks And Its Applications You will learn: Contemporary theoretical principles and paradigms of Neuroinformatics, Mathematical models and algorithms of neural network techniques for experimentation, Applications of Neuroinformatics to engineering and sciences problems, Computer-Aided Technology for Instrumentation Theory Practice Self-Paced Work 12 What we will learn and do? • General principles of artificial neural networks • General principles of learning algorithms • Feedforward neural network and backpropagation learning • Multi-valued neurons and a feedforward neural network based on multi-valued neurons • Basic ideas of image processing • Edge detection on noisy images using a neural network 13 Symbol Manipulation or Pattern Recognition ? Ill-Formalizable Tasks: •Sound and Pattern recognition •Decision making •Knowledge discovery •Context-Dependent Analysis What is difference between human brain and traditional computer via specific approaches to solution of illformalizing tasks (those tasks that can not be formalized directly)? Symbol manipulation Which way of imagination is best for you ? Pattern recognition Dove flies Lion goes Tortoise scrawls Donkey sits Shark swims 14 Principles of Brain Processing How our brain manipulates with patterns ? A process of pattern recognition and pattern manipulation is based on: Massive parallelism Connectionism Brain computer as an information or signal processing system, is composed of a large number of a simple processing elements, called neurons. These neurons are interconnected by numerous direct links, which are called connection, and cooperate which other to perform a parallel distributed processing (PDP) in order to soft a desired computation tasks. Brain computer is a highly interconnected neurons system in such a way that the state of one neuron affects the potential of the large number of other neurons which are connected according to weights or strength. The key idea of such principle is the functional capacity of biological neural nets determs mostly not so of a single neuron but of its connections Associative distributed memory Storage of information in a brain is supposed to be concentrated in synaptic connections of brain neural network, or more precisely, in the pattern of these connections and strengths (weights) of the synaptic connections. 15 Brain-like Computer Artificial Neural Network – Mathematical Paradigms of Brain-Like Computer The new paradigm of computing mathematics consists of the combination of such artificial Neurons and Neural Net neurons into some artificial neuron net. Brain-Like Computer Brain-like computer – is a mathematical model of humane-brain principles of computations. This computer consists of those elements which can be called the biological neuron prototypes, which are interconnected by direct links called connections and which cooperate to perform parallel distributed processing (PDP) in order to solve a desired computational task. ? 16 Principles of Neurocomputing Connectionizm NN is a highly interconnected structure in such a way that the state of one neuron affects the potential of the large number of another neurons to which it is connected accordiny to weights of connections Not Programming but Training NN is trained rather than programmed to perform the given task since it is difficult to separate the hardware and software in the structure. We program not solution of tasks but ability of learning to solve the tasks w11 w 11 w11 w11 w11 w11 w11 w11 w11 w11 w11 w11 w11 w11 w11 w11 Distributed Memory NN presents an distributed memory so that changing-adaptation of synapse can take place everywhere in the structure of the network. 17 Principles of Neurocomputing Learning and Adaptation NN are capable to adapt themselves (the synapses connections between units) to special environmental conditions by changing their structure or strengths connections. y x 2 Non-Linear Functionality Every new states of a neuron is a nonlinear function of the input pattern created by the firing nonlinear activity of the other neurons. Robustness of Assosiativity NN states are characterized by high robustness or insensitivity to noisy and fuzzy of input data owing to use of a highly redundance distributed structure 18