Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The Measure of a Man (Star Trek: The Next Generation) wikipedia , lookup
Genetic algorithm wikipedia , lookup
Neural modeling fields wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Fuzzy concept wikipedia , lookup
Gene expression programming wikipedia , lookup
Mathematical model wikipedia , lookup
Convolutional neural network wikipedia , lookup
Data (Star Trek) wikipedia , lookup
Fuzzy logic wikipedia , lookup
K-nearest neighbors algorithm wikipedia , lookup
Data-driven methods in Environmental Sciences Exploration of Artificial Intelligence Techniques [email protected] [email protected] 1 Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing [email protected] 2 What is AI? Machines that perceive, understand and react to their environment Goal of Babbage, etc. Oldest endeavor in computer science Machines that think Robots: factory floors, home vacuums Still quite impractical [email protected] 3 AI vs. humans AI applications built on Aristotlean logic Computers never as good as humans Induction, semantic queries, system of logic Human reasoning involves more than just induction In reasoning and making sense of data In obtaining a holistic view of a system Computers much better than humans In processing reams of data Performing complex calculations [email protected] 4 Successful AI applications Targeted tasks more amenable to automated methods Build special-purpose AI systems Determine appropriate dosage for a drug Classify cells as benign or cancerous Called “expert systems” Methodology based on expert reasoning Quick and objective ways to obtain answers [email protected] 5 Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing [email protected] 6 Fuzzy logic Fuzzy logic addresses key problem in expert systems How to represent domain knowledge Humans use imprecisely calibrated terms How to build decision trees on imprecise thresholds [email protected] 7 Fuzzy logic example Source: Matlab fuzzy logic toolbox tutorial http://www.mathworks.com/access/helpdesk/help/toolbox/fuzzy/fp350.html [email protected] 8 Advantages of fuzzy logic Considerable skill for little investment Fuzzy logic systems piggy bank on human analysis Humans encode rules after intelligent analysis of lots of data Verbal rules generated by humans are robust Simple to create Not much need for data or ground truth Logic tends to be easy to program Fuzzy rules are human understandable [email protected] 9 Where not to use fuzzy logic Do not use fuzzy logic if: Humans do not understand the system Different experts disagree Knowledge can not be expressed with verbal rules Gut instinct is involved Not just objective analysis A fuzzy logic system is limited Piece-wise linear approximation to a system Non-linear systems can not be approximated Many environment applications are non-linear [email protected] 10 Neural Networks Neural networks can approximate nonlinear systems Evidence-based Weights chosen through optimization procedure on known dataset (“training”) Works even if experts can’t verbalize their reasoning, or if there is ground truth [email protected] 11 A example neural network Diagram from: http://www.codeproject.com/useritems/GA_ANN_XOR.asp [email protected] 12 Advantages of neural networks Can approximate any smooth function Can yield true probabilities Training process is well understood Fast in operations If output node is a sigmoid node Not hard to train The three-layer neural network Training is slow, but once trained, the network can calculate the output for a set of inputs quite fast Easy to implement Just a sum of exponential functions [email protected] 13 Disadvantages of neural networks A black box Measure of skill needs to be differentiable The final set of weights yields no insights Magnitude of weights doesn’t mean much RMS error, etc. Can not use Probability of Detection, for example Training set has to be complete Unpredictable output on data unlike training Need lots of data Need expert willing to do lot of truthing [email protected] 14 Recap: Fuzzy logic Humans provide the rules Not optimal Neural network Humans can not understand system Optimal Middle ground? Genetic Algorithms Decision Trees [email protected] 15 Genetic algorithms In genetic algorithms One fixes the model (rule base, equations, class of functions, etc.) Optimize the parameters to model on training data set Use optimal set of parameters for unknown cases [email protected] 16 An example genetic algorithm Sources: http://tx.technion.ac.il/~edassau/web/genetic_algorithms.htm http://cswww.essex.ac.uk/research/NEC/ [email protected] 17 Advantages of genetic algorithms Near-optimal parameters for given model Human-understandable rules Best parameters for them Cost function need not be differentiable The process of training uses natural selection, not gradient descent Requires less data than a neural network Search space is more limited [email protected] 18 Disadvantages of genetic algorithms Highly dependent on class of functions If poor model is chosen, poor results Optimization may not help at all Known model does not always lead to better understanding Magnitude of weights, etc. may not be meaningful if inputs are correlated Problem may have multiple parametric solutions [email protected] 19 Decision trees Can automatically build decision trees from known data Root Prune trees 30 50 Select thresholds Choose operators Disadvantages T < 10C T > 10C Piece-wise linear, so typically less skilled than neural networks 20 15 10 35 Large decision trees are effectively a blackbox Can not do regression, only classification Z > 45 Z < 45 V<5 V>5 Advantages: 18 2 2 13 82 2 33 Fast to train New advances: bagged, boosted decision trees approach skill of neural networks, but are no longer fast to train [email protected] 20 Radial Basis Functions Diagram from: A. W. Jayawardena & D. Achela K. Fernando 1998: Use of Radial Basis Function Type Artificial Neural Networks for Runoff Simulation, Computer-Aided Civil and Infrastructure Engineering 13:2 Radial Basis Functions are a form of neural network Localized gaussians Linear sum of non-linear functions Advantage: Can be solved by inverting a matrix, so very fast Disadvantage: Not a general-enough model [email protected] 21 Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing [email protected] 22 Typical data-driven application Which features? Input Data How do we find f() Features AI application in run-time f(features) Result [email protected] 23 What is the role of the data? Validation Test known model Technique: Calibration Find parameters to model with desired structure Technique: Difference between model output and ground truth helps to validate the model Tuned fuzzy logic method Genetic algorithms Induction Find model and parameters from just data Technique: Neural network methods, bagged/boosted decision trees, support vector machines, etc. [email protected] 24 What is the problem to solve? Do you have a bunch of data and want to: Estimate an unknown parameter from it? Classify what the data correspond to? True rainfall based on radar observations? Amount of liquid content from in-situ measurements of temperature, pressure, etc? Regression A water surge? A temperature inversion? A boundary? Classification Regression and classification aren’t that different Classification: estimate probability of an event A function from 0-1 [email protected] 25 Which AI technique? Do you have expert knowledge? Humans have a “model” in their head? Should the final f() be understandable? Create fuzzy logic rules from experts’ reasoning Aggregate the individual fuzzy logic rules Can tune the fuzzy rules based on data Many times the original rules are just fine Do you already know the model? A power-law relationship? Gaussian? Quadratic? Rules? Just need to find parameters to this model? Using regression, decision trees or neural networks for RMS error criterion Genetic algorithms for error criteria like ROC, economic cost, etc. If linear, just use linear regression If non-linear: use genetic algorithms Use continuous GAs Both of these can be used for regression (therefore, also classification) [email protected] 26 Which AI technique (contd.) Do you know nothing about the data? Not the suspected equation/model (GA)? Not the suspected rules (fuzzy logic)? Use a AI technique that supplies its equations/rules For classification, use: Bagged decision trees or Support Vector Machines “black box”. If output is probabilistic, remember to apply Platt scaling Summary statistics on bagged DTs can help answer “why” Neural Networks For regression, use: Neural networks [email protected] 27 Where do your data come from? Observed data Compute features Choose AI technique The 4 choices in the previous two slides Simulated data: Example: trying to replicate a very complex model Throw randomly-generated data at model Compute features Choose AI technique: GA for parametric approximations NN when you don’t know how to approximate [email protected] 28 Where do you get your inputs? What type of data do you have? Individual observations? Sparse observations in a time series? Sample them (choose at random) and use directly Generate time-based features (1D moving windows) Signal processing features from time series Data from remotely sensed 2D grids? Generate image-based features using convolution filters Do you need: Pixel-based regression/classification? Use convolution features directly Object-based regression/classification? Identify regions using region growing Use region-aggregate features [email protected] 29 Typical data-driven application Observed data Signal/image processing;sampling Features normalize/create chromosome/ determine confidences f() FzLogic/GenAlg/NN/DecTree Platt method/region-average/threshold A data-driven application in run-time Result [email protected] 30 Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing [email protected] 31 Preprocessing Often can not use pixel data directly Different data sets may not be collocated Need to interpolate to line them up Mapping, objective analysis Noise in data may need to be reduced Too much data, too highly correlated May need to segment pixels into objects and use features computed on the objects Smoothing Present statistic of data, rather than data itself Features need to be extracted from data Human experts often good source of ideas on signatures to extract from data [email protected] 32 Postprocessing The output of an expert system may be grid point by grid point May need to provide output on objects Storms, forests, etc. Can average outputs over objects’ pixels May need probabilistic output Scale output of maximum marginal techniques Use a sigmoid function Called Platt scaling [email protected] 33 Summary What is Artificial Intelligence? Common AI techniques Fuzzy logic, neural networks, genetic algorithms, decision trees Choosing between AI techniques Data-driven methods to perform specific targeted tasks Understand the role of your data Do experts understand the system? (have a model) Do experts expect to understand the system? (readability) Pre and post processing Image processing techniques on spatial grids [email protected] 34