Download Data Mining & Analysis

yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Nonlinear dimensionality reduction wikipedia, lookup

K-nearest neighbors algorithm wikipedia, lookup

K-means clustering wikipedia, lookup

Cluster analysis wikipedia, lookup

Machine Learning
Márk Horváth
Morgan Stanley
Institutional Securities
AI Paradigm
Data Mining
Application Areas
• Introduce many fields and the whole
– No time for details
AI Paradigm
• “The area of computer science which deals with
problems, that we where not able to cope with
– Computer science is a branch of mathematics, btw.
• “Algorithms solving problems mainly through
interaction with the problem. The programmer
does not have to understand the solution to the
problem itself, but only the details of the learning
AI Paradigm
• Why AI?
– new, fast expanding science, applicable at most of other
• it also deals with explaining evidence
– interdisciplinar
computer science
applied math
philosophy of science
biology (many naturally inspired algorithms, thinking machine)
• Why Machine Learning / Data Mining?
– it can be applied on any data (financial, medical, demographical,
AI Paradigm
1965 John McCarthy => 42 years
Hilbert, theorem proving machine
Occam (XIV.)
Many distinct fields
Many algorithms at each field
• => 1 hour is nothing….
Empirical and theoretical science
Intuition needed to use and hybridize
Few proves
Area too big to grasp everything in detail, but concepts are important
– => BIG PICTURE, no formulas!
AI Taxonomy
Logic /
Expert Sys
Machine Learning /
Optimization Control Clustering Model / AGI
Data Mining /
Function Approximation
Kernel Based /
Decision Tree / Linear Regression /
Naiive Bayes
0R, 1R
Nearest Neighbor Covering
Gradient Methods …
(max likelihood)
Data Mining vs. Statistics
• Statistics
– ~ hypothesis testing
• DM
– search through hypothesizes
• Empirical side
– Many methods work which are proven to not
– Some methods do not work while they should (due to
computation power problems, slow convergence)
Relation, Attribute, Class
(Ω, A, P)
X = MYCT x MMIN x MMAX x CACH x CHMIN x CHMAX (Attribute, Feature)
Y = class (Class, Target)
ρ( Y | X ) = ?
@relation 'cpu‘
@attribute MYCT real
@attribute MMIN real
@attribute MMAX real
@attribute CACH real
@attribute CHMIN real
@attribute CHMAX real
@attribute class real % performance
General View of Data Mining
• Language
• Build model / search over the Language
Simple Cases
• 0R
• 1R (nominal class)
• Max likelihood
• Linear Regression
Data Mining Taxonomy
• Regression vs. Classification
• Deterministic vs. Stochastic
(~exchangeable: Chebyshev)
• Batch driven vs. Updateable
(~exchangeable, but with cost)
• Symbolic vs. Subsymbolic
Clean data
Try many methods
Optimize good methods
Hybridize good methods, make meta
Evaluation Measures
• Mean Absolute Error / Root Mean Squared
• Correlation Coefficient
• Information gain
• Custom (e.g. weighted)
• Significance analysis (Bernoulli process)
Overfitting, Learning Noise
• Philosophical question
– When do we accept or deny a model?
– No chance to prove, only to reject
• Train / (Validation) / Test
• Cross-validation, leave one out
• Minimum Description Length principle
– Occam
– Kolmogorov complexity
Nearest Neighbor / Kernel
Instance based
Statistical (k neighbors)
Distance: Euclidian, Manhattan / Evolved
Missing Attribute: maximal distance
• KD-tree (log(n)), ball tree, metric tree
Decision Trees / Covering
• Divide and Conquer
• Split by the best feature
• User Classifier / REP Tree
Naiive Bayes
• Independent Attributes
• P(X | Y) = P(Y | X) * P(X) / P(Y) =
= Π P(Y | Xi) * P(X) / P(Y)
• Discrete Class
Artificial Neural Networks
• Structure (Weka)
– Theoretical limitations (Minsky, AI winter)
• Recurrent networks for time series
Feedforward Learning Rules
• Learning rules
– Perceptron / Winnow (very simple rules for special cases)
– Various gradient descent methods
• Slower than perceptron
• Faster than doing derivation of the whole expression
• Local search
– Evolution
• Global search
• Bit slower, but easy to hybridize with local search
• Can evolve:
Transfer functions
Recurrent networks
Perceptron / Winnow
• Perceptron
– Add the misclassified instance to the weight
– Converges if the space is separable
• Winnow
– Binary
– Increase or decrease non zero attribute
Feature extraction
Various state space transitions
Evolving features
Meta / Hybrid Methods
Vote (many ways)
Use meta algorithm to predict based on base methods
– Apply regression in the leaves of decision trees
– Embed decision tree, or training samples in ANN
• Unify
– Choose a general purpose language
– Use conventional training methods to build models
– Hybridize training methods, evolve
• Easy to write articles, countless new ideas
Practical Uses
• New paradigm
• Countless applications
• At all natural sciences
– finance, psychology, sociology, biology,
medicine, chemistry, …
– actually discovering and explaining evidence
is science itself
• Business
– predictive enterprise
Applications in AI
• Optimal Control (model building)
• Using in other AI methods
– Speech recognition
– Speech synthesis
– Vision, recognition
– AGI (logic, DM, evolution, clustering,
reinforcement learning, …)
TDK, Article
• Any topic you’ve found interesting…