Download Learning - WSU EECS

Learning
• Webster
– To gain knowledge or understanding of or skill in by study,
instruction or experience
– To memorize
– Synonym: discover
∗ To obtain knowledge of for the first time
∗ May imply acquiring knowledge with little effort or conscious intention (as by simply being told) or it may imply
study and practice
– Knowledge
∗ Knowing something with familiarity gained through experience or association
∗ Facts or ideas acquired by study, investigation, observation, or experience
• Deduction? (6!)
• Knowledge representation?
• Performance measure?
1
Machine Learning
• Simon
– Any process by which a system improves its performance
• Expert systems
– Acquisition of explicit knowledge
• Psychologists
– Skill acquisition
• Scientists
– Theory formation, hypothesis formation and inductive inference
2
Machine Learning: Motivations
• Automated knowledge engineering
– Expertise is scarce
– Codification of expertise is difficult
– Expertise frequently consists of a set of test cases
– Data from measurements, but no information or knowledge
• Only one computer has to learn, then copy
• Understand human learning
3
Machine Learning: Applications
• Speech recognition
• Object recognition
• Language learning
• Autonomous navigation
• Data mining
• Intelligent agents
• Cognitive modeling
4
History of Machine Learning
Exploration (1950s and 1960s)
• Neurophysiological
– Rosenblatt’s perceptron
• Biological
– Simulated evolution
• Psychological
– Symbol processing systems
• Statistical
– Control and pattern recognition
– Samuel’s checkers program
• Theoretical
– Gold’s identification in the limit
– Minsky and Papert’s criticism of the perceptron
5
History of Machine Learning
Development of practical algorithms (1970s)
• Winston’s ARCH
– Learned concept of a blocks-world arch
• Buchanan and Mitchell’s Meta-Dendral
– Learned mass-spectrometry prediction rules
• Michalski’s AQ11
– Learned soybean disease diagnosis rules
• Quinlan’s ID3
– Learned chess end-game rules
• Fikes, Hart and Nilsson’s MACROPS
– Learned macro-operators in blocks-world planning
• Lenat’s AM
– Discovered interesting mathematical concepts
6
History of Machine Learning
Explosion of research directions (1980s)
• Learning theory
• Symbolic learning algorithms
• Connectionist (neural network) learning algorithms
• Clustering and discovery
• Explanation-based learning
• Knowledge-guided inductive learning
• Analogical and case-based reasoning
• Genetic algorithms
7
History of Machine Learning
Maturity of the field (1990s)
• Statistical comparisons of algorithms
• Theoretical analyses of algorithms
• Machine learning = Data mining (?)
• Successful applications
• Multi-relational learning
• Ensemble and Kernel Methods
8
Mitchell’s Book
• Practical approach to study of machine learning
• Methodology snapshot (good one for 1997)
9