Download Introduction to Machine Learning

Lecture Slides for
ETHEM ALPAYDIN
© The MIT Press, 2010
[email protected]
http://www.cmpe.boun.edu.tr/~ethem/i2ml2e
Why “Learn” ?
 Machine learning is programming computers to optimize a
performance criterion using example data or past experience.
 There is no need to “learn” to calculate payroll
 Learning is used when:
 Human expertise does not exist (navigating on Mars),
 Humans are unable to explain their expertise (speech recognition)
 Solution changes in time (routing on a computer network)
 Solution needs to be adapted to particular cases (user biometrics(生物
統計學))
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
3
What We Talk About When We
Talk About “Learning”
 Learning general models from a data of particular examples
 Data is cheap and abundant; knowledge is expensive and
scarce (不足).
 Example in retail (零售):
Customer transactions(交易) to consumer behavior:
People who bought “X” also bought “Y”
 Build a model that is a good and useful approximation to the
data.
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
4
What is Machine Learning?
 Optimize a performance criterion using example data or past
experience.
 Role of Statistics:
 Inference from a sample
 Role of Computer science:
 Efficient algorithms to


Solve the optimization problem
Representing and evaluating the model for inference
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
5
Applications
 Association(聯想)
 Auto-association
 Hetero-association
 Supervised Learning(監督式學習)
 Classification (Recognition)
 Regression
 Unsupervised Learning(非監督式學習)
 Clustering (Grouping)
 Reinforcement Learning
…
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
6
Learning Associations
 Basket analysis
 To find associations between products bought by customers
 Learning a conditional probability
 P (Y | X )
probability that somebody who buys X also buys Y
where X and Y are products/services.
 Example
 P ( chips | beer ) = 0.7
70 percent of customers who buy beer also buy chips.
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
7
Classification
 Example:
Credit scoring
 Differentiating between low-
risk and high-risk customers
from their income and savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
8
Classification: Applications
 Pattern recognition
 Character recognition: Different handwriting styles.
 Face recognition: Pose, lighting, occlusion, make-up, hair
style
 Speech recognition: Temporal dependency.

Use of a dictionary or the syntax of the language.

Sensor fusion: Combine multiple modalities; eg, visual (lip image)
and acoustic(聽覺的) for speech
 Medical diagnosis: From symptoms(症狀) to illnesses
 ...
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
9
Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UK
http://www.uk.research.att.com/facedatabase.html
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
10
Regression
 Example:
Price of a used car
 y = g (x | θ )
y : price
x : car attributes
e.g. milage (英里數)
g ( ): model
θ: parameters
e.g. age(年份)
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
y = wx+w0
11
Regression: Applications
 Navigating a car
 Angle of the steering wheel (CMU NavLab)
 Kinematics (運動學) of a robot arm
(x,y)
α2
α1= g1(x,y)
α2= g2(x,y)
α1
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
12
Supervised Learning: Uses
 Prediction of future cases:
 Use the rule to predict the output for future inputs
 Knowledge extraction:
 The rule is easy to understand
 Compression:
 The rule is simpler than the data it explains
 Outlier detection:
 Exceptions that are not covered by the rule
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
13
Unsupervised Learning
 Learning “what normally happens”
 The aim is to find the regularities in the input.
 Density estimation: we want to see what generally happens and what
does not.
 Clustering:
 Grouping similar instances
 Example applications
 Customer segmentation in CRM (customer relationship management)
 Image compression: Color quantization
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
14
Reinforcement Learning
 Learning a policy: A sequence of outputs
 The output of the system is a sequence of actions.
 An action is good if it is part of a good policy.
 No supervised output but delayed reward
 Examples:
 Game playing
 Robot in a maze
 Partial observability ...
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
15
Exercise
 What is learning?
 What is supervised learning? Unsupervised learning?
Reinforcement learning?
Explain them and give an example of each type of learning.
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
16
Resources: Journals
 Journal of Machine Learning Research (www.jmlr.org)
 Machine Learning
 Neural Computation
 Neural Networks
 IEEE Transactions on Neural Networks
 IEEE Transactions on Pattern Analysis and Machine Intelligence
 Annals of Statistics
 Journal of the American Statistical Association
 ...
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
17
Resources: Conferences








International Conference on Machine Learning (ICML)
European Conference on Machine Learning (ECML)
Neural Information Processing Systems (NIPS)
Uncertainty in Artificial Intelligence (UAI)
Computational Learning Theory (COLT)
International Joint Conference on Artificial Intelligence (IJCAI)
International Conference on Neural Networks (Europe)
...
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
18