Download Machine Learning Introduction

Machine Learning
Introduction
Ingmar Schuster
Patrick Jähnichen
Institut für Informatik
This lecture covers
●
●
●
●
●
●
Machine Learning Overview
Example applications of
Machine Learning
Distinction Supervised vs.
Unsupervised
Beyond Supervised /
Unsupervised
Distinction Generative vs.
Discriminative Models
Basic Probability Theory
Machine Learning Introduction
2
Machine Learning: foundations and definitions
●
Definitions
●
●
●
“[Giving] computers the ability to learn without being explicitly programmed”
- Arthur Samuel
“A computer program is said to learn from experience E with respect to some
class of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E”
- Tom M. Mitchell
Common foundations
●
Probability Theory
●
Decision Theory (making “good” decisions based on data)
●
Optimization (optimize model of data based on decision objective)
independent of application domain
Machine Learning Introduction
3
Applications of Machine Learning
Machine Learning Introduction
4
Application: Panorama Stitching (using Markov Random Fields)
Machine Learning Introduction
5
Application: Brain Computer Interface (classification algorithm)
[youtu.be/jXpjRwPQC5Q]
Machine Learning Introduction
6
Application: Self-driving car
[youtu.be/cdgQpa1pUUE]
Machine Learning Introduction
7
Application Domains of Machine Learning
●
Computer Vision
●
Gene Analysis
●
Network security
●
Textmining & NLP: every domain
●
Parsing
●
POS-Tagging, Typing prediction
●
Modelling Language Semantics
●
Topic Models
Machine Learning Introduction
8
Supervised vs. Unsupervised
Machine Learning Introduction
9
Supervised vs. unsupervised learning
●
●
General distinction: Supervised
vs. Unsupervised Learning
Supervised Learning (given
expected answer, build
prediction model)
●
●
Classification
e.g. training data contains
tumor size, growth rate &
diagnosis
Regression
e.g. Training data contains
review text & product rating
Rating:
Rating: 88 (out
(out of
of 10)
10)
II really
really liked
liked the
the
picture.
picture.
The
The story
story of
of Johns
Johns
struggle
struggle against
against the
the
gremlins
gremlins touched
touched my
my
heart.
heart.
Machine Learning Introduction
10
Unsupervised learning
●
Unsupervised learning (find structure in data)
●
Clustering
Find which datapoints relate to one another
[Blei, Ng & Jordan 2003]
11
Supervised vs. unsupervised learning
●
Unsupervised learning (find structure in data)
●
Anomaly Detection
Find datapoints that strongly deviate from the majority
Possible
Possible
attacks
attacks
Regular
Regular
network
network
traffic
traffic
12
Supervised vs. unsupervised learning
●
Unsupervised learning (find structure in data)
●
Dimensionality Reduction, Source Separation
Find most important sources of variance in data
[cnl.salk.edu/~tewon/Blind/blind_audio.html]
13
Beyond
Supervised / Unsupervised
Machine Learning Introduction
14
Beyond supervised/unsupervised (1)
●
Active learning
(child constantly asking)
●
●
In between supervised
and unsupervised
Algorithm asks for labels
of only most important
data points
Machine Learning Introduction
15
Beyond supervised/unsupervised (2)
●
Reinforcement learning
(training a dog)
●
Algorithm is told when it
is right, but not why
Machine Learning Introduction
16
Beyond supervised/unsupervised (3)
●
Semi-Supervised learning &
Transfer learning
(you after college)
●
●
In between supervised
and unsupervised
Some data is labeled, but
most of it isn't
Machine Learning Introduction
17
Beyond supervised/unsupervised (4)
… and all kinds of remixes.
Machine Learning Introduction
18
Generative vs. Discriminative
Machine Learning Introduction
19
Discriminative Models
●
Discriminative Models
●
●
●
●
●
Smokes
Good if used for only one
specific prediction task
Models target distribution
directly
Doesn't model datagenerating process
Cancer
Xray
diagnosis
Breathing
difficulty
Can't produce
synthetic data samples
Good prediction
performance often
sufficient
Machine Learning Introduction
20
Generative Models
●
Generative models
●
●
●
●
Smokes
Models data-generating
process in real world
Enables generation of
synthetic data
Models joint distribution
of random variables
(often through distinct
conditional probabilities)
Cancer
Xray
diagnosis
Breathing
difficulty
Very flexible
Machine Learning Introduction
21
Basic Probability Theory
Machine Learning Introduction
22
●
●
●
●
Random variables can take different
values, written in uppercase letters
(
)
Values (events) for random variables
written in lowercase letters (
)
Events can be
●
Continuous or
●
Discrete
NameType
p(x)
person
0.4
location
0.3
other
0.3
… can be
●
Numbers
●
named values
●
Trees
●
any other type of object
Machine Learning Introduction
23
●
Probability of an event
or
●
Distribution of random
variable is function mapping
event to probability
NameType
p(x)
person
0.4
location
0.3
other
0.3
Machine Learning Introduction
24
Joint probability of two
events
●
Conditional probability
●
or
Smokes
Cancer
p(s,c)
Smokes
Cancer
p(c|s)
yes
yes
0.1
yes
yes
0.25
no
yes
0.06
no
yes
0.1
yes
no
0.3
yes
no
0.75
no
no
0.54
no
no
0.9
Machine Learning Introduction
25
This lecture covered
●
●
Probabilites sum to 1
Marginalization
Summing/Integrating out
random variable
S
C
p(s,c)
yes
yes
0.1
no
yes
0.06
yes
no
0.3
no
no
0.54
Machine Learning Introduction
26
This lecture covered
●
●
●
●
●
●
Machine Learning Overview
Example applications of
Machine Learning
Distinction Supervised vs.
Unsupervised
Beyond Supervised /
Unsupervised
Distinction Generative vs.
Discriminative Models
Basic Probability Theory
Machine Learning Introduction
27
Pictures
●
●
Tumor picture by flickr-user bc
the path, License CC SA NC
Other pictures from
openclipart.org, public domain
Machine Learning Introduction
28