Download Bayesian Estimation

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

page
1
page
2
page
3
page
4
page
5
page
6
page
7
page
8
page
9
page
10
page
11
page
12
page
13
page
14
page
15
page
16
page
17
Document related concepts
no text concepts found
Transcript 
INTRODUCTION TO
MACHINE LEARNING
Bayesian Estimation
Bayesian Estimation
2

Estimating parameters of a model from the data
 Regression
 Classification

Have some prior knowledge on possible parameter
range
 Before
looking at the data
 Distribution of the parameter
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Generative Model
3
Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)
Bayes Rule
4
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Multinomial variable
5



Sample of multinomial data taking one of K state
Sample Likelihood
Good way to specify prior distribution on state
probabilities q
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Dirichlet Distribution
6


Probability of each combination of state
probabilities
Parameters: approximate proportions of data in
state qi
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Posteriori
7

Likelihood

Posteriori
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Conjugate Prior
8


Posteriori and prior have the same form
Sequential learning
 Instance
by instance
 Calculate posteriori for the current item
 Make it prior for the next item
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Continuous Variable
9


Instances are Gaussian Distributed with unknown
parameters
Conjugate prior
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Continuous Variable
10
•Posteriori Mean is weighted combination of sample mean and
prior mean
•More samples, estimate is closer to m
•Little prior uncertainty=>closer to prior mean
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Precision/Variance Prior
11

More convenient to work with precision

Conjugate prior is a Gamma Distribution
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Precision
12

Posteriori is a weighted sum of prior and sample
statistics
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Parameter Estimation
13


Used prior to refine distribution parameter
estimates
User prior to refine parameter of some function of
the input
 Regression
 Classification
discriminant
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Regression
14
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Regression
15

Maximum Likelihood

Prediction

Gaussian Prior
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Prior on weights
16
Based on E Alpaydın 2004 Introduction to Machine Learning ©
The MIT Press (V1.1)
Examples
17
Related documents