Download Bayesian Estimation - Department of Business Administration

Stochastic Models
Bayesian Estimation
Walt Pohl
Universität Zürich
Department of Business Administration
March 31, 2015
Bayesian Estimation
Bayesian estimation is an likelihood-based alternative to
maximum likelihood estimation.
Imagine each parameter is a random variable,
chosen once at the beginning of time.
We don’t know the value of the variable, but we
know its probability distribution, the prior.
Using the data, we can infer a new distribution, the
posterior. We use the posterior to produce
estimates.
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
2/1
Conditional probability
Suppose x is a draw from a random variable that depends
on a parameter, α. Let p(x|α) be the PDF. We think of
p as being the conditional probability of x given α.
We assume α is a random variable, with PDF π(α).
The joint distribution of the two variables is
p(x|α)π(α).
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
3/1
The Posterior
We don’t observe α, but we can determine its
conditional distribution, by Bayes Rule,
p(α|x) = R
p(x|α)π(α)
.
p(x|α)π(α)dα
This distribution is the posterior distribution of α, given
x.
(Note that the denominator is just a normalizing
constant – frequently we don’t even need to calculate it.)
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
4/1
What is the Prior?
If α really was a random variable, then p(α|x) would be
the distribution of α, given that we observed x. But α is
a parameter. Where does the prior come from?
The prior is an example of subjective probability. It
represents your subjective belief in the likelihood of some
outcomes.
For long samples, the posterior gets narrower and
narrower, and converges on the MLE.
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
5/1
We Can We Do With The Posterior?
The posterior mean provides a single point estimate,
which also converges to the MLE.
So how do we compute the posterior mean? The most
general method: simulate draws from the posterior, and
then compute the mean.
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
6/1
Bayesian Estimation for the EWMA Model
We use a “flat” prior: we consider every possible value of
λ between 0 and 1 equally likely,
π(λ) = 1.
If p(S1 , . . . , SN |λ) is the PDF of the EWMA model,
then the posterior for λ is just
R
p(S1 , . . . , SN |λ)
.
p(S1 , . . . Sn |λ)dλ
How do we simulate from that distribution? Markov
chain Monte Carlo.
Walt Pohl (UZH QBA)
Stochastic Models
March 31, 2015
7/1