Download Gaussian Processes for Regression CKI Williams and CE Rasmussen

Gaussian Processes for
Regression
CKI Williams and CE Rasmussen
Summarized by Joon Shik Kim
12.05.10.(Fri)
Computational Models of
Intelligence
Introduction
• In the Bayesian approach to neural
networks a prior distribution over the
weights induces a prior distribution over
functions. This prior is combined with a
noise model, which specifies the
probability of observing the target t
given function value y, to yield a
posterior over functions which can then
be used for predictions.
Prediction with Gaussian
Processes (1/3)
• A stochastic process is a collection of random
variables {Y(x)|x∈X) indexed by a set X. In our
case X will be the input space with dimension d,
the number of inputs. The stochastic process is
specified by giving the probability distribution
for every finite subset of variables Y(x(1)),…,Y(x(k))
in a consistent manner. A Gaussian process is a
stochastic process which can be fully specified
by its mean function μ(x)=E[Y(x)] and its
covariance function C(x,x’)=E(Y(x)-μ(x))(Y(x’)μ(x’)). We consider Gaussain processes which
have μ(x)=0.
Prediction with Gaussian
Processes (2/3)
• The training data consists of n pairs of
inputs and targets {(x(i),t(i)). i=1…n}. The
input vector for a test case is denoted x
(with no superscript). The inputs are ddimensional x1,…,xd and the targets are
scalar.
Prediction with Gaussian
Processes (3/3)
yˆ (x)  k (x)K t
-1
T
 ( x)  C (x, x)  k (x)K k(x)
T
2
yˆ
-1
k(x)  (C (x, x ),..., C (x, x )
(1)
K ij  C (x , x )
(i)
(j)
t  (t ,..., t )
(1)
(n) T
(n) T
Illustration of Prediction using GP
Proof of Prediction Model (1/3)
Proof of Prediction Model (2/3)
Proof of Prediction Model (3/3)