Download Convolution integrals of Normal distribution functions

Convolution integrals of Normal distribution
functions
Susana Vinga
September 23, 2004
Supplementary material to S.Vinga and JS.Almeida (2004) “Rényi continuous entropy of DNA sequences”. J.Theor.Biol. 231(3):377–388.
Contents
1 Convolution
1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1
1
2 Normal distribution function
2
3 Convolution of Normal distribution functions
3.1 Integral simplification . . . . . . . . . . . . . .
3.1.1 Properties of determinants . . . . . . .
3.1.2 Factorization of quadratic forms . . . .
3.2 Result . . . . . . . . . . . . . . . . . . . . . . .
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
4
4
4
Convolution
1.1
Definition
The convolution f ∗ g of two functions f (x) and g(x) defined in R is given by:
Z
f ∗ g(z) =
f (x)g(z − x)dx
(1)
R
1.2
Properties
Some of the properties of f ∗ g are described below.
1. f ∗ g = g ∗ f (commutative)
2. f ∗ (g ∗ h) = (f ∗ g) ∗ h (associative)
1
3. f ∗ (g + h) = (f ∗ g) + (f ∗ h)
4.
d(f ∗g)
dx
=
df
dx
∗g =f ∗
R
R
R
5. f ∗ g = f · g
dg
dx
6. laplace transform1 L [f ∗ g] = L(f )L(g)
7. in probability theory, the convolution of two functions has a special relation with the distribution of the sum of two independent random variables.
If the two random variables X and Y are independent, with pdf’s f and g
respectively, the distribution h(z) of Z = X + Y is given by h(z) = f ∗ g.
This result is obtained below.
H(z) =
=
=
=
=
R
d( FX (z − y) · g(y)dy)
dH(z)
=
dz
Z dz
d(FX (z − y))
· g(y)dy
dz
Z
f (z − y) · g(y)dy
=
f ∗g
h(z) =
=
2
P (Z ≤ z) = P (X + Y ≤ z)
Z
P (X + Y ≤ z|Y = y) · g(y)dy
Z
P (X ≤ z − y) · g(y)dy
Z
FX (z − y) · g(y)dy
Normal distribution function
The Gaussian or Normal p-dimensional distribution with mean µ and covariance
matrix Σ is given by the following equation 2, where x ∈ Rp is a p-dimensional
random vector, xT is the transpose vector of x and |Σ| is the determinant of Σ:
µ
¶
1
1
T −1
(2)
gp (x; µ, Σ) =
exp − (x − µ) Σ (x − µ)
2
(2π)p/2 |Σ|1/2
When a random variable X, taking values in Rp , has a probability density
function (pdf) given by the former equation we say that X ∼ Np (µ, Σ).
1 Laplace
transform of function f(t) is defined as L[f (t)](s) =
2
R∞
0
f (t)e−st dt
3
Convolution of Normal distribution functions
Given two p-dimensional normal probability density functions G1 ≡ gp (x; a, A)
and G2 ≡ gp (x; b, B) (see eq. 2) we will prove that the convolution of these two
functions is a normal probability density distribution function with mean a + b
and variance A + B, i.e. gp (x; a + b, A + B):
G1 ∗ G2 (z) = gp (z; a + b, A + B)
The next sections demonstrate this result by first presenting an algebraic
simplification of integrals using some properties of determinants and the factorization of quadratic forms.
3.1
Integral simplification
R
The following deduction represents the simplification of the integral G1 G2 dx
where G1 and G2 are the pdf of the normal distribution described above.
Z
gp (x; a, A) · gp (x; b, B)dx
Z
=
Z
=
=
=
=
=
=
=
(2π)
0
1
|A|
1/2
|A|
1/2
e− 2 (x−a) A
(2π)
−1
1
p/2
(2π)
(2π)
1/2
1
p/2
1
|B|
e− 2 ((x−a) A
1
|B|
1
(x−a)
1
1
p/2
Z
=
1
p/2
0
−1
e− 2 ((x−c) (A
1
0
1/2
0
e− 2 (x−b) B
−1
(x−b)
(x−a)+(x−b)0 B −1 (x−b))
−1
dx
dx
+B −1 )(x−c)+(a−b)0 C(a−b))
dx
|A| (2π) |B|
¯¡
¢−1 ¯¯1/2
¯ −1
¯ A + B −1
¯
0
1
e− 2 (a−b) C(a−b) ·
p/2
1/2
1/2
(2π) |A| |B|
Z
1
− 1 (x−c)0 (A−1 +B −1 )(x−c)
dx
·
¯
¯1/2 e 2
p/2 ¯
−1 ¯
−1
−1
(2π) ¯(A + B ) ¯
¯¡
¢−1 ¯¯1/2
¯ −1
¯ A + B −1
¯
0
1
e− 2 (a−b) C(a−b)
(3)
p/2
1/2
1/2
(2π) |A| |B|
0
−1
1
1
e− 2 (a−b) (A+B) (a−b)
p/2
1/2
−1
−1
(2π) (|A| |B| |A + B |)
0
−1
1
1
e− 2 (a−b) (A+B) (a−b)
p/2
1/2
−1
−1
(2π) |ABA + ABB |
0
−1
1
1
e− 2 (a−b) (A+B) (a−b)
p/2
1/2
−1
(2π) |ABA + A|
0
−1
1
1
e− 2 (a−b) (A+B) (a−b)
p/2
1/2
(2π) |A (B + A) A−1 |
p/2
(2π)
1/2
p/2
1/2
3
=
3.1.1
1
p/2
(2π)
0
1
|A + B|
1/2
−1
e− 2 (a−b) (A+B)
(a−b)
Properties of determinants
1. |AB| = |A| |B|
¯
¯
1
2. ¯A−1 ¯ = |A|
3. |cA| = cn |A|
¯
¯
¯
¯
4. ¯BAB −1 ¯ = |B| |A| ¯B −1 ¯= |B||A|
|B| = |A|
¯
¯ ¯
¯ ¯
¯
5. ¯B −1 AB − λI ¯ = ¯B −1 AB − B −1 λIB ¯ = ¯B −1 (A − λI) B ¯ = |A − λI|
3.1.2
Factorization of quadratic forms
Give x, a and b vectors of dimension p, A and B symmetric matrices of order p
positively defined such as A + B is not singular, we have:
0
0
(x − a) A (x − a) + (x − b) B (x − b) =
0
0
(x − c) (A + B) (x − c) + (a − b) C (a − b)
(4)
where
c = (A + B)
C
3.2
−1
(Aa + Bb)
¡
¢−1
−1
= A (A + B) B = A−1 + B −1
Result
Let G1 (x) and G2 (x) be the probability density function of the p-dimensional
normal distributions N (a, A) and N (b, B) respectively. The convolution G1 ∗G2
is defined as:
Z
G1 ∗ G2 (z)
=
G1 (x)G2 (z − x)dx
Z
=
1
(2π)
·
Z
=
=
=
p/2
|A|
1
p/2
(2π)
0
1
1/2
e− 2 (x−a) A
1
|B|
1/2
−1
(x−a)
0
e− 2 (z−x−b) B
−1
·
(z−x−b)
dx
gp (x; a, A) · gp (x; z − b, B)dx
1
p/2
1
1/2
(2π) |A + B|
gp (z; a + b, A + B)
4
0
−1
e− 2 (z−(a+b)) (A+B)
(z−(a+b))
(5)
This means that the convolution G1 ∗ G2 (z) is the pdf of the normal distribution N (a + b, A + B).
References
[1] Richard A. Johnson and Dean W. Wichern. Applied multivariate statistical
analysis. Prentice Hall, New Jersey, 4th edition, 1998.
[2] Kyle Siegrist.
Virtual laboratories in probability and statistics.
http://www.math.uah.edu/stat/, 1997–2004.
[3] Gilbert Strang. Linear Algebra and Its Applications. International Thomson
Publishing, 3rd edition, 1988.
5