Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Convolution integrals of Normal distribution functions Susana Vinga September 23, 2004 Supplementary material to S.Vinga and JS.Almeida (2004) “Rényi continuous entropy of DNA sequences”. J.Theor.Biol. 231(3):377–388. Contents 1 Convolution 1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 Normal distribution function 2 3 Convolution of Normal distribution functions 3.1 Integral simplification . . . . . . . . . . . . . . 3.1.1 Properties of determinants . . . . . . . 3.1.2 Factorization of quadratic forms . . . . 3.2 Result . . . . . . . . . . . . . . . . . . . . . . . 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 4 4 4 Convolution 1.1 Definition The convolution f ∗ g of two functions f (x) and g(x) defined in R is given by: Z f ∗ g(z) = f (x)g(z − x)dx (1) R 1.2 Properties Some of the properties of f ∗ g are described below. 1. f ∗ g = g ∗ f (commutative) 2. f ∗ (g ∗ h) = (f ∗ g) ∗ h (associative) 1 3. f ∗ (g + h) = (f ∗ g) + (f ∗ h) 4. d(f ∗g) dx = df dx ∗g =f ∗ R R R 5. f ∗ g = f · g dg dx 6. laplace transform1 L [f ∗ g] = L(f )L(g) 7. in probability theory, the convolution of two functions has a special relation with the distribution of the sum of two independent random variables. If the two random variables X and Y are independent, with pdf’s f and g respectively, the distribution h(z) of Z = X + Y is given by h(z) = f ∗ g. This result is obtained below. H(z) = = = = = R d( FX (z − y) · g(y)dy) dH(z) = dz Z dz d(FX (z − y)) · g(y)dy dz Z f (z − y) · g(y)dy = f ∗g h(z) = = 2 P (Z ≤ z) = P (X + Y ≤ z) Z P (X + Y ≤ z|Y = y) · g(y)dy Z P (X ≤ z − y) · g(y)dy Z FX (z − y) · g(y)dy Normal distribution function The Gaussian or Normal p-dimensional distribution with mean µ and covariance matrix Σ is given by the following equation 2, where x ∈ Rp is a p-dimensional random vector, xT is the transpose vector of x and |Σ| is the determinant of Σ: µ ¶ 1 1 T −1 (2) gp (x; µ, Σ) = exp − (x − µ) Σ (x − µ) 2 (2π)p/2 |Σ|1/2 When a random variable X, taking values in Rp , has a probability density function (pdf) given by the former equation we say that X ∼ Np (µ, Σ). 1 Laplace transform of function f(t) is defined as L[f (t)](s) = 2 R∞ 0 f (t)e−st dt 3 Convolution of Normal distribution functions Given two p-dimensional normal probability density functions G1 ≡ gp (x; a, A) and G2 ≡ gp (x; b, B) (see eq. 2) we will prove that the convolution of these two functions is a normal probability density distribution function with mean a + b and variance A + B, i.e. gp (x; a + b, A + B): G1 ∗ G2 (z) = gp (z; a + b, A + B) The next sections demonstrate this result by first presenting an algebraic simplification of integrals using some properties of determinants and the factorization of quadratic forms. 3.1 Integral simplification R The following deduction represents the simplification of the integral G1 G2 dx where G1 and G2 are the pdf of the normal distribution described above. Z gp (x; a, A) · gp (x; b, B)dx Z = Z = = = = = = = (2π) 0 1 |A| 1/2 |A| 1/2 e− 2 (x−a) A (2π) −1 1 p/2 (2π) (2π) 1/2 1 p/2 1 |B| e− 2 ((x−a) A 1 |B| 1 (x−a) 1 1 p/2 Z = 1 p/2 0 −1 e− 2 ((x−c) (A 1 0 1/2 0 e− 2 (x−b) B −1 (x−b) (x−a)+(x−b)0 B −1 (x−b)) −1 dx dx +B −1 )(x−c)+(a−b)0 C(a−b)) dx |A| (2π) |B| ¯¡ ¢−1 ¯¯1/2 ¯ −1 ¯ A + B −1 ¯ 0 1 e− 2 (a−b) C(a−b) · p/2 1/2 1/2 (2π) |A| |B| Z 1 − 1 (x−c)0 (A−1 +B −1 )(x−c) dx · ¯ ¯1/2 e 2 p/2 ¯ −1 ¯ −1 −1 (2π) ¯(A + B ) ¯ ¯¡ ¢−1 ¯¯1/2 ¯ −1 ¯ A + B −1 ¯ 0 1 e− 2 (a−b) C(a−b) (3) p/2 1/2 1/2 (2π) |A| |B| 0 −1 1 1 e− 2 (a−b) (A+B) (a−b) p/2 1/2 −1 −1 (2π) (|A| |B| |A + B |) 0 −1 1 1 e− 2 (a−b) (A+B) (a−b) p/2 1/2 −1 −1 (2π) |ABA + ABB | 0 −1 1 1 e− 2 (a−b) (A+B) (a−b) p/2 1/2 −1 (2π) |ABA + A| 0 −1 1 1 e− 2 (a−b) (A+B) (a−b) p/2 1/2 (2π) |A (B + A) A−1 | p/2 (2π) 1/2 p/2 1/2 3 = 3.1.1 1 p/2 (2π) 0 1 |A + B| 1/2 −1 e− 2 (a−b) (A+B) (a−b) Properties of determinants 1. |AB| = |A| |B| ¯ ¯ 1 2. ¯A−1 ¯ = |A| 3. |cA| = cn |A| ¯ ¯ ¯ ¯ 4. ¯BAB −1 ¯ = |B| |A| ¯B −1 ¯= |B||A| |B| = |A| ¯ ¯ ¯ ¯ ¯ ¯ 5. ¯B −1 AB − λI ¯ = ¯B −1 AB − B −1 λIB ¯ = ¯B −1 (A − λI) B ¯ = |A − λI| 3.1.2 Factorization of quadratic forms Give x, a and b vectors of dimension p, A and B symmetric matrices of order p positively defined such as A + B is not singular, we have: 0 0 (x − a) A (x − a) + (x − b) B (x − b) = 0 0 (x − c) (A + B) (x − c) + (a − b) C (a − b) (4) where c = (A + B) C 3.2 −1 (Aa + Bb) ¡ ¢−1 −1 = A (A + B) B = A−1 + B −1 Result Let G1 (x) and G2 (x) be the probability density function of the p-dimensional normal distributions N (a, A) and N (b, B) respectively. The convolution G1 ∗G2 is defined as: Z G1 ∗ G2 (z) = G1 (x)G2 (z − x)dx Z = 1 (2π) · Z = = = p/2 |A| 1 p/2 (2π) 0 1 1/2 e− 2 (x−a) A 1 |B| 1/2 −1 (x−a) 0 e− 2 (z−x−b) B −1 · (z−x−b) dx gp (x; a, A) · gp (x; z − b, B)dx 1 p/2 1 1/2 (2π) |A + B| gp (z; a + b, A + B) 4 0 −1 e− 2 (z−(a+b)) (A+B) (z−(a+b)) (5) This means that the convolution G1 ∗ G2 (z) is the pdf of the normal distribution N (a + b, A + B). References [1] Richard A. Johnson and Dean W. Wichern. Applied multivariate statistical analysis. Prentice Hall, New Jersey, 4th edition, 1998. [2] Kyle Siegrist. Virtual laboratories in probability and statistics. http://www.math.uah.edu/stat/, 1997–2004. [3] Gilbert Strang. Linear Algebra and Its Applications. International Thomson Publishing, 3rd edition, 1988. 5