Download COSC 3213: Computer Networks

COSC 6221: Statistical Signal Processing Theory
Assignment # 3: Higher Order Moments and Introduction to Estimation
Due Date: October 16, 2003
In the last few lectures, we defined the expectation operation for a random variable (RV), X , and showed that the expected
value of any function of X including the higher order moments can be calculated directly from the probability density function
f X (x) of X . Calculating the moments individually is a tedious process, however, the moment generating function,  X (t ) ,
or alternatively the characteristic function,  X () , can be used to simplify the process. Important bounds including the
Chernoff bound and the Chebyshev and Schwartz inequalities that provides upper limit to the probabilities of a RV exceeding
its expected value, were introduced. Finally, we had our first taste of estimation by covering problems in linear regression and
the minimum mean square (MMSE) estimators for calculating the mean and variance of the observed process. We noted that
an optimal estimator is usually unbiased and consistent. In the next week, we will expand the theory of random variables to
multiple random variables and stochastic processes.
Please review chapter 4 from the Woods text to cover the intrinsic details about the random variables.
1.
(Expectation) Let X be a Poisson RV
PX k  
1
exp  a a k
k!
with parameter a . Compute EY   X 2  b.
2.
(Conditional Expectation) A random sample of 20 households shows the following numbers of children per household:
{3, 2, 0, 1, 0, 0, 3, 2, 5, 0, 1, 1, 2, 2, 1, 0, 0, 0, 6, 3}.
(a) For this set, calculate the average number of children per household.
(b) What is the average number of children per households given that there is at least one child in the family?
3.
(Conditional Expectation) A particular TV model is manufactured in three different plants, say A, B, and C of the
same company. Because the workers at A, B, and C are not equally experienced, the quality of the sets differs from a
plant to another. The pdf’s of the time of failure, X , in years are
Plant A :
Plant B :
Plant C :
f X ( x)  1 5 exp  x 5U ( x)
f X ( x)  1 6.5 exp  x 6.5U ( x)
f X ( x)  1 10 exp  x 10 U ( x)
where U (x) is the unit step function. Plant A produces three times as many sets as B , which produces twice as many
sets as C. Assuming that the sets are all sent to a central warehouse, intermingled, and shipped to retail stores without
any distinction, calculate the expected lifetime of a set purchased at random.
4.
(Conditional Expectation) A source transmits a signal  with pdf

21 0    2
f    

otherwise
0
Because of additive Gaussian noise, the pdf of the received signal Y when    is
f Y   y  
Compute the expected value of Y .
  ( y  ) 2
exp 
 2 2

2 2
1




5.
(Moment Generating function) Let X and Y be two independent Poisson RV’s with
PX k  
1
exp  2 2 k
k!
1
PX k   exp  3 3 k
k!
Compute the PMF of Z  ( X  Y ) using the moment generating function.
6.
(Characteristic Function) Let
1 N
 Xi
N i 1
Y
where X i are independent identically distributed (iid) Cauchy RV’s with

1
1


 1  x   2 
f X i x  
By calculating the characteristic function of Y , show that the pdf of Y is identical to that of X i and independent of N .
Hint: The characteristic function for the RV X i with Cauchy distribution is given by  X i ()  exp   .

7.

(Chernoff Bound) Compute the Chernoff bound PX  k  for a RV X that has an exponential pdf
f X x    exp x  U x 
8.
(Optimum Linear Prediction) Show that the minimum value of the mean square error (MSE) is given by
 2min   Y2 (1   2 ).
Explain why the minimum value is zero when  equals 1.
9.
(Moving Average Regression) Consider the recursion known as the first order moving average given by
X n  Z n  Z n1
 1
where X n , Z n , and Z n1 are all RV’s for   n   . Assume
E Z n   0
E

  
Z n2

for all n
2
for all n
E Z n X n j  0
for j  1,2,...
and E Z n Z j  0
for n  j


Calculate the correlation R X [k ]  EX n X nk  for   k  .
10. (Swartz Inequality) Derive the inequality


 X P X   X  E X   X
2
that holds true if f X ( x)  f X ( x) with  x being the variance of X .