Download + P(X = 1) - 동국대학교 OCW

디지털통신
Random Process
임민중
동국대학교 정보통신공학과
Minjoong Rim, Dongguk University
1
Random Process
Random Processes
Probability
• Definitions

Sample space S
-

Event
-

a single sample point or a set of sample points in the space S
Elementary event
-

the set of all possible outcomes of the experiment
a single sample point
Mutually exclusive
-
the occurrence of one events precludes the occurrence of the other event
• Axioms

0  P(A)  1 for an event A

P(S) = 1

If A and B are two mutually exclusive events, then P(AB) = P(A) + P(B)
Minjoong Rim, Dongguk University
3
Random Process
Conditional Probability
• P(B|A)

The probability of event B, given that A has occurred
P( A  B)
P( B | A) 
P( A)
P(A  B) = P(A) P(B|A)
• Bayes’ rule
P( A  B)  P( B | A) P( A)  P( A | B) P( B)
P( A | B) P( B)
P( B | A) 
P( A)
• Example: binary data transmission
Transmitted Data
0
X
Y
1

P(X = 0) = 0.6
P(X = 1) = 0.4

P(Y = 0 | X = 0) = 0.9
P(Y = 1 | X = 0) = 0.1

P(Y = 1 | X = 1) = 0.8
P(Y = 0 | X = 1) = 0.2

0
1
P(error) = P(X  Y) = P(X = 0  Y = 1) + P(X = 1  Y = 0)
= P(X = 0) P(Y = 1 | X = 0) + P(X = 1) P (Y = 0 | X = 1)
Minjoong Rim, Dongguk University
4
Received Data
= 0.6 0.1 + 0.4 0.2 = 0.14
Random Process
Random Variables - 1
• Random variable, X(A)

the function relationship between a random event, A, and a real number
• Cumulative distribution function (CDF)

Properties of CDF
FX ( x )  P( X  x )
0  FX ( x )  1
FX ( x1 )  FX ( x2 ) if x1  x2
FX ( )  0
FX ()  1
• Probability density function (PDF)

p X ( x) 
dFX ( x )
dx
P( x1  X  x2 )   p X ( x )dx
x2
Properties of PDF
x1
p X ( x)  0



Minjoong Rim, Dongguk University
5
p X ( x )dx  1
Random Process
Random Variables - 2
• Mean (Ensemble average)
X

mX  E{ X }   xpX ( x)dx
E(X)

• Variance

  V ( X )  E{( X  mx ) }   ( x  mx ) 2 p X ( x)dx
2
X
2
X

X

Relationship between variance and mean-square value
 X2  E { X 2  2mX X  m2X }
 E { X 2 }  2mX E { X }  m2X
 E { X 2 }  m2X
Minjoong Rim, Dongguk University
6
Random Process
Random Variables - 3
• Example: Rolling a dice

pdf, pmf (probability mass function)
The sum should be one
1/6
1

2
3
4
5
6
time average
cdf
Rolling a dice
...
repeatedly
increasing from 0 to 1
1
1/6
1
2
3
4
5
6
3 1 5 4 5 2 6 3 1 1 4 6 2 ...
6
2
ensemble average
3
1

E(X) =
1 (1/6) + 2 (1/6) + 3 (1/6) + 4 (1/6) + 5 (1/6) + 6 (1/6) = 3.5

V(X) =
12 (1/6) + 22 (1/6) + 32 (1/6) + 42 (1/6) + 52 (1/6) + 62 (1/6) - 3.52 = 2.9167
Minjoong Rim, Dongguk University
7
Random Process
Random Variables - 4
• Example: Uniform random variable

pdf
1/(b-a)
f(x)
area = 1
a

cdf
b
1
F(x)
a

E(X) =



V(X) =
b
xf ( x)dx   x


b
  x  m
Minjoong Rim, Dongguk University
a
1
ba
dx 
ba
2
b  a

1 b
ab 
f ( x)dx 
x
 dx 
b  a a 
2 
12
2
2
8
2
Random Process
Random Variables - 5
• Average Symbol Energy E  T s(t ) 2 dt 
0



1
0
0
1
0
0
1
1
0
E(0) = E(1) = 1/2
A
t
A 2T  ( A) 2 T
P
 A 2T
2
-A
T = symbol duration
10
11
01
10
00
01
11
3A
00
10
E(00) = E(01) = E(10)
= E(11) = 1/4
A
t
-A
-3A
A 2T  (3 A) 2 T  ( A) 2 T  (3 A) 2 T
P
 5 A 2T
4
T = symbol duration
Minjoong Rim, Dongguk University
9
Random Process
Random Variables - 6
• Average Symbol Energy
0  0
1  A
0
1
0
A
 
assuming E(0) = E(1) = 1/2
E | s |2
E(00) = E(01) = E(10) = E(11) = 1/4
0  B
1  -B
Symbol Energy = (0 + A2) / 2 = A2 / 2
00
01
10
11




C
3C
-C
-3C
11
10
00
01
-C
C
0
-B
B
Symbol Energy = ((-B)2 + B2) / 2 = B2
00
01
10
11




D
D
D
D
j 2  1
-3C
1
(1 + j)
(-1 + j)
(1 - j)
(-1 - j)
01
D
-D
00
D
3C
11
Symbol Energy = ((-3C)2 + (-C)2 + C2 + (3C)2) / 4 = 5C2
-D
10
Symbol Energy = 2D2
Minjoong Rim, Dongguk University
10
Random Process
Gaussian Random Variables - 1
• Gaussian Random Variable
 1 n2
1
p( n) 
exp    
 2
 2  
• 2-dimensional Gaussian
Random Variable
2-dimensional
amplitude
Rayleigh = Amplitude of Zero-Mean
Complex Gaussian
Complex Gaussian
Minjoong Rim, Dongguk University
11
Rayleigh distribution in amplitude
Random Process
Gaussian Random Variables - 2
• Example

Central Limit Theorem
- Probability distribution of the sum
of j statistically independent random
variables approaches the Gaussian
distribution as j  
x
y = (x1 + x2) / sqrt(2)
Minjoong Rim, Dongguk University
y = (x1 + x2 + x3) / sqrt(3)
12
y = (x1 + x2 + ... + x100) / sqrt(100)
Random Process
Gaussian Random Variables - 3
x
y = (x1 + x2 + x3 + x4) / sqrt(4)
Minjoong Rim, Dongguk University
y = (x1 + x2) / sqrt(2)
y = (x1 + x2 + x3 + x4 + x5) / sqrt(5)
13
y = (x1 + x2 + x3) / sqrt(3)
y = (x1 + x2 + ... + x100) / sqrt(100)
Random Process
Functions of Random Variables - 1
• Functions of Random Variables
Y  g( X )

Expected Value

E (Y )   g ( x) f X ( x)dx
X

• Properties
Y  aX  b
0
X 1
X: random variable
a, b: constants
E (aX  b)  aE ( X )  b
1
2X
V (aX  b)  a V ( X )
2
0
• Example

Y = 2X + 1, E(X) = 1, V(X) = 1

E(Y) = ?

V(Y) = ?
Minjoong Rim, Dongguk University
2X 1
1
14
Random Process
Functions of Random Variables - 2
• Q function
Random Variable
mean = 0

X: Gaussian RV with zero mean and unit variance

Q(x)  P(X > x)

Table is given
-

X
variance = 1
Q  x
x
0
Q(0.5) = 0.309, Q(1) = 0.159, Q(1.5) = 0.0668, Q(2) = 0.0228, Q(3) = 0.00135
Q(0) = ?, Q() = ?, Q(-) = ?, Q(-1) = ?
-1
0
• Example

Y: Gaussian random variable with mean = b and variance = a2

P(Y > T) = Q((T – b) / a)
Y  aX  b
b
Minjoong Rim, Dongguk University
T
X
Y b
a
T b 
Q

 a 
0 (T-b)/a
15
Random Process
Functions of Random Variables - 3
• Example
• Example
P(Y > 0) where Y is a Gaussian
RV with mean = -1 and variance =
22 ?
P(Y < 0) where Y is a Gaussian
RV with mean = 1 and variance =
(1/2)2 ?


1
2
2 X 1
-1
0
0
1
1
2
2X
0
X
1
-1
0
X
0
X 1
X
-2
0.5
0
X
Q(0.5) = 0.309, Q(1) = 0.159, Q(1.5) = 0.0668,
Q(2) = 0.0228, Q(3) = 0.00135
Minjoong Rim, Dongguk University
0
16
2
Random Process
Functions of Random Variables - 4
0
• Example
0
Xb

Y=X+N

X: P(X = -1) = P(X = 1) = 1/2
Yb
1

N: Gaussian RV with mean 0 and variance 1

X, N: independent
1
0  -1
1 1
Xb
X
Y
<00
>01
Yb
N
P(error) = P(Xb = 0  Yb = 1) + P(Xb = 1  Yb = 0)
P( A  B)  P( B | A) P( A)  P( A | B) P( B)
= P(X = -1  Y > 0) + P(X = 1  Y < 0)
= P(X = -1) P(Y > 0 | X = -1) + P(X = 1) P(Y < 0 | X = 1) = ?

P(Y > 0 | X = -1)
= P(X + N > 0 | X = -1)
= P(-1 + N > 0)
= P(N > 1)
= Q(1)
0.5
0.5
P(X = 1) f(Y | X = 1)
P(X = -1) f(Y | X = -1)
P(X = 1) P(Y < 0 | X = 1)
P(X = -1) P(Y > 0 | X = -1)
= 0.5 Q(1)
0.5
-1
Minjoong Rim, Dongguk University
P(Y < 0 | X = 1)
= P(X + N < 0 | X = 1)
= P(1 + N < 0)
= P(N < -1)
= Q(1)
0.5
0
0
17
= 0.5 Q(1)
1
Random Process
Correlation
• Correlation of X and Y
How much X and Y are correlated
E[XY]
Correlation can be also affected by
the mean and variance of X and Y
• Covariance of X and Y
zero mean
cov[XY ]  E[( X  E[ X ])(Y  E[Y ])]
 E[ XY ]  E[ X ]E[Y ]
• Correlation coefficient of X and Y
unit variance
 X ,Y 
cov( X , Y )
 XY
Minjoong Rim, Dongguk University
18
Uncorrelated
Random Process
Power Spectral Density - 1
• Random Process
Deterministic Signal (Fourier Transform) Frequency-domain Representation
Random Signal  Autocorrelation (Fourier Transform) Power Spectral Density
the outcome of a random experiment is mapped into a waveform that is
a function of time

• Autocorrelation
X (t )
RX ( )  E[ X (t   ) X (t )]

• Power Spectral Density

A measure of the frequency distribution of a single random process

S X ( f )   RX ( )e j 2 f  d

• Cross-correlation
X (t )
RXY ( )  E[ X (t   )Y (t )]
• Cross Spectral Density

A measure of the frequency inter-relationship between two random
processes


S XY ( f )   RXY ( )e j 2f d

Minjoong Rim, Dongguk University
19
Random Process
Y (t )
Power Spectral Density - 2
• Random process
slowly fluctuating
random process


• Autocorrelation

Rapidly fluctuating
random process

• Power spectral density
narrow bandwidth
wide bandwidth
f
f
Minjoong Rim, Dongguk University
20
Random Process
Example: Power Spectral Density - 1
Signal
Minjoong Rim, Dongguk University
Autocorrelation
21
Power Spectral Density
Random Process
Example: Power Spectral Density - 2
Signal
Minjoong Rim, Dongguk University
Autocorrelation
22
Power Spectral Density
Random Process