Download Basics of Probability Theory and Random Processes

$
'
Basics of Probability Theory and Random
Processes
Basics of probability theory
a
• Probability of an event E represented by P (E) and is given by
NE
P (E) =
NS
(1)
where, NS is the number of times the experiment is performed
and NE is number of times the event E occured.
Equation 1 is only an approximation. For this to represent the
exact probability NS → ∞.
The above estimate is therefore referred to as Relative
Probability
Clearly, 0 ≤ P (E) ≤ 1.
&
a required
for understanding communication systems
%
$
'
• Mutually Exclusive Events
Let S be the sample space having N events E1 , E2 , E3 , · · · , EN .
Two events are said to be mutually exclusive or statistically
N
[
Ai = S for all i and j.
independent if Ai ∩ Aj = φ and
i=1
• Joint Probability
Joint probability of two events A and B represented by
P (A ∩ B) and is defined as the probability of the occurence of
both the events A and B is given by
NA∩B
P (A ∩ B) =
NS
• Conditional Probability
Conditional probability of two events A and B represented as
&
%
'
P (A|B) and defined as the probability of the occurence of
event A after the occurence of B.
P (A|B) =
$
NA∩B
NB
Similarly,
NA∩B
P (B|A) =
NB
This implies,
P (B|A)P (A) = P (A|B)P (B) = P (A ∩ B)
&
%
$
'
• Chain Rule
Let us consider a chain of events A1 , A2 , A3 , · · · , AN which are
dependent on each other. Then the probability of occurence of
the sequence
P (AN , AN −1 , AN −2 ,
· · · , A 2 , A1 )
= P (AN |AN −1 , AN −2 , · · · , A1 ).
P (AN −1 |AN −2 , AN −3 , · · · , A1 ).
· · · .P (A2 |A1 ).P (A1 )
&
%
'
$
Bayes Rule
A1
A5
A4
A
B
2
A3
Figure 1: The partition space
In the above figure, if A1 , A2 , A3 , A4 , A5 partition the sample space
S, then (A1 ∩ B), (A2 ∩ B), (A3 ∩ B), (A4 ∩ B), and (A5 ∩ B)
partition B.
Therefore,
&
%
'
P (B) =
=
n
X
i=1
n
X
$
P (Ai ∩ B)
P (B|Ai ).P (Ai )
i=1
In the example figure here, n = 5.
P (Ai |B) =
=
P (Ai ∩ B)
P (B)
P (B|Ai ).P (Ai )
n
X
P (B|Ai ).P (Ai )
i=1
In the above equation, P (Ai |B) is called posterior probability,
&
%
'
$
P (B|Ai ) is called likelihood, P (Ai ) is called prior probability and
n
X
P (B|Ai ).P (Ai ) is called evidence.
i=1
&
%
'
Random Variables
$
Random variable is a function whose domain is the sample space
and whose range is the set of real numbers Probabilistic description
of a random variable
• Cummulative Probability Distribution:
It is represented as FX (x) and defined as
FX (x) = P (X ≤ x)
If x1 < x2 , then FX (x1 ) < FX (x2 ) and 0 ≤ FX (x) ≤ 1.
• Probability Density Function:
It is represented as fX (x) and defined as
&
%
$
'
fX (x) =
dFX (x)
dx
This implies,
P (x1 ≤ X ≤ x2 ) =
fX (x)
&
=
Z
x2
fX (x) dx
x1
dFX (x)
dx
%