Download Digital Communications

Digital Communications
Lecture # 5&6
Probability and Random Processes
Sidra Shaheen Syed
[email protected]
Review of Probability and Random Processes
2
Importance of Random Processes
• Random variables and processes talk about quantities and
signals which are unknown in advance
• The data sent through a communication system is modeled as
random variable
• The noise, interference, and fading introduced by the channel
can all be modeled as random processes
• Even the measure of performance (Probability of Bit Error) is
expressed in terms of a probability
3
Random Events
• When we conduct a random experiment, we can
use set notation to describe possible outcomes
• Examples: Roll a six-sided die
Possible Outcomes:
• An event is any subset of possible outcomes:
S  {1, 2,3, 4,5, 6}
A  {1, 2}
4
Random Events (continued)
• The complementary event: A  S  A  {3, 4,5, 6}
• The set of all outcomes in the certain event:
S
• The null event: 
• Transmitting a data bit is also an
experiment
5
Probability
• The probability P(A) is a number which measures the
likelihood of the event A
Axioms of Probability
• No event has probability less than zero: P( A)  0
P ( A)  1 and P( A)  1  A  S
• Let A and B be two events such that: A  B  
Then: P( A  B)  P( A)  P( B)
6
Mutually exclusive outcomes
Outcomes are mutually exclusive if they cannot happen
at the same time.
For example, when you toss a single coin either it will land on heads or it will land on
tails. There are two mutually exclusive outcomes.
Outcome A: Head
Outcome B: Tail
When you roll a dice either it will land on an odd number or it will land on an even
number. There are two mutually exclusive outcomes.
Outcome A: An odd number
Outcome B: An even number
The sum of all mutually exclusive outcomes is 1.
Relationships Between Random Events
• Joint Probability: P( AB)  P( A  B )
- Probability that both A and B occur
P( AB )
• Conditional Probability: P( A | B) 
P( B)
- Probability that A will occur given that B has occurred
8
Relationships Between Random Events
• Statistical Independence:
- Events A and B are statistically independent if:
P( AB)  P( A) P( B)
- If A and B are independence than:
and P( B | A)  P( B)
P( A | B)  P( A)
9
Independence
• The probability of independent events A, B
and C is given by:
P(A,B,C) = P(A)P(B)P(C)
A and B are independent, if knowing that A has happened
does not say anything about B happening
10
Random Variables
• A random variable X(S) is a real valued function of the
underlying even space: s  S
• A random variable may be:
-Discrete valued: range is finite (e.g.{0,1}) or countable
infinite (e.g.{1,2,3…..})
-Continuous valued: range is uncountable infinite (e.g. )
• A random variable may be described by:
- A name: X

- Its range: X
- A description of its distribution

11
Cumulative Distribution Function
• Definition: FX ( x)  F ( x)  P( X  x)
• Properties:
 FX ( x) is monotonically non decreasing
 F ()  0
 F ( )  1
 P(a  X  b)  F (b)  F (a )
• While the CDF defines the distribution of a random
variable, we will usually work with the pdf or pmf
• In some texts, the CDF is called PDF (Probability
Distribution function)
12
Probability Density Function
• Definition:
dFX ( x)
PX ( x) 
dx
dF ( x)
or P( x) 
dx
• Interpretations: pdf measures how fast the CDF is increasing
or how likely a random variable is to lie around a particular
value
• Properties:
P( x)  0



P( x)dx  1
b
P(a  X  b)   P( x)dx
a
13
14