Download Document

Random Variables and
Stochastic Processes – 0903720
Dr. Ghazi Al Sukkar
Email: [email protected]
Office Hours: will be posted soon
Course Website:
http://www2.ju.edu.jo/sites/academic/ghazi.alsukkar
Most of the material in these slide are based on slides prepared by Dr. Huseyin Bilgekul
http://faraday.ee.emu.edu.tr/ee571/
EE 720
1
Relations of Events
 Subset
An event E is said to be a subset of the event F if,
whenever E occurs, F also occurs.  E  F *
 Equality
Events E and F are said to be equal if the occurrence
of E implies the occurrence of F, and vice versa.
 E = F  E  F and F  E
 Intersection
An event is called the intersection of two events E and F
if it occurs only whenever E and F occur simultaneously.
It is denoted by E  F or EF. General Form： 𝑘𝑖=1 𝐸𝑖
*
The following three statements are always satisfied: A ⊂ S, ∅ ⊂ A and A ⊂ A
EE 720
2
Relations of Events (Cont’d)
 Union
An event is called the union of events E and F if it
occurs whenever at least one of them occurs.
It is denoted by E  F or E+F. General Form： 𝑘𝑖=1 𝐸𝑘
 Complement
An event is called the complement of the event E if it
only occurs whenever E does not occur, denoted by E or EC
𝑺 = ∅, ∅ = 𝑺, 𝑨 = 𝑨
 Difference
An event is called the difference of two events E and F
if it occurs whenever E occurs but F does not, and is
denoted by EF .
Notes: EC = SE and EF = EFC
EE 720
3
Relations of Events (Cont’d)
 Certainty
An event is called certain if it its occurrence is
inevitable. The sample space is a certain event.
 Impossibility
An event is called impossible if there is certainty in
its non-occurence. The empty set is an impossible event.
 Mutually Exclusiveness
If the joint occurrence of two events E and F is
impossible, we say that E and F are mutually exclusive
(disjoint). That is, EF =  .
EE 720
4
Venn Diagrams of Events
EF
EF
S
S
E
E
F
F
S
S
F
F
E
EE
G
(ECG)  F
EC
EE 720
5
Examples
 Example 1.7
At a busy international airport, arriving planes land on
a first-come first-served basis. Let
E = there are at least 5 planes waiting to land,
F = there are at most 3 planes waiting to land,
H = there are exactly 2 planes waiting to land. Then
EC is the event that at most 4 planes are waiting to land.
FC is the event that at least 4 planes are waiting to land.
E is a subset of FC. That is, EFC = E
H is a subset of F. That is, FH = H
E and F, E and H are mutually exclusive.
FHC is the event that the number of planes waiting to
land is 0, 1, or 3.
EE 720
6
Useful Laws
 Commutative Laws: EF = F  E, E  F = F  E
 Associative Laws:
E(F  G) = (E  F)  G, E (F  G) = (E  F)  G
 Distributive Laws:
(EF)H = (EH)(FH), (EF)H = (E H) (FH)
 De Morgan’s Laws:
C
C
C ( n E ) C
(EF) = E  F ,
i 1
i
  ni1 E iC
If in a set identity we replace all sets by their complements, all unions by intersections,
and all intersections by unions, the identity is preserved.
e.g., if 𝐴 𝐵 ∪ 𝐶 = 𝐴𝐵 ∪ 𝐴𝐶, then 𝐴 ∪ 𝐵𝐶 = (𝐴 ∪ 𝐵)(𝐴 ∪ 𝐶), proof: take the complement of
both sides and apply De Morgan’s laws.
-
EE 720
7
 De Morgan’s Second Laws:
n
C
n
C
(

E
)


E
(E  F) = E  F ,
i 1 i
i 1 i
C
C
C
If in a set identity we replace all unions by intersections, all intersections by unions, S
by ∅ , and ∅ by S the identity is preserved.
e.g., 𝐴 𝐵 ∪ 𝐶 = 𝐴𝐵 ∪ 𝐴𝐶, 𝑡ℎ𝑒𝑛 𝐴 ∪ 𝐵𝐶 = (𝐴 ∪ 𝐵)(𝐴 ∪ 𝐶)
Also 𝑆 ∪ 𝐴 = 𝑆, 𝑡ℎ𝑒𝑛 ∅𝐴 = ∅
-
EE 720
8
1.3 Axioms of Probability
 Definition： Probability Axioms
Let S be the sample space of a random phenomenon.
Suppose that to each event A of S, a number denoted
by P(A) is associated with A. If P satisfies the
following axioms, then it is called a probability and the
number P(A) is said to be the probability of A.
Equally likely： P(A) = P(B)
Axiom 1 P(A)  0
P({𝜻𝟏 }) = P({𝜻𝟐 })
Axiom 2 P(S) = 1
(when an experiment is conducted there has to be an outcome)
Axiom 3 if {A1, A2, A3, …} is a sequence of
mutually exclusive events (The events A1, A2, A3, … are
mutually exclusive if 𝐴𝑖 ∩ 𝐴𝑗 = ∅ for all 𝑖 ≠ 𝑗), then
 
 
P   Ai    P ( Ai )
 i 1  i 1 EE 720
9
Properties of Probability
 The probability of the empty set
 is 0. That is,
P() = 0 (something has to happen).
 Countable additivity & finite additivity
 
 
 n
 n
P   Ai    P ( Ai ) and P   Ai    P ( Ai )
 i 1  i 1
 i 1  i 1
 The probability of the occurrence of an event is
always some number between 0 and 1. That is,
0  P(A)  1.
 Probability is a real-value, nonnegative, countably
additive set function.
EE 720
10
Field
• A collection of subsets of a nonempty set S forms
a field F if:
i. 𝑆 ∈ 𝐹
ii. If 𝐴 ∈ 𝐹, 𝑡ℎ𝑎𝑛 𝐴 ∈ 𝐹
iii. If 𝐴 ∈ 𝐹 and 𝐵 ∈ 𝐹, than 𝐴 ∪ 𝐵 ∈ 𝐹
• Using (i) - (iii), it is easy to show that 𝐴 ∩ 𝐵, 𝐴 ∩ 𝐵, ∅
etc., also belong to F.
• Thus if 𝐴 ∈ 𝐹, and 𝐵 ∈ 𝐹 then
𝐹 = 𝑆, ∅, 𝐴, 𝐵, 𝐴, 𝐵, 𝐴 ∪ 𝐵, 𝐴 ∩ 𝐵, 𝐴 ∪ 𝐵, 𝐴 ∪ 𝐵, …
• From here on wards, we shall reserve the term
‘event’ only to members of F.
EE 720
11
Examples
Let P be a probability defined on a sample space
S. For events A of S define Q(A) = [P(A)]2 and
R(A) = P(A)/2. Is Q a probability on S ? Is R a
probability on S ? Why or why not?
Sol：
EE 720
12
Examples
 Example 1.9
A coin is called unbiased or fair if, whenever it is flipped,
the probability of obtaining heads equals that of obtaining
tails. When a fair coin is flipped, the sample space is
S = {T, H}.
Since {H} and {T} are equally likely & mutually exclusive,
1 = P(S) = P({T,H}) = P({T}) + P({H}) .
Hence, P({T}) = P({H}) = 1/2.
When a biased coin is flipped, and the outcome of tails is
twice as likely as heads. That is, P({T}) = 2P({H}).Then
1 = P(S) = P({T,H}) = P({T}) + P({H}) =3P({H}) .
Hence, P({H}) = 1/3 and P({T}) = 2/3.
EE 720
13
Theorem 1.1
(Classical Definition of Probability)
Let S be the sample space of an experiment. If S has N
points that are equally likely to occurs, then for any event
A of S,
N ( A)
P( A) 
N
where N(A) is the number of points of A.
EE 720
14
Examples
 Example 1.10
Let S be the sample space of flipping a fair coin three
times and A be the event of at least two heads; then
S={
}
and A = {
}.
So N =
and N(A) = .
The probability of event A is.
P(A) = N(A)/N =
.
EE 720
15
Examples
 Example 1.11
An elevator with 2 passengers stops at the second, third,
and fourth floors. If it is equally likely that a passenger
gets off at any of the 3 floors, what is the probability
that the passengers get off at different floors?
Sol：Let a and b denote the two passenger and a2b4 mean
that a gets off at the 2nd floor and b gets off at the 4th
floor. So S = {
}
and A = {
}.
So N = and N(A) = . The probability of event A is.
P(A) = N(A)/N =
.
EE 720
16
Examples
 Example 1.12
A number is selected at random from the set of integers
{1, 2, …, 1000}. What is the probability that the number is
divisible by 3?
Sol：
Let A be the set of all numbers between 1 and 1000 that
are divisible by 3.
 N = 1000, and N(A) =
.
Hence, the probability of event A is.
P(A) = N(A)/N =
.
EE 720
17
Basic Theorems
S
 Theorem 1.2
For any event A, P(AC) = 1  P(A).
B
A
B-A
 Theorem 1.3
If A  B, then P(BA) = P( BAC ) = P(B)P(A).
Corollary： If AB, then P(A)  P(B). This says that if
event A is contained in B then occurrence of B means
A has occurred but the converse is not true.
EE 720
18
S
 Theorem 1.4
P ( A  B)  P ( A)  P ( B)  P ( A  B).
• Note that if two events A and B are A AB
mutually exclusive then
P(A ∪ B) = P(A) + P(B)
• In General: 𝑃 𝐴 ∪ 𝐵 ≤ 𝑃 𝐴 + 𝑃(𝐵)
 Example 1.13
B
Suppose that in a community of 400 adults. 300 bike or swim or do both. 160
swim, and 120 swim and bike. What is the probability that an adult, selected
at random from this community, bike?
Ans： 0.65
EE 720
19
Examples
 Example 1.14
A number is chosen at random from the set of integers {1,2, …, 1000}. What
is the probability that it is divisible by 3 or 5?
Ans： 0.467.
 Example 1.15
Suppose that 25% of the population of a city read newspaper A, 20% read
newspaper B, 13% read C, 10% read both A and B, 8% read both A and C, 5%
read B and C, and 4% read all three. If a person from this city is selected at
random, what is the probability that he or she does not read any of these
newspaper?
𝑃 𝐴∪𝐵∪𝐶
=𝑃 𝐴 +𝑃 𝐵 +𝑃 𝐶 −𝑃 𝐴∩𝐵 −𝑃 𝐴∩𝐶 −𝑃 𝐵∩𝐶 +𝑃 𝐴∩𝐵∩𝐶
Ans= 1 − 𝑃 𝐴 ∪ 𝐵 ∪ 𝐶
Ans： 0.61
EE 720
20
Inclusion-Exclusion Principle
 For 3 events
P ( A1  A2  A3 )  P ( A1 )  P ( A2 )  P ( A3 )
 P ( A1  A2 )  P ( A2  A3 )  P ( A1  A3 )
 For n events
 P ( A1  A2  A3 )
n 1 n
n  2 n 1
n
n  n
P   Ai    P ( Ai )    P ( Ai  A j )     P ( Ai  A j  Ak )
 i 1  i 1
i 1 j  i 1
i 1 j  i 1 k  j 1
   (1) n1 P ( A1  A2    An )
 Theorem 1.5
P ( A)  P ( A  B)  P ( A  BC )
EE 720
21
 Theorem 1.6:
If 𝑨𝒊 𝒌𝒊=𝟏 are k random events such that:
(1) 𝑨𝒊 ∩ 𝑨𝒋 = ∅, for 𝒊 ≠ 𝒋, they are said to be
mutually exclusive.
= 𝑺, they are said to be exhaustive
• Then: These events are called partition of S.
(2)
𝒌
𝒊=𝟏 𝑨𝒊
𝒌
𝑷 𝑨 =𝑷 𝑨∩𝑺 =𝑷 𝑨∩
𝒌
𝑨𝒊
=
𝒊=𝟏
𝒊=𝟏
A1
Aj
EE 720
𝑷(𝑨 ∩ 𝑨𝒊 )
A2
Ai
An
22