Download 2 Probability

STAT511 — Spring 2014
Lecture Notes
1
Chapter 2: Events and Laws of Probability
January 24, 2014
2
Probability
Chapter Overview
This chapter focuses on “Probability”
• Random Experiments, Sample Space and Events.
– Random Experiment (R.E.)
– Sample Space
– Events
– Examples
• Introduction to Probability
– Axiom
– Properties
– Examples
• Counting Techniques
– Combinations
– Permutations
• Conditional Probability
– Definition
– Related rules and theorem
• Independence
2.1
Random Experiment, Sample Space & Events
Random Experiment, Sample Space & Events
• Random Experiment (R.E.) A R.E. is any action or process whose outcome is
subject to uncertainty; i.e. many outcomes, each has a certain chance to happen.
• Sample Space of a R.E. Denoted S, is the set of all possible outcomes of the
R.E.
• Event of a R.E. Denoted with capital letters (event A, B, E1 , E2 , etc), an event
is any collection (subset) of outcomes contained in the sample space S of the R.E.
– An event is said to be simple if it contains of exactly one outcome.
– An event is said to be compound if it contains more than one outcome.
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
2
R.E. Example 2.1.1
Example 1 (Roll a Die).
What is the sample space? Outcomes: Landing 1, 2, 3, 4, 5, 6 face up. Sample
Space: S = {1, 2, 3, 4, 5, 6}. Events: Let E1 be the event of getting a 2, then
E1 = {2} (simple event). Let E2 be the event of getting a number greater than 3, then
E2 = {4, 5, 6} (compound event).
Exercise
1. Let E3 be the event of rolling a seven on a single die, what is E3 ? 2. Let E4 be the
event of rolling an even number on a die, what is E4 ?
Example 2 (2.1.2 Tossing a Coin).
Toss a fair coin until a head appears, what is the sample space? Sample Space:
S = {H, T H, T T H, T T T H, T T T T H, · · · }. Events:
1. Let A be the event of tossing exactly 2 tosses, then A = {T H}. (simple event)
2. Let B be the event of tossing less than 4 tosses, then B = {H, T H, T T H}
(compound event).
3. Let C be the event of tossing ≥ 2 and ≤ 5 tosses, then C = {T H, T T H, T T T H, T T T T H}
(compound event).
4. Let D be the event of tossing more than 3 tosses, then D = {T T T H, T T T T H, T T T T T H, · · · }
(compound event).
Relations From Set Theory
An event is a set of outcomes, we may use set theory to form new events.
• Union A ∪ B: is the event containing all outcomes in A or B.
• Intersection A ∩ B: is the event containing all outcomes in both A and B.
• Compliment Ac : is the event containing all outcomes in the sample space S that
are not contained in A.
A ∪ Ac = S
• Mutually Exclusive (Disjoint): A and B are said to be mutually exclusive if
they contain no outcome in common.
A∩B =φ
φ is the empty set.
Disjoint Events
Example 3 (Drawing Cards). When drawing a single card from a standard deck of cards,
if event A = {heart, diamond} (red) and event B = {spade, club} (black), then A and
B are mutually exclusive.
Example 4 (Rolling a Die). When rolling a die, if event A = {2, 4, 6} (evens) and event
B = {1, 3, 5} (odds), then A and B are mutually exclusive.
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
3
Lecture Notes
Venn Diagram
Venn diagram is a visual representation of events and their operations.
Set Operations
Example 5 (2.1.2 Continued). A = {T H}. B = {H, T H, T T H}. C = {T H, T T H, T T T H, T T T T H}.
D = {T T T H, T T T T H, T T T T T H, · · · }.
1. A ∪ B ∪ C = {H, T H, T T H, T T T H, T T T T H}.
2. B ∩ C = {T H, T T H}.
3. C ∩ D = {T T T H, T T T T H}.
4. B c = {T T T H, T T T T H, · · · } = D.
Exercise
A ∩ (B ∪ C) =
. (B ∩ C) ∪ A =
.
DeMorgan’s Law
(A ∪ B)c = Ac ∩ B c
Purdue University
(A ∩ B)c = Ac ∪ B c
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
2.2
4
Lecture Notes
Axioms, Interpretations, and Properties of Probability
Axioms of Probability
Probability is a concept that can be used to model randomness of events. For a given
event A, denote the probability of A by P (A).
1. For any given event A, P (A) ≥ 0.
2. P (S) = 1.
3. If A1 , A2 , A3 , · · · is a countably infinite collection of mutually exclusive events,
then:
∞
X
P (A1 ∪ A2 ∪ A3 ∪ · · · ) =
P (Ai )
i=1
This indicates that, if we have a finite collection of mutually exclusive events
A1 , A2 , · · · , Ak , then:
P (A1 ∪ A2 ∪ A3 ∪ · · · ∪ Ak ) =
k
X
P (Ai )
i=1
Properties of Probability
Proposition
1. For any event A, P (A) = 1 − P (Ac ).
2. Probability of the empty set is P (φ) = 0.
3. For mutually exclusive events A and B, P (A ∩ B) = P (φ) = 0.
4. For any two events A and B,
P (A ∪ B) = P (A) + P (B) − P (A ∩ B).
This is called the general addition rule.
Exercises
1. A card is drawn from a deck of 52 cards. What is the probability that it is a queen
or a heart?
2. Two dice are thrown, what is the probability of getting at least one “6” on both
dice or an absolute difference of 4?
Example 2.1.3
Example 6 (Drawing Cards). A card is drawn from a well-shuffled deck of 52 playing
cards. What is the probability that it is an ace or a diamond? Let A be the event of
drawing an ace, D be the event of drawing a diamond. We are looking for P (A or D) =
4
P (A ∪ D) = P (A) + P (D) − P (A ∩ D) But, P (A) = 52
, P (D) = 13
52 , and P (A ∩ D) =
1
P (ace of diamond) = 52 . Thus,
P (A ∪ D) =
Purdue University
4
13
1
4
+
−
= .
52 52 52
13
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
2.3
5
Lecture Notes
Counting Techniques
Counting Techniques
• Product Rule
– Product rule for ordered pairs
– Product rule for k−tuples
• Permutations
– Example
– Permutation Formula
• Combinations
– Example
– Combination Formula
• Use of counting techniques
– Examples
Equally Likely Outcomes
• When outcomes of a R.E. are equally likely, then the probability of an event A
happening is
N (A)
count of outcome in A
P (A) =
=
N
N
Here N (A) is the number of outcomes contained in event A, and N is the number
of outcomes in the samples space.
Example 7 (Drawing Cards).
Draw a card from a standard deck of cards.
Let A be the event that the card is ace, then
N (A) = 4,
N = 52,
P (A) =
4
.
52
Product Rule for Ordered Pairs
An ordered pair refers to a pair of objects O1 , O2 and (O2 , O1 ) is considered different
than (O1 , O2 ); i.e. order matters.
Proposition
If the first element or object of an ordered pair can be selected in n1 ways, and for each
of these n1 ways the second element of the pair can be selected in n2 ways, then the
number of pairs is n1 n2 .
Example 8 (2.2.1). Two bags, bag #1 contains two balls marked 1 and 2 respectively;
bag #2 contains three balls marked 1, 2, and 3 respectively. Pick a ball from bag #1,
and then pick another ball from bag #2, how many different possibilities will we get?
(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3). Bag #1 can be selected in 2 ways, n1 = 2, bag
#2 can be selected in 3 ways, n2 = 3, so the number of pairs is n1 n2 = 2 · 3 = 6.
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
6
Product Rule for k−Tuples
A k−tuple is an ordered collection of k objects.
Proposition
An ordered collections of k elements (k−tuple), n1 choices for the first element; n2
choices for the second element, · · · ; nk choices for the k th element. Then there are
n1 n2 · · · nk possible k−tuples.
Example 9 (2.2.2). Three bags, bag #1 contains 2 balls marked 1 and 2 respectively;
bag #2 contains 3 balls marked 1, 2, 3 respectively; bag #3 contains 3 balls marked
a, b, c respectively. Pick one ball from bag #1, one ball from bag #2, and one from
bag #3. How many different pairs will we get? (1, 1, a), (1, 1, b), (1, 1, c), (1, 2, a), (1, 2,
b), (1, 2, c), (1, 3, a), (1, 3, b), (1, 3, c), (2, 1, a), (2, 1, b), (2, 1, c), (2, 2, a), (2, 2, b), (2, 2,
c), (2, 3, a), (2, 3, b), (2, 3, c). We are looking at a 3−tuple. Bag #1 can be selected in 2
ways, n1 = 2, bag #2 can be selected in 3 ways, n2 = 3, bag #3 is selected in 3 ways,
n3 = 3, so the number of 3−tuples is n1 n2 n3 = 2 · 3 · 3 = 18.
Permutations (Ordered Sequence)
Definition 10. Any ordered sequence of k objects taken from a set of n distinct objects
is called a permutation of size k. The number of permutations of size k constructed
from n objects is denoted Pk,n .
Example 11 (2.2.3). Take one suit of 13 cards, all spades, how many different permutations are there if we draw 4, 5, and 13 cards at random without replacement? 4
cards: 13 choices for 1st card, 12 choices for 2nd card, 11 choices for 3rd card, and
10 choices for 4th card. Thus, there 13 · 12 · 11 · 10 = 17160 different ways. 5 cards:
13 · 12 · 11 · 10 · 9 = 154440 different ways. 13 cards: 13 · 12 · 11 · 10 · · · · · 3 · 2 · 1 = 13!
different ways.
Permutation Formula
Formula for Pk,n
Pk,n = n · (n − 1) · (n − 2) · · · · · (n − k + 1)
⇒ Pk,n =
n · (n − 1) · · · · · (n − k + 1) · (n − k) · (n − k − 1) · · · · · 2 · 1
(n − k) · (n − k − 1) · · · · · 2 · 1
⇒ Pk,n =
n!
(n − k)!
Note
In a permutation order matters.
Exercise
A boy has 4 beads red, white, blue, and yellow. How different ways can three of the
beads be strung together in a row?
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
7
Combinations (Unordered Sequence)
Definition 12. Given a set of n distinct objects, any unordered subset of size k of the
objects is called a combination.
The number of combinations of size k from n objects
n
denoted Ck,n or
.
k
Example 13 (2.2.4). 5 balls marked 1, 2, 3, 4, and 5 respectively. Take out 3 balls,
how many different combinations are there? If we were to calculate the number of
permutations, we have P3,5 = 5 · 4 · 3. Order does not matter here, so any permutation
of the same collection of balls should be counted as the same combination. There are 3!
permutations of 3 balls, so the number of combinations is
5·4·3
5
= 10 different ways.
=
3
3!
Combination Formula
n
Formula for
k
Pk,n
n
=
k
k!
⇒
n · (n − 1) · · · · · (n − k + 1)
n
=
k
k!
n!
n
⇒
=
k
(n − k)·!k!
Note
In a combination order does not matter.
Combinations
Exercises
1. A boy has 4 beads red, white, blue, and yellow. How different ways can three of
the beads be chosen to trade away?
2. Three balls are selected at random without replacement from the jar below. Find
the probability that one ball is red and two are black.
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
8
Use of Counting Techniques
The general steps include:
• Construct the sample space where each outcome that is equally likely to happen.
• Determine the event of interest.
• Count the number of outcomes in that event.
• Find the probability using
N (event)
N
Counting Techniques to Calculate Probability
Exercises
1. Roll 2 dice, what is the probability that the sum of the two numbers is 5?
2. Draw two cards from a suit of 13 cards (say, hearts), what is the probability that
the sum of two cards is even? (A=1, J=11, Q=12, K=13).
3. Draw two cards from a deck of 52 cards, what is the probability that the two cards
are both 3?
4. Draw two cards from a deck of 52 cards, what is the probability that the two cards
are of the same number?
5. Draw two cards from a deck of 52 cards, what is the probability that the two cards
are not of the same number or they are both not hearts?
2.4
Conditional Probability
Conditional Probability
Example 14 (2.3.1). Draw a card from a deck of 52 cards, what is the probability of
getting a heart? Suppose now we already know that the card is not a club, what is the
probability of getting a heart? If we don’t know any other information, the probability
13
is 52
. Now, given that the card is not a club, the probability becomes 13 .
Conditional probability is used to model the probability of events when certain information is provided.
Definition 15. For any two events A and B with P (B) > 0, the conditional probability of A given B has occurred denoted P (A|B), is defined by
P (A|B) =
P (A ∩ B)
.
P (B)
General Multiplication Rule
Rearranging the terms, we get the product rule for P (A ∩ B):
P (A ∩ B) = P (A|B) · P (B).
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
9
Lecture Notes
Example 16 (2.3.1). Let A be the event of getting a hear, B be the event of getting a 4.
What is the conditional probability of A given B?
P (A|B) =
P (A ∩ B)
1/52
1
=
= .
P (B)
4/52
4
Example 17 (2.3.2). Deal 2 cards from a deck of 52 cards (well shuffled, without replacement). Find the probability that they are both hearts. What if we deal 3, and there are
12
2 hearts and a club? P (1st H) = 13
52 , P (2nd H | 1st H) = 51
P (1st H and 2nd H) =
13 12
· .
52 51
Deal 3, we have
3 13 12 13
.
P (2H and 1C) =
1 52 51 50
Bayes’ Theorem
Theorem 18 (Law of total Probability). Let A1 , A2 , · · · , Ak be a sequence of mutually
exclusive events and A1 ∪ A2 ∪ · · · ∪ Ak = S. Then for any other event B, we have
P (B) = P (B|A1 )P (A1 ) + · · · + P (B|Ak )P (Ak ) =
k
X
P (B|Ai ) P (Ai ) .
i=1
Theorem 19 (Bayes’ Theorem). Let A1 , A2 , · · · , Ak be a collection of mutually
exclusive and exhaustive events with P (Ai ) > 0 for i = 1, · · · , k, and A1 ∪A2 ∪· · ·∪Ak =
S. Then for any other event B with P (B) > 0, we have
P (Aj |B) =
P (B|Aj )P (Aj )
P (Aj ∩ B)
= Pk
.
P (B)
i=1 P (B|Ai ) P (Ai )
Bayes’ Theorem
Exercises
1. Test for diabetes, suppose we know that 5% of the total population has diabetes.
If a person is sick, diabetes test successfully test it with 95% correct rate, and
fails 5% of the time. If a person is not sick, 1% of the time the test will misjudge.
What is the probability that when the test shows a person is sick, the person is
really sick?
2. Flip a coin. If head, draw a ball from a bag with 2 white and 2 black balls; if tail,
draw a ball from a bag with 1 white and 2 black balls. Now if the ball is white,
what is the probability that the coin was a head?
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
10
Bayes’ Theorem
Exercises
3 A store stocks light bulbs from three suppliers. Suppliers A, B, and C supply 10%,
20%, and 70% of the bulbs respectively. It has been determined that company A’s
bulbs are 1% defective while company B’s are 3% defective and company C’s are
4% defective. If a bulb is selected at random and found to be defective, what is
the probability that it came from supplier B?
2.5
Independence
Independence of Events
Definition 20 (Independence between two events A and B). Two events A and B are
independent if P (A|B) = P (A), provided P (B) 6= 0; i.e. knowing B happening does
not affect the possibility of A happening.
Definition 21 (Alternative definition). Two events A and B are independent if P (A∩
B) = P (A) · P (B). Easy to show that these two are equivalent using P (A|B) = P P(A∩B)
(B) .
Definition 22 (Mutually Independence). Events A1 , A2 , · · · , An are said to be mutually independent if for every k, k = 2, 3, · · · , n and every subset of indices i1 , i2 , · · · , ik ,
P (Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P (Ai1 ) · P (Ai2 ) · · · · · P (Aik ).
i.e., any subset of events are mutually independent.
Independence and Mutually Exclusiveness
• Independence 6= mutually exclusiveness.
• Independence indicates the events knowing one does not give more information
about the other (in terms of probability).
• Mutually exclusiveness indicates no common outcome(s).
Exercise
Deal a card, event A = card is heart, event B = card is not heart. Are A, B independent?
Mutually exclusive?
Independence Examples
Exercise
1. Deal two cards, without replacement, event A = 1st card is heart, B = 2nd is
heart. Is A, B independent? event C = 2nd is club, is A, C independent?
2. All components in the network functions independently. P (A) = 0.9, P (B) = 0.92,
P (C) = 0.8, P (D) = 0.95, what is the probability that the network will fail?
Purdue University
Chapter2˙print.tex; Last Modified: January 24, 2014 (W. Sharabati)