Download Probability and Probability Distributions Probability Concepts

Chapter 4
Probability Concepts
Probability and
Probability Distributions
Definitions
• Experiment:
– An act or process that leads to a single
outcome that cannot be predicted with
certainty.
• Event:
– A collection of one or more simple events.
Simple event - outcome of an experiment that
cannot be decomposed into a simpler
outcome.
• Probability:
– We now assume the population parameters are
known and calculate the chances of obtaining certain
samples from this population.
– This is the reverse of statistics and statistical
measurements.
– The ability to measure the probability of occurrence of
a certain event or events is the basis for inference.
Example of Simple Events
• Experiment:
– Toss two coins and observe the up faces.
• Simple events:
– Observe H1, H2; or
– Observe H1, T2; or
– Observe T1, H2; or
– Observe T1, T2.
1
Chapter 4
Example of Events
• Sample space of an experiment:
• Experiment:
– The collection of all of its simple events.
– Toss a die and observe the up face.
• Simple events:
–
–
–
–
Definitions
• Probability of a simple event (outcome):
Observe a 1; or
- Observe a 2; or
Observe a 3; or
- Observe a 4; or
Observe a 5; or
- Observe a 6.
A event would be “Observe an even number” since it
can be decomposed into the three simple events in
the right column above.
Venn Diagram
Probability
• A graphical method for showing a sample space
and its associated simple events.
• Example:
– The experiment of “Toss a die and observe the up
face”.
– The associated Venn Diagram is:
1
2
3
4
5
6
– The likelihood that the event will occur when
the experiment is performed.
– An important property of simple events is that
with one performance of the experiment, one
and only one of the simple events will occur.
• A number that represents the chance that
a particular outcome will occur if the
experiment is conducted.
• Three types
– A priori - each outcome equally likely.
– Relative Frequency - proportion of past
experiments where the outcome occurred.
– Subjective - best estimate of an expert.
S
2
Chapter 4
Simple Event Example
• Experiment:
Event Example
• Experiment:
– Toss a coin and observe the up face.
• Venn Diagram:
– Toss a die and observe the up face.
• Venn Diagram:
H
1
3
5
T
S
2
4
6
– Probability of “Obtaining an even number” (an event)
equals the probability of obtaining a 2 plus the
probability of obtaining a 4 plus the probability of
obtaining a 6 (the sum of three simple events).
– Probability of obtaining a “Heads” on one toss of Sthe
coin equals 0.5;
– Probability of obtaining a “Tails” on one toss of the
coin equals 0.5.
Steps to Calculate Probabilities
of Events
Probability Notes
• For simple events:
– All simple event probabilities must lie between 0 (0%)
and 1 (100%) inclusive. (Simple events either
happen with certainty, don’t happen at all, or
somewhere in between.)
– The probabilities of all simple events in the sample
space must sum to 1 (100%).
– The probability of an event is calculated by summing
the probabilities of the simple events which compose
that event.
•
•
•
•
Define the experiment.
List the simple events.
Assign probabilities to the simple events.
Determine the collection of simple events
contained in the event of interest.
• Sum the simple event probabilities to obtain the
event probability.
3
Chapter 4
Coin Toss Example
Coin Toss Example, cont’d
• Experiment:
• Event A:
– Toss two coins and observe the up faces.
• Venn diagram:
–
–
–
–
H1, H2
H1, T2
T1, H2
P(H1, H2) = 1/4 or 0.25;
P(H1, T2) = 1/4 or 0.25;
P(T1, H2) = 1/4 or 0.25;
P(T1, T2) = 1/4 or 0.25.
T1, T2
– Probability of observing exactly one head.
– P(A) = P(H1, T2) + P(T1, H2) = 0.25 + 0.25
– P(A) = 0.50
S
• Event B:
– Probability of observing at least one head.
– P(B) = P(H1, H2) + P(H1, T2) + P(T1, H2)
– P(B) = 0.25 + 0.25 + 0.25 = 0.75
Determining the Number of
Simple Events
Venn Diagram
• Simple enumeration:
– Four possibilities (X1, X2, X3, X4) and we
need to choose two:
– Simple Events (combinations):
T1, T2
H1, H2
H1, T2
A
B
• X1, X2
• X1, X3
• X1, X4
T1, H2
S
-X2, X3
-X2, X4
-X3, X4
– Exponentially complex as the number of
possibilities increases.
4
Chapter 4
Combinatorial Mathematics
• A way to calculate the total number of
possible combinations for x samples from
a population N:
N
N!
  
x !( N  x ) !
x 
– N is the number of elements in the population.
– x is the number of elements in each simple
event.
Compound Events:
Union
Combinatorial Mathematics
Example
• Four possibilities (X1, X2, X3, X4) and we
need to choose two:
N
N!
4!
(1 * 2 * 3 * 4 )


 6
  
x !( N  x ) !
2 !( 4  2 ) !
( 1 * 2 )( 1 * 2 )
x 
• Ten possibilities and we need to choose
six:
N
N!
10!

 210
  
x !( N  x ) !
6 !(1 0  6 ) !
x 
Compound Events:
Intersection
• Union:
• Intersection:
– All outcomes (events) that are either part of A
or part of B or both.
– Symbol:
– Venn Diagram: A  B 
A
B
– All outcomes (events) that are part of both A
and B.
– Symbol:
– Venn Diagram: A  B 
A
S
B
S
5
Chapter 4
Example
Example, cont’d
• Experiment:
• Union:
– Toss a die and observe the up face.
– An even number or a number less than or equal to 3,
or both.
–  A  B  ={1, 2, 3, 4, 6}.
– P A  B  =P(1)+P(2)+P(3)+P(4)+P(6)=5/6
• Define the following events:
– A: {Toss an even number}
– B: {Toss a number less than or equal to 3}
• Intersection:
• Venn Diagram:
B
1
5
3
– Both an even number and a number less than or
equal to 3.
–  A  B  ={2}
– P  A  B  =P(2)=1/6
A
2
4
6
S
Additive Rule of Probability
• Additive Rule of Probability:
– The probability of the union of events A and B
is the sum of the probabilities of events A and
B minus the probability of the intersection of
events A and B.
– Symbolically:
P ( A  B )  P ( A )  P ( B )  P A  B 
– Subtract out the intersection because it was
included twice.
Mutually Exclusive Events
• Events A and B are mutually exclusive if the
intersection contains no simple events.
– Venn Diagram:
A
B
S
– Symbolically:
P(A  B)  P(A )  P( B)
6
Chapter 4
Example
Venn Diagram
• Toss two fair coins:
• Define events:
T1, T2
– A: {Observe at least one head}
– B: {Observe exactly one head}
– C: {Observe exactly two heads}
H1, H2
A
H1, T2
• So:
A = B C
P(A) = P ( B  C)  P ( B)  P(C)
Complimentary Events
• Compliment:
– The compliment of any event A is the event
that a does not occur, i.e.”not A”.
– Symbolically: A c
– The sum of the probabilities of complimentary
events equals 1:
P (A )  P (A c )  1
CB
B
T1, H2
S
Using a Complimentary Event to
Calculate Probability
• Toss two fair coins:
– Let event A: {Observe at least one head}, i.e.
A={H1, H2; H1, T2; T1, H2}.
– The compliment of event A is:
A c  {T1, T2}
– Rewriting:
1
3
P( A )  1  P ( A c )  1  ( ) 
4
4
7
Chapter 4
Conditional Probability
• Conditional probability:
Conditional Probability Formula
• Calculated as:
– The probability that event A occurs given that
event B occurs.
– Symbolically: P ( A B )
– Venn Diagram:
P (A
B ) =
– For die example:
P (A B ) =
B
1
3
A
2
P ( A  B )
P ( B )
1
( )
P (2 )
1
 6 
3
P (1 )  P ( 2 )  P ( 3 )
3
( )
6
4
6
5
S
Older Child Paradox
• A random family of two children, assuming all
four gender combinations are equally likely:
– P(FF)=P(FM)=P(MF)=P(MM)=0.25
– What is the conditional probability that FF will occur
given that B occurred, where B is the event that at
least one of the children is a girl?
– What is the conditional probability that FF will occur
given that B occurred, where B is the event that the
older child is a girl?
Venn Diagrams
• At least one girl=1/3
• Oldest is a girl=1/2
M1F2
M1M2
M1F2
F1M2 B
M1M2
F1M2 B
F1F2
F1F2
A
A
S
S
8
Chapter 4
Multiplicative Rules of
Probability
• Multiplicative Rule:
– or
P (A  B)  P ( B) P (A B)
P (A  B)  P (A ) P ( B A )
Independence
• Independence:
– Events A and B are said to be independent if
the assumption that B has occurred does not
alter the probability that A occurs.
P (B A )  P (B )
P ( A B )  P ( A )
P( A  B)  P( A) P(B)
Random Sampling
• Random sample - select a group of n units
in such a way that each sample of size n
has the same chance of being selected.
• Random number table - the numbers
occur randomly and with equal probability
no matter where you start or how you
move.
9