Download pptx file

Probability and
Probability Distributions
Probability Concepts
• Probability:
– We now assume the population parameters are
known and calculate the chances of obtaining certain
samples from this population.
– This is the reverse of statistics and statistical
measurements.
– The ability to measure the probability of occurrence of
a certain event or events is the basis for inference.
Definitions
• Experiment:
– An act or process that leads to a single
outcome that cannot be predicted with
certainty.
• Event:
– A collection of one or more simple events.
Simple event - outcome of an experiment that
cannot be decomposed into a simpler
outcome.
Example of Simple Events
• Experiment:
– Toss two coins and observe the up faces.
• Simple events:
– Observe H1, H2; or
– Observe H1, T2; or
– Observe T1, H2; or
– Observe T1, T2.
Example of Events
• Experiment:
– Toss a die and observe the up face.
• Simple events:
–
–
–
–
Observe a 1; or
- Observe a 2; or
Observe a 3; or
- Observe a 4; or
Observe a 5; or
- Observe a 6.
A event would be “Observe an even number” since it
can be decomposed into the three simple events in
the right column above.
Definitions
• Sample space of an experiment:
– The collection of all of its simple events.
• Probability of a simple event (outcome):
– The likelihood that the event will occur when
the experiment is performed.
– An important property of simple events is that
with one performance of the experiment, one
and only one of the simple events will occur.
Venn Diagram
• A graphical method for showing a sample space
and its associated simple events.
• Example:
– The experiment of “Toss a die and observe the up
face”.
– The associated Venn Diagram is:
1
2
3
4
5
6
S
Probability
• A number that represents the chance that
a particular outcome will occur if the
experiment is conducted.
• Three types
– A priori - each outcome equally likely.
– Relative Frequency - proportion of past
experiments where the outcome occurred.
– Subjective - best estimate of an expert.
Simple Event Example
• Experiment:
– Toss a coin and observe the up face.
• Venn Diagram:
H
T
– Probability of obtaining a “Heads” on one toss of Sthe
coin equals 0.5;
– Probability of obtaining a “Tails” on one toss of the
coin equals 0.5.
Event Example
• Experiment:
– Toss a die and observe the up face.
• Venn Diagram:
1
3
5
2
4
6
S
– Probability of “Obtaining an even number” (an event)
equals the probability of obtaining a 2 plus the
probability of obtaining a 4 plus the probability of
obtaining a 6 (the sum of three simple events).
Probability Notes
• For simple events:
– All simple event probabilities must lie between 0 (0%)
and 1 (100%) inclusive. (Simple events either
happen with certainty, don’t happen at all, or
somewhere in between.)
– The probabilities of all simple events in the sample
space must sum to 1 (100%).
– The probability of an event is calculated by summing
the probabilities of the simple events which compose
that event.
Steps to Calculate Probabilities
of Events
•
•
•
•
Define the experiment.
List the simple events.
Assign probabilities to the simple events.
Determine the collection of simple events
contained in the event of interest.
• Sum the simple event probabilities to obtain the
event probability.
Coin Toss Example
• Experiment:
– Toss two coins and observe the up faces.
• Venn diagram:
–
–
–
–
H1, H2
H1, T2
T1, H2
P(H1, H2) = 1/4 or 0.25;
P(H1, T2) = 1/4 or 0.25;
P(T1, H2) = 1/4 or 0.25;
P(T1, T2) = 1/4 or 0.25.
T1, T2
S
Coin Toss Example, cont’d
• Event A:
– Probability of observing exactly one head.
– P(A) = P(H1, T2) + P(T1, H2) = 0.25 + 0.25
– P(A) = 0.50
• Event B:
– Probability of observing at least one head.
– P(B) = P(H1, H2) + P(H1, T2) + P(T1, H2)
– P(B) = 0.25 + 0.25 + 0.25 = 0.75
Venn Diagram
T1, T2
H1, H2
H1, T2
A
B
T1, H2
S
Determining the Number of
Simple Events
• Simple enumeration:
– Four possibilities (X1, X2, X3, X4) and we
need to choose two:
– Simple Events (combinations):
• X1, X2
• X1, X3
• X1, X4
-X2, X3
-X2, X4
-X3, X4
– Exponentially complex as the number of
possibilities increases.
Combinatorial Mathematics
• A way to calculate the total number of
possible combinations for x samples from
a population N:
 N
N!
  
x !( N  x )!
x 
– N is the number of elements in the population.
– x is the number of elements in each simple
event.
Combinatorial Mathematics
Example
• Four possibilities (X1, X2, X3, X4) and we
need to choose two:
 N
N!
4!
(1 * 2 * 3 * 4)


6
  
x !( N  x )! 2 !( 4  2)!
(1 * 2)(1 * 2)
x 
• Ten possibilities and we need to choose
six:
 N
N!
10!

 210
  
x
x !( N  x )! 6!(10  6)!
 
Compound Events:
Union
• Union:
– All outcomes (events) that are either part of A
or part of B or both.
– Symbol:
– Venn Diagram:A  B
A
B
S
Compound Events:
Intersection
• Intersection:
– All outcomes (events) that are part of both A
and B.
– Symbol:
– Venn Diagram:A  B
A
B
S
Example
• Experiment:
– Toss a die and observe the up face.
• Define the following events:
– A: {Toss an even number}
– B: {Toss a number less than or equal to 3}
• Venn Diagram:
B
1
5
A
2
3
4
6
S
Example, cont’d
• Union:
– An even number or a number less than or equal to 3,
or both.
– A  B ={1, 2, 3, 4, 6}.
– PA  B =P(1)+P(2)+P(3)+P(4)+P(6)=5/6
• Intersection:
– Both an even number and a number less than or
equal to 3.
– A  B ={2}
– P A  B =P(2)=1/6
Additive Rule of Probability
• Additive Rule of Probability:
– The probability of the union of events A and B
is the sum of the probabilities of events A and
B minus the probability of the intersection of
events A and B.
– Symbolically:
P(A  B)  P(A )  P( B)  PA  B
– Subtract out the intersection because it was
included twice.
Mutually Exclusive Events
• Events A and B are mutually exclusive if the
intersection contains no simple events.
– Venn Diagram:
A
B
S
– Symbolically:
P(A  B)  P(A )  P( B)
Example
• Toss two fair coins:
• Define events:
– A: {Observe at least one head}
– B: {Observe exactly one head}
– C: {Observe exactly two heads}
• So:
A = B C
P(A) = P( B  C)  P( B)  P(C)
Venn Diagram
T1, T2
H1, H2
CB
A
H1, T2
T1, H2
B
S
Complimentary Events
• Compliment:
– The compliment of any event A is the event
that a does not occur, i.e.”not A”.
– Symbolically: A c
– The sum of the probabilities of complimentary
events equals 1:
c
P(A )  P(A )  1
Using a Complimentary Event to
Calculate Probability
• Toss two fair coins:
– Let event A: {Observe at least one head}, i.e.
A={H1, H2; H1, T2; T1, H2}.
– The compliment of event A is:
A c  {T1, T2}
– Rewriting:
1
3
P(A )  1  P(A )  1  ( ) 
4
4
c
Conditional Probability
• Conditional probability:
– The probability that event A occurs given that
event B occurs.
– Symbolically: P(A B)
– Venn Diagram:
B
1
A
2
3
4
6
5
S
Conditional Probability Formula
• Calculated as:
P ( A  B)
P (A B) =
P ( B)
– For die example:
1
( )
P ( 2)
1
6
P(A B) =


3
P(1)  P( 2)  P(3)
3
( )
6
Older Child Paradox
• A random family of two children, assuming all
four gender combinations are equally likely:
– P(FF)=P(FM)=P(MF)=P(MM)=0.25
– What is the conditional probability that FF will occur
given that B occurred, where B is the event that at
least one of the children is a girl?
– What is the conditional probability that FF will occur
given that B occurred, where B is the event that the
older child is a girl?
Venn Diagrams
• At least one girl=1/3
• Oldest is a girl=1/2
M1F2
M1M2
M1F2
F1M2 B
M1M2
F1M2 B
F1F2
F1F2
A
A
S
S
Multiplicative Rules of
Probability
• Multiplicative Rule:
– or
P(A  B)  P( B) P(A B)
P(A  B)  P(A) P( BA)
Independence
• Independence:
– Events A and B are said to be independent if
the assumption that B has occurred does not
alter the probability that A occurs.
P ( B A)  P ( B )
P ( A B )  P ( A)
P( A  B)  P( A) P( B)
Random Sampling
• Random sample - select a group of n units
in such a way that each sample of size n
has the same chance of being selected.
• Random number table - the numbers
occur randomly and with equal probability
no matter where you start or how you
move.