Download P(A and B)

Basic Probability
Theory
Probability Appears Everywhere
Probability Appears Everywhere
• Happening of unexpected events around us is not
uncommon. It therefore makes sense to abandon the
idea of a world with certainties and accept a world where
we associate likelihood to each event.
• Probability is a quantitative measure of likelihood of an
event. We all have a fair understanding of :
– “Probability of getting a head in a toss of a fair coin is 1/2”.
– “Probability of a human getting infected by HIV during blood
transfer is 0.000000001”
– “Probability that six appears in two consecutive throws of a fair
dice is 1/36”.
• For each such random experiment, there is a well
defined set of all possible events (outcomes) which the
experiment can results in.
Sample Space Ω
• The possible outcomes of an experiment
are called the elementary events.
• Sample space := set of all elementary
events.
• Event is a set of outcomes – a set of
elementary events
(a subset of the sample space)
• If the subset is a singleton,
then the event is an elementary event
– Ω = { head, tail }
– Ω = { 1, 2, 3, 4, 5, 6 }
Examples
1. The event: {toss of a die is even}.
– Denote this event as { 2,4,6 }.
2. We flip a coin twice.
– Sample space = { HH, HT, TH, TT }.
– The subset { HT, TH, TT } is one of the
16 events defined by this sample
space, namely “the event that we get at
least one T”.
Probability Distribution
• A sample space Ω comes with a
probability distribution:
a mapping P: 2Ω → R such that:
• 1. P(A) ≥ 0 for all events A  Ω
• 2. P(Ω) = 1
• 3. P(A U B) = P(A) + P(B) for any two
events A,B  Ω that are mutually
exclusive: A ∩ B = 
Non-intersecting events
• A U B = the event in which A or B or both occur
• A and B are mutually exclusive if A and B
cannot occur at the same time: P(A ∩ B) = 
• Elementary events are mutually exclusive.
• Example: a die cannot be both 4 and 6 at the
same time.
• Examples:
– The probability for the event {2,4,6}
is 1/6+1/6+1/6=½.
A
– The probability of the die being even or odd is 1
• Therefore, the sum of P(w) for all
elementary events w should be 1.
B
Random Variable
• Given a probability space, we can have random
variables: the outcome of a coin toss, for
example, is a variable which gets a random
value.
• A random variable is a function that assigns a
real number to each elementary event. (Note
that it does not assign a number to a nonelementary event.)
• discrete random variables: random variables
whose possible values come from a discrete set
Expectation
• The Expectation is the average or mean of a
random variable.
• The expected value of X is denoted by E(X).
E(X) = Σ P(X=m) m
• For example: The expected value of a toss of a
die is 1/6 +2/6+ 3/6+4/6+ 5/6+ 6/6 = 21/6=3.5
• The expected value of a toss of a fair coin is
• ½*0+1/2*1=1/2.
• The expected number of heads coin which has
P(tail)=1/3, P(head)=2/3 is 2/3.
• Let X be the random variable which is the sum of
two tosses of a die. What is its expectation?
Markov Inequality
• What is the probability that a non-negative
random variable is much greater than its
expectation?
• The Russian mathematician Andrey Markov gave an upper bound for
this question.
• Let X is any non negative random variable and
a > 0, then
E[ x ]
P( x  a) 
a
Markov Inequality - Proof


E[ X ]   xP( x )   xP( x )   xP( x )
x 0

  xP( x )
xa

  aP ( x )
xa

 a  P( x)
xa
 aP ( x  a )
xa
xa
Intersection of events A ∩ B
• The intersection of events A and B is the event
in which both A and B occur.
• Example:
• A = the event in which the toss of a die is even,
P(A)=½.
• B = the event in which the toss
is at most 3, P(B)=1/2.
• P(A ∩ B) =
P(Toss is both even and less
than 3) = P(Toss=2) = 1/6.
A
A and B
B
Union of events A U B
• The union of events A and B is the event
in which A or B or both occur.
• In words:
The probability of {A or B} is found by
adding their individual probabilities, then
subtracting the probability of both (which
has been counted twice).
• In symbols:
P(AUB)=P(A)+P(B)-P(A and B)
A
B
Conditional Probability: B|A
• How does the probability of an event B
change, given that we know that an event
A has occurred ? It is the proportion of
{A and B} out of A.
• P(B|A)=
Pr(A and B)/Pr(A).
A
A and B
B
Example
• Suppose we know that the toss of the die
was odd. What is the probability that it is 3,
given that it is odd?
A is the event of the toss being Odd. Pr(A)=1/2.
B is the event that the toss is 3.
P(B)=1/6.
• P({3} given {odd})=
P( {3}{odd} )/P({odd}) =
P({3})/P({odd})=1/3.
A
A and B
B
Independent events:
P(A ∩ B) = P(A)P(B).
• If P( B|A )=P(B) we say that the event B is
independent of A.
• In the independent case we have on one
hand what we had in the last slide:
• P(B|A)=P(A ∩ B) / P(A).
• And because of independence:
P(B|A)= P(B).
• Hence, for independent events:
P(A ∩ B) = P(A)P(B).
Example
• A Toss of two fair coins.
• Pr(0,0)=Pr(1,0)=Pr(0,1)=Pr(1,1)=1/4.
• Let A be the event that the first coin is 0.
Pr(A)=Pr(0,0)+Pr(0,1)=1/2
• Let B be the event that the second coin is 1.
Pr(B)=Pr(0,1)+Pr(1,1)=1/2
Pr(A  B) =Pr(0,1)=1/4
Pr(B|A)=Pr(A and B)/Pr(A) =(1/4)/(1/2)=1/2.
And indeed, this is equal to Pr(B), and we have
Pr(A  B)=pr(B)Pr(A)
Hence, the events A and B are independent.
This is true for any value that the first and second coin can
take
• The random variables which are the value of the first and
second coin toss are independent random variables.
•
•
•
•
•
•
Linearity of Expectation
• For any two random variables X and Y
• E(X+Y)=E(X)+E(Y)
• (whether they are independent or not!)
The average behaves linearly.
We will not prove it here…
The Complement of an event
• The complement of event A is the event {A does not
occur}.
• Example, the complement of the even {2,4,6} (the toss is
even) is {1,3,5} (the Toss is odd).
The probabilities of an event and its complement add to 1:
P(A) + P(not A) = 1
• Example:
• A fact: 37% of gingis have type O blood.
• What is the chance a gingi will not have
type O blood?
Answer: 100% - 37% = 63% do not have
type O blood.
The Complement
• Note:
• The complement of right wing people is
NOT-Right wing as
opposed to left wing.
not A
• There may be
another political party.
A
B