Download probability

AMS 5
PROBABILITY
Basic Probability
Probability: The part of Mathematics
devoted to quantify uncertainty
Frequency Theory
Bayesian Theory
Game: Playing Backgammon. The chance of getting
(6,6) is 1/36. If we rolled the dice many times
independently and under the same conditions, the (6,6)
would turn up about 1/36 of the time.
Frequency Theory
Basic Probability
Therefore according to the Frequency Theory
everyone should get the same answer when
calculating chances and thus probability is
Objective.
Basic Probability
This interpretation is fine when you think about
events that can be repeated.
If in October 2003 you had been asked to calculate
the chance that the Marlins won the World Series,
it would have been very hard to think of that
problem in terms of frequencies.
Here is a Subjective (Bayesian) interpretation:
Properties
Given the frequentist interpretation we observe that:
This is because frequencies can not be negative and can not be
above 100%.
On the other hand, if you think about the subjective
interpretation, you see that, unless you are willing to loose money
for sure, you will never bet more than one unit. On the other
hand, nobody will accept a negative bet from you, unless that
person is willing to loose money for sure. So the subjective
definition also implies that chance is between 0 and 1.
Properties
From the frequentist perspective this is justified since, if A is an
event, then every time A happens the opposite does not happen
and vice-versa. So the proportion of times that the opposite of A
happens is 1 minus the proportion of times A happens.
From the subjective perspective, if you are willing to bet p on A
happening to get a prize of 1, then you must be willing to bet 1-p
on the opposite of A.
That is, if you are willing to bet 30 cents on Florida, you must be
willing to bet 70 cents on NY (this is not about your favorite team,
this is about your rational quantification of each team's chances).
Example
A box contains red marbles and blue marbles. One marble is
drawn at random from the box. If it is red you win $1, if it is
blue you win nothing. If you can choose between the following
two boxes, which one you would prefer?
1. The box contains 3 red marbles and 2 blue ones.
2. The box contains 30 red marbles and 20 blue ones
In both cases the answer is 3/5. This is because what counts is
the ratio
(number of red marbles)/(total number of marbles)
which is the same for both boxes.
This is a general rule when all possible outcomes are equally
likely.
Events
An event is a collection of possible outcomes in a specific
situation.
• Duke wins the NCAA tournament.
• A dice is rolled and the outcome is greater than 4.
• In a sample of 20 students from this class the oldest student is
less than 22 years of age.
• The temperature tomorrow is higher than 60 degrees.
If all possible outcomes are equally likely then
Events
You have an experiment you are thinking of
performing, e.g. tossing a coin. The collection of
all possible ways the experiment could come out
is a set called the sample space, e.g. Ω =
{heads, tails}. There is some particular event,
i.e. some particular way the experiment could
turn out, whose probability interests you, e.g.
H={head}. Then the probability of getting head
is:
number of ways favorable to the event (H)
1
=
total number of ways the experiment could come out 2
Probability vs. Statistics
Inductive vs. deductive reasoning.
Induction: the process of reasoning from the part (sample) to the
whole (population).
Deduction: the process of reasoning from the whole (population)
to the part (sample).
If you know the population fully and you choose a sampling
method like random sampling, you can use probability to
accurate predict what kind of sample you will get (Deduction).
If all you have is a sample and you try to infer from it back to the
population that’s statistics (Induction).
Probability vs. Statistics
Probability (Deduction)
POPULATION
SAMPLE
Statistics (Induction)
Example (Probability): According to the latest census the 52.2%
of the population of Boston are female. If we randomly select 100
Bostonians what is the probability of having less than 50 women?
• Example (Statistics): A random sample of 1000 Bostonians is
chosen, and 550 of them are female. Estimate the percentage of
the females in the whole population.
•
Conditional Probabilities
Very often it happens that the occurrence of a certain event
affects the chances of another even happening. How do we deal
with that? We use conditional probabilities.
Consider a deck of cards with 4 suits: clubs, diamonds, hearts
and spades. Each suit has 13 cards: 2 through 10, jack, queen,
king and ace. In total there are 52 cards.
A deck of cards is shuffled and the top two cards are put over a
table, face down.
• What are the chances that the second card is the queen of
hearts?
• What are the chances that the second card is the queen of
hearts given that the first card was the seven of clubs?
Conditional Probabilities
To answer the first question think that shuffling the cards is like
putting them in a random order. There are 52 possible positions
for the queen of hearts and they are all equally likely. In
particular the second position is one of those. So the chances are
1/52.
To answer the second question we observe that we have only 51
positions left, since the seven of clubs is occupying one of them.
So there is a chance of 1/51 that the second card will be the
queen of hearts. The answer to the second question is a
conditional probability. There is a condition on the event seven
of clubs happening before we think of the event queen of hearts.
In contrast the first question puts no condition on the first card.
Multiplication Rule
Consider the following problem: a box has three tickets colored
red, white and blue. We draw two tickets without replacement.
What are the chances of drawing the red ticket and then the
white?
Since at the beginning there are three tickets and they all have
the same chances, then the probability of drawing the red ticket is
1/3. Suppose the first ticket is drawn and turns out to be the red
one, then there are two tickets left, both with the same chances
of being drawn. So the probability of drawing the white one is
1/2. There are 3 ways of drawing the first ticket and 2 ways of
drawing the second for each one of the ways of drawing the first.
So there are 6 possible combinations of first and second tickets of
which only one is red-white. Implying that the probability if 1/6.
1 1 1
= ×
6 3 2
Multiplication Rule
Examples:
1. What is the probability that the first card taken from a deck
of well shuffled cards will be the seven of clubs and the second
will be the queen of hearts?
1/52 is the chance that the first one is the seven of clubs. 1/51
is the chance that the second will be the queen of hearts. So
the probability is:
1
1 1
= ×
2652 52 51
Multiplication Rule
2. What is the probability that the first and the second cards are
aces?
There are 4 aces out 52 cards. So the probability of the first
card being an ace is 4/52. If the first one is an ace, there are 3
aces left in the deck, so the chance that the second is also an
ace is 3/51. So the probability is: 12
4 3
= ×
2652 52 51
3. A coin is tossed twice. What are the chances that the result
is a head followed by a tail?
The chance of a head is 1/2. No matter what the first toss is,
the probability of a tail in the second toss is 1/2. So the
probability is:
1 1 1
= ×
4 2 2
Independence
Notice that in the last example is doesn't really matter that the
first toss resulted in heads or tails. The results of the first toss
do not affect the probability of the second toss. This is because
the two tosses are independent.
Example: A box contains five tickets labeled 1, 1, 2, 2 and 3.
Two draws are made at random, with replacement, from the
box. Suppose the first draw is a 1, what are the chances of
getting a 2 in the second draw?
Independence
When drawing with replacement the ticket that is drawn the first
time is replaced back into the box. So the chances of getting a 2
in the second draw are the same for any outcome of the first
draw. Suppose the draws are done without replacement. Then
after the first draw there is one less ticket in the box and this
affects the probability of drawing a 2 in the second draw.
When two events are independent the probability that both will
happen is the product of their unconditional probabilities. This is
a special case of the multiplication rule.
Notation
Consider an event A. The probability of A is
denoted as P(A).
Consider two events A and B. The conditional
probability of A given B is denoted as P(A|B).
The multiplication rule can be written as:
P(A & B) = P(A)P(B|A)=P(B)P(A|B).
The events A and B are independent if:
P(A|B)=P(A) and P(B|A)=P(B)
When the events A and B are independent then
the multiplication rule can be written as:
P(A & B) = P(A)P(B).
Counting outcomes
As we saw previously, when sampling at random, the
probability of an event can be calculated by counting the
number of possible outcomes that correspond to the event and
dividing that number by the total number of outcomes. The
former implies listing the ways something can happen. Let's
consider the problem of rolling two dice. There are 6 possible
outcomes for the first die and, for each of those, there are 6
possible outcomes of the second die to pair with the results of
the first. So we have 6 x 6 = 36 possible pairs as outcomes of
the experiment. We can use the following 6 x 6 table to answer
a few questions regarding rolling two dice.
Counting outcomes
Q: What is the
probability that the
sum will be equal to
2?
A: There is only one
way of getting 2 as
the sum of the two
dice, so
the probability is
1/36.
Q: What is the probability that the sum will be 7?
A: There are 6 possible ways of getting a sum of 7, thus the
probability is 6/36 = 1/6.
Counting outcomes
With three dice life is even more complicated. To start with we
have 6 possibilities for each die. That produces 6 x 6 x 6 = 63 =
216 possible outcomes. Suppose we want to know the chances
that the sum of the three dice is 9. We can do that by obtaining
the following combinations
126 135 144 234 225 333
If we think of the chances that the sum will be equal to 10 we
also get six possible combinations
145 136 226 235 244 334
Nevertheless the chances of a 9 and a 10 are different since the
listed combinations can be obtained in a different number of
ways. In fact the following table counts the number of triplets
that produce each outcome.
Counting outcomes
So the probability that the sum will be 9 is 25/216 whilst the
probability that sum will be 10 is 27/216.
Addition Rule
We saw that the multiplicative rule is useful when looking at two
events that occur jointly. So it is related to the problem of
observing A and B. Let's consider the or case.
Two dice are rolled and the sum of the two is observed. The
events: the sum is greater than 6 and the sum is smaller than 3
are disjoint.
Addition Rule
Q: What are the chances that a card from a well shuffled deck
will be either hearts or spades?
A: We either think that there are 26 out of the 52 cards that are
either spades or hearts, and so the probability is 26/52 = 1/2.
Or we can think that the chance of the card being hearts is 1/4
and the chance of it being spades is 1/4. Since the two events
are disjoint, the chance of either spades or hearts is 1/4+1/4 =
1/2.
Q: What are the chances of getting at least one 1 when two dice
are rolled?
A: Consider the events: 1 in the red die and one in the blue die.
If we use the addition rule then the chances are 1/6+1/6 = 1/3
= 12/36. But if we observe the table we have that there are 11
ways of getting a 1. Thus the chances are just 11/36.
Addition Rule
The addition rule does not apply in this case since the two
events are not disjoint. The addition rule is counting the
outcome (1, 1) twice.
The right answer is obtained by subtracting the probability that
the events happen simultaneously. In the dice example we have
1/6 + 1/6 - 1/36 = 11/36, which is the correct answer.
The mathematical notation for this is:
P(A or B) = P(A) + P(B) - P(A and B)
Independence vs. Disjoint
Q: If two events are disjoint, are they also independent?
A: No. Actually when two events are disjoint, the occurrence of
one gives a lot of information about the occurrence of the other.
In particular, the other can not happen!
Remember that two events are independent when the probability
of one is unaffected by the other event happening. Since this is
not the case when events are disjoint, then they are not
independent.
Suppose A is an event and denote Ac as the opposite of A. Then
A and Ac are disjoint but, for any value of P(A), P(A|Ac) = 0, so
they are not independent.
Examples
Q1: What is the probability that in four rolls of a die at least one 1
will turn up?
A: In one roll of a die there is 1/6 probability of getting a 1. In 4
rolls there is:
1 2
4× =
6 3
Q2: What is the probability that in 24 rolls of a pair of dice, at least one
double 1 will turn up?
A: In one roll of a pair of dice there is 1/36 chance of getting a
double 1. In 24 rolls the chances are: 24 ×
1 2
=
36 3
We are adding probabilities for non mutually exclusive events.
To see how wrong our argument is, think of rolling one die 8 times, then we
would get 8 ×
1
>1
6
probability of observing a 1!
Examples
What is the right calculation?
First problem: Think of the opposite event. The gambler looses if
none of the four rolls come up 1. What are the chances of not
getting a 1 in a specific roll? This can be calculated as 1 - 1/6 =
5/6. For the gambler to loose this has to happen the first and the
second and the third and the fourth. The rolls are all
independent, thus we use the multiplication rule and obtain
(5/6)4 = 0.482. Thus the chance of winning is 1 - 0.482 = 0.518.
Second problem: In a similar way we get the chances of the
event which is the opposite of getting a double 1 in one
particular roll: 35/36. We want this to happen 24 times, so we
get (35/26)24 = 0.509. The chance of winning is 1 - 0.509 =
0.491.
Components in Series/Parallel
A system of three components in series is such that the whole
system works if all the components work. That corresponds to
the figure
If all three components are independent and they have
probability P1, P2 and P3 of working properly, then the system
will work properly if all three components work and that, using
the multiplication rule, happens with probability
P = P1× P2× P3
This can be generalized to an arbitrary number of components.
Components in Series/Parallel
Suppose that a system is made of three component and it is
such that it will work if any of the three components work
properly. Then the components are in parallel. This corresponds
to the figure below.
To work out the
probability that the
system will function,
consider the opposite
event: the system will
not function. This only
happens if all three
components do not
function.
Components in Series/Parallel
The probability that component 1 will not function is 1-P1. For
component 2 we have 1-P2 and for component 3 we have 1-P3.
Using the multiplicative rule we obtain that the system will not
function with probability (1-P1)(1-P2)(1-P3). So the
probability that the system will function is:
P = 1-(1-P1)(1-P2)(1-P3)
Again, this can be generalized to an arbitrary number of
components.
Suppose P1 = P2 = P3 = 1/3 then:
Series: P = (1/3)3 = 1 / 27.
Parallel: P = 1-(1-1/3)3 =19 / 27.
Summary of Probability Rules
0 ≤ P(A) ≤ 1.
P(not A) = 1 – P(A).
P(A or B) = P(A) + P(B) –P(AB) (addition rule).
If A, B are disjoint then P(A or B) = P(A) + P(B).
P(A and B) = P(A)P(B|A)=P(B)P(A|B) (multiplication
rule).
If A, B are independent then P(A and B) = P(A)P(B).
From the multiplication rule we get the definition of
the conditional probability:
P(A and B)
P(A | B) =
P(B)