Download Chapter 5: Probability

Chapter 5: Probability
Randomness
●
●
●
A random phenomenon is a situation in which we know
what outcomes could happen, but we don't know which
particular outcome did or will happen
However, we can calculate the probability with which
each outcome will happen
People are not good at identifying truly random samples
or random experiments, so we need to rely on outside
mechanisms such as coin flips or random number tables
Simulating Randomness
●
●
●
●
Flip a coin to generate random 0's and 1's
Pick a card to generate random numbers
Pick a number out of a hat
Use a Random Number table...
Simulating Randomness
●
●
●
●
Use a Random Number table
Ex. Simulate rolling a die 10 times
Pick any line to start, say line 30 to start
Go through the numbers, picking numbers 1-6 and ignoring
0,7,8,9.
The“random”numbers are: 5, 4, 5, 3, 4, 6, 2, 5, 3, 1
16% of cars fail pollution tests in California
●
●
●
We can represent this in many ways:
In a bag with 100 chips, 16 are red and represent cars
that fail pollution tests, the remaining 84 are black and
represent cars that pass the tests.
In a bag with 50 marbles, 8 are orange and represent
cars that fail pollution tests, the remaining 42 are blue
and represent cars that pass the tests.
On a random number table numbers 00-15 represent
cars that fail pollution tests and 16-99 represent cars
that pass the tests.
Estimating Probabilities via Simulation
●
●
●
●
We would like to estimate the probability that an entire fleet
of seven cars would pass using a random number table. We
assume each car is independent.
Start at row 1 of the random number table and read two
digits at a time.
If the random number is between 00-15, the car will fail the
pollution test. If the number is between 16-99, the car will
pass the test.
A fleet of cars is comprised of seven cars, i.e. seven 2-digit
random numbers. If all seven cars pass the test, record a 1,
if not record a 0.
Repeat many times and calculate the proportion 1s among
the total number of runs, i.e. number of fleets where all cars
passed the pollution test.
Run 1
Run 2
Run 3
Did all the cars in run 3 pass?
a) Yes?
b) No?
Run 4
Estimation of Probability
Based on the simulation results, estimate the probability that
an entire fleet of seven cars would pass the pollution test.
Notes
●
●
●
4 runs is usually not considered sufficient, for homework use
at least 5 runs.
You can start at any row you like on the random number
table, but you should make sure to note it when you’re
writing up your simulation scheme.
You should not arbitrarily pick numbers from the random
number table. Just pick a row and follow across. Otherwise
the numbers you’re using won’t be random (they will
instead be your choices).
Estimation of Probability
Based on the simulation results, estimate the average
number of cars that fail the pollution test in a fleet of seven
cars.
a) 0 b) 1 c) 2 d) 3 e) 4
Randomness
●
●
A random phenomenon
is a situation in which
we know what
outcomes could happen,
but we don't know
which particular
outcome did or will
happen
However, we can
calculate the probability
with which each
outcome will happen
Randomness to Probability
Let's say there are 1000
songs in your iTunes
library. Only 5 of these
songs are by Justin Bieber.
What is the probability
that next time you hit
shuffle you get a Justin
Bieber song?
Prob(JB song) = 5/1000
= 0.005
Definitions of Probability
●
A Frequentist: The probability of an event is its long-run
relative frequency.
●
●
●
Theoretical: We may not be able to predict which song we play
each time we hit shuffle, however we know that in the long run 5
out of 1000 songs will be Justin Bieber songs.
Empirical: We listen to 100 songs on shuffle and 4 of them are
Justin Bieber songs. The empirical probability is 4/100=0.04
A Bayesian:
●
●
Prior probability of an event is the best guess by the observer of
an event's probability.
Posterior probability of an event is the probability of an event
after collecting some empirical data. It is obtained by updating
information from the prior probability with additional data related
to the event in question. The prior probability may be subjective,
but with enough data, has little impact on the posterior
probability.
Definitions of Probability
Note: In this course we will use the frequentist definition of
probability, though we will make some use of Bayesian
tools.
More Definitions
●
For any random phenomenon, each attempt is called a
trial and each trial generates an outcome
●
●
●
Sample space is the collection of all possible outcomes of
a trial
●
●
Each time you hit shuffle is a trial.
The song that plays as a result of hitting shuffle is an outcome.
Sample space is the entire iTunes library of 1000 songs.
A combination of outcomes is called an event
● For example playing two Justin Bieber songs in a row
in shuffle
Sample Space
A couple has two kids, what is the sample space for the
gender of these kids?
Calculating Probabilities
If a couple has two kids what is the probability that
both are boys?
If a couple has two kids, the list of possible scenarios
are:
BB, GG, BG, GB
Since each scenario is equally likely:
Prob(BB) = ¼ = 0.25
Independence
●
●
●
●
When thinking about what happens with combinations of
outcomes, things are simplified if the individual trials are
independent.
Roughly speaking, this means that the outcome of one trial
doesn't influence or change the outcome of another.
If the iTunes shuffle is truly random then the songs played
are independent of each other.
In other words, the iTunes shuffle is memoryless, it does not
say to itself“Wait, I just played an Justin Bieber song, I
shouldn't play one again”
Similarly, if the genders of kids a couple has are
independent, then the probability of having a boy for a
second child doesn't change based on whether or not the
first child of the couple was a girl.
Definitions – Coin toss example
●
●
●
●
●
Trial: each coin toss
Outcome: Heads or Tails
Probability: P(H) = 0.5 and
P(T) = 0.5
Note: Each time we toss a coin we
can't tell which side will come up,
however in the long run tails will
come up 50% of the time and
heads will come up 50% of the
time
Sample Space:
Tossed once: S = {H,T}
Tossed twice: S = {HT,TH,HH,TT}
Independence: The outcome of one
coin toss does not affect the
outcome of the next
Dice
●
●
●
●
●
Trial: each die roll
Outcome: A die has six sides
Probability: With a fair die the
probability of getting each side is
the same and is 1/6
Sample Space:
Rolled once: S = {1, 2, 3, 4, 5, 6}
Independence: The outcome of one
die roll does not affect the outcome
of the next
Deck of Cards
●
●
●
●
●
A deck is made up of 52 cards
There are 4 suits: clubs, spades,
diamonds, hearts
Each suit has 13 cards
Within each suit there is one ace,
numbers 2-10, a jack, a queen,
and a king
Clubs and spades are black cards,
diamonds and hearts are red cards
Law of Large Numbers
●
●
●
The Law of Large Numbers (LLN) says that the long-run
relative frequency of repeated independent events gets closer
and closer to the true relative frequency as the number of
trials increases
● If a coin is tossed many times, the overall percentage of
heads should settle down to about 50% as the number of
tosses increases
This is why we define probability as the long run relativefrequency of an event
The common (mis)understanding of the LLN is that random
phenomena are supposed to compensate for whatever
happened in the past; this is just not true and if also called
gambler's fallacy (or law of averages)
Law of Large Numbers
When tossing a fair coin, if heads comes up on each of the
first 10 toss, what do you think the chance is that another
head will come up on the next toss?
HHHHHHHHHHHHHH?
●
●
●
The probability is still 0.5, or there is still a 50% chance that
another head will come up on the next toss
Prob(H on 11th toss) = Prob(T on 11th toss) = 0.5
The coin is not due for a tails
Equally Likely Outcomes
●
●
Equally likely outcomes have the same probability of
happening
● It's equally likely to get any one of six outcomes from
the roll of a fair die
● It's equally likely to get heads or tails from the toss
of a fair coin
Not all outcomes are equally likely
● Its not equally likely to draw an ace as a red card
● A skilled basketball player has a better than 50-50
chance of making a free throw
Personal Probability
●
●
WIn everyday speech, when we express a degree of
uncertainty without basing it on long-run relative
frequencies, we are stating subjective or personal
probabilities
Personal probabilities don't display the kind of consistency
that we will need probabilities to have, so we will stick with
formally defined probabilities
Impossibility to a Certainty
●
●
●
Probabilities must be between 0 and 1, inclusive.
A probability of 0 indicates impossibility
A probability of 1 indicates certainty
Note: If you calculate a probability and get a negative
number or a number greater than 1 you are making a
mistake, go back and check your work.
Something has to happen rule
The probability of the seat of all possible outcomes of a trial
must be 1
Prob(sample space) = 1
●
●
Coin toss: Prob(H) + Prob(T) = 0.5 + 0.5 =1
Cards: Prob(face card) + Prob(non – face card)
= 12/52
+
40/52 = 52/52 = 1
In a class 30% of the students are freshmen, 20% are
sophomores, 10% are juniors and the rest are seniors. If we
randomly pick a student what is the probability that he or she
is a senior?
a) 0.3
b) 0.4
c) 0.2
c) 0.1
Complement Rule
The set of outcomes that are not in the event A is called the
complement of A, denoted
. The probability of an event
occurring is 1 minus the probability that it does not occur.
In a survey, 52% of respondents said they are Democrats.
What is the probability that a randomly selected respondent
from this sample is a Republican?
a) greater than 48%
b) less than 48%
c) equal to 48%
d) cannot be computed from the information
Disjoint Events
Events that have no outcomes in common (and, thus,
cannot occur together) are called disjoint (or mutually
exclusive).
●
●
●
The outcome of a coin toss cannot be a head and a tail
A student cannot fail and pass a class
A card drawn from a deck cannot be an ace and a queen
Addition Rule
For two disjoint events A and B, the probability that one
OR the other occurs is the sum of the probabilities of the
two events.
P(A or B) = P(A) + P(B)
Do the sum of two disjoint events always add up to one?
– Yes?
– No?
Multiplication Rule
For two independent events A and B, the probability that
both A AND B occur is the product of the probabilities
of the two events.
P(A or B) = P(A) x P(B)
In a multiple choice exam there are 5 questions and 4
choices for each question (a, b, c, d). Nancy has not studied
for the exam at all, and decided to randomly guess the
answers. What is the probability that she gets the first
question right?
a) 1
c) 0.2
b) 0.25
d) 0.75
What is the probability that she gets the second question
right?
a) 1
c) 0.25
b) 0.2
d) 0.4
Example – Guessing at Random
What is the probability that the first question she gets right
is the 5th question?
Prob(Wrong Wrong Wrong Wrong Right)
= 0.75 x 0.75 x 0.75 x 0.75 x 0.25
= 0.0791
What is the probability that she gets all the questions right?
a) 1.25
c) 0.0039
b) 0.0010
d) 0
Example – Guessing at Random
What is the probability that the first question she gets at
least one question right?
If there are 5 questions, the possible number of questions
she gets right are:
S = {0,1,2,3,4,5}
We are interested in instances where she gets at least 1
question right:
S = {0,1,2,3,4,5}
So we can divide up the sample space into two
categories:
S = {0, at least 1}
Example – Guessing at Random
Independent vs. Disjoint
Can two events be independent and disjoint?
Remember: Independence means that the outcome of one
trial doesn't influence the outcome of another and disjoint
means that two events cannot occur together.
If we know that the outcome of a coin toss is heads,
then we know that it is not tails. So whether or not the
outcome is tails depends on whether or not the outcome
is heads. Therefore these two events are disjoint and
dependent.
Independent vs. Disjoint
●
Disjoint events cannot be independent
Since we know that disjoint events have no outcomes in common,
knowing that one occurred means the other didn't happen.
● Thus, the probability of the one event occurring changed based on
our knowledge that the other occurred.
● Therefore the two events are not independent.
Non-disjoint events may or may not be independent
●
●
●
●
●
Just because two events can occur at the same time does not
mean they need to also be dependent.
It is possible that two randomly selected students get an A in a
class, but how they do in the class may have nothing to do with
each other.
However if you and a close friend of yours that you study with
both get an A in a class, how well the two of you do in that class
are possibly dependent on each other.
Properties of Probability - Summary
●
●
●
When thinking about what happens with combinations of
outcomes, things are simplified if the individual trials are
independent.
Roughly speaking, this means that the outcome of one trial
doesn't influence or change the outcome of another.
If the iTunes shuffle is truly random then the songs played
are independent of each other.
IIn other words, the iTunes shuffle is memoryless, it does not
say to itself“Wait, I just played an Justin Bieber song, I
shouldn't play one again”
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. What is the
probability that a student chosen at random doesn't miss
any days of school due to sickness?
Prob(no misses) = 1 - (0.20 + 0.13 + 0.05) = 0.62
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. What is the
probability that a student chosen at random doesn't miss
any days of school due to sickness?
a) 0.38
c) 0.18
b) 0.33
d) 0.62
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. What is the
probability that a student chosen at random misses no more
than one day?
Prob(at most 1 miss) =
Prob(no misses) + Prob(1 miss) =
0.62 + 0.20 = 0.82
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. If a parent has two
kids at this middle school, what is the probability that
neither will miss any school?
Prob(neither miss any) =
Prob(no miss) x Prob(no miss)
0.62 x 0.62 = 0.3844
Example – Absence from School
We used the multiplication rule to calculate the probability
in the previous slide.
a) What must be true about these kids to make that
approach valid?
b) Do you think this assumption is reasonable?
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. If a parent has two
kids at this middle school, assuming that the kids are
independent with respect to missing school, what is the
probability that both will miss some school, i.e. at least one
day? Choose the closest answer.
a) 0.14
c) 0.38
b) 0.24
d) 0.76
Example – Absence from School
A middle school estimates that 20% of its students miss one
day of school per semester due to sickness, 13% miss two
days, and 5% miss three or more days. If a parent has two
kids at this middle school, assuming that the kids are
independent with respect to missing school, what is the
probability that one kid misses some school and the other
doesn't miss any?
General Addition Rule
Example – Second Language
In a class where everyone's native language is English, 70%
of students speak Spanish, 45% speak French as a second
language and 20% speak both. Assume that there are no
students who speak another second language. What is the
probability that a randomly selected students speaks Spanish
or French?
Prob(Spanish or French) =
Prob(Spanish) + Prob(French) - Prob(Spanish and French) =
0.70 + 0.45 - 0.20 = 0.95
Picturing Probabilities
The most common kind of picture we will make to picture
probabilities is called Venn Diagram.
Example – Second Language
In a class where everyone's native language is English, 70%
of students speak Spanish, 45% speak French as a second
language and 20% speak both. Assume that there are no
students who speak another second language.
Conditional Probability
Example – Second Language
In a class where everyone's native language is English, 70%
of students speak Spanish, 45% speak French as a second
language and 20% speak both. Assume that there are no
students who speak another second language. What is the
probability that a randomly selected student speaks French
given that they speak Spanish?
Example – Second Language
In a class where everyone's native language is English, 70%
of students speak Spanish, 45% speak French as a second
language and 20% speak both. Assume that there are no
students who speak another second language. What is the
probability that a randomly selected student speaks Spanish
given that they speak French?
a) 0.45
c) 2.25
b) 0.44
d) 0.05
Example – Burgers in LA
A survey conducted in 2010 by Survey USA for KABC-TV
Los Angeles asked 500 Los Angeles residents“What is the
best hamburger place in Southern California? Five Guys
Burgers? In-N-Out Burger? Fat Burger? Tommys
Hamburgers? Umami Burger? Or somewhere else?”
Consider the results of this survey, shown in the table below,
in each of the following questions.
Example – Burgers in LA
●
Refer to the table on the previous slide:
What is the probability that a randomly chosen male likes
In-N-Out the best?
●
What is the probability that a randomly chosen female likes
In-N-Out the best?
●
What is the probability that a man AND a woman who are
dating both like In-N-Out the best?
Example – Burgers in LA
●
What is the probability that a randomly chosen person likes
Umami best or that the person is a female?
a) 0.006
c) 0.514
b) 0.504
d) 0.516
General Multiplication Rule
Checking for independence using
conditional probabilities
Example – Second Language
In a class where everyone's native language is English, 70%
of students speak Spanish, 45% speak French as a second
language and 20% speak both. Assume that there are no
students who speak another second language. Are the
variables speaking French and speaking Spanish
independent?
Independent vs disjoint - recap
●
●
If A and B are disjoint:
● Prob(A and B) = 0
● A and B are not independent
If A and B are not disjoint:
● Prob(A and B) > 0
● A and B may or may not be independent
Example
An online survey shows that 25% of bloggers own a DSLR
camera, 90% own a point-and-shoot camera, and 20% own
both types of cameras. Are owning a DSLR and point-andshoot camera disjoint?
Example
An online survey shows that 25% of bloggers own a DSLR
camera, 90% own a point-and-shoot camera, and 20% own
both types of cameras. Are the variables owning a DSLR
and owning a point-and-shoot camera independent?