Download Basic Probability - Arizona State University

Basic Probability
Theoretical versus Empirical


Theoretical probabilities are those that
can be determined purely on formal or
logical grounds, independent of prior
experience.
Empirical probabilities are estimates of
the relative frequency of an event
based by our past observational
experience.
Theoretical Probability
Probability of
 A tossed coin landing on heads
 Drawing a spade from a poker deck
 Observing a three when rolling a die
Empirical Probability



Empirical probability can be subdivided
into two categories:
Objective versus Subjective
Probability that conception will result in
twins (Objective)
Probability of an insurance applicant
filing a claim (Objective)
Objective Probability

The previous examples can be
considered objective in the sense
that they are based on observations
of past occurrences of events, under
what are hopefully the same
conditions that currently prevail.
Subjective Probability


Empirical in the sense that they are ultimately
based on past observation
Subjective in the sense that the particular
observation(s) upon which the particular
probability estimate(s) are based, is not well
defined, that is, a independent observer could
not be instructed on how to arrive at the
same probability
Subjective Probability




What is the probability that a space satellite
will fall out of orbit and land on Tucson?
What is the probability that a direct-response
advertisement will draw a profitable
response?
What is the probability of extra-terrestrial life?
What is the probability that upon graduation,
you will be offered a position on your first job
interview?
Basic Probability Concepts
Probability Experiments

Whenever we manipulate or make an
observation on our environment with an
uncertain outcome, we have conducted an
experiment.
Examples



Taking an exam
Tossing a coin
Delivering a sales pitch
Probability Experiment



Can be repeated many times
(at least in theory)
Each such repetition is called a trial
When an experiment is performed it can
result in one or more outcomes, which
are called events.
Sample Space




The set of all possible outcomes of an
experiment is called the sample space, S, for
the experiment
The outcomes in the sample space are called
sample points
The outcomes forming the sample space
must be mutually exclusive and exhaustive
The sample space and sample points depend
on what the experimenter chooses to observe
Example – Toss a Coin Twice



Can record the sequence of heads (H)
and tails (T), then S= {HH, HT, TH, TT}
Can record the total number of tails
observed, then S= {0, 1, 2}
Can record whether the two tosses
match (M) or do not match (D), so
S= {M, D}
Exercise (Sample Spaces)

Determine the sample space of the following
experiments:




Toss a die and recording the number on the top
face
Let a light bulb burn until it burns out
Observe General Electric common stock and
recording whether it increased, decreased or
remained unchanged during one market day
Record the sex of successive children in a threechild family
Events




An event, E, is a subset of the sample space
and it denoted by E  S
An event E is said to occur if the outcome of an
experiment is an element of E
Consider the experiment of tossing a die once
and recording the number on the top face.
The sample space, S= {1, 2, 3, 4, 5, 6}
Example (Events)





Some events associated with this
experiment are:
E1={1}
We observe a 1
E2={2}
We observe a 2
E3={1,3,5} We observe an odd
number
E4={1,2,3} We observe a number less
than 4.
Simple vs Compound Events


A simple event cannot be decomposed.
A compound event is an event that can
be decomposed into other events.
E1 and E2 are simple events.
E3 and E4 are compound events.
Exercise



Consider the experiment of flipping a
balanced coin three times.
Determine the sample space for the
experiment
List two events that correspond to this
experiment
Teminology








Experiment
Sample space
Sample points
Probability model
Events
Certain event
Impossible event
Mutually exclusive (disjoint) events
Discrete Sample Space


A discrete sample space consists of a
finite number of sample points or a
countable number of sample points.
Throughout Project 1, we will be
concerned with finite discrete sample
spaces.
Probability of an Event



Given an event, we would like to assign it a
number, P(E), called the probability of E
This number indicates the likelihood that the
event will occur.
We can find this number by determining the
value of the ratio:
number of ways event can occur
total number of outcomes
Relative Frequency


Suppose that we repeat the die tossing
experiment n times and notice that the event
E1 occurs f times. We call the ratio f / n the
relative frequency of the event after n
repetitions.
If we repeat this experiment indefinitely and
if the ratio f / n approaches a number, p, as n
becomes larger and larger, then p is called
the probability of the event.
Law of Large Numbers


The more repetitions we take, the better the
approximation
p f/n
This is sometimes referred to as the Law of
Large Numbers, which states that if an
experiment is repeated a large number of
times, the relative frequency of the outcome
will tend to be close to the probability of the
outcome.
Summary of 20,000 Coin Tosses
Num of Tosses Num. of Heads Relative Freq.
n
f
f/n
10
8
.8000
100
62
.6200
1,000
473
.4730
5,000
2,550
.5100
10,000
5,098
.5098
15,000
7,649
.5099
20,000
10,038
.5019
Fundamental Properties
Upon analyzing the relative frequency
concept, we see the following must hold:
1. Negative relative frequencies do not make
sense
2. The relative frequency of the sample space
must be 1
3. If two events are mutually exclusive, the
relative frequency of their union must be the
sum of their relative frequencies.
Fundamental Properties Cont.
1. For an event E, 0P(E) 1
2. P(S)=1, where S is the sample space
3. P(E  F)= P(E)+ P(F), where E and F
are mutually exclusive events
4. P(E1  E2 …  Ek)
= P(E1)+ P(E2)+…+ P(Ek), where
the Ek’s are mutually exclusive.
Calculating P(E)
1. Define the experiment and clearly
determine how to describe one simple
event
2. List the simple events associated with
the experiment and test each to be
certain
that
they
cannot
be
decomposed. This defines the sample
space S.
Calculating P(E) Continued.
3. Assign probabilities to the sample
points in S making certain that the
Fundamental Properties for a discrete
sample space are preserved.
4. Define the event, E, as a specific
collection of sample points.
5. Find P(E) by summing the
probabilities of the sample points in E.
Example

A balanced coin is tossed three times.
Calculate the probability that exactly
two of the three tosses results in heads.
Example
A balanced coin is tossed three times.
 Let E1 be the event that you observe at
least two heads. What is P(E1)?
 Let E2 be the event that you observe at
exactly two heads. What is P(E2)?
 Let E3 be the event that you observe at
most most heads. What is P(E3)?
 What can you say about E1 and E3
Basic Theorems of Probability
Let S be a discrete sample space.
 Theorem 1: P()=0, where  is the
empty set.
 Theorem 2: For any two events E and F
in S, P(E  F)= P(E) + P(F) - P(E  F)
 Theorem 3: If E is an event in S, then
P(EC)= 1 - P(E)
Mutually Exclusive

Two events are mutually exclusive if AB=.
A

B
If A and B are mutually exclusive, then
P  A  B   P( A)  P( B)
Mutually Exclusive

If no two events E1, E2, . . . , En can
happen at the same time, then
P  E1  E2  .....  En   P(E1 )  P(E2 )  .....  P(En )