Download Essentials of Mathematical Statistics

Chapter 1
Basics of Probability
Chapter 1 – Basics of Probability
1.
2.
3.
4.
5.
Introduction
Basic Concepts and Definitions
Counting Problems
Axioms of Probability and the Addition Rule
Conditional Probability and the
Multiplication Rule
6. Bayes’ Theorem
7. Independent Events
1.1 - Introduction
• Probability deals with describing random
experiments
– An activity in which the result is not known until it
is performed
– Most everything in life is a random experiment
• Informally
– Probability is a measure of how likely something is
to occur
• Probabilities take values between 0 and 1
1.2 – Basic Concepts and Definitions
• Definition 1.2.1
– An outcome is a result of a random experiment
– The sample space of a random experiment is the
set of all possible outcomes
– An event is a subset of the sample space
• Notation
– Events are typically denoted by capital letters
– P(A) = probability of event A occurring
Calculating Probabilities
Relative Frequency Approximation: To
estimate the probability of an event A, repeat the
random experiment several times (each
repetition is called a trial) and count the number
of times the event occurred. Then
number of times A occurred
P  A 
number of trials
The fraction on the right is called a relative
frequency.
Calculating Probabilities
Theoretical Approach: If all outcomes of a
random experiment are equally likely, S is the
finite sample space, and A is an event, then
n  A
P  A 
nS 
where n (A) is the number of elements in the set
A and n (S) is the number of elements in the set
S.
Relation
Law of Large Numbers: As the number of trials
gets larger, the relative frequency approximation
of P (A) gets closer to the theoretical value.
A probability is an average in the long run.
Example
• Bag with 4 blue, 3 red, 2 green, and 1 yellow
cube
• Question: “How likely is it that we get a green
cube?”
– Let G = event we get a green cube
– We want to know P(G)
Relative Frequency
• Choose a cube, record its color, replace, and
repeat 50 times
Student 1
Student 2
Theoretical Approach
• Let
S   B1 , B2 , B3 , B4 , R1 , R2 , R3 , G1 , G2 , Y1 , and
G  G1 , G2  ,
then
n( S )  10, n(G )  2
2
 P(G )   0.2
10
Law of Large Numbers
• Combine students’ results
– 50 + 50 = 100 trials
– 12 + 7 = 19 green cubes
19
Rel Freq 
 0.19  0.2
100
Example 1.2.3
• Roll two “fair” six-sided dice and add the
results
– Let A = event the sum is greater than 7
– Find P(A)
Sample Space
15 5
P  A 
  0.417
36 12
Random Variable
• Let X = the sum of the dice
– Called a random variable
– Table is called the distribution of X
Subjective Probabilities
• Subjective Probabilities: P(A) is estimated by
using knowledge of pertinent aspects of the
situation
Example 1.2.4
• Will it rain tomorrow?
– Let
S  rain, not rain and A  rain
then
1
P( A) 
2
since the outcomes are not equally likely
Example 1.2.4
• What is P(A)?
– Weather forecasters use knowledge of weather and
information from radar, satellites, etc. to estimate a
likelihood that it will rain tomorrow
Example 1.2.5
• Suppose we randomly choose an integer
between 1 and 100
• Let
S  1, , 100 and A  75, , 100
then
n  A  26
P  A 

 0.26
n  S  100
Example 1.2.5
• Suppose we randomly choose any number
between 1 and 100
• Let
S  1, 100 and B   75, 100
then a reasonable assignment of probability is
length of interval B 25
P  B 

 0.253
length of interval S 99
1.3 – Counting Problems
• Fundamental Counting Principle: Suppose a
choice has to be made which consists of a
sequence of two sub-choices. If the first choice
has n1 options and the second choice has n2
options, then the total number of options for
the overall choice is n1 × n2.
• Definition 1.3.1 Let n > 0 be an integer. The
symbol n! (read “n factorial”) is
n !  n   n  1   n  2  
 2 1
For convenience, we define 0! = 1.
Permutations
Definition 1.3.2 Suppose we are choosing r objects
from a set of n objects and these requirements are
met:
1. The n objects are all different.
2. We are choosing the r objects without replacement.
3. The order in which the choices are made is important.
Then the number of ways the overall choice can be
made is called the number of permutations of n
objects chosen r at a time.
n!
n Pr 
 n  r !
Combinations
Definition 1.3.3 Suppose we are choosing r objects
from a set of n objects and these requirements are
met:
1. The n objects are all different.
2. We are choosing the r objects without replacement.
3. The order in which the choices are made is not
important.
Then the number of ways the overall choice can be
made is called the number of combinations of n
objects chosen r at a time.
n
n!
n Cr    
 r  r ! n  r  !
Arrangements
Definition 1.3.4 Suppose we are arranging n
objects, n1 are identical, n2 are identical, … , nr
are identical. Then the number of unique
arrangements of the n objects is
n!
n1 ! n2 ! nr !
Example 1.3.7
A local college is investigating ways to improve the
scheduling of student activities. A fifteen-person
committee consisting of five administrators, five
faculty members, and five students is being formed.
A five-person subcommittee is to be formed from
this larger committee. The chair and co-chair of the
subcommittee must be administrators, and the
remainder will consist of faculty and students. How
many different subcommittees could be formed?
Example 1.3.7
• Two sub-choices:
1. Choose two administrators.
2. Choose three faculty and students.
• Number of choices:
10 
5 P2  
  20 120  2400
3
Binomial Theorem
 n  nk k
( a  b)     a b
k 0  k 
n
n
• Example 1.3.12
 4  4k k
( x  y)     x y
k 0  k 
 4  40 0  4  41 1  4  4 2 2  4  43 3  4  44 4
  x y  x y  x y  x y  x y
0
1
 2
 3
 4
4
4
 x 4  4 x3 y  6 x 2 y 2  4 xy 3  y 4
1.4 – Axioms of Probability and the
Addition Rule
Axioms of Events Let S be the sample space of a
random experiment. An event is a subset of S.
Let E be the set of all events, and assume that E
satisfies the following three properties:
1. S  E
2. If A  E , then A  E (where A is the complement of A)
3. If A1 , A2 ,  E , then A1  A2   E
1.4 – Axioms of Probability and the
Addition Rule
Axioms of Probability Assume that for each
event 𝐴 ∈ 𝐸, a number P(A) is defined
in such a way that the following three properties
hold:
1. 0  P( A)  1
2. P  S   1
3. If A1 , A2 ,  E for which Ai  A j   for i  j , then
P  A1  A2 
 An   P  A1   P  A2  
for each integer n  0, and
P  A1  A2 
  P  A1   P  A2  
 P  An 
Theorem 1.4.2
Theorem 1.4.2 : P  A   P  A   1
Proof : Note that S  A  A and that A  A  . So,
using axioms 2 and 3, we have
1  P  S   P  A  A   P  A  P  A 
Theorem 1.4.3
Theorem 1.4.3 : Let A and B be events. If A  B,
then P( A)  P( B).
Proof : By elementary set theory,
B  A   B  A
so by axiom 3,

and
  A   B  A

P( B)  P A   B  A   P ( A)  P  B  A   P ( A)
since P  B  A   0 by axiom 1.
The Addition Rule
Theorem 1.4.4 : P( A  B)  P( A)  P( B)  P( A  B)
Example 1.4.4: Randomly choose a student
E = event student did not complete the problems
F = event student got a C or below
Example 1.4.4
38
34
26
P( E ) 
 0.475 P( F ) 
 0.425 P( E  F ) 
 0.325
80
80
80
P( E  F )  0.475  0.425  0.325  0.575
1.5 – Conditional Probability and the
Multiplication Rule
• Definition 1.5.1 The conditional probability of
event A given that event B has occurred is
P( A  B)
P( A | B) 
P( B)
provided that P( B)  0
Example 1.5.2
If we randomly choose a family with two
children and find out that at least one child is a
boy, find the probability that both children are
boys.
S  boy boy, boy girl, girl boy, girl girl
– A = event that both children are boys
– B = event that at least one is a boy
Example 1.5.2
– A = event that both children are boys
– B = event that at least one is a boy
S  boy boy, boy girl, girl boy, girl girl
3
1
P( B)  and P( A  B) 
4
4
P( A  B) 1/ 4 1
P( A | B) 


P( B)
3/ 4 3
Multiplication Rule
Definition 1.5.2 The probability that events A
and B both occur is one trial of a random
experiment is
P( A  B )  P ( A) P( B | A)
Example 1.5.4
Randomly choose two different students. Find the
probability they both completed the problems
A = event first student completed the problems
B = event second student completed the problems
Example 1.5.4
42 21
P( A) 

80 40
41
P( B | A) 
79
21 41
P( A  B)  P( A) P( B | A)    0.272
40 79
1.6 – Bayes’ Theorem
Definition 1.6.1 Let S be the sample space of a
random experiment and let A1 and A2 be two
events such that
A1  A2  S and A1  A2  
The collection of sets {A1, A2} is called a
partition of S.
1.6 – Bayes’ Theorem
Theorem 1.6.1 If A1 and A2 form a partition of S,
and B ⊂ S is any event, then for i = 1, 2
P  Ai  P  B | Ai 
P  Ai | B  
P  A1  P  B | A1   P  A2  P  B | A2 
Example 1.6.1
Medical tests for the presence of drugs are not perfect.
They often give false positives where the test indicates the
presence of the drug in a person who has not used the
drug, and false negatives where the test does not indicate
the presence of the drug in a person who has actually used
the drug.
Suppose a certain marijuana drug test gives 13.5% false
positives and 2.5% false negatives and that in the general
population 0.5% of people actually use marijuana. If a
randomly selected person from the population tests
positive, find the conditional probability that the person
actually used marijuana.
Example 1.6.1
Define the events
U  used marijuana, NU  not used marijuana,
T   tested positive, and T   tested negative
Note that {U, NU} forms a partition.
P T  | NU   0.135, P T  | NU   0.865,
P T | U   0.025,

P T | U   0.975

Example 1.6.1
P U | T   
P U  P T  | U 
P U  P T | U   P  NU  P T | NU 


(0.005)(0.975)

(0.005)(0.975)  (0.995)(0.135)
 0.035
Example 1.6.1
• “Tree diagram”
(0.005)(0.975)
P U | T  
(0.005)(0.975)  (0.995)(0.135)

1.7 – Independent Events
Definition 1.7.1 Two events A and B are said to
be independent if
P  A  B   P ( A) P ( B )
If they are not independent, they are said to be
dependent.
Informally: Independence means the occurrence of
one event does not affect the probability of the other
event occurring
Example 1.7.2
Suppose a student has an 8:30 AM statistics test
and is worried that her alarm clock will fail and
not ring, so she decides to set two different
battery-powered alarm clocks. If the probability
that each clock will fail is 0.005. Find the
probability that at least one clock will ring.
– Let F1 and F2 be the events that the first and
second clocks fail
– Assume independence
Example 1.7.2
P(at least one rings)  1  P(both fail)
 1  P  F1  F2 
 1  P  F1  P  F2 
 1   0.005  0.005 
 0.999975
Mutual Independence
Definition 1.7.2 Three events A, B, and C are
said to be pairwise independent if
P( A  B)  P( A) P( B), P( A  C )  P( A) P(C ),
and P( B  C )  P( B) P(C )
They are said to be mutually independent (or
simply independent) if they are pairwise
independent and
P( A  B  C )  P ( A) P ( B ) P (C )
Example 1.7.7
Suppose that on a college campus of 1,000
students (referred to as the population), 600
support the idea of building a new gym and 400
are opposed. The president of the college
randomly selects five different students and talks
with each about their opinion. Find the
probability that they all oppose the idea.
Example 1.7.7
Let
– A1 = event that the first student opposes the idea,
– A2 = event that the second student opposes it, etc.
These events are dependent since the selections
are made without replacement
400 399 398 397 396
P  A1  A2  A3  A4  A5  




1000 999 998 997 996
 0.010087
Example 1.7.7
Now suppose the selections are made with
replacement
– The events are independent
400 400 400 400 400
P  A1  A2  A3  A4  A5  




1000 1000 1000 1000 1000
 (0.4)5  0.01024.
Example 1.7.7
• Without replacement: Prob ≈ 0.010087
• With replacement: Prob = 0.01024
– Practically the same
• 5% Guideline: If no more than 5% of the
population is being selected, then the
selections may be treated as independent, even
though they are technically dependent