Download Probability 1

Probability 1
Random Experiment
A experiment with more than one outcome.
Outcomes vary in a unpredictable way when the experiment is repeated.
Sample space:
denoted by S, the collection of all possible outcomes of the random experiment.
Events are subsets of the sample space.
we usually denote them be A,B,C
elementarily events (outcomes) are single outcomes from the sample space, denoted usually by
a,b,c.
Set-Ops (algebra on events)
vendiagram
B
B
A
A
B
A
B
A
Union of A and B 𝐴 ∪ 𝐵, A or B or both occur
(maroon)
Interception of A and B 𝐴 ∩ 𝐵 both A and B occur
(darker section)
Complement: 𝐴𝐶
(Yellow)
𝑜𝑟 𝐴̅ A does not occur
Difference: 𝐴\𝐵 = 𝐴 ∩ 𝐵̅
A occurs, B does not
(light purple)
Rules:
̅̅̅̅̅̅̅
𝐴 ∪ 𝐵 = 𝐴̅ ∩ 𝐵̅
̅̅̅̅̅̅̅
𝐴 ∩ 𝐵 = 𝐴̅ ∪ 𝐵̅
Interception is done before union, Operation order.
The above rules daisy chain, with themselves but not with each other
Probability:
𝑃: {𝑠𝑢𝑏𝑠𝑒𝑡𝑠: 𝑜𝑓 𝑆} ∈ [0,1]
Axioms of probability:
1) P(A)≥0 for each A
2) P(S)=1
3) 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) if A and B are disjoint or mutrally exclusive ie 𝐴 ∩ 𝐵 = { } ==
𝑒𝑚𝑝𝑡𝑦 𝑠𝑝𝑎𝑐𝑒
Rules:
4) 𝑃(𝐴1 ∪ 𝐴2 ∪ … 𝐴𝑛 ) = 𝑃(𝐴1 ) + 𝑃(𝐴2 ) + 𝑃(𝐴𝑛 )
if Ai ∩ Aj = {} for each pair of i,j: i≠j
5) 𝑃(𝜙) = 0
as 𝐴 ∪ 𝜙 = 𝐴 and 𝐴 ∩ ∅ = ∅
so 𝑃(𝐴 ∪ 𝜙) = 𝑃(𝐴) = 𝑃(𝐴) + 𝑃(𝜙) therefore 𝑃(𝜙) = 0
6) P(A’)=1-P(A)
7) if 𝐴 ⊂ 𝐵 then 𝑃(𝐵\𝐴) = 𝑃(𝐴 ∩ 𝐵̅) = 𝑃(𝐵) − 𝑃(𝐴)
8) 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)
Example: page 77 prob 3.48
Know:
P(A)=0.35
P(B)=0.73
𝑃(𝐴 ∩ 𝐵) = 0.14
a)
𝐴 ∪ 𝐵 = 𝐴 ∪ (𝐵\(𝐴 ∩ 𝐵))
𝐴 ∩ (𝐵\(𝐴 ∩ 𝐵) = {}
𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴 ∪ (𝐵\(𝐴 ∩ 𝐵))) = 𝑃(𝐴) + P(B\(A∩ 𝐵))= 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)
b)𝑃(𝐴 ∩B’)
c) 𝑃(𝐴′ ∪ 𝐵′ ) = 1 − 𝑃((𝐴′ ∪ 𝐵′ )′ ) = 1 − 𝑃(𝐴′′ ∩ 𝐵′′ ) = 1 − 𝑃(𝐴 ∩ 𝐵) = 1 − 0.14
Conditional Probability.
what are the chances of something happening given something else has happened.
𝑃(𝐴|𝐵)𝑖𝑠 𝑃(𝐴)𝑔𝑖𝑣𝑒𝑛 𝐵 𝑜𝑐𝑐𝑢𝑟𝑠
Let 𝑛𝐴 be the number of times A occurs in n experiments
Let 𝑛𝐵 be the number of times B occurs in n experiments
Let 𝑛𝐴∩𝐵 be the number of times A and B both occurs in n experiments
𝑃(𝐴|𝐵) ≈
𝑛𝐴∩𝐵
𝑛𝐵
≈
𝑃(𝐴∩𝐵)
𝑃(𝐵)
𝑃(𝐴 ∩ 𝐵)
𝑃(𝐵)
𝑃(𝐴|𝐵) = 0 if P(B)=0
𝑃(𝐴|𝐵) =
Independence of events
Roughly about If 𝑃(𝐴|𝐵) = 𝑃(𝐴) A is independent of B.
𝑃(𝐴|𝐵) = 𝑃(𝐴) = 𝑃(𝐴|𝐵′ )
𝑃(𝐴′ |𝐵) = 𝑃(𝐴′ ) = 𝑃(𝐴′ |𝐵′ )
Definition of independence
A and B are said to be independent if 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵)
Properties:
𝑃(𝐴|𝐵) = 𝑃(𝐴) = 𝑃(𝐴|𝐵′ )
𝑃(𝐴′ |𝐵) = 𝑃(𝐴′ ) = 𝑃(𝐴′ |𝐵′ )
Similar hold true for B, in respect to prior knowledge of A.
From: 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵) (A and B are independent) it follows
𝑃(𝐴′ ∩ 𝐵) = 𝑃(𝐴′)𝑃(𝐵)
𝑃(𝐴 ∩ 𝐵′) = 𝑃(𝐴)𝑃(𝐵′)
𝑃(𝐴′ ∩ 𝐵′) = 𝑃(𝐴′ )𝑃(𝐵′)
Example (3.62 p89)
population of all individuals with certain income.
58% invest in money market
25% invest in stocks
19% invest in both
If you randomly pick a person who invests in money market, what is the probability that they
will also invest in stocks.
ie:
What is the probability of a person investing in stocks given that they invest in the money
market
Solution:
Relevant Events:
A: event that the person picked is investing in money market
B: event that the person picked is investing in stocks
We Know:
P(A)=0.58
P(B)=0.25
𝑃(𝐴⋂B)=0.19
We want to know:
P(B|A)
Example:
Let A and be mutually exclusive (disjoint).
𝐴⋂B={} .’. P(𝐴⋂B)=0
P(A)>0, P(B)>0
question: are A and B independent?
No as P(A)P(B)=\=0
Impendence of more than two events
three events; A,B,C are indendent if (all of the following are true)
i) 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵)
ii) 𝑃(𝐴 ∩ 𝐶) = 𝑃(𝐴)𝑃(𝐶)
iii) 𝑃(𝐵 ∩ 𝐶) = 𝑃(𝐵)𝑃(𝐶)
iv) 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶) = 𝑃(𝐴)𝑃(𝐵)𝑃(𝐶)
Properties:
P(A’∪ B|C)=P(A’∪ B)
P(C|A∩ B’)=P(C)
Similar for more than 3 events. (more than three won’t be ascessed)
Total Probability formula
Let 𝐴1 , 𝐴2 … 𝐴𝑛 be a partition of the sample space (S).
s.t.
𝐴𝑖 ∩ 𝐴𝑗 = { } for each pair i,j
𝑛
S = ⋃ 𝐴𝑖 = 𝐴1 ∪ 𝐴2 … ∪ 𝐴𝑛 = 𝑆
𝑖=1
Let B be an event which overlaps with some of the Partitions A_i (it must, as they cover all of S)
𝑛
𝑃(𝐵) = ∑ 𝑃(𝐵 ∩ 𝐴𝑖 )
𝑖=1
Remember: 𝑃(𝐴|𝐵) =
𝑃(𝐴∩𝐵)
=>
𝑃(𝐵)
P(A∩ B)=P(B)P(A|B)=P(A)P(B|A)
𝑛
𝑛
𝑃(𝐵) = ∑ 𝑃(𝐵 ∩ 𝐴𝑖 ) = ∑ 𝑃(𝐵|𝐴𝑖 )𝑃(𝐴𝑖 )
𝑖=1
𝑖=1
Example; (P90 Q3.73)
let
D be event of renting a car from agency D
E renting from agency E
F renting from agency F
B be the event of the car rented having bad tyres
P(D)=0.2
P(E)=0.2
P(F)=0.6
these are mutually exclusive
and D∪ E∪ F=S, so is a partition of S
P(B|D)=0.1,
P(B|E) =0.12
P(B|F)=0.04
10% of cars from D have bad tyres
Total Probability Formula
P(B)=P(B|D)P(D)+P(B|E)P(E)+P(B|F)P(F)
= 0.1x0.2 + 0.12x0.2 + 0.04x0.6= proability of getting bad tyres
Q3.74
What is the proability that a car with bad tyres that is rented, has been rented froim agency F.
𝑃(𝐹 ∩ 𝐵) 𝑃(𝐵|𝐹)𝑃(𝐹) 0.04 ∙ 0.6
𝑃(𝐹|𝐵) =
=
=
𝑃(𝐵)
𝑃(𝐵)
𝑃(𝐵)
Bayes Formula
𝐴1 , 𝐴2 … , 𝐴𝑛 forms a partition of S
𝑃(𝐴𝑖 |𝐵) =
𝑃(𝐵|𝐴𝑖 )𝑃(𝐴𝑖 )
𝑛
∑𝑘=1(𝑃(𝐵|𝐴𝑘 )𝑃(𝐴𝑘 ))
Random Variable (R.V)
A random variable, say X, is a real function on a sample space.
X:S→ℝ.
Roughly Maps partitions on to real numbers.
Discrete R.V: finite number of possible values of Random Variables>
Continuous RV: the possibilities cover an interval from ℝ
Discrete RVs
p(x)=P(X=x), x is a possible value of X
p(x) is called the probability the probability mass function of X. (can also be written 𝑃𝑋 (𝑥) )
Properties of Probaility Mass Function:
1) p(x)≥0 for all x
2) ∑all x p(x) = 1
Examples of important discrete R.V’s
1) beruoulli r.v: two possible outcomes: X has a sample space 𝑆𝑋 = 𝑎𝑙𝑙 𝑝𝑜𝑠𝑖𝑏𝑙𝑒 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑋
𝑆𝑋 = {0,1}
P(1)=p
called probability of “successes”
P(0)=1-p
called probability of “failure”
2) indicator function: (also a beruoulli r.v:)
if you divide a sample space into A (1) and A’ (0)
1 𝑤∈𝐴
𝐼𝐴 (𝑤) = {
0 𝑤∉𝐴
𝑃(𝐼𝐴 = 1) = 𝑃(𝐴)
Bernoulli RVs
let 𝑋1 , 𝑋2 , 𝑋3 , … , 𝑋𝑛 be n indiepent Bernoulli RVs, with probability of success p
(𝑋𝑖 ~ 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝑝) )
𝑌 = 𝑋1 + 𝑋2 + 𝑋3 + … 𝑋𝑛
Y is the count of successes in “n” indepenedent Bernoulli(p) r.v’s.
Defininition:y is calle Binominal r. (Y~Bin(n,p))
Find the pmf (probability Mass function) of Y.
𝑃𝑌 (𝑦) = 𝑃(𝑌 = 𝑦);
𝑃𝑌 (0) = 𝑃(𝑎𝑙𝑙 𝑡𝑟𝑖𝑎𝑙𝑠𝑒𝑞𝑢𝑎𝑙 𝑧𝑒𝑟𝑜) = (1 − 𝑝)(1 − 𝑝) … (1 − 𝑝) = (1 − 𝑝)𝑛 ;
𝑃𝑌 (1) = 𝑃(1 𝑠𝑢𝑐𝑐𝑒𝑠𝑠 𝑎𝑛𝑑 (𝑛 − 1) 𝑓𝑎𝑖𝑙𝑢𝑟𝑒𝑠) = 𝑛𝑝(1 − 𝑝)𝑛−1 ;
𝑃𝑌 (𝑘) = 𝑃(𝑘 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑒𝑠, (𝑛 − 𝑘)𝑓𝑎𝑖𝑙𝑢𝑟𝑒𝑠) = (𝑛𝑘)𝑝𝑘 (1 − 𝑝)(𝑛−𝑘) ; k=0,1,2,3…,n
Example: p112 Q4.11=
family individuals 13 individuals attend. Count how main have a cold.
is not modalable with a binomial, because they aren’t independent because other family
member could give each other colder.
A meeting of strangers, who don’t live near each other, etc.
We can now model with binomial how many have colds.
Because is independent.
A shop has 8 projecter, 2 of which are broken (but not marked).
If we pick 2 what are the chances of how many of the 2 are defective.
Not modalable with binomial as we change the same space by removing one.
Cumulative Distribution function
Let Z be a r.v with 𝑃𝑍 (𝑧)
with pmf:
𝐹(𝑧) = 𝑃(𝑍 ≤ 𝑧) ,
z∈ ℝ
𝐹(𝑧) = ∑ 𝑝𝑍 (𝑥)
𝑎𝑙𝑙 𝑥≤𝑧
we will use tables.
Example Q4.26 p 114
(for very large sample doesn’t matter if no replacing)
2 out of 20 new buildings (10%) in a city violate building code.
the building inspector who checks 4 buildings:
Let Y=# of sampled buildings, out of 4, violating the buildingcode.
𝑌~𝐵𝑖𝑛(4, 0.1)
A) What is the probability that none of these (checked buildings ) violate the building code?
4
𝑃(𝑌 = 0) = ( ) 0.10 (1 − 0.1)4 = 0.94
0
4
𝑃(𝑌 = 1) = ( ) 0.11 (1 − 0.1)3 =
1
Expected Value of a discrete r.v
expected value = expectation =mean
Let X be a discrete r.v with p.m.f 𝑝𝑋 (𝑥)
Defn:
𝐸(𝑋) = ∑ 𝑥 𝑝𝑋 (𝑥)
𝑎𝑙𝑙 𝑥
Example:
𝑥
𝑝𝑋 (𝑥)
-1
0.2
0
0.5
2
0.3
𝐸(𝑋) = ∑ 𝑥 𝑝𝑋 (𝑥) = (−1)0.2 0(0.5) + 2(0.3) = 𝟎. 𝟒
𝑎𝑙𝑙 𝑥
EXs:
X~bernoulli(p): 𝐸(𝑋) = 1𝑝 + 0(1 − 𝑝) = 𝑝
X~Bin(n,p): ∑𝑛𝑥=0(𝑥 (𝑛𝑥)𝑝 𝑥 (1 − 𝑝)𝑛−𝑥 = 𝑛𝑝
Long Run interpretation of E(X)
The random experiment is repeated n times,
and the value of X are recorded as 𝑥1 , 𝑥2 , … . , 𝑥𝑛 .
Then
𝑥1 +𝑥2 + …+𝑥𝑛
→ 𝐸(𝑋)
𝑛
𝑛→∞
X – descrete r.v
let Y=f(X)
Y is therefore a random varirable
Find E(Y)
𝐸(𝑌) = ∑ 𝑦 𝑝𝑌 (𝑦)
𝑎𝑙𝑙 𝑦
Proposition:
𝐸(𝑌) = ∑ 𝑓(𝑥) 𝑝𝑋 (𝑥)
𝑎𝑙𝑙 𝑥
Variabnce if a discrerte r.v
𝑉𝑎𝑟(𝑋) = 𝐸[(𝑋 − 𝐸(𝑋)2 ]
2
𝑓(𝑥) = (𝑥 − 𝐸(𝑋)) , 𝐸(𝑋)𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 = 𝐸𝑥𝑝𝑒𝑐𝑡𝑎𝑡𝑖𝑜𝑛
.’.
𝑉𝑎𝑟(𝑋) = 𝐸[(𝑋 − 𝐸(𝑋)2 ] = 𝐸(𝑓(𝑋)) = ∑ 𝑓(𝑥) 𝑝𝑋 (𝑥)
𝑎𝑙𝑙 𝑥
Example:
X~bernoulli(p)
1
2
𝑉𝑎𝑟(𝑋) = ∑(𝑥 − 𝐸(𝑋)) 𝑝𝑥 (𝑥) = (0 − 𝑝)2 (1 − 𝑝) + (1 − 𝑝)2 𝑝 = 𝑝(1 − 𝑝)
𝑥=0
Expection Properties:
𝐸(𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 ) = 𝐸(𝑋1 ) + 𝐸(𝑋2 ) + ⋯ + 𝐸(𝑋𝑛 )
𝐸(𝑎𝑋 + 𝑏) = 𝑎𝐸(𝑋)+b
2
2
𝐸[𝑋 − 𝐸(𝑋)) ] = 𝐸(𝑋 2 ) − (𝐸(𝑋))
Moments of r.v’s
𝐸(𝑋 𝑘 ) − 𝑐𝑎𝑙𝑙𝑒𝑑 𝑡ℎ𝑒 𝑘𝑡ℎ 𝑚𝑜𝑚𝑒𝑚𝑒𝑛𝑡 𝑜𝑓 𝑋
E(X) is the first moment of X
𝐸(𝑋 2 ) is the second moment of X
Variance of a biniomial:
𝑛
𝑛
𝑉𝑎𝑟(𝑋) = ∑ ((𝑥 − 𝑛𝑝)2 ( ) 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥 ) = 𝑛𝑝(1 − 𝑝)
𝑥
𝑥=0
Poisson RV’s
𝑝𝑋 (𝑥) = 𝑃(𝑋 = 𝑥) =
𝑆𝑋 = {0,1,2, … }
𝜆𝑥 −𝜆
𝑒 ,
𝑥!
where λ >0
X~Poi(λ)
Axiom 3 expended:
If 𝐴1 , 𝐴2 , …. mutrally exclusive
∞
∞
𝑃 (⋃ 𝐴𝑖 ) = ∑(𝑃(𝐴𝑖 ))
𝑖=1
𝑖=1
∞
∞
𝑥=0
𝑥=0
𝜆𝑥 −𝜆
𝜆𝑥
−𝜆
∑ ( 𝑒 ) = 𝑒 ∑ ( ) = 𝑒 −𝜆 𝑒 𝜆 = 1
𝑥!
𝑥!
Macrabian expansion:
∞
𝑦
𝑒 = ∑(
𝑘=1
𝑦𝑘
)
𝑘!
Properties:
𝑋~𝑃𝑜𝑖(𝜆)
E(X)=λ
∞
∞
∞
𝑥=0
𝑥=0
𝑥=1
𝜆𝑥
𝜆𝑥
𝐸(𝑋) = ∑ 𝑥𝑝𝑋 (𝑥) = ∑ 𝑥 𝑒 −𝜆 = 𝑒 −𝜆 ∑
= 𝑒 −𝜆 𝜆𝑒 𝜆 = 𝜆
𝑥!
(𝑥 − 1)!