Download Introduction to Coding Theory

Chap 1 Axioms of probability
Ghahramani 3rd edition
Outline
1.1
1.2
1.3
1.4
1.5
1.6
1.7
Introduction
Sample space and events
Axioms of probability
Basic Theorems
Continuity of probability function
Probabilities 0 and 1
Random selection of points from intervals
p2.
1.1 Introduction
Advent of Probability as a math discipline
1. Ancient Egypt
4-sided die 3500 B.C.
6-sided die 1600 B.C.
2. China
playing card 7th-10th centuries
p3.
Introduction
Studies of Chances of Events
Italy
Luca Paccioli(1445-1514)
Niccolo Tartaglia(1499-1557)
Girolamo Cardano(1501-1576)
Galileo Galielei(1564-1642)
France
Blaise Pascal(1623-1662)
Pierre de Fermat(1601-1665)
Christian Huygens(1629-1695) Dutch
1657 first prob. book
“On Calculations in Games of Chance"
p4.
Introduction
James Bernoulli(1654-1705)
Abraham de Moivre(1667-1754)
Pierre-Simon Laplace(1749-1827)
Simeon Denis Poisson(1781-1840)
Karl Friedrich Gauss(1777-1855)
Russia
Pafnuty Chebyshev(1821-1894)
Andrei Markov(1856-1922)
Aleksandr Lyapunov(1857-1918)
p5.
Introduction
1900 David Hilbert(1862-1943) pointed out the
problem of the axiomatic treatment of the theory
of probability
Emile Borel(1871-1956)
Serge Bernstein(1880-1968)
Richard von Mises(1883-1953)
*1933 Andrei Kolmogorov(1903-1087) Russian
successfully axiomatized the theory of probability
p6.
1.2 Sample space and events
Experiment (eg. Tossing a die)
Outcome(sample point)
Sample space={all outcomes}
Event: subset of sample space
•
Ex1.1 tossing a coin once
sample space S = {H, T}
• Ex1.2 flipping a coin and tossing a die if T
or flipping a coin again if H
S={T1,T2,T3,T4,T5,T6,HT,HH}
p7.
Sample space and events
•
Ex1.3 measuring the lifetime of a light bulb
S={x: x
E={x: x
 0}
 100} is the event that the light
bulb lasts at least 100 hours
•
Ex1.4 all families with 1, 2, or 3 children
(genders specified)
S={b,g,bg,gb,bb,gg,bbb,bgb,bbg,bgg,
ggg,gbg,ggb,gbb}
p8.
Sample space and events


Event E has occurred in an experiment:
If the outcome of an experiment belongs to E.
Take events E, F as sets and sample space S then
E  F , EF  E  F , E  F , E  F , E c  S  E

can be defined straightforward.
Can also define
 E
ni1 Ei ,ni1 Ei ,
E
,

i 1 i i 1 i
if {E1, E2, … } is a set of events
p9.
Sample space and events

Associative laws EU(FUG)=(EUF)UG

Distributive laws (EF)UH=(EUH)(FUH)
(EUF)H=(EH)U(FH)

De Morgan’s 1st law: (E U F)c = EcFc

De Morgan’s 2nd law: (EF)c = Ec U Fc

E = ES = E(FUFc) = EF U EFc
p10.
1.3 Axioms of probability
Definition(Probability Axioms)
S: sample space
A: event, A  S
Pr: a function for each event A, i.e. Pr: 2S  R
Pr(A) is said to be the probability of A if
Axiom 1 Pr(A) >= 0
Axiom 2 Pr(S) = 1
Axiom 3 If {A1, A2, A3, … } is a sequence of
mutually exclusive events then
Pr( 
i 1 Ai ) 

Pr( Ai )
i 1

p11.
1.4 Basic Theorem

Theorem 1.4 P(Ac) = 1 – P(A)

Theorem 1.5 If A  B, then
P(B-A)=P(BAc)=P(B)-P(A)

Corollary If A  B, then P(A) <= P(B)

Theorem 1.6 P(AUB) = P(A)+P(B)-P(AB)
p12.
Basic Theorem

Ex 1.15 In a community of 400 adults, 300 bike or
swim or do both, 160 swim, and 120 swim and bike.
What is the probability that an adult, selected at
random from this community, bike?
Sol: A: event that the person swims
B: event that the person bikes
P(AUB)=300/400, P(A)=160/400,
P(AB)=120/400
P(B)=P(AUB)+P(AB)-P(A)
= 300/400+120/400-160/400=260/400= 0.65
p13.
Basic Theorem

Ex 1.16 A number is chosen at random from the
set of numbers {1, 2, 3, …, 1000}. What is the
probability that it is divisible by 3 or 5(I.e. either
3 or 5 or both)?
Sol: A: event that the outcome is divisible by 3
B: event that the outcome is divisible by 5
P(AUB)=P(A)+P(B)-P(AB)
=333/1000+200/1000-66/1000
=467/1000
p14.
Basic Theorem

Inclusion-Exclusion Principle
P( A1  A2  ...  An ) 


 P( Ai )   P( Ai A j )
P( Ai A j Ak )  ...  ( 1)n 1 P( A1A2 ... An )
p15.
1.5 Continuity of probability
function

Recall the continuity of a function f: RR
lim f ( xn )  f ( lim xn )


n
n
fro every convergent seq {xn} in R.
The continuity of probability function is similar.
Def. A seq {En, n>=1} of event of a sample space
is called increasing if
E1  E2  E3   En  En 1  ;
it is called decreasing if
E1  E2  E3   En  En 1  .
p16.
Continuity of probability function

Thm 1.8(continuity of probability function)
For any increasing or decreasing sequence of
events, {En, n>=1}:
lim P(En)=P(lim En)
p17.
1.6 Probabilities 0 and 1

If E and F are events with probabilities 1 and 0,
then it is not correct to say that E is the sample
space S and F is the empty set.

Example: selecting a random point from (0,1)
1.
A={1/3}, P(A)=0
2.
B=(0,1)-A, P(B)=1
p18.
1.7 Random selection of points
from intervals

Def. A point is randomly selected from an interval
(a, b). The probability the subinterval (c, d)
contains the point is defined to be (d-c)/(b-a).
p19.