Download Lecture 1

Lecture 1: Probability Spaces
1. Monty Hall problem
2. Course details (office hours, grades, etc.)
3. Motivation: why study probability?
Deterministic regularity: If we repeat an experiment under the same conditions, then we
will observe the same outcome. (Ex.: drop a dice from a fixed position in a vacuum.) (Laplace’s
demon)
Limitations:
• imprecise knowledge of the initial conditions
• imprecise knowledge of the laws governing the system
• inability to accurately solve the system (chaos)
• intrinsic stochasticity (quantum mechanics)
Statistical regularity: If we repeat an experiment a large number of times, then the proportion of outcomes that satisfy a given condition will tend to a limit.
Example with dice.
4. Mathematical formulation of probability
- limiting frequencies (R. von Mises)
- measure theory (A. N. Kolmogorov)
Suppose that we perform an experiment.
The sample space is the set of all possible outcomes.
• Ω = {1, 2, 3, 4, 5, 6} (roll of a dice)
• Ω = {(i, j); 1 ≤ i, j ≤ 6} (two ordered tosses)
• Ω = {m, f } (sex of a child)
• Ω = [0, ∞) (lifespan of an individual)
• Ω = {(xn ; n ≥ 1), xn ∈ {h, t}} (sequence of coin tosses)
1
An event is a subset of the sample space. An event corresponds to a collection of possible
outcomes for the experiment.
• E = {1}, {2, 4, 6}, ∅, Ω
• E = {(i, j) : i + j = 6}
• E = {(xn ; n ≥ 1) : limn→∞ n−1 #{k ≤ n : xk = h} = 1/2}
Operations with events:
• union: E ∪ F is the set of outcomes that are in E or F (non-exclusive)
• intersection: E ∩ F is the set of outcomes that are in both E and F
– Ross uses the notation EF
– E and F are said to be mutually exclusive if E ∩ F = ∅
• unions and intersections can be extended to more than two sets.
• complements: E c is the set of outcomes in Ω that are not in E
Unions and intersections satisfy several important algebraic laws:
• Commutative laws: E ∪ F = F ∪ E, E ∩ F = F ∩ E
• Associative laws: (E ∪ F ) ∪ G = E ∪ (F ∪ G), (E ∩ F ) ∩ G = E ∩ (F ∩ G)
• Distributive laws: (E ∪ F ) ∩ G = (E ∩ G) ∪ (F ∩ G), (E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G)
DeMorgan’s laws:
n
[
Ek
c
=
k=1
n
\
Ek
c
=
k=1
n
\
k=1
n
[
Ekc
Ekc
k=1
To formalize our notion of probability, we need to specify the collection of events that have
probabilities.
Recall that a set is countably infinite if it can be put into one-to-one correspondence with N.
Definition: A σ-algebra F on a set Ω is a collection of subsets of Ω that satisfies the following
three conditions:
1. Ω ∈ F
2. F is closed under complements
2
3. F is closed under countable unions and countable intersections.
The pair (Ω, F) is called a measurable space.
Examples:
• for any set Ω, the power set of Ω is a σ-algebra.
• Ω = {1, 2, 3, 4}, F = {∅, Ω, {1, 3}, {2, 4}}
To have a probability space, we need to assign each event a probability. Using our intuition
about relative frequencies, these numbers should have the following properties:
• P(E) ∈ [0, 1]
• P(Ω) = 1
• if E and F are disjoint sets (mutually exclusive events), then P(E ∪ F ) = P(E) + P(F )
In fact, for many applications, we need to insist on countable additivity. If En , n ≥ 1 is a
countable collection of disjoint events, then
[ X
En =
P
P(En ).
n
n≥1
Definition: A probability measure on a measurable space (Ω, F) is a real-valued function
P : F → R such that:
1. P(Ω) = 1
2. P(E) ≥ 0 for all E ∈ F
3. P is countable additive
A probability space is a triple (Ω, F, P).
NB: P is undefined on sets E not in F.
Examples:
• Ω = {h, t}, F = 2Ω , P(h) = 1 − P(t) = p
• Ω = {1, 2, · · · }, F = 2Ω , P(n) = 2−n
Why do we need to restrict to countable additivity?
Example: Ω = [0, 1], F contains all open and closed intervals, P([a, b]) = b − a
• P(x) = 0
3
• 1 = P([0, 1]) 6=
P
x P(x)
= 0.
The problem is that [0, 1] is uncountable. (Proof by Cantor diagonalization.)
Some properties of countable sets:
• a finite union of countable sets is countable
• a countable union of countable sets is countable
• a finite product of countable sets is countable
• a countably infinite product of finite sets is not countable
Monty Hall problem revisited (describe the sample space).
Important Remark: In general, we won’t operate at this level of abstraction in the remainder
of this course, i.e., we won’t explicitly construct our probability spaces or prove theorems using
measure theoretic machinery. However, as mathematicians (or even as consumers of mathematics), it is good to be aware that a fully rigorous theory of probability can be built from
the ground up using measure theory. In fact, one of the remarkable features of Kolmogorov’s
formulation of probability is that one can prove that sequences of random experiments behave
the way that we expect them to, i.e., the Strong Law of Large Numbers will tell us that relative
frequency of an event does converge to the probability of that event.
5. Some properties of probabilities
Complements: P(E c ) = 1 − P(E)
Ex: What is the probability of obtaining at least one head if we toss a coin three times?
Subsets: If E ⊂ F , then P(E) ≤ P(F ).
Unions: P(E ∪ F ) = P(E) + P(F ) − P(E ∩ F ).
Ex: Suppose that the frequency of HIV infection in a community is 0.05, that the frequency of
tuberculosis infection is 0.1, and that the frequency of dual infection is 0.01. What proportion
of individuals have neither HIV nor tuberculosis?
P(HIV or tuberculosis) = P(HIV) + P(tuberculosis) − P(both) = 0.05 + 0.1 − 0.01 = 0.14
P(neither) = 1 − P(HIV or tuberculosis) = 0.86
4