Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Indeterminism wikipedia , lookup
History of randomness wikipedia , lookup
Random variable wikipedia , lookup
Dempster–Shafer theory wikipedia , lookup
Probability box wikipedia , lookup
Infinite monkey theorem wikipedia , lookup
Birthday problem wikipedia , lookup
Boy or Girl paradox wikipedia , lookup
Risk aversion (psychology) wikipedia , lookup
Inductive probability wikipedia , lookup
Lecture 1: Probability Spaces 1. Monty Hall problem 2. Course details (office hours, grades, etc.) 3. Motivation: why study probability? Deterministic regularity: If we repeat an experiment under the same conditions, then we will observe the same outcome. (Ex.: drop a dice from a fixed position in a vacuum.) (Laplace’s demon) Limitations: • imprecise knowledge of the initial conditions • imprecise knowledge of the laws governing the system • inability to accurately solve the system (chaos) • intrinsic stochasticity (quantum mechanics) Statistical regularity: If we repeat an experiment a large number of times, then the proportion of outcomes that satisfy a given condition will tend to a limit. Example with dice. 4. Mathematical formulation of probability - limiting frequencies (R. von Mises) - measure theory (A. N. Kolmogorov) Suppose that we perform an experiment. The sample space is the set of all possible outcomes. • Ω = {1, 2, 3, 4, 5, 6} (roll of a dice) • Ω = {(i, j); 1 ≤ i, j ≤ 6} (two ordered tosses) • Ω = {m, f } (sex of a child) • Ω = [0, ∞) (lifespan of an individual) • Ω = {(xn ; n ≥ 1), xn ∈ {h, t}} (sequence of coin tosses) 1 An event is a subset of the sample space. An event corresponds to a collection of possible outcomes for the experiment. • E = {1}, {2, 4, 6}, ∅, Ω • E = {(i, j) : i + j = 6} • E = {(xn ; n ≥ 1) : limn→∞ n−1 #{k ≤ n : xk = h} = 1/2} Operations with events: • union: E ∪ F is the set of outcomes that are in E or F (non-exclusive) • intersection: E ∩ F is the set of outcomes that are in both E and F – Ross uses the notation EF – E and F are said to be mutually exclusive if E ∩ F = ∅ • unions and intersections can be extended to more than two sets. • complements: E c is the set of outcomes in Ω that are not in E Unions and intersections satisfy several important algebraic laws: • Commutative laws: E ∪ F = F ∪ E, E ∩ F = F ∩ E • Associative laws: (E ∪ F ) ∪ G = E ∪ (F ∪ G), (E ∩ F ) ∩ G = E ∩ (F ∩ G) • Distributive laws: (E ∪ F ) ∩ G = (E ∩ G) ∪ (F ∩ G), (E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G) DeMorgan’s laws: n [ Ek c = k=1 n \ Ek c = k=1 n \ k=1 n [ Ekc Ekc k=1 To formalize our notion of probability, we need to specify the collection of events that have probabilities. Recall that a set is countably infinite if it can be put into one-to-one correspondence with N. Definition: A σ-algebra F on a set Ω is a collection of subsets of Ω that satisfies the following three conditions: 1. Ω ∈ F 2. F is closed under complements 2 3. F is closed under countable unions and countable intersections. The pair (Ω, F) is called a measurable space. Examples: • for any set Ω, the power set of Ω is a σ-algebra. • Ω = {1, 2, 3, 4}, F = {∅, Ω, {1, 3}, {2, 4}} To have a probability space, we need to assign each event a probability. Using our intuition about relative frequencies, these numbers should have the following properties: • P(E) ∈ [0, 1] • P(Ω) = 1 • if E and F are disjoint sets (mutually exclusive events), then P(E ∪ F ) = P(E) + P(F ) In fact, for many applications, we need to insist on countable additivity. If En , n ≥ 1 is a countable collection of disjoint events, then [ X En = P P(En ). n n≥1 Definition: A probability measure on a measurable space (Ω, F) is a real-valued function P : F → R such that: 1. P(Ω) = 1 2. P(E) ≥ 0 for all E ∈ F 3. P is countable additive A probability space is a triple (Ω, F, P). NB: P is undefined on sets E not in F. Examples: • Ω = {h, t}, F = 2Ω , P(h) = 1 − P(t) = p • Ω = {1, 2, · · · }, F = 2Ω , P(n) = 2−n Why do we need to restrict to countable additivity? Example: Ω = [0, 1], F contains all open and closed intervals, P([a, b]) = b − a • P(x) = 0 3 • 1 = P([0, 1]) 6= P x P(x) = 0. The problem is that [0, 1] is uncountable. (Proof by Cantor diagonalization.) Some properties of countable sets: • a finite union of countable sets is countable • a countable union of countable sets is countable • a finite product of countable sets is countable • a countably infinite product of finite sets is not countable Monty Hall problem revisited (describe the sample space). Important Remark: In general, we won’t operate at this level of abstraction in the remainder of this course, i.e., we won’t explicitly construct our probability spaces or prove theorems using measure theoretic machinery. However, as mathematicians (or even as consumers of mathematics), it is good to be aware that a fully rigorous theory of probability can be built from the ground up using measure theory. In fact, one of the remarkable features of Kolmogorov’s formulation of probability is that one can prove that sequences of random experiments behave the way that we expect them to, i.e., the Strong Law of Large Numbers will tell us that relative frequency of an event does converge to the probability of that event. 5. Some properties of probabilities Complements: P(E c ) = 1 − P(E) Ex: What is the probability of obtaining at least one head if we toss a coin three times? Subsets: If E ⊂ F , then P(E) ≤ P(F ). Unions: P(E ∪ F ) = P(E) + P(F ) − P(E ∩ F ). Ex: Suppose that the frequency of HIV infection in a community is 0.05, that the frequency of tuberculosis infection is 0.1, and that the frequency of dual infection is 0.01. What proportion of individuals have neither HIV nor tuberculosis? P(HIV or tuberculosis) = P(HIV) + P(tuberculosis) − P(both) = 0.05 + 0.1 − 0.01 = 0.14 P(neither) = 1 − P(HIV or tuberculosis) = 0.86 4