Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture 8 5.2 Discrete Probability 5.2 Recap • Sample space: space of all possible outcomes. • Event: subset of of S. • p(s) : probability of element s of S: p(s) 1 s 0 p( s) 1 sS • p( E ) p( s) sE • Probability of complement p( E ) 1 p( E ) n • Disjoint events: p( i 1 n Ei ) p ( Ei ) • Overlapping events: p( E1 i 1 E2 ) p( E1 ) p( E2 ) p( E1 E2 ) • Three doors problem (see ex. 39 p. 362 + matlab demo) 5.2 We flip a coin 3 times with Heads and Tails equal probability. There are 8 possibilities: |S| = 8 S HHH E HTT THT HHT THH TTH TTT HTH p( E ) 1 2 P( F ) F 1 2 P( E F) E: event that an odd number of tails occurs F: first flip comes up tails 3 4 P( E F) 1 4 5.2 S E HHT THT THH HHH p( E ) THH E|F TTH F TTT HTH THT HTT TTT TTH 1 2 P( F ) 1 2 P( E F) 3 4 P( E F) 1 4 F What is the probability of odd number of tails (i.e. of event E) if we know that the first flip was tails (i.e. F happened) ? If we know that F happened the total number of possible outcomes shrinks to |F| F = THH, THT, TTH, TTT. Of those there are two that make event E happen: THH, TTT. P(E|F) = 1/2 5.2 Theorem: If E and F are two events and p(F)>0, then the conditional probability of E given F is given by: p( E F ) P( E | F ) P( F ) Example: What is the probability that a family with 2 kids has two boys, given that they have at least one boy? (all possibilities are equally likely). S: all possibilities: {BB, GB, BG, GG}. E: family has two boys: {BB}. F: family has at least one boy: {BB, GB, BG}. E F = {BB} p(E|F) = (1/4) / (3/4) = 1/3 GG F E BB BG GB 5.2 S GG S F E BB HTT E HHT BG GB HTH THT THH TTT TTH F HHH p(E) = ¼ P(E|F) = 1/3 By knowing F has happened, I have changed the probability that E has happened: they are dependent P(E)=1/2 P(E|F) = 1/2 By knowing F has happened, I have NOT changed the probability that E has happened: they are independent 5.2 Equivalent statement: The probability of the intersection of 2 events is the product of the probabilities of the separate events. P( E | F ) P( E ) P( E F ) P( E ) P( E P( F ) F ) P( E ) P( F ) Example: A family has 2 children (|S|=4). Is the event E that the family has children of both sexes independent from the event F that they have at most one boy? E: {BG, GB} F: {GG, BG, GB} E F: {BG, GB} P( E F ) 1/ 2 P( E ) P( F ) 1/ 2 3 / 4 3 / 8 dependent 5.2 What about a family with 3 kids ? |S| = 8 |E| = 8 – 2 = 6 (only BBB and GGG violate E). |F| =| {GGG, GGB, BGG, GBG} | = 4 |E F | =| {BGG, GBG, GGB}| = 3 P( E F) 3/8 P( E ) P( F ) 3 / 4 1/ 2 3 / 8 independent ! (again: it’s hard to get any intuition for this). 5.2 Problem: We toss a coin 7 times, but the coin is biased with probability 2/3 heads. What is the probability that we find 4 heads in those 7 tosses? There are C(7,4) ways to generate a sequence with 4 heads and 3 tails. (think of a bit-string with 0’s and 1’s). Each one of them has probability (2/3)^4 x (1/3)^3. Total: probability: C(7,4) x (2/3)^4 x (1/3)^3 More general: What is the probability of finding k 1’s in a bit-string of length n, when the probability of finding a 1 is p? P(k , n, p) n! p k (1 p) n k k !(n k )! Binomial distribution. (matlab demo) 5.2 Does it sum to 1? n Recall: n! ( p q) p k q nk k 0 k !( n k )! n Use q = 1-p to prove the result. 5.2 Random Variables Many problems deal with real numbers rather than set memberships. E.g. What is the sum of the outcomes of 2 dice? How many times do we expect to 7 1 in bit-strings of length 12? We were able to answer those questions before, but we now introduce some formal machinery: Definition: A random variable X is a function (not a variable!) from sample space to the real numbers. It assigns a real number to each possible outcome. (and is not random!). Example: S={apple, pear, banana} X(apple) = 1, X(pear) = 2, X(banana) = 3. X: the number of times heads comes up when we toss a coin 2 times: X(TT) = 0; X(HT) = 1; X(TH) = 1; X(HH) = 2; 5.2 Definition: Let P(X=r) be the probability that X takes a value r. A distribution of a random variable X on a sample space S is the set of all pairs (r,P(X=r)) for all r in X(S). Thus, we specify the distribution by providing all possible P(X=r). Example: X is the number of heads of 2 coin tosses: P(X=0) = ¼ P(X=1) = ½ P(X=2) = ¼ 5.3 Definition: The expected value of a random variable X(s) on the sample space S is given by: E ( X ) X ( s) p( s) sS Sometimes, it’s more efficient to compute expectations by clumping together all elements of S that result in the same value for X(s). p( X r ) p( s) generally true sS , X ( s ) r K (r ) p( s) only true when all p(s) s.t. X(s)=r have equal probability. number of elements in S such that X(s)=r 5.3 Example: X is number of heads in 2 coin tosses. Expected value? E(X) = 0 x P(X=0) + 1 x P(X=1) + 2 x P(X=2) = 0 +½ + 2x ¼ = 1. We expect on average that we see 1 head. Example: X is the value of the number that comes up on a die. Expected value? E(X) = 1 x 1/6 + 2 x 1/6 + 3 x 1/6 + 4 x 1/6 + 5 x 1/6 + 6 x 1/6 = 21/6 = 7/2. (matlab demo2)