Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
1.5 Bernoulli trials and binomial probabilities 1.5.1 Independent and identically distributed random variables A stochastic process X1, ..., Xn is said to be a sequence of independent and identically distributed (iid for short) random variables or a sequence of repeated independent trials if (1) X1, ..., Xn are independent of each other, i.e. Pr{ X1 = x1, ..., Xn = xn } = Pr{X = x } ... Pr{X = x } for any x , ..., x . 1 (2) 1 n n 1 n X1, ..., Xn all have the same probability distribution, i.e. for each xj, the value of Pr{Xi = xj} is the same for each i. It follows from (2) that the Xi all have the same probability mass function. Let’s denote this simply by f(a) so f(a) = Pr{X1 = a} = Pr{X2 = a} = ... = Pr{Xn = a}. It follows from the independence of the Xi that (3) f(x1 ..., xn) = Pr{ X1 = x1, ..., Xn = xn } = f(x1) ... f(xn). where f(x1 ..., xn) is the joint pmf of X1, ..., Xn. Example 1. Suppose, as in Example 9 of section 1.2.5, we roll a die n times. Let Xj be the result of the jth roll. The outcomes are (x1, ..., xn) where x1, ..., xn can each be 1, 2, 3, 4, 5 or 6. Suppose the n rolls of the die are a sequence of independent trials with Pr{Xj = k} = 1/6 for each i and j. Then Pr{(x1, ..., xn)} = (1/6)n for each outcome (x1, ..., xn). Problem 1. Under the assumptions of Example 1 a. What is the probability that we don't get a 6 in n rolls of a die? Ans: (5/6)n. b. What is the probability that we get at least one 6 in n rolls of a die? Ans: 1 - (5/6)n. 1.5.2 Bernoulli trials. A Bernoulli trial is a series of repeated independent trials X1, ..., Xn where each observation can only have two possible outcomes. The two outcomes are often called success and failure. It is also common to let S or 1 represent success and F or 0 represent failure and to denote the probability of success on any one trial by p. Thus, the probability of failure is 1-p. For Bernoulli trials, formula (6.3) becomes (4) Pr{ (x1, ..., xn) } = Pr{ X1 = x1, ..., Xn = xn) } = pk (1-p)n-k, where k is the number of successes in x1, ..., xn. Note that each xi is either success or failure. 1.5 - 1 Example 2. During January the probability that any child at Washington Elementary School is sick is 0.4. Furthermore whether one child is sick is independent of whether any other child is sick. Suppose five children are selected at random. What is the probability they are all sick? Suppose we let S or 1 indicate that a child is sick and W or 0 indicate that a child is well and Xj be the condition of the jth child. So Xj = 1 indicats the jth child is sick and Xj = 0 indicates the jth child is well. The sample space consists of the 32 possible outcomes SSSSS, SSSSW, SSSWS, …, WWWWW Each outcome has an equivalent representation in terms of random variables. For example, the outcome SSWSW could be represented by X1 = 1, X2 = 1, X3 = 0, X4 = 1, X5 = 0. The probability that there are all five children are sick is Pr{SSSSS} = Pr{X1 = 1, ..., X5 = 1}. Since X1, ..., X5 are independent, this is equal to Pr{X = 1} ... Pr{X = 1} = (0.4)5 ≈ 0.01024. 1 5 Example 3. A company produces transistors. They estimate that the probability of any one of the transistors is defective is 0.1. Suppose a box contains 10 transistors. What is the probability that there are no defective transistors? Assume that whether each transistor is defective is independent of whether the others are are not. Let Xi be the random variable which is 1 if the i-th transistor is defective and 0 if it is not defective. The probability that there are no defectives is Pr{X1 = 0, ..., X10 = 0}. Since X1, ..., X10 are independent, this is equal to Pr{X = 0} ... Pr{X = 0} = (0.9)10 ≈ 0.3487. 1 10 Let N be the number of successes in a sequence of n Bernoulli trials X1, ..., Xn. If success is denoted by 1 and failure by 0 then (5) N = X 1 + + Xn We are interested in the distribution of N, i.e. probability that N has a given value k. Example 4. In the context of Example 2, what is the probability that exactly 2 of the 5 children are sick? Solution. One possiblility is that the first two children are sick and the last three are well. This corresponds to SSWWW or X1 = 1, X2 = 1, X3 = 0, X4 = 0, X5 = 0. The probability of this outcome is Pr{SSWWW} = Pr{X1 = 1, X2 = 1, X3 = 0, X4 = 0, X5 = 0} = (0.4)2 (0.6)3 = 0.03456. However, there are other outcomes where two of the five children are sick. In this case it is not hard to list them all. They are SSWWW, SWSWW, SWWSW, SWWWS, WSSWW, WSWSW, WSWWS, WWSSW, WWSWS, WWWSS. So there are 10 different outcome where two of the five children are sick. Each of these outcomes has the same probability, namely (0.4)2 (0.6)3 = 0.03456. So the probability of exactly two of the five children sick is (10)(0.03456) = 0.3456. Here is a way to think about the outcomes where two of the five children are sick. To each such outcome we list the positions where the two S’s occur. Thus SSWWW corresponds to 12 and SWSWW corresponds to 13. Thus the 10 outcomes where there are 2 S’s can be listed as 12, 13, 14, 15, 23, 24, 25, 34, 34, 45. 1.5 - 2 Let's return to the general case where we are interested in the probability that exactly k outcomes are success in a sequence of n Bernoulli trials. We need to count the number of sequences (x1, ..., xn) which have k successes. Each such sequence gives a set {i1, ..., ik} of k numbers from the set {1, ..., n}, namely the numbers i where xI is success. In the previous section, 1.4.2, we showed that the number of ways we can select k different objects from a set of n objects was (nk ) (6) = n! k! (n-k)! This quantity is called the number of combinations of n things taken k at a time and often denoted by one of (n ) the symbols k . It is also called the binomial coefficient of n things taken k at a time. (7) n Pr{ N = k } = Pr{k successes in a sequence of n Bernoulli trials} = k pk (1-p)n-k. n The set of probabilities pk = k pk qn-k where k runs 0, 1, ..., n, is called a binomial probability distribution and a random variable N with a probability mass function of this form is called a binomial random variable. Example 5. In the context of Example 3, what is the probability pk that exactly k of the 5 children are sick for each of the values k = 0, 1, 2, 3, 4, and 5? 5 Solution. pk = k (0.4)k (0.6)5-k. Here is a table of values of this distribution and its graph. 0.35 0.3 k 0 1 2 3 4 5 pk 0.07776 0.2592 0.3456 0.2304 0.0768 0.01024 0.25 0.2 0.15 0.1 0.05 1 2 3 4 5 This graph illustrates one of the general features of a binomial probability distribution. For a fixed n and p n the values of pk = k pk (1-p)n-k increase with k until a value m at which point the decrease with k. The value of m is the largest integer less than or equal to (n+1)p. An extreme case is when (n+1)p < 1 in which case the values of pk decrease with k for k = 0, 1, ..., n. The other extreme is when (n+1)p > n in which case the values of pk increase with k for k = 0, 1, ..., n. 1.5 - 3 Example 6. What is the probability of getting exactly 5 heads in 10 flips of a fair coin 10? This is a series of Bernoulli trials where the probability of success is p = ½. We want the probability that the number N of successes is k = 5. By (7) this is (105 ) (½) 10 = 252/210 0.2461. n Problem 2. a) Show that the probability of getting exactly r heads in n tosses of fair coin is r 2-n. n b) Suppose n is even. Show that the probability, n/2 2-n, of getting exactly n/2 heads in n tosses is approximately 2 n . Use Stirling's approximation: n! 2 n (n/e)n. Example 7. If we expand the binomial (x + y)n then we get the sum of the 2n different terms of the form z1...zn where each zi is either x or y. The number of different terms which have exactly r x's and (n-r) y's is n . These terms are all equal to xryn-r. So r n (x + y)n = nr x y r n-r r=0 This is called the binomial theorem. Example 8. What is the probability pk of getting exactly k defective transistors in a box of 10 in Example 3? Solution. pk = 10 (0.1)k (0.9)10-k. Here is a graph of this distribution. k 0.3 0.2 0.1 2 4 6 8 10 1.5.3 Random walks. A random walk is closely related to a sequence of Bernoulli trials. The most common way of thinking about a random walk is as follows. We start at z = 0 on the z axis. We flip a (perhaps unfair) coin with 1.5 - 4 probability of getting a head equal to p. If the coin comes up heads we take a step to the right to z = 1 while if the coin comes up tails we take a step to the left to z = - 1. We continue in this fashion flipping the coin and taking steps to the right or left depending on whether the coin comes up heads or tails. We are interested in the probability of being at position z after n flips. Let Zn be the location after n flips. We are interested in the distribution of Zn. Let pz = pn,k be the probability of being at location z after n flips. After n = 1 flip we are either at location z = 1 or z = - 1 with probabilities p1 = Pr{Z1 = 1} = p and p-1 = Pr{Z1 = - 1} = 1 - p. After n = 2 flips we are either at location – 2, 0 or 2 with probabilities p2 = Pr{X2 = 2} = Pr{HH} = p2 and p0 = Pr{X2 = 0} = Pr{HH, TH} = 2p(1-p) and p-2 = Pr{X2 = -2} = Pr{TT} = (1-p)2. After n flips we could be at any of the location – n, - n+2, - n+4, …, n-2 or n. If we get k heads and n-k tails in the n flips we will be at location z = k – (n-k) = 2k – n. By (7), the probability of this occurring is n pk (1-p)n-k. Since k = (z+n)/2 one has k (8) pk = Pr{Zn = z} = n p(z+n)/2 (1-p)(n-z)/2. (z+n)/2 Example 9. We go to the casino and play a game where we can either win a dollar or lose a dollar on each play. Suppose the probability of winning is p = 0.4 and the probability of losing is 0.6. Suppose we play the game n = 5 times. What are the probabilities of that the net winning over these five plays will be 5 z = - 5, - 3, - 1, 1, 3, 5? According to the above formula this will be p5,z = (z+5)/2 (0.4)(z+5)/2 (0.6)(5-z)/2. Here is a table of values and graph of this distribution. 0.35 0.3 z -5 -3 -1 1 3 5 Pz 0.07776 0.2592 0.3456 0.2304 0.0768 0.01024 0.25 0.2 0.15 0.1 0.05 -4 -2 2 4 A slightly different way of expressing the above derivation if (8) is as follows. Note that Zn = Y1 + … + Yn where Y1, …, Yn are independent with Pr{Yj = 1} = p and Pr{Yj = - 1} = 1 – p for j = 1, …, n. If we let Xj = (Yj + 1)/2 then X1, …, Xn are independent with Pr{Xj = 1} = p and Pr{Xj = 0} = 1 – p. So X1, …, Xn are a series of Bernoulli trials. Since Yj = 2Xj – 1, one has Zn = (2X1 – 1) + … + (2Xn – 1) = 2(X1 + … + Xn) – n = 2N – n where N = X1 + … + Xn is given by (5). Then Pr{Zn = z} = Pr{2N – n = z} = Pr{N = (z+n)/2}. Using (7) this is equal to (8). 1.5 - 5