Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Probability Probability 1 Classical Interpretation of Probability Before we dive into the basic results and theorems of probability theory, it is worth considering what we mean by “classical” probability theory. In essence, the classical interpretation of probability is concerned with undertaking an “experiment” that has a number of possible outcomes, one of which must occur as the outcome of the experiment. For example, when a die is rolled, the outcome of rolling the die must be that either the number 1, 2, 3, 4, 5 or 6 is on the uppermost face of the die. In this case the “experiment” is rolling the die, and the numbers 1 through to 6 form the possible outcomes of the experiment. At the heart of the classical interpretation of probability is the assumption that each of the possible outcomes of an experiment is equally likely to occur. With this assumption, if there are N possible outcomes of an experiment, then the probability P of any particular outcome O is given by 1 P (O ) = N Thus, the probability of throwing a die and getting a “1” is equal to 1/6, since there are six possible outcomes, each of which is considered to be equally likely to occur. Basic Definitions In order to proceed, we need to make some basic definitions with respect to an experiment that we undertake, and which leads to a number of possible outcomes. Experiment: Experiment is an activity that is either observed or measured, such as tossing a coin, or drawing a card. The Sample Space – symbol S – denotes the set of all possible outcomes of the experiment. For example, when a die is rolled and the uppermost number is noted, the sample space S consists of six elements, written as follows: S = {1,2,3,4,5,6} An Event – symbol A – represents the subset of outcomes in S that satisfy some criterion for success in the experiment. For example, if a particular event A is that the number obtained by rolling a die is even, then the event A is A = {2,4,6} Since some events are impossible (e.g. rolling a “7” on a six-sided die), an event may contain no elements at all. A set with no elements is referred to as an Empty Set. Elementary Events: Elementary events are those types of events that cannot be broken into other events. For example, suppose that the experiment is to roll a die. The elementary events for this experiment are to roll a 1 or a 2, and so on, i.e., there are six elementary events (1, 2, 3, 4, 5, 6). Note that rolling an even number is an event, but it is not an elementary event, because the even number can be broken down further into events 2, 4, and 6. Mutually Exclusive Events : Those events that cannot happen together are called mutually exclusive events. For example, in the toss of a single coin, the events of heads and tails are mutually exclusive. The probability of two mutually exclusive events occurring at the same time is zero. Independent Events : Two or more events are called independent events when the occurrence or nonoccurrence of one of the events does not affect the occurrence or nonoccurrence of the others. Thus, when two events are independent, the probability of attaining the second event is the same regardless of the outcome of the first event. For example, the probability of tossing a head is always 0.5, regardless of Probability 2 what was tossed previously. Note that in these types of experiments, the events are independent if sampling is done with replacement. Collectively Exhaustive Events : A list of collectively exhaustive events contains all possible elementary events for an experiment. For example, for the die-tossing experiment, the set of events consists of 1, 2, 3, 4, 5, and 6. The set is collectively exhaustive because it includes all possible outcomes. Thus, all sample spaces are collectively exhaustive. Complementary Events (Ac): The complement of an event such as A consists of all events not included in A. For example, if in rolling a die, event A is getting an odd number, the complement of A is getting an even number. Thus, the complement of event A contains whatever portion of the sample space that event A does not contain. Equally likely events : When the events have equal chance to happen in an experiment , they are called Equally likely events. Ex. In tossing a fair coin appearance of Head and Tail have equal chance. Now consider two events A and B. Two important concepts are the union and intersection of A and B. The Union of events A and B is the event that contains all the outcomes of A alone, all the outcomes of B alone, and all the outcomes that belong to both A and B. The union of events A and B is written as ( A ∪ B ) OR (A or B) and can be displayed diagrammatically as follows: S In this figure, the rectangle represents the total sample space S, and the ellipses represent the subsets of S covered by the events A and B. The event that represents the union of A and B is shaded blue in the figure. The Intersection of events A and B is the event that contains all the outcomes that belong to both A and B. The intersection of events A and B is written as ( A ∩ B ) OR (A and B) and can be represented diagrammatically as follows: A B A B S The event that represents the intersection of A and B is shaded blue in the figure. Finally, the Complement of an event A – written AC – is the event that contains all the elements of the sample space S that do not belong to A. The complement of an event A can be represented diagrammatically as follows: S A The event that represents the complement of A is shaded blue in the above figure. As an example, consider the experiment of rolling a die, and let the event A be that the number rolled is even, and let the event B be that the number rolled is three or less. Then: A ∪ B = {1,2,3,4,6} , A ∩ B = {2}, AC = {1,3,5} Mathematical Probability : Consider a sample space that contains N outcomes, such that the probability of any one of the outcomes is 1/N. Now consider an event A that is a subset of S and contains m outcomes. The probability of the event A is given by m P ( A) = N As an example, consider rolling a die and let the event A be that the number rolled is even. Since there are three outcomes in A (2,4 and 6) and six outcomes in S, it follows that m = 3 and N = 6, so that P(A) = 3/6 = 1/2. Now, two basic axioms of probability theory are as follows: P ( A) ≥ 0 P (S ) = 1 Probability 3 P(Ac) = 1 – P(A) Further important results are as follows: P ( AC ) = 1 − P ( A) P ( A ∪ B ) = P ( A) + P (B ) − P ( A ∩ B ) The last of these results can be visualised by considering the diagram above for the union of two events. The probability is clearly related to the sum of the probabilities of the two events, but the probability of the intersection of the two events needs to be subtracted out, since the intersection is counted twice in the sum P(A) + P(B). If the events A and B are independent of one another, then P ( A ∩ B ) = P ( A )P (B ) Probability Laws: 1. When two or more events will happen at the same time, and the events are not mutually exclusive, then: P(X Y) = P(X) + P(Y) - P(X Y) For example, what is the probability that a card chosen at random from a deck of cards will either be a king or a heart? P(King or Heart) = P(X or Y) = 4/52 + 13/52 - 1/52 = 30.77% 2. When two or more events will happen at the same time, and the events are mutually exclusive, then: P(X Y) = P(X) + P(Y) For example, suppose we have a machine that inserts a mixture of beans, broccoli, and other types of vegetables into a plastic bag. Most of the bags contain the correct weight, but because of slight variation in the size of the beans and other vegetables, a package might be slightly underweight or overweight. A check of many packages in the past indicate that: Weight.................Event............No. of Packages.........Probability Underweight..........X.......................100.........................0.025 Correct Weight.......Y....................3600.........................0.9 Overweight............Z.......................300..........................0.075 Total..............................................4000..........................1.00 What is the probability of selecting a package at random and having the package be under weight or over weight? Since the events are mutually exclusive, a package cannot be underweight and overweight at the same time. The answer is: P(X or Z) = P(0.025 + 0.075) = 0.1 3. When two or more events will happen at the same time, and the events are dependent, then the general rule of multiplication law is used to find the joint probability: P(X Y) = P(X) . P(Y|X) For example : What is the probability of selecting one card at random from a deck of cards and finding the probability that the card is an 8 and a diamond. P(8 and diamond) = (4/52) . (1/4) = 1/52 which is = P(diamond and 8) = (13/52) . (1/13) = 1/52. 4. When two or more events will happen at the same time, and the events are independent, then the special rule of multiplication law is used to find the joint probability: Probability 4 P(X Y) = P(X) . P(Y) If two coins are tossed, what is the probability of getting a tail on the first coin and a tail on the second coin? P(T T) = (1/2) . (1/2) = 1/4 = 25%. This can be shown by listing all of the possible outcomes: T T, or T H, or H T, or H H. Conditional Probability In many situations, an experiment may consist of a number of components that can be considered to be independent. For example, consider an experiment in which two dice are rolled. If the two throws of the dice are random, then the probability that the second die lands as a six (or any other number) is completely independent of the outcomes of the throw of the first die, and is equal to 1/6. However, consider now two events A and B. Suppose that event B occurs first. If A and B are independent, then the probability that A occurs does not depend on the probability that B occurs. However, there may be some circumstances in which the probability that A occurs is contingent on whether the event B has occurred or not. That is, if we know that the event B has occurred, then that knowledge influences the probability that A occurs. In this case, we talk of the conditional probability that A occurs, given that event B has occurred. This conditional probability is written as follows: P( A B) and is usually termed the “conditional probability of A given B”. The conditional probability of A given B is evaluated from the following expression: P( A ∩ B) P(A B) = P (B ) The plausibility of this formula can be confirmed by rewriting it slightly as P ( A ∩ B ) = P ( A B )P (B ) This makes the not unreasonable assertion that the probability of A and B occurring is equal to the probability of B occurring, multiplied by the probability of A given that B has occurred. One can also write this in the form P ( A ∩ B ) = P (B A)P ( A) This result follows by swapping A and B around in expression (A). Note that if A and B are independent then P ( A ∩ B ) = P ( A)P (B ) P ( A B ) = P ( A) Similarly, if A and B are mutually exclusive: P( A ∩ B) = 0 P( A B) = 0 Example 1: Consider the rolling of two dice. Compute the probability that (a) the sum of the rolled numbers is 7 (b) both dice show the same number (c) both dice show an even number. In this case, the sample space S contains 36 outcomes. This is because the first die can give six values, as can the second die. The sample space is therefore composed of the following 36 pairs of possible outcomes: S = (11, 12, 13, , 63, 64, 65, 66) To evaluate the required probabilities, we define the appropriate event A and hence work out how many of the 36 outcomes are present in A. Probability 5 (a) Let the event A be the set of outcomes for which the sum of the numbers is 7. The event A is equal to A = {16, 25, 34, 43, 52, 61} and hence contains 6 elements. The required probability is therefore 6/36 = 1/6. (b) Let the event A be the set of outcomes for which the numbers on the dice are the same. In this case, the event A is equal to A = {11, 22, 33, 44, 55, 66} and so the probability of A is once again 6/36 = 1/6. (c) We could once again define an event A and count all the possible outcomes for which both numbers are even. However, an alternative way to solve the problem is to note that the two throws of the dice are independent. Hence we can define events A and B to be equal to the outcomes that the first die gives an even number, and the second die gives an even number, respectively. Hence P(A) = 1/2 and P(B) = 1/2. Thus the required probability is P ( A ∩ B ) = P ( A )P (B ) = 1/ 4 Example2 (Conditional Prob.): Two dice are rolled and the sum of the two numbers is odd. What is the probability that the sum is less than 8? (This example is taken from the book Probability and Statistics, by Morris DeGroot, 1989). We begin by letting A be the event that the sum is less than 8, and B be the event that the sum is odd. Now, in this case, A = {2,3,4,5,6,7,8} B = {3,5,7,9,11} A ∩ B = {3,5,7} Now, there are two ways in which the sum can be 3 or 11, four ways in which the sum can be 5 or 9, and six ways in which the sum can be 7. Therefore, P (B ) = 1 / 2 P ( A ∩ B ) = 1/ 3 Therefore the probability that we seek, namely the probability that the sum is less than 8 given that the sum is odd, is equal to P ( A ∩ B ) 1/ 3 2 = = P(A B) = 1/ 2 3 P (B )