Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
1 Probability Theory 1.1 Experiment, Outcomes, Sample Space Example 1 An psychologist examined the response of people standing in line at a copying machines. Student volunteers approached the person first in line and asked for permission to skip line, ”because I only have 5 pages to copy, and I am really in a rush”. The person at the front of the line, could now give permission or not. This is an example for an experiment! Where permission yes/no are the possible outcomes. The set of all possible out comes is called the sample space, S = {yes, no}. Definition: 1. The sample space S is the set of all possible outcomes. 2. An event is a set of one or more outcomes for an experiment. It is a sub set of S. 3. An event that includes one and only one of the outcomes is called a simple event. 4. A compound event is an event that includes at least 2 outcomes for an experiment. Examples: Experiment Outcomes Sample Space toss a coin Head, Tail S = {H, T} roll a die 1,2,3,4,5,6 S = {1, 2, 3, 4, 5, 6} toss 2 coins HH, HT, TH, TT S = {HH, HT, TH, TT} Use Venn Diagrams to illustrate experiments. The outer box represents the sample space, which contains all possible outcomes: E1 , . . . , E6 . Is A = {E1 , E2 } the collection of outcomes the appropriate outcomes are circled and labelled with the letter A. S E3 A E1 E4 E5 E2 E6 Or use Tree Diagrams to illustrate experiments. We start at one point we draw a branch for each possible outcome. 1 Tree diagram are especially useful for multiple step experiments Example 2 Tossing two coins, then S = {HH, HT, T H, T T } Example 3 Consider the experiment of tossing a coin, then S = {1, 2, 3, 4, 5, 6}. Consider the simple event of rolling a 6. E = {6} and the compound event of rolling an even number A = {2, 4, 6}. 1.2 Calculating Probabilities Definition 1 The Probability of an event is a numerical measure of the likelihood that the event will occur. Probability Rules: 2 1. The probability is always a number between 0 and 1. – For all events A, 0 ≤ P (A) ≤ 1. 2. The sum of the probabilities of all simple events for an experiment is always 1. ∑ – Let Ei denote the simple events of an experiment, then P (Ei ) = 1 The probability for the impossible event is zero and the probability of the sure event is one. Example 4 Assume an experiment with equally likely outcomes (like rolling a fair die). Then the probability of each simple event should be the same and those probabilities need to add up to one, so for all simple events Ei P (Ei ) = 1 total number of possible outcomes If you roll a fair die all outcomes are equally likely, there is a total of 6 possible outcomes, so P ({1}) = . . . = P ({6}) = 1/6 In this case the probability of a compound event is calculated by P (A) = number of outcomes favourable to A total number of possible outcomes The probability event, A, to roll an even number with a fair die is P (A) = 3/3 = 1/2 But how do we assign probabilities in the cases where the outcomes do not have the same probabilities? Relative frequency interpretation of probability The probability of an event E, denoted as P (E), is the long-run proportion of the time that the event will occur when the experiment is performed repeatedly under identical conditions. This is, if n is the number of same experiments and the event E occurred fn times in n experiments, then P (E) = n→∞ lim fn n This property is called the law of large numbers. Example 5 Assume there 6 of the 60 people enrolled in this class participate in competitive sports. Let A be the event that a randomly chosen person from this class participates in sports, then P (A) = 6/60 = 0.1 (using the the rule equally likely outcomes). But we also could conduct the experiment of picking randomly a person from the class, determine sport yes/no, and by keeping doing this we might find 5 ”yes” after 40 people being asked and ”estimate” 5/40=1/8=0.125 as the probability. Or do it 1000 times and get 102 ”yes” and estimating 102/1000=0.102. 3 1.2.1 Probability Rules Definition: • The union of events A and B, denoted by A or B, is the event that either A or B or both occur. A B • The intersection of events A and B, denoted by A and B, is the event that both A and B occur. A B • The complement of an event A, denoted by Ā = Ac , is the event that A does not occur. Ac A Example: Continue the die rolling example. C is the event to roll an odd number. Let D be the event to roll a number smaller than 4. Then D = {1, 2, 3}. The probability of every simple event is 1/6. With the definition follows that P (C) = P ({2}) + P ({4}) + P ({6}) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2. 4 P (C or D) = P ({1, 3, 5, 2}) = 4/6 = 2/3. P (C and D) = P ({1, 3}) = 2/6 = 1/3. Addition Rule Let A and B be two events. The probability of the union can then be calculated by: P (A or B) = P(A ∪ B) = P(A) + P(B) − P(A and B) Remark: The subtraction of P (A and B) is necessary because this area is counted twice by the addition of P (A) and P (B), once in P (A) and once in P (B). Check the diagram below. B A A and B Rule for Complements The probability of the complement of an event is 1 minus the probability of this event. Mathematically: If P (Ac ) = 1 − P (A) Example: Let A = {1, 2} and B = {1, 3, 5}, with P (A) = P ({1}) = 61 . 1 , P (B) 3 = 1 , 2 and P (A and B) = With the Addition Rule we get P (A or B) = P (A) + P (B) − P (A and B) = 1/3 + 1/2 − 1/6 = 2/3. With the Rule for Complements is P (Ac ) = 1 − 1/3 = 2/3. 1.3 Independence of Two Events and Conditional Probabilities Two events are considered independent, when the occurrence of one of the events has no impact on the probability for the second event to occur. They don’t have an impact on each other. In this section we will define and do examples to illustrate this concept. For defining the independence of events properly we first introduce the concept of a conditional probability. 5 Definition If A and B are events with P (B) > 0, the conditional probability of A given B is defined by P (A|B) := P (A and B) . P (B) Bc B S A and B c A and B A The interpretation of the conditional probability P (A|B) is as follows: Given that you know already event B occurred, what is the probability that A occurs. Example: Consider the event B to roll an even number with a fair 6 sided die. The Probability for the event A to roll a 1, given B equals 0: P (A|B) = 0 Because: If you roll a die, someone peeks and tells you that the number is even, than this number can not be a 1. So the conditional probability of A given B must be 0. In probability theory two event are considered independent, if the knowledge that one of them has occurred does not change the probability for the second to occur. Definition: Events A and B, with P (B) > 0, are independent if P (A|B) = P (A), (which is equivalent to P (A and B) = P (A) · P (B)). Remark: From this definition we also learn that: If we know that two events A and B are independent then P (A and B) = P (A)P (B) This is only true for INDEPENDENT events. The following example shows how to apply these two two concepts to the experiment of rolling a die. Example: Consider the experiment to roll a fair die. Let A = {1, 2, 3} B = {4} C = {3, 4} Then we can calculate, by finding the simple events contained by the described events and adding their probabilities: 6 • P (A) = 1 2 • P (B) = 1 6 • P (C) = 1 3 • P (A ∩ B) = 0 • P (A ∩ C) = • P (B|A) = 1 6 P (B and A) 0 = 1 = 0 (by definition of the conditional probability) P (A) 2 Since P (B) ̸= P (B|A), we find that B and A are not independent (apply the definition for independence). • P (C|A) = P (C and |A) = P (A) 1 6 1 2 = 1 3 Since P (C) = P (C|A), we find that C and A are independent. Example 6 In a company all 100 employees were asked whether they are in favour of or against paying high salaries to CEOs of large companies. The following table show the results depending on the gender of the employees In Favour Against Male 15 45 Female 4 36 In order to complete this table we should add row and column totals In Favour Against Male 15 45 Female 4 36 Total 19 81 Total 60 40 100 Suppose one employee is selected at random, we can ask what is the probability that the employee is a male (female)? 60/100 (40/100). We say that theses numbers give the marginal probabilities of the variable gender. We only consider one variable, and neglect the other . The marginal probabilities of the variable ”response” is: P(In Favour)=19/100, P(Against)=81/100 Suppose now we select randomly an employee and ask, given the person is a male, what is the probability is that he is in favour? This is asking for the conditional probability of ”In Favour” given ”male”: P(In Favour | male)= 15/60=0.25 and P(In Favour | female)= 4/40=0.1 7 Example: A satellite has two power systems, a main and an independent backup system. Suppose the probability of failure in the first ten years for the main system is 0.05 and for the backup system 0.08. What is the probability that both systems fail in the first 10 years and the satellite will be lost? Let M be the event that the main system fails and B the event that the backup systems fails. Since the systems are independent we get: P (M ∩ B) = P (M ) · P (B) = 0.05 · 0.08 = 0.004 The event O that at least one of the systems is still operational after 10 years is the complement event of both systems failing. We calculate: P (O) = 1 − P (Oc ) = 1 − 0.004 = 0.996 The following example shows the practical relevance of these concepts. In this example they are applied to an AIDS test, but please be aware that the same arguments apply to other clinical tests. Example: • ELISA is a test for detecting HIV antibodies in blood samples. • The test is positive in 99.7% of blood samples containing antibodies. • The test is positive in 1.5% of blood samples containing no antibodies. Assuming that 0.18% of the population do have HIV antibodies in their blood (Statistics Canada for 2005), how big is the probability that a person has HIV if his/her blood tests positive? For finding an answer to this question we translate the number given above into probability theory. First choose the denotation: • Let H+ be the event of a blood sample containing antibodies. • Then is H− = H+C the event of a blood sample not containing antibodies. • Let E+ be the event of a positive ELISA test result. • Then is E− = E+C the event of a negative ELISA test result. Now “translate” the facts stated above into conditional probabilities. We get from the introductory statements and the definition: P (E+ |H+ ) = 0.997, P (E+ |H− ) = 0.015, and P (H+ ) = 0.01 and want to know P (H+ |E+ ). We solve the problem by giving the table showing the probabilities for events H+ , H− and E+ , E− 8 H+ H− E+ E− Total P (H+ ) = 0.0018 Total P (H− ) = 1 − P (H+ ) = 0.9982 1 The entries in this table are P (E+ and H+ ), P (E+ and H− ), P (E− and H+ ), and P (E− and H− ) Using the definition: P (E+ |H+ ) = P (E+ and H+ ) P (H+ ) so we get P (E+ and H+ ) = P (E + |H+)P (H+ ) and similarly P (E+ and H− ) = P (E + |H−)P (H− ) After this we just find row totals and use the knowledge of column totals to deduct the values still missing. H+ E+ P (E+ andH+ ) =(0.997)0.0018 =0.0017946 E− 0.0000054 Total 0.0018 H− P (E+ andH− ) =(0.015)0.9982 =0.014973 0.983227 0.9982 Total 0.0167676 0.9832324 1 From the table we now get: P (H+ |E+ ) = P (H+ and E+ ) 0.0017946 = ≈ 0.1070 P (E+ ) 0.0167676 This probability is very low and a test result would have to be confirmed with additional tests. This shows that ELISA is rather a test to exclude HIV then to test for HIV. Try on your own to calculate P (H− |E− ). 9