Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Survey

Document related concepts

no text concepts found

Transcript

PROBABILITY AND BAYES THEOREM 1 PROBABILITY SAMPLE POPULATION STATISTICAL INFERENCE 2 • PROBABILITY: A numerical value expressing the degree of uncertainty regarding the occurrence of an event. A measure of uncertainty. • STATISTICAL INFERENCE: The science of drawing inferences about the population based only on a part of the population, sample. 3 PROBABILITY • CLASSICAL INTERPRETATION If a random experiment is repeated an infinite number of times, the relative frequency for any given outcome is the probability of this outcome. Probability of an event: Relative frequency of the occurrence of the event in the long run. – Example: Probability of observing a head in a fair coin toss is 0.5 (if coin is tossed long enough). • SUBJECTIVE INTERPRETATION The assignment of probabilities to event of interest is subjective – Example: I am guessing there is 50% chance of rain today. 4 PROBABILITY • Random experiment – a random experiment is a process or course of action, whose outcome is uncertain. • Examples Experiment • Flip a coin • Record a statistics test marks • Measure the time to assemble a computer Outcomes Heads and Tails Numbers between 0 and 100 Numbers from zero and above 5 PROBABILITY • Performing the same random experiment repeatedly, may result in different outcomes, therefore, the best we can do is consider the probability of occurrence of a certain outcome. • To determine the probabilities, first we need to define and list the possible outcomes 6 Sample Space • Determining the outcomes. – Build an exhaustive list of all possible outcomes. – Make sure the listed outcomes are mutually exclusive. • The set of all possible outcomes of an experiment is called a sample space and denoted by S. 7 Sample Space Countable Uncountable (Continuous ) Finite number of elements Infinite number of elements 8 EXAMPLES • Countable sample space examples: – Tossing a coin experiment S : {Head, Tail} – Rolling a dice experiment S : {1, 2, 3, 4, 5, 6} – Determination of the sex of a newborn child S : {girl, boy} • Uncountable sample space examples: – Life time of a light bulb S : [0, ∞) – Closing daily prices of a stock S : [0, ∞) 9 Sample Space • Multiple sample spaces for the same experiment are possible • E.g. with 5 coin tosses we can take: S={HHHHH, HHHHT, …} or if we are only interested in the number of heads we can take S*={0,1,2,3,4,5} 10 EXAMPLES • Examine 3 fuses in sequence and note the results of each experiment, then an outcome for the entire experiment is any sequence of N’s (non-defectives) and D’s (defectives) of length 3. Hence, the sample space is S : { NNN, NND, NDN, DNN, NDD, DND, DDN, DDD} 11 Assigning Probabilities – Given a sample space S ={O1,O2,…,Ok}, the following characteristics for the probability P(Oi) of the simple event Oi must hold: 1. 0 POi 1 for each i k 2. PO 1 i i 1 – Probability of an event: The probability P(A), of event A is the sum of the probabilities assigned to the simple events contained in A. 12 Assigning Probabilities • P(A) is the proportion of times the event A is observed. total outcomes in A P( A) total outcomes in S 13 Set theory: Definitions • Set: a set A is a collection of elements (or outcomes) • Membership: xA (x is in A), or x A (x is not in A) C • Complement: A {x : x A} • Union: A B {x : x A or x B} • Intersection: A B {x : x A and x B} • Difference: A \ B ABC {x : x A and x B} • Subset: A B ( x A x B) A is contained in B • Equality: A B if A B and B A • Symmetric difference: AB {x : x A or x B, but not both} 14 Algebraic laws • commutative: A ∪ B = B ∪ A A∩B=B∩A • associative: (A ∪ B) ∪ C = A ∪ (B ∪ C) A ∩ (B ∩ C) = (A ∩ B) ∩ C • distributive: A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) • DeMorgan’s: (A ∪ B)' = A' ∩ B' (' is complement) (A ∩ B)' = A' ∪ B' 15 Intersection • The intersection of event A and B is the event that occurs when both A and B occur. • The intersection of events A and B is denoted by (A and B) or AB. • The joint probability of A and B is the probability of the intersection of A and B, which is denoted by P(A and B) or P(AB). 16 Union • The union event of A and B is the event that occurs when either A or B or both occur. • At least one of the events occur. • It is denoted “A or B” OR AB 17 Addition Rule For any two events A and B P(A B) = P(A) + P(B) - P(A B) 18 Complement Rule • The complement of event A (denoted by AC) is the event that occurs when event A does not occur. • The probability of the complement event is calculated by A and AC consist of all the simple events in the sample space. Therefore, P(A) + P(AC) = 1 P(AC) = 1 - P(A) 19 MUTUALLY EXCLUSIVE EVENTS • Two events A and B are said to be mutually exclusive or disjoint, if A and B have no common outcomes. That is, A and B = (empty set) •The events A1,A2,… are pairwise mutually exclusive (disjoint), if Ai Aj = for all i j. 20 EXAMPLE • The number of spots turning up when a six-sided dice is tossed is observed. Consider the following events. A: The number observed is at most 2. B: The number observed is an even number. C: The number 4 turns up. 21 VENN DIAGRAM • A graphical representation of the sample 1 space. A AB S 1 2 B 4 A 3 6 5 C 4 2 1 B 6 AB A 22 4 AC = A and C are mutually exclusive B 6 22 AXIOMS OF PROBABILTY (KOLMOGOROV AXIOMS) Given a sample space S, the probability function is a function P that satisfies 1) For any event A, 0 P(A) 1. 2) P(S) = 1. 3) If A1, A2,… are pairwise disjoint, then P ( Ai ) i 1 P( A ) i 1 i 23 Probability P : Probability function S domain [0,1] range 24 THE CALCULUS OF PROBABILITIES • If P is a probability function and A is any set, then a. P()=0 b. P(A) 1 c. P(AC)=1 P(A) 25 THE CALCULUS OF PROBABILITIES • If P is a probability function and A and B any sets, then a. P(B AC) = P(B)P(A B) b. If A B, then P(A) P(B) c. P(A B) P(A)+P(B) 1 (Bonferroni Inequality) d. P i 1 Ai P A for any sets A , A , i 1 2 i 1 (Boole’s Inequality) 26 Principle of Inclusion-Exclusion • A generalization of addition rule n n i 1 i 1 P( Ai ) P( Ai ) P( Ai Aj ) ... (1) n1 P( A1 A2 ... An ) i j • Proof by induction 27 EQUALLY LIKELY OUTCOMES • The same probability is assigned to each simple event in the sample space, S. • Suppose that S={s1,…,sN} is a finite sample space. If all the outcomes are equally likely, then P({si})=1/N for every outcome si. 28 ODDS • The odds of an event A is defined by P( A) P( A) C P( A ) 1 P( A) •It tells us how much more likely to see the occurrence of event A. •P(A)=3/4P(AC)=1/4 P(A)/P(AC) = 3. That is, the odds is 3. It is 3 times more likely that A occurs as it is that it does not. 29 ODDS RATIO • OR is the ratio of two odds. • Useful for comparing the odds under two different conditions or for two different groups, e.g. odds for males versus females. • If odds of event A is 4.2 for males and 2 for females, then odds ratio is 2.1. The odds of observing event A is 2.1 times higher for males compared to females. 30 CONDITIONAL PROBABILITY • (Marginal) Probability: P(A): How likely is it that an event A will occur when an experiment is performed? • Conditional Probability: P(A|B): How will the probability of event A be affected by the knowledge of the occurrence or nonoccurrence of event B? • If two events are independent, then P(A|B)=P(A) 31 CONDITIONAL PROBABILITY P(A B) P(A | B) if P(B) 0 P(A | B) 1 P(B) 0 P(A | B) 1 P(A C | B) P(A | A) 1 P(A1 A 2 | B) P(A1 | B) P(A 2 | B) P(A1 A 2 | B) 32 Example • • • • Roll two dice S=all possible pairs ={(1,1),(1,2),…,(6,6)} Let A=first roll is 1; B=sum is 7; C=sum is 8 P(A|B)=?; P(A|C)=? • Solution: • P(A|B)=P(A and B)/P(B) P(B)=P({1,6} or {2,5} or {3,4} or {4,3} or {5,2} or {6,1}) = 6/36=1/6 P(A|B)= P({1,6})/(1/6)=1/6 =P(A) A and B are independent 33 Example • P(A|C)=P(A and C)/P(C)=P(Ø)/P(C)=0 A and C are disjoint Out of curiosity: P(C)=P({2,6} or {3,5} or {4,4} or {5,3} or {6,2}) = 5/36 CONDITIONAL PROBABILITY P( AB) P( A) P( B | A) P( B) P( A | B) P( A1 A2 ... An ) P( A1 ) P( A2 | A1 ) P( A3 | A1 , A2 )...P( An | A1 ,..., An1 ) 35 Example • Suppose we pick 4 cards at random from a deck of 52 cards containing 4 aces. • A=event that we pick 4 aces • Ai=event that ith pick is an ace (i=1,2,3,4) 52 P( A) 1 / 1 / 270,725 4 4 3 2 1 P( A) P( A1 A2 A3 A4 ) P( A1 ) P( A2 | A1 )... 1 / 270,725 52 51 50 49 36 BAYES THEOREM • Suppose you have P(B|A), but need P(A|B). P(A B) P(B | A)P(A) P(A | B) for P(B) 0 P(B) P(B) 37 Example • Let: – D: Event that person has the disease; – T: Event that medical test results positive • Given: – Previous research shows that 0.3 % of all Turkish population carries this disease; i.e., P(D)= 0.3 % = 0.003 – Probability of observing a positive test result for someone with the disease is 95%; i.e., P(T|D)=0.95 – Probability of observing a positive test result for someone without the disease is 4%; i.e. P(T| D C )= 0.04 • Find: probability of a randomly chosen person having the disease given that the test result is positive. 38 Example • Solution: Need P(D|T). Use Bayes Thm. P(D|T)=P(T|D)*P(D)/P(T) P(T)=P(D and T)+P(D C and T) = 0.95*0.003+0.04*0.997 = 0.04273 P(D|T) =0.95*0.003 / 0.04273 = 6.67 % Test is not very reliable! 39 BAYES THEOREM • Can be generalized to more than two events. • If Ai is a partition of S, then, P( B | Ai ) P( Ai ) P( B | Ai ) P( Ai ) P( Ai | B) P( B) P( B | A j ) P( A j ) j • Can be rewritten in terms of odds – Suppose A1,A2,… are competing hypotheses and B is evidence or data relevant to choosing the correct hypothesis P( Ai | B) P( B | Ai ) P( Ai ) P( A j | B) P( B | A j ) P( A j ) Posterior odds = likelihood ratio x prior odds 40 Independence • A and B are independent iff – P(A|B)=P(A) or P(B|A)=P(B) – P(AB)=P(A)P(B) • A1, A2, …, An are mutually independent iff P ( Ai ) P ( Ai ) for every subset j of {1,2,…,n} i j i j E.g. for n=3, A1, A2, A3 are mutually independent iff P(A1A2A3)=P(A1)P(A2)P(A3) and P(A1A2)=P(A1)P(A2) and P(A1A3)=P(A1)P(A3) and P(A2A3)=P(A2)P(A3) 41 Independence • If n=4, then the number of conditions for independence is 4 4 4 4 3 2 11 • Find these conditions. 42 Sequences of events • A sequence of events A1, A2, … is increasing iff A1 A2 A3 ... • A sequence of events A1, A2, … is decreasing iff A1 A2 A3 ... An Ai • If {An} is increasing, then lim n i 1 An Ai • If {An} is decreasing, then lim n i 1 43 Examples • Let S=(0,1) and An=(1/n,1) {An} is increasing. What is limit of An as n goes to infinity? • Let S=(0,1) and Bn=(0,1/n) {Bn} is decreasing. What is limit of Bn as n goes to infinity? 44 Problems 1. Show that two nonempty events cannot be disjoint and independent at the same time. Hint: First, prove that if they are disjoint, then they are not independent. Second, prove that if they are independent, then they are not disjoint. Problems 2. If P(A)=1/3 and P(Bc)=1/4, can A and B be disjoint? Explain. Problems 3. Either prove the statement is true or disprove it: If P(B|A)=P(B|AC), then A and B are independent. Problems 4. An insurance company has three types of customers – high risk, medium risk, and low risk. Twenty percent of its customers are high risk, and 30% are medium risk. Also, the probability that a customer has at least one accident in the current year is 0.25 for high risk, 0.16 for medium risk, and 0.1 for low risk. a) Find the probability that a customer chosen at random will have at least one accident in the current year. b) Find the probability that a customer is high risk, given that the person has had at least one accident during the current year. Problems 5. Eleven poker chips are numbered consecutively 1 through 10, with two of them labeled with a 6 and placed in a jar. A chip is drawn at random. i) Find the probability of drawing a 6. ii) Find the odds of drawing a 6 from the jar. iii) Find the odds of not drawing a 6. 49 Problems 6. If the odds in favor of winning a horse race are 3:5, find the probability of winning the race. 50 Problems 7. In a hypothetical clinical study, the following results were obtained. Treatment Ibuprofen 400 mg Placebo Total number of patients treated Number who achieved at least 50% pain relief Number who did not achieve at least 50% pain relief 40 22 18 40 7 33 Find the odds ratio and interpret. 51

Related documents