Download probability

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts
no text concepts found
Transcript
PROBABILITY AND
BAYES THEOREM
1
PROBABILITY
SAMPLE
POPULATION
STATISTICAL
INFERENCE
2
• PROBABILITY: A numerical value
expressing the degree of uncertainty
regarding the occurrence of an event. A
measure of uncertainty.
• STATISTICAL INFERENCE: The science
of drawing inferences about the population
based only on a part of the population,
sample.
3
PROBABILITY
• CLASSICAL INTERPRETATION
If a random experiment is repeated an infinite
number of times, the relative frequency for any given
outcome is the probability of this outcome.
Probability of an event: Relative frequency of the
occurrence of the event in the long run.
– Example: Probability of observing a head in a fair coin toss is 0.5 (if coin is
tossed long enough).
• SUBJECTIVE INTERPRETATION
The assignment of probabilities to event of interest is
subjective
– Example: I am guessing there is 50% chance of rain today.
4
PROBABILITY
• Random experiment
– a random experiment is a process or course of action,
whose outcome is uncertain.
• Examples
Experiment
• Flip a coin
• Record a statistics test marks
• Measure the time to assemble
a computer
Outcomes
Heads and Tails
Numbers between 0 and 100
Numbers from zero and above
5
PROBABILITY
• Performing the same random experiment
repeatedly, may result in different
outcomes, therefore, the best we can do is
consider the probability of occurrence of
a certain outcome.
• To determine the probabilities, first we
need to define and list the possible
outcomes
6
Sample Space
• Determining the outcomes.
– Build an exhaustive list of all possible
outcomes.
– Make sure the listed outcomes are mutually
exclusive.
• The set of all possible outcomes of an
experiment is called a sample space and
denoted by S.
7
Sample Space
Countable
Uncountable
(Continuous )
Finite number
of elements
Infinite number of
elements
8
EXAMPLES
• Countable sample space examples:
– Tossing a coin experiment
S : {Head, Tail}
– Rolling a dice experiment
S : {1, 2, 3, 4, 5, 6}
– Determination of the sex of a newborn child
S : {girl, boy}
• Uncountable sample space examples:
– Life time of a light bulb
S : [0, ∞)
– Closing daily prices of a stock
S : [0, ∞)
9
Sample Space
• Multiple sample spaces for the same
experiment are possible
• E.g. with 5 coin tosses we can take:
S={HHHHH, HHHHT, …} or if we are only
interested in the number of heads we can
take S*={0,1,2,3,4,5}
10
EXAMPLES
• Examine 3 fuses in sequence and note the
results of each experiment, then an
outcome for the entire experiment is any
sequence of N’s (non-defectives) and D’s
(defectives) of length 3. Hence, the
sample space is
S : { NNN, NND, NDN, DNN, NDD,
DND, DDN, DDD}
11
Assigning Probabilities
– Given a sample space S ={O1,O2,…,Ok}, the
following characteristics for the probability P(Oi)
of the simple event Oi must hold:
1.
0  POi   1 for each i
k
2.
 PO   1
i
i 1
– Probability of an event: The probability P(A),
of event A is the sum of the probabilities
assigned to the simple events contained in A.
12
Assigning Probabilities
• P(A) is the proportion of times the event A
is observed.
total outcomes in A
P( A) 
total outcomes in S
13
Set theory: Definitions
• Set: a set A is a collection of elements (or
outcomes)
• Membership: xA (x is in A), or x  A (x is not in A)
C
• Complement: A  {x : x  A}
• Union: A  B  {x : x  A or x  B}
• Intersection: A  B  {x : x  A and x  B}
• Difference: A \ B  ABC  {x : x  A and x  B}
• Subset: A  B ( x  A  x  B) A is contained in B
• Equality: A  B if A  B and B  A
• Symmetric difference: AB  {x : x  A or x  B, but not both}
14
Algebraic laws
• commutative: A ∪ B = B ∪ A
A∩B=B∩A
• associative: (A ∪ B) ∪ C = A ∪ (B ∪ C)
A ∩ (B ∩ C) = (A ∩ B) ∩ C
• distributive: A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
• DeMorgan’s: (A ∪ B)' = A' ∩ B' (' is complement)
(A ∩ B)' = A' ∪ B'
15
Intersection
• The intersection of event A and B is the event
that occurs when both A and B occur.
• The intersection of events A and B is denoted by
(A and B) or AB.
• The joint probability of A and B is the probability
of the intersection of A and B, which is denoted
by P(A and B) or P(AB).
16
Union
• The union event of A and B is the event
that occurs when either A or B or both
occur.
• At least one of the events occur.
• It is denoted “A or B” OR AB
17
Addition Rule
For any two events A and B
P(A  B) = P(A) + P(B) - P(A  B)
18
Complement Rule
• The complement of event A (denoted by
AC) is the event that occurs when event A
does not occur.
• The probability of the complement event is
calculated by
A and AC consist of all the
simple events in the
sample space. Therefore,
P(A) + P(AC) = 1
P(AC) = 1 - P(A)
19
MUTUALLY EXCLUSIVE EVENTS
• Two events A and B are said to be
mutually exclusive or disjoint, if A and B
have no common outcomes. That is,
A and B =  (empty set)
•The events A1,A2,… are pairwise mutually
exclusive (disjoint), if
Ai  Aj =  for all i  j.
20
EXAMPLE
• The number of spots turning up when a
six-sided dice is tossed is observed.
Consider the following events.
A: The number observed is at most 2.
B: The number observed is an even number.
C: The number 4 turns up.
21
VENN DIAGRAM
• A graphical representation of the sample
1
space.
A
AB
S
1
2
B
4
A
3
6
5
C
4
2
1
B
6
AB
A
22
4
AC = A and C are mutually exclusive
B
6
22
AXIOMS OF PROBABILTY
(KOLMOGOROV AXIOMS)
Given a sample space S, the probability
function is a function P that satisfies
1) For any event A, 0  P(A)  1.
2) P(S) = 1.
3) If A1, A2,… are pairwise disjoint, then

P ( Ai ) 
i 1

 P( A )
i 1
i
23
Probability
P
:
Probability
function
S
domain

[0,1]
range
24
THE CALCULUS OF
PROBABILITIES
•
If P is a probability function and A is any
set, then
a. P()=0
b. P(A)  1
c. P(AC)=1  P(A)
25
THE CALCULUS OF
PROBABILITIES
•
If P is a probability function and A and B
any sets, then
a. P(B  AC) = P(B)P(A  B)
b. If A  B, then P(A)  P(B)
c. P(A  B)  P(A)+P(B)  1 (Bonferroni Inequality)

d. P 



i 1

Ai  



 P  A  for any sets A , A ,
i
1
2
i 1
(Boole’s Inequality)
26
Principle of Inclusion-Exclusion
• A generalization of addition rule
n
n
i 1
i 1
P( Ai )   P( Ai )   P( Ai Aj )  ...  (1) n1 P( A1 A2 ... An )
i j
• Proof by induction
27
EQUALLY LIKELY OUTCOMES
• The same probability is assigned to each simple
event in the sample space, S.
• Suppose that S={s1,…,sN} is a finite sample
space. If all the outcomes are equally likely, then
P({si})=1/N for every outcome si.
28
ODDS
• The odds of an event A is defined by
P( A)
P( A)

C
P( A ) 1  P( A)
•It tells us how much more likely to see the
occurrence of event A.
•P(A)=3/4P(AC)=1/4 P(A)/P(AC) = 3.
That is, the odds is 3. It is 3 times more
likely that A occurs as it is that it does not.
29
ODDS RATIO
• OR is the ratio of two odds.
• Useful for comparing the odds under two
different conditions or for two different
groups, e.g. odds for males versus
females.
• If odds of event A is 4.2 for males and 2
for females, then odds ratio is 2.1. The
odds of observing event A is 2.1 times
higher for males compared to females.
30
CONDITIONAL PROBABILITY
• (Marginal) Probability: P(A): How likely is it
that an event A will occur when an
experiment is performed?
• Conditional Probability: P(A|B): How will
the probability of event A be affected by
the knowledge of the occurrence or
nonoccurrence of event B?
• If two events are independent, then
P(A|B)=P(A)
31
CONDITIONAL PROBABILITY
P(A  B)
P(A | B) 
if
P(B)
0  P(A | B)  1
P(B)  0
P(A | B)  1  P(A C | B)
P(A | A)  1
P(A1  A 2 | B)  P(A1 | B)  P(A 2 | B)  P(A1  A 2 | B)
32
Example
•
•
•
•
Roll two dice
S=all possible pairs ={(1,1),(1,2),…,(6,6)}
Let A=first roll is 1; B=sum is 7; C=sum is 8
P(A|B)=?; P(A|C)=?
• Solution:
• P(A|B)=P(A and B)/P(B)
P(B)=P({1,6} or {2,5} or {3,4} or {4,3} or {5,2} or {6,1})
= 6/36=1/6
P(A|B)= P({1,6})/(1/6)=1/6 =P(A)
A and B are
independent
33
Example
• P(A|C)=P(A and C)/P(C)=P(Ø)/P(C)=0
A and C are disjoint
Out of curiosity:
P(C)=P({2,6} or {3,5} or {4,4} or {5,3} or {6,2})
= 5/36
CONDITIONAL PROBABILITY
P( AB)  P( A) P( B | A)  P( B) P( A | B)
P( A1 A2 ... An )  P( A1 ) P( A2 | A1 ) P( A3 | A1 , A2 )...P( An | A1 ,..., An1 )
35
Example
• Suppose we pick 4 cards at random from
a deck of 52 cards containing 4 aces.
• A=event that we pick 4 aces
• Ai=event that ith pick is an ace (i=1,2,3,4)
 52 
P( A)  1 /   1 / 270,725
4 
4 3 2 1
P( A)  P( A1 A2 A3 A4 )  P( A1 ) P( A2 | A1 )... 
 1 / 270,725
52 51 50 49
36
BAYES THEOREM
• Suppose you have P(B|A), but need
P(A|B).
P(A  B) P(B | A)P(A)
P(A | B) 

for P(B)  0
P(B)
P(B)
37
Example
• Let:
– D: Event that person has the disease;
– T: Event that medical test results positive
• Given:
– Previous research shows that 0.3 % of all Turkish
population carries this disease; i.e., P(D)= 0.3 % = 0.003
– Probability of observing a positive test result for someone
with the disease is 95%; i.e., P(T|D)=0.95
– Probability of observing a positive test result for someone
without the disease is 4%; i.e. P(T| D C )= 0.04
• Find: probability of a randomly chosen person having the
disease given that the test result is positive.
38
Example
• Solution: Need P(D|T). Use Bayes Thm.
P(D|T)=P(T|D)*P(D)/P(T)
P(T)=P(D and T)+P(D C and T)
= 0.95*0.003+0.04*0.997 = 0.04273
P(D|T) =0.95*0.003 / 0.04273 = 6.67 %
Test is not very reliable!
39
BAYES THEOREM
• Can be generalized to more than two events.
• If Ai is a partition of S, then,
P( B | Ai ) P( Ai )
P( B | Ai ) P( Ai )
P( Ai | B) 

P( B)
 P( B | A j ) P( A j )
j
• Can be rewritten in terms of odds
– Suppose A1,A2,… are competing hypotheses and B is evidence
or data relevant to choosing the correct hypothesis
P( Ai | B) P( B | Ai ) P( Ai )

P( A j | B) P( B | A j ) P( A j )
Posterior odds = likelihood ratio x prior odds
40
Independence
• A and B are independent iff
– P(A|B)=P(A) or P(B|A)=P(B)
– P(AB)=P(A)P(B)
• A1, A2, …, An are mutually independent iff
P ( Ai )   P ( Ai ) for every subset j of {1,2,…,n}
i j
i j
E.g. for n=3, A1, A2, A3 are mutually independent
iff P(A1A2A3)=P(A1)P(A2)P(A3) and
P(A1A2)=P(A1)P(A2) and P(A1A3)=P(A1)P(A3) and
P(A2A3)=P(A2)P(A3)
41
Independence
• If n=4, then the number of conditions for
independence is
 4  4  4

 4

3 

 2
  11
     
• Find these conditions.
42
Sequences of events
• A sequence of events A1, A2, … is
increasing iff A1  A2  A3 ...
• A sequence of events A1, A2, … is
decreasing iff A1  A2  A3 ...

An   Ai
• If {An} is increasing, then lim
n 
i 1

An   Ai
• If {An} is decreasing, then lim
n 
i 1
43
Examples
• Let S=(0,1) and An=(1/n,1)
{An} is increasing. What is limit of An as n
goes to infinity?
• Let S=(0,1) and Bn=(0,1/n)
{Bn} is decreasing. What is limit of Bn as n
goes to infinity?
44
Problems
1. Show that two nonempty events cannot
be disjoint and independent at the same
time.
Hint: First, prove that if they are disjoint,
then they are not independent. Second,
prove that if they are independent, then
they are not disjoint.
Problems
2. If P(A)=1/3 and P(Bc)=1/4, can A and B
be disjoint? Explain.
Problems
3. Either prove the statement is true or disprove it:
If P(B|A)=P(B|AC), then A and B are independent.
Problems
4. An insurance company has three types of customers –
high risk, medium risk, and low risk. Twenty percent of
its customers are high risk, and 30% are medium risk.
Also, the probability that a customer has at least one
accident in the current year is 0.25 for high risk, 0.16
for medium risk, and 0.1 for low risk.
a) Find the probability that a customer chosen at random
will have at least one accident in the current year.
b) Find the probability that a customer is high risk, given
that the person has had at least one accident during
the current year.
Problems
5. Eleven poker chips are numbered consecutively
1 through 10, with two of them labeled with a 6
and placed in a jar. A chip is drawn at random.
i) Find the probability of drawing a 6.
ii) Find the odds of drawing a 6 from the jar.
iii) Find the odds of not drawing a 6.
49
Problems
6. If the odds in favor of winning a horse
race are 3:5, find the probability of winning
the race.
50
Problems
7. In a hypothetical clinical study, the following
results were obtained.
Treatment
Ibuprofen 400
mg
Placebo
Total
number
of
patients
treated
Number who
achieved at
least 50% pain
relief
Number who
did not achieve
at least 50%
pain relief
40
22
18
40
7
33
Find the odds ratio and interpret.
51
Related documents