Download Chapter 2: The Concept of Probability

Chapter 2, Probability
2.1 Sampling space
(1) Definition: A sampling space is a set of all possible outcomes of experiments, listed
in a mutually exclusive and exhaustive manner
(2) In the above example (the students in the front row of the class described in Chapter
1), the sample space contains all the records:
S1 = {1, 2, 3, ..., 12}
(3) Note that depending on the formation of the problem, the sampling space may be
different.
- In the above example, if we are interested in whether the student is male or female,
then:
S2 = {M, F}
-
If we are interested in how many hours they study per week, then:
S3 = {10, 12, 14, 15, 20, 25}
Note that 11, 13, ... are not listed because they will not be the outcomes of the
experiments. Also, 15 is listed just once also they occur several times.
-
As another example, the sampling space of the time of the universe is (-, ).
(4) Sampling spaces can be divided into various types, such as:
finite and infinite
discrete and continuous
2.2. Events
(1) The definition: an event is any subset of a sample space
(2) In the example above, for instance,
the event of picking the students who study 15 hours per week is E1 = {1, 2, 11}
the event of picking the students who study 20 hours or more is E2 = {7, 8, 10}
(3) Events are represented by sets and hence, we shall be familiar to the important set
concepts, such as:
empty set
universal set
mutually exclusive sets
as well as set operations, such as
complement (A’)
intersection (A  B)
union (A  B)
2-1
-
The set concepts and set operations can be demonstrated using Venn diagram.
Following are some examples:
(A  B)
A’
(A  B)
A
2.3. Counting
(1) One can determine the occurrence of events by counting - though it is applicable to
finite discrete sampling spaces only. For instance, in the above example, the number
of students who study more than 19 hours per week is 3.
(2) The counting can be extended to more complicated cases using the counting rules,
including multiplication rule, permutation rule, combination rule and so on.
(3) The multiplication rule is as follows:
if
experiment A1 can be conducted in n1 ways,
experiment A2 can be conducted in n2 ways,
....
experiment Am can be conducted in nm ways,
then there are n1 • n2 • ... • nm ways to conduct the experiment that consists of
experiments A1, A2, ... Am.
Note that the multiplication rule can be demonstrated by tree diagrams
(4) The permutation rule
- An example: pick a president, a treasurer, and a secretary from the 12 students
there are 12 ways to pick the president
there are 11 ways to pick the treasurer (one has been picked for president)
there are 10 ways to pick the secretary (two have been picked)
thus, there 12 • 11 • 10 = 1320 ways to pick (to permute)
-
In general, pick r from n, the permutation is:
nPr = n(n-1)(n-2) ... (n-r+1) = n! / (n - r)!
(5) The combination rule
- An example: pick a three-person committee from the 12 students From another
aspect, consider the case of permutation, we have nPr ways to pick. However, in these
picks, there could be PTS (President, Treasurer, Secretary), or PST, or TPS, or TSP or
STP, or SPT. In the case of combination, all these picks are the same since P = C
(committee), T = C and S = C. Therefore, there are only 1320 / (3)(2)(1) = 220 ways
to pick.
-
In general, pick r from n, the combination is
2-2
n Cr
= (n/r)[(n-1)/(r-1)] ... [(n-r+1)/1] = n! / r! (n - r)!
(6) The partition rule:
pick r1, r2, ..., rk (r1 + r2 + ... + rk = n) from n objects, the number of partition is:
n! / n1! n2! ... nk!
(7) The use of counting rules really needs practice.
2.4. Probability
(1) An informal definition: the probability of an event A occur is defined as:
n
number of times A occur
P(A)  lim A 
n n
number of experiments
-
An example:
the probability of picking a student who study 15 hours per week: n = 12, nA = 3,
thus, P(A) = 1/4
the probability of picking a student who study less than 15 hours per week: n = 12, nA
= 5, thus, P(A) = 5/12.
(2) The formal definition: a probability is a numerical valued function that assigns a
number P(A) to every event A so that the following axioms hold:
a. P(A)  0
b. P(S) = 1
c. if A1, A2, ..., is a sequence of mutually exclusive events, that is AiAj = 0, if i  j,
then:
P(

i 1

Ai )   P( Ai )
i1
note: it does not show how to assign the number
(3) how to assign probability to uncertain events
* by subjective assignment
* by counting
* by probability models
2.5. Additive rules
- The formula: if A and B are any two events; then
P(AB) = P(A) + P(B) - P(AB)
This formula can be illustrated using Venn diagram
-
An example: the probability of picking a student who study 15 hours or less
A - study 15 hours, P(A) = 3/12
B - study less than 15 hours, P(B) = 5/12
P(A B) = 0 (because he / she cannot do both)
2-3
P(A B) = 8/12 = 2/3
2.6. Conditional rules
(1) The example: given that a female student is picked what is the probability the she
studies 15 hours per week? We can illustrate this using Venn diagram:
M
F
(2) The formula for conditional rule: if A and B are any events and P(B)  0, the
conditional probability of A given B is:
P(AB)
P(A / B) 
P(B)
note that P(A/B) < P(AB) because P(B) < 1.
(4) In the above example:
A - a student who studies 15 hours per week
B - a female student
P(B) = 4/12 (the probability of picking a female student)
P(AB) = 2/12 (the probability of a female student who studies 15 hours per week)
P(A/B) = P(AB) / P(B) = (2/12)(4/12) = 2/4 = 1/2
(5) Independence
if A and B are independent, then P(AB) = P(A)P(B)
under the independence the conditional probability is:
P(A/B) = P(A), P(B/A) = P(B)
The concept of independence is very important.
2.7. Multiplication rules
(1) The formula: if A and B are any events in S, then
P(AB) = P(A)P(B|A)
if P(A) 0
= P(B)P(A|B)
if P(B) 0
In particular, if A and B are independent events, then
P(A B)=P(A)P(B)
2-4
(2) The example: the probability of picking a female student who study 15 hours per
week
A - a student who studies 15 hours per week
B - a female student
P(B) = 4/12 (the probability of picking a female student)
P(A/B) = 1/2 (given a female student the probability that she studies 15 hours per
week)
P(AB) = (4/12)(1/2) = 2/12 = 1/6
or we can do it the other way:
P(A) = 3/12 (the probability of picking a student who study 15 hours per week)
P(B/A) = 2/3 (given a student who studies 15 hours per week is picked, the
probability that student is a female)
P(AB) = (3/12)(2/3) = 2/12 = 1/6
(3) Rule of elimination: if B1, B2, ..., Bn are mutually exclusive events of which one must
occur, then
n
P(A)   P(Bi )P( A / Bi )
i1
2.8. Bayes's rule
(1) The formula: if B1, B2, ..., Bn are mutually exclusive events of which one must occur,
then the probability of Bj given that A occurs is
P(B j / A) 
P(Bj )P(A / Bj )
n
 P(Bi )P(A / Bi )
i1
(2) The student example: pick a student who study 15 hours per week, is the student male
or female?
B1 - the student is a male, P(B1) = 8/12
B2 - the student is a female, P(B2) = 4/12
A - the student who study 15 hours, P(A) = 3/12
P(A/B1) = P(AB1)/P(B1) = (1/12)(8/12) = 1/8
P(A/B2) = P(AB2)/P(B2) = (2/12)(4/12) = 1/2
therefore:
P(B1)P(A/B1) = (8/12)(1/8) = 1/12
P(B2)P(A/B2) = (4/12)(1/2) = 2/12
P(B1/A) = (1/12) / [(1/12) + (2/12)] = 1/3
P(B2/A) = (2/12) / [(1/12) + (2/12)] = 2/3
The answer: we don't know for sure, but most likely, it
is a female (B2).
2-5
(3) Another example: communication with noise.
- An communication system as shown below:
noise
Input:
X = {0, 1}
Communication
channel
-
the known information:
P(X = 0) = 0.4, P(X = 1) = 0.6
P(Y = 0 / X = 0) = 0.95, P(Y = 0 / X = 1) = 0.05
P(Y = 1 / X = 1) = 0.9, P(Y = 1 / X = 0) = 0.1
-
the question: P(Y = 1) = ? P(X = 1 / Y = 1) = ?
First, use the probability formula:
P(Y = 1) = P(Y=1/X=1)P(X=1) + P(Y=1/X=0)P(X=0)
= (0.9)(0.6) + (0.05)(0.4)
= 0.56
-
Then, use Bayes formula:
P(X=1/Y=1) = P(Y=1/X=1)P(X=1) / P(Y=1)
= (0.9)(0.6) / (0.56)
= 0.9643
2-6
Output
Y = 0? or 1?