Download Topic 3: Introduction to Probability

Topic 3:
Introduction to
Probability
1
Contents
1.
2.
3.
4.
5.
6.
7.
8.
9.
Introduction
Simple Definitions
Types of Probability
Theorems of Probability
Probabilities under conditions of statistically
independent events
Probabilities under conditions of statistically
dependent events
Counting Rule
permutation
combinations
2
Introduction

If an experiment is repeated under essentially
homogeneous & similar conditions we generally
come across 2 types of situations:

Deterministic/ Predictable: - The result of what
is usually known as the ‘outcome’ is unique or
certain.

Example:- The velocity ‘v’ of a particle after time ‘t’
is given by
v = u + at
Equation uniquely determines v if the right-hand
quantities are known.
3

Unpredictable/ Probabilistic: - The result is not
unique but may be one of the several possible
outcomes.

Examples: (i) In tossing of a coin one is not sure if a head or a
tail will be obtained.
(ii) If a light tube has lasted for t hours, nothing can
be said about its further life. It may fail to
function any moment.
4
Simple Definitions

Trial & Event

Example: - Consider an experiment which, though
repeated under essentially identical conditions, does
not give unique results but may result in any one of the
several possible outcomes.

Experiment is known as a Trial & the outcomes are
known as Events or Cases.
 Throwing a die is a Trial & getting (1,2,3,…,6) is an
event.
 Tossing a coin is a Trial & getting Head (H) or Tail (T)
is an event.
5

Exhaustive Events: - The total number of possible
outcomes in any trial.


In tossing a coin there are 2 exhaustive cases, head &
tail.
In throwing a die, there are 6 exhaustive cases since
any one of the 6 faces 1,2,…,6 may come uppermost.
Experiment
Collectively Exhaustive Events
In a tossing of an unbiased coin
Possible solutions – Head/ Tail
Exhaustive no. of cases – 2
In a throw of an unbiased cubic
die
Possible solutions – 1,2,3,4,5,6
Exhaustive no. of cases – 6
In drawing a card from a well
shuffled standard pack of playing
cards
Possible solutions – Ace to King
Exhaustive no. of cases – 52
6

Favorable Events/ Cases: - It is the number of
outcomes which entail the happening of an event.



In throwing of 2 dice, the number of cases favorable to
getting the sum 5 is:
(1,4), (4,1), (2,3), (3,2).
In drawing a card from a pack of cards the number of
cases favorable to drawing an ace is 4, for drawing a
spade is 13 & for drawing a red card is 26.
Independent Events: - If the happening (or nonhappening) of an event is not affected by the
supplementary knowledge concerning the occurrence
of any number of the remaining events.

In tossing an unbiased coin the event of getting a head in
the first toss is independent of getting a head in the
second, third & subsequent throws.
7

Mutually exclusive Events: - If the happening of any
one of the event precludes the happening of all the
others.


In tossing a coin the events head & tail are mutually
exclusive.
In throwing a die all the 6 faces numbered 1 to 6 are
mutually exclusive since if any one of these faces comes,
the possibility of others, in the same trial, is ruled out.
Experiment
Mutually Exclusive Events
In a tossing of an unbiased
coin
Head/ Tail
In a throw of an unbiased
cubic die
Occurrence of 1 or 2 or 3 or 4 or 5 or
6
In drawing a card from a well Card is a spade or heart
shuffled standard pack of
Card is a diamond or club
playing cards
Card is a king or a queen
8

Equally likely Events: - Outcomes of a trial are said to
be equally likely if taken into consideration all the relevant
evidences, there is no reason to expect one in preference
to the others.


In tossing an unbiased coin or uniform coin, head or tail are
equally likely events.
In throwing an unbiased die, all the 6 faces are equally likely
to come.
Experiment
Collectively Exhaustive Events
In a tossing of an unbiased
coin
Head is likely to come up as a Tail
In a throw of an unbiased
cubic die
Any number out of 1,2,3,4,5,6 is
likely to come up
In drawing a card from a well
shuffled standard pack of
playing cards
Any card out of 52 is likely to come
up
9

Probability: Probability of a given event is an
expression of likelihood of occurrence of an event.



Probability is a number which ranges from 0 to 1.
Zero (0) for an event which cannot occur and 1 for an
event which is certain to occur.
Importance of the concept of Probability



Probability models can be used for making predictions.
Probability theory facilitates the construction of
econometric model.
It facilitates the managerial decisions on planning and
control.
10
Types of Probability
There are 3 approaches to probability, namely:
1.
2.
3.
The Classical or ‘a priori’ probability
The Statistical or Empirical probability
The Axiomatic probability
11
Mathematical/ Classical/ ‘a priori’ Probability

Basic assumption of classical approach is that the
outcomes of a random experiment are “equally
likely”.

According to Laplace, a French Mathematician:
“Probability, is the ratio of the number of ‘favorable’ cases
to the total number of equally likely cases”.

If the probability of occurrence of A is denoted by
p(A), then by this definition, we have:
Number of favorable cases
m
p = P(E) = ------------------------------ = ---Total number of equally likely cases
n
12

Probability ‘p’ of the happening of an event is also
known as probability of success & ‘q’ the nonhappening of the event as the probability of failure.

If P(E) = 1, E is called a certain event &
if P(E) = 0, E is called an impossible event

The probability of an event E is a number such that
0 ≤ P(E) ≤ 1, & the sum of the probability that an
event will occur & an event will not occur is equal to
1.
i.e., p + q = 1
13
Limitations of Classical definition

Classical probability is often called a priori probability
because if one keeps using orderly examples of
unbiased dice, fair coin, etc. one can state the
answer in advance (a priori) without rolling a dice,
tossing a coin etc.

Classical definition of probability is not very
satisfactory because of the following reasons:


It fails when the number of possible outcomes of the
experiment is infinite.
It is based on the cases which are “equally likely” and
as such cannot be applied to experiments where the
outcomes are not equally likely.
14

It may not be possible practically to enumerate
all the possible outcomes of certain experiments
and in such cases the method fails.

Example it is inadequate for answering questions
such as: What is the probability that a man aged
45 will die within the next year?
Here there are only 2 possible outcomes, the
individual will die in the ensuing year or he will live.
The chances that he will die is of course much
smaller than he will live.
How much smaller?
15
Relative/ Statistical/ Empirical Probability

Probability of an event is determined objectively by repetitive
empirical observations/ Experiments. Probabilities are assigned
a posterior.

According to Von Mises “If an experiment is performed
repeatedly under essentially homogeneous conditions and
identical conditions, then the limiting value of the ratio of the
number of times the event occurs to the number of trials, as the
number of trials becomes indefinitely large, is called the
probability of happening of the event, it being assumed that the
limit is finite and unique”.

Example: - When a coin is tossed, what is the probability that the
coin will turn heads?

Suppose coin is tossed for 50 times & it falls head 20 times, then
the ratio 20/50 is used as an estimate of the probability of heads
of this coin.
16

Symbolically, if in n trials an event E happens m
times, then the probability ‘ p’ of the happening of E
is given by
m
p = P(E) = Lim
N -> 
---N

In this case, as the number of trails increase
probabilities of outcomes move closer to the real
probabilities and tend to be real probabilities as the
number of trails tends to infinity (a large number).

The empirical probability approaches the classical
probability as the number of trails becomes
indefinitely large.
17
Limitations of Statistical/ Empirical method

The Empirical probability P(A) defined earlier can never
be obtained in practice and we can only attempt at a
close estimate of P(A) by making N sufficiently large.

The experimental conditions may not remain essentially
homogeneous and identical in a large number of
repetitions of the experiment.

The relative frequency of m/N, may not attain a unique
value, no matter however large N may be.
18
The Axiomatic Approach

Modern theory of probability is based on the axiomatic
approach introduced by the Russian Mathematician A. N.
Kolmogorov in 1930’s.

Classical approach restricts the calculation of probability to
essentially equally likely & mutually exclusively events.

Empirical approach requires that every question be
examined experimentally under identical conditions, over a
long period of time considering repeated observations.

Axiomatic approach is largely free from the inadequacies of
both the classical & empirical approaches.
19
Given a sample space of a random experiment, the
probability of the occurrence of any event A is defined as a
set function P(A) satisfying the following axioms.

1.
2.
3.
Axiom 1: - P(A) is defined, is real and non-negative i.e.,
P(A) ≥ 0 (Axiom of non-negativity)
Axiom 2: - P(S) = 1 (Axiom of certainty)
Axiom 3: - If A1, A2, …., An is any finite or infinite sequence of
disjoint events of S, then
n
n
P ( U Ai) = ∑ P( Ai )
i=1
i=1
20
Theorems of Probability

There are 2 important theorems of probability
which are as follows:


The Addition Theorem and
The Multiplication Theorem
21
Addition theorem when events are Mutually Exclusive


Definition: - It states that if 2 events A and B are
mutually exclusive then the probability of the occurrence
of either A or B is the sum of the individual probability of
A and B.
Symbolically
P(A or B) or P(A U B) = P(A) + P(B)

The theorem can be extended to three or more mutually
exclusive events. Thus,
P(A or B or C) = P(A) + P(B) + P(C)
22
Addition theorem when events are not Mutually
Exclusive (Overlapping or Intersection Events)


Definition: - It states that if 2 events A and B are not
mutually exclusive then the probability of the occurrence
of either A or B is the sum of the individual probability of
A and B minus the probability of occurrence of both A
and B.
Symbolically
P(A or B) or P(A U B) = P(A) + P(B) – P(A ∩ B)
23
Multiplication theorem

Definition: States that if 2 events A and B are
independent, then the probability of the occurrence of
both of them (A & B) is the product of the individual
probability of A and B.

Symbolically,
Probability of happening of both the events:
P(A and B) or P(A ∩ B) = P(A) x P(B)

Theorem can be extended to 3 or more independent
events. Thus,
P(A, B and C) or P(A ∩ B ∩ C) = P(A) x P(B) x P(C)
24
How to calculate probability in case of Dependent
Events
Case
1.
1.
2.
Probability of occurrence of at least A or B
When events are mutually
When events are not mutually exclusive
Formula
P(A U B) = P(A) + P(B)
P(A U B) = P(A) + P(B) – P(A ∩ B)
2.
Probability of occurrence of both A & B
P(A ∩ B) = P(A) + P(B) – P(A U B)
3.
Probability of occurrence of A & not B
P(A ∩ B) = P(A) - P(A ∩ B)
4.
Probability of occurrence of B & not A
P(A ∩ B) = P(B) - P(A ∩ B)
5.
Probability of non-occurrence of both A & B
P(A ∩ B) = 1 - P(A U B)
6.
Probability of non-occurrence of atleast A or B
P(A U B) = 1 - P(A ∩ B)
25
How to calculate probability in case of Independent
Events
Case
Formula
1.
Probability of occurrence of both A & B
P(A ∩ B) = P(A) x P(B)
2.
Probability of non-occurrence of both A &
B
P(A ∩ B) = P(A) x P(B)
P(A ∩ B) = P(A) x P(B)
3.
Probability of occurrence of A & not B
P(A ∩ B) = P(A) x P(B)
4.
Probability of occurrence of B & not A
P(A U B) = 1 - P(A ∩ B) = 1 – [P(A) x P(B)]
5.
Probability of occurrence of atleast one
event
6.
Probability of non-occurrence of atleast
one event
P(A U B) = 1 - P(A ∩ B) = 1 – [P(A) x P(B)]
P(A ∩ B) + P(A ∩ B) = [P(A) x P(B)] +
7.
Probability of occurrence of only one
event
[P(A) x P(B)]
26
Problem

An inspector of the Alaska Pipeline has the task of
comparing the reliability of 2 pumping stations. Each
station is susceptible to 2 kinds of failure: Pump failure &
leakage. When either (or both) occur, the station must be
shut down. The data at hand indicate that the following
probabilities prevail:
Station
P(Pump failure)
P(Leakage) P(Both)
1
0.07
0.10
0
2
0.09
0.12
0.06
Which station has the higher probability of being shut
down.
27
Solution
P(Pump failure or Leakage)
= P(Pump Failure) + P(Leakage Failure)
– P(Pump Failure ∩ Leakage Failure)
Station 1: 0.07 + 0.10 – 0
= 0.17
Station 2: 0.09 + 0.12 – 0.06
= 0.15
Thus, station 1 has the higher probability of being shut down.
28
Probability Rules
29
Probabilities under conditions of Statistical
Independence

Statistically Independent Events: - The
occurrence of one event has no effect on the
probability of the occurrence of any other event.

Most managers who use probabilities are
concerned with 2 conditions.
1.
2.
The case where one event or another will occur.
The situation where 2 or more events will both occur.
30

There are 3 types of probabilities under statistical
independence.

Marginal

Joint

Conditional

Marginal/ Unconditional Probability: - A single probability
where only one event can take place.

Joint probability: - Probability of 2 or more events
occurring together or in succession.

Conditional probability: - Probability that a second event
(B) will occur if a first event (A) has already happened.
31
Example: Marginal Probability - Statistical Independence

A single probability where only one event can take place.
Marginal Probability of an Event
P(A) = P(A)

Example 1: - On each individual toss of an biased or unfair
coin, P(H) = 0.90 & P(T) = 0.10. The outcomes of several
tosses of this coin are statistically independent events too,
even though the coin is biased.

Example 2: - 50 students of FST-MU drew lottery to see which
student would get a free trip to the Carnival at Dar es salaam.
Any one of the students can calculate his/ her chances of
winning as:
P(Winning) = 1/50 = 0.02
32
Example: Joint Probability - Statistical Independence

The probability of 2 or more independent events occurring
together or in succession is the product of their marginal
probabilities.
Joint Probability of 2 Independent Events
P(A then B) = P(AB)= P(A) * P(B)

Example: - What is the probability of heads on 2 successive
tosses?
P(H1H2) = P(H1) * P(H2)
= 0.5 * 0.5 = 0.25
The probability of heads on 2 successive tosses is 0.25, since
the probability of any outcome is not affected by any
preceding outcome.
33

We can make the probabilities of events even more explicit using a Probabilistic
Tree.
1 Toss
2 Toss
3 Toss
H1
0.5
H1H2
0.25
H1H2H3
0.125
T1
0.5
H1T2
0.25
H1H2T3
0.125
T1H2
0.25
H1T2H3
0.125
T1T2
0.25
H1T2T3
0.125
T1H2H3
0.125
T1H2T3
0.125
T1T2H3
0.125
T1T2T3
0.125
34
Example: Conditional Probability - Statistical Independence

For statistically independent events, conditional probability of
event B given that event A has occurred is simply the
probability of event B.
Conditional Probability for 2 Independent Events
P(B|A) = P(B)

Example: - What is the probability that the second toss of a
fair coin will result in heads, given that heads resulted on the
first toss?
P(H2|H1) = 0.5
For 2 independent events, the result of the first toss have
absolutely no effect on the results of the second toss.
35
Probabilities under conditions of Statistical
Dependence

Statistical Dependence exists when the probability of
some event is dependent on or affected by the
occurrence of some other event.

The types of probabilities under statistical dependence
are:
•
Marginal
•
Joint
•
Conditional
36
Consider the following example

Assume that a box contains 10 balls distributed as follows: 



3 are colored & dotted
1 is colored & striped
2 are gray & dotted
4 are gray & striped
Event
Probability of Event
1
0.1
2
0.1
3
0.1
4
0.1
5
0.1
6
0.1
7
0.1
8
0.1
Colored & Dotted
Colored & Striped
Gray & Dotted
Gray & Striped
9
0.1
10
0.1
37
Marginal Probability - Statistically Dependent

It can be computed by summing up all the joint events in
which the simple event occurs.

From example compute the marginal probability of the
event colored.
It can be computed by summing up the probabilities of
the two joint events in which colored occurred:
P(C) = P(C and D) + P(C and S)
= 0.3 + 0.1
=0.4
38
Example: Joint Probability - Statistically Dependent

Joint probabilities under conditions of statistical
dependence is given by
Joint probability for Statistically Dependent Events
P(A and B) = P(A|B) * P(B)
• What is the probability that this ball is dotted and
colored?
Probability of colored & dotted balls =
P(D and C) = P(D|C) * P(D)
= (0.75) * 0.5
= 0.3 (Approximately)
39
Example: Conditional Probability - Statistically Dependent

Given A & B to be the 2 events then,
Conditional probability for Statistically Dependent Events
P(A ∩ B)
P(A|B) = ---------P(B)

What is the probability that this ball is dotted, given that it
is colored?
The probability of drawing any one of the ball from this box is 0.1
(1/10) [Total no. of balls in the box = 10].
40
We know that there are 4 colored balls, 3 of which
are dotted & one of it striped.
P(D ∩ C)
0.3
P(D|C) = ------------- =
P(C)
-----0.4
= 0.75
P(D ∩ C) = Probability of colored & dotted balls
(3 out of 10 --- 3/10)
P(C) = 4 out of 10 --- 4/10
41
The Multiplication Rule for Counting
Multiplication Rule for counting :
In a sequence of n events in which the first
one has k1 possibilities of occuring and the
second event has k2 and the third has k3,
and so forth, the total possibilities of the
sequence will be k1k2k3kn.

The Multiplication Rule for Counting – Example 1

A nurse has three patients to visit. How
many different ways can she make her
rounds if she visits each patient only
once?
The Multiplication Rule for Counting – solution 1

She can choose from three patients for the first
visit and choose from two patients for the
second visit, since there are two left. On the
third visit, she will see the one patient who is
left. Hence, the total number of different
possible outcomes is 3 2 1= 6.
The Multiplication Rule for Counting – Example 2

Employees of a large corporation are to be
issued special coded identification cards.
The card consists of 4 letters of the
alphabet. Each letter can be used up to 4
times in the code. How many different ID
cards can be issued?
The Multiplication Rules for Counting – Solution 2

Since 4 letters are to be used, there are 4
spaces to fill ( _ _ _ _ ). Since there are
26 different letters to select from and each
letter can be used up to 4 times, then the
total number of identification cards that
can be made is 26 2626 26=
456,976.
The Multiplication Rule for Counting – Example 3
The digits 0, 1, 2, 3, and 4 are to be used in a
4-digit ID card. How many different cards are
possible if repetitions are permitted?
Solution 3:
 Since there are four spaces to fill and five
choices for each space, the solution is 5  5 
5  5 = 54 = 625.

The Multiplication Rule for Counting – Example 4
What if the repetitions were not permitted in
the previous example?
 Solution 4:
The first digit can be chosen in five ways. But
the second digit can be chosen in only four
ways, since there are only four digits left; etc.
Thus the solution is 5  4  3  2 = 120.

Permutations



Consider the possible arrangements of the letters
a, b, and c.
The possible arrangements are: abc, acb, bac, bca,
cab, cba.
If the order of the arrangement is important then we
say that each arrangement is a permutation of the
three letters. Thus there are six permutations of
the three letters.
Permutations
An arrangement of n distinct objects in a
specific order is called a permutation of the
objects.
 Note:
To determine the number of possibilities
mathematically, one can use the multiplication
rule to get:
3  2  1 = 6 permutations.

Permutations
Permutation Rule :
The arrangement of n objects in a specific
order using r objects at a time is called a
permutation of n objects taken r objects at a
time. It is written asType equation here. nPr
and the formula is given by

nPr
=
!
!
Permutations - Example


How many different ways can a chairperson
and an assistant chairperson be selected for a
research project if there are seven scientists
available?
Solution: Number of ways
= 7P2 = 7! / (7 – 2)! = 7!/5! = 42.
Permutations - Example


How many different ways can four books
be arranged on a shelf if they can be
selected from nine books?
Solution: Number of ways
=9P4 = 9! / (9 – 4)! = 9!/5! = 3024.
Combinations



Consider the possible arrangements of the letters
a, b, and c.
The possible arrangements are: abc, acb, bac, bca,
cab, cba.
If the order of the arrangement is not important
then we say that each arrangement is the same.
We say there is one combination of the three
letters.
Combinations
Combination Rule :
The number of combinations of of r objects
from n objects is denoted by nCr and the
formula is given by

nCr
=
!
! !
Combinations – Example 1
How many combinations of four objects
are there taken two at a time?
Solution 1
Number of combinations is 4C2
4C2 = 4! / [(4 – 2)! 2!] = 4!/[2!2!] = 6.

Combinations – Example 2

In order to survey the opinions of customers at
local malls, a researcher decides to select 5 malls
from a total of 12 malls in a specific geographic
area. How many different ways can the selection
be made?
Solution 2:
Number of combinations is 12C5
12C5 = 12! / [(12 – 5)! 5!] = 12!/[7!5!] = 792.
Combinations – Example 3
In a club there are 7 women and 5 men. A
committee of 3 women and 2 men is to be chosen.
How many different possibilities are there?
Solution 3
Number of possibilities are as follows:
(number of ways of selecting 3 women from 7)
(number of ways of selecting 2 men from 5) =
7C3 5C2 = (35)(10) = 350.

Combinations – Example 4

A committee of 5 people must be
selected from 5 men and 8 women. How
many ways can the selection be made if
there are at least 3 women on the
committee?
Combinations – Solution 4
The committee can consist of 3 women and 2
men, or 4 women and 1 man, or 5 women.
To find the different possibilities, find each
separately and then add them:
8C3 5C2 + 8C4 5C1 + 8C5 5C0
= (56)(10) + (70)(5) + (56)(1)
= 966.
Application of Combinations in
probability
A committee of 5 people must be selected
from 5 men and 8 women. What is the
probability of forming a committee which
consist of at least 3 women?
MORE EXAMPLES
1. Suppose you want to buy two lines for your double lines mobile
phone. How many choices do you have from vodacom, Tigo,
Airtel and Zantel mobile companies?
2. If a box contains 2 black balls and 3 red balls and two
balls are selected at randomly. Find the probability that:
(a)both are red
(b)both are of the same color
(c)one is red and one is black
62