Download h-out-probability_statistics

NIRMA UNIVERSITY
INSTITUTE OF TECHNOLOGY
DEPARTMENT OF MATHEMATICS & HUMANITIES
MCA, SEM II
3CA1254, PSNA
HANDOUT
PROBABILITY THEORY
.
Objectives:
(i)
(ii)
(iii)
(iv)
To know various types of probabilities
To apply them in various problems
To understand theorems based on probability
To implement these definitions and theorems in their field
(i) Random experiment: If in each trail of an experiment conducted under identical
conditions, the out come is not unique, but may be any one of the possible
outcomes, the such an experiment is called random experiment.
(ii) Sample Space & Events: Consider a statistical experiment, which may consist of
finite or infinite number of trails each trail results into an outcome such as tossing
three coins at a time. We have 8 outcomes. The set of all possible outcomes of
an experiment is called sample space and is denoted by  or S . At the same
time, the collection of all outcomes favorable to a phenomenon or happening is
called an event and is denoted be A, B, C, etc.
(iii) Complementary events: An event A is said to be complementary to an event A in 
if A consists of all those points which are not in A .
(iv) Simple or elementary events: An event having only one point is called simple or
elementary event
(v) Transitivity of an events: if A , B and C are three events such that A  B and B  C ,
it implies that A  C . Such a property of events is called transitivity of events.
(vi) Compound events: An event, which is not simple, is called a compound event. Every
compound event can be uniquely represented by the union of a set of elementary
events.
(vii) Mutually exclusive events: Two events A and B are said to be mutually exclusive
if there is no point is common between in between the points belonging to A and B .
Consider the trail of tossing a coin thrice. Let the event A is that there are two
heads in three tossing of a coin. A has points: HHT , HTH , THH . Again let the event
B is that there are at least two tails. Event B has points: HTT , THT , TTH , TTT there
is no common point amongst the events A and B . Hence, A and B are mutually
exclusive events.
(viii) Derived events: Two or more events joined by the conjunction ‘or’ are called
derived events. For two events A and B the event A or B  A  B  is a derived
event.
(ix) Intersection of events: Intersection of two events A and B lead to an event which
conforms to the occurrence of A as well as B . Hence it consist of those points which
are common to A and B . It is denoted as A  B .
(x) Impossible events: An event, which is certain to not occur, is called an impossible
event.
(xi) Probability (Classical or Mathematical): If a random experiment or a trial results in ‘n’
exhaustive, mutually exclusive and equally likely out comes, out of which ‘m’ are
favorable to the occurrence of an event E, then the probability ‘p’ of occurrence (or
happening) of E, usually denoted by P (E ) is given by:
number of Favourable cases
m
p  P( E ) 

Total number of exastivecases n
Limitations of Classical probability: the definition of classical probability will fail on the
following cases.
a) If the various out comes of the random experiment are not equally likely. For example
i) The probability that a ceiling fan in a room will fall is not 1 / 2 , Since the events of the fan
‘falling ‘ and ‘not falling’ though mutually exclusive and exhaustive, are not equally likely.
ii) If a persons jump from the a running train, then the probability of his survival will not be
1 / 2 , since in his case the events survival & death, though mutually exclusive and exhaustive,
are not equally likely.
b) If the exhaustive number of out comes are infinite or unknown.
Statistical probability, Probability function and conditional probabilities
(a) Statistical (or Empirical) Probability: If an experiment perform repeatedly under
essentially homogeneous and identical conditions, then the limiting value of the ratio of the
number of times the event occurs to the number of trails, as the number of trail become
indefinitely large, is called probability of happening of the event, it being assume that limit is
finite and unique.
Symbolically, if in N trails an event E happens M times, then the probability of happening of
E , denoted by P (E ) , is given by:
M
P ( E )  lim .
N  N
(b) Definition of Probability function. If P( A) is the probability function defined on a  field  of events if the following axioms (properties) hold.
(i)
(ii)
(iii)
P( A)  0 , For each A   , P( A) is defined real
Axioms of non-negative
P( S )  1 ,
Axioms of certainty
If An  is any finite or infinite sequence of disjoint events in  , then
 n
 n
P  An    P( An )
 i 1  i 1
Simply  -field  is taken to be the collection of all subset of S .
(c) Conditional Probability: Many times the information is available that an event A has
occurred and one is required to find out the probability of occurrence of another event B
utilizing the information about A . Such probability is known as conditional probability and is
denoted by P( B A) i.e. the probability of B given A .
The formula is,
P( A  B)
.
P( A)
If A & B are independent then
P( B | A) 
P( B | A) 
P( A  B) P( A).P( B)

 P( B)
P( A)
P( A)
Multiplication Theorem: For two events A and B ,
P( A  B)  P( A).P( A | B)
P( A)  0
 P( B).P( B | A)
P( B)  0
Where P ( A | B ) & P ( B | A) represents the conditional probability
Multiplication Theorem of probability to n events: For n events A1, A2 ,, An we have
P( Ai  A2    An )  P( A1 ).P( A2 | A1 ).P( A3 | A1  A2 ) P( An | A1  A2    An 1 )
where P( Ai | Aj  Ak    A1 represents the conditional probability of the event Ai given that
the events A j , A k ,  , A1 have already happen
Multiplication Theorem: For two events A and B ,
P( A  B)  P( A).P( A | B)
P( A)  0
 P( B).P( B | A)
P( B)  0
Where P ( A | B ) & P ( B | A) represents the conditional probability
Multiplication Theorem of probability to n events: For n events A1, A2 ,, An we have
P( Ai  A2    An )  P( A1 ).P( A2 | A1 ).P( A3 | A1  A2 ) P( An | A1  A2    An 1 )
where P( Ai | Aj  Ak    A1 represents the conditional probability of the event Ai given that
the events A j , A k ,  , A1 have already happen
Independent Events: Two events A and B are said to be independent if the occurrence of
one does not effect the occurrence of other.
Theorem: if A and B are independent events, then
A and B
(i)
(ii)
(iii)
A and B
A and B are also independent
Pair wise independent: If A1 , A2 ,, Ak are k events, they are said to be pair wise
independent iff
PAi  A j   P( Ai ). P( A j )
 i j
Multiplication Theorem of probability to n independent events: For n events
A1, A2 ,, An are independent iff
P( Ai  A2    An )  P( A1 ).P( A2 ).P( A3 ) P( An )
Bayes’ Probability: Bayes’ probability is also known as inverse probability. The problem of
inverse probability arises when we have an out comes and we want to know the probability of
its belonging to a specified population out of many alternative population. For example, three
urns containing white (W), black (B) and red (R) balls as follows.
URN I:
URN II:
URN III:
2W, 3B and 4R balls
3W, 1B and 2R balls
4W, 2B and 5R balls
Two balls are drawn from a urn and the happen to be one white and one red balls. Now the
interest lies to know the probability that the balls are drawn from URN III. Such a probability is
Bayes’ probability.
Bayes’ theorem: If E1 , E2 , , En are mutually disjoint events with P( Ei )  0, (i  1,2,, n), then
n
for any event A , which is a subset of  Ei such that P( A)  0 , we have
i 1
P( Ei | A) 
P( Ei ).P( A | Ei )
n
 P( E ).P( A | E )
i 1
i
,
i  1,2,3, , n
i
(A) Random Variable: A rule that assigns a real number to each out come (Sample point)
of a random experiment is called random variable (r.v.). It is governed by a function of the
variable. Hence, a random variable is a real valued function X (x ) of the elements of the
sample space S where x is an element S . Further, the range of the variable will be a set of
real values. For example, in tossing a coin, x  1 , if the coins fall with head, and x  0 , if the
coins fall with tail.
(B) Type of random variables: There are two types of random variables (i) discrete random
variable & (ii) continuous random variable.
(C) Discrete random variable: A random variable X , which can take a finite number of
values in an interval of the domain, is called discrete random variable. For example, if we toss
a coin, the variable can take only two values 0 & 1 assigned to tail and head respectively i.e.
0 if x is T
X ( x)  
1 if x is H
In rolling a die, only six values of the variable X i.e. 1, 2, 3, 4, 5 and 6 are possible. Hence, the
variable X is discrete. Here the variable,
X ( x)  x : x 1, 2, 3, 4, 5 and 6
(D) Continuos random variable: A random variable X , which can take any values in its
domain or in an interval or the union of intervals on the real line is called continuos random
variable. For example the weight of middle aged people in India lying between 50 kg and 140
kg is a continuous variable. Notationally,
X ( x)  x : 50  x  140 
Properties of random variable: The properties of random variables are:
i)
If X is a random variable and a, b are any two constants, then aX  b is also a
random variable.
ii)
If X is a random variable then X 2 is also a random variable.
iii)
If X is a random variable then 1 / X is also a random variable.
iv)
If X & Y are two random variables defined over a sample space, then
X  Y , X  Y , aX , bY & aX  bY , are also random variable where a, b are any two
constants, except that a and b both are not zero.
v)
If X 1 , X 2 , , X n are n random variables then
U n  max( X 1 , X 2 , , X n ) and
U n  min( X 1 , X 2 , , X n ) are also random variable
Distribution function: A function FX (x) of a random variable X for a real value x giving the
probability of the event ( X  x) is called a cumulative distribution function (c.d.f) or simply
distribution function. Symbolically,
FX ( x)  P ( x) .
X lies in the interval ( , x)
Properties of distribution function:
i)
If a and b are two constant values such that a  b and F is the distribution function
then
P(a  X  b)  F (b)  F (a)
ii)
If F (x) is the distribution function of a mono variate X then 0  F ( x)  1 .
iii)
If X  Y , then F ( x)  F ( y ) .
iv)
If F (x) is the distribution function of a mono variate X then ,
F ()  lim F ( x)  0
x  
F ()  lim F ( x) 1
x 
Probability mass function (Discrete density function): If X 1 , X 2 ,  are the variate values
in the sample space S of a single dimensional variable X with probabilities of the occurrence
p1 , p2 , respectively, i.e. pi  P( X  xi )  P( xi ) such that P( xi )  0  i and  P( xi ) 1 ,
all x i S
P (x ) is called the probability mass (density) function.
Binomial Distribution: Consider n (finite) independent Bernoulli trails with probability p
(constant for each trail) of success and q 1  p of a failure. The probability of r success out
of n trails is given by
P ( X  r)  n Cr p r q n  r .
Physical conditions for Binomial distribution - We get the binomial distribution under the
following conditions:
i)
Each trail results in two exhaustive and mutually exclusive and mutually disjoint
out comes, termed as success and failure.
ii)
The number of trials “ n ” is finite.
iii)
The trails are independent to each other.
iv)
The probability of success “ p ” is constant for each trails.
Important features of Binomial Distribution:
i)
If n  1 , the binomial distribution reduces to Bernoulli distribution.
ii)
Binomial distribution has two parameters n and p .
iii)
Mean of the binomial distribution is np and variance is npq .
iv)
The first four moments are 1  np , 2  npq , 3  npq(n  p)
and
4  npq[1  3 pq(n  2)]
v)
(1  2 p) 2
. The value of  2 indicate the binomial
npq
1
1
distribution is positive skewed if p  and symmetric if p  .
2
2
The measure of Skewnes, 1 
Poisson Distribution: Poisson distribution is a limiting case of Binomial distribution. If a
discrete random variable X is such that the constant probability p of success for each trial is
very small ( p  0) and the number of trials n is very large (n  ) and np   is finite, the
probability of x successes is given by the probability mass function,
e   x
P( X  x) 
,
for r  0,1,2,
x!
 0,
other wise
It is denoted by P ( x;  )
Some examples of Poisson variate are:
i)
Number of deaths in a city due to suicides.
ii)
Number of defective items in a box of 1000 items.
iii)
Number plane accidents per week.
iv)
Number of mistake in typing.
Normal distribution:- Normal distribution was first discovered by De-Moiver in 1733 and was
also known to Laplace 1774. Later it was derived by Kark Friedrich Gauss in 1809 and used it
for the study of errors in astronomy. Any how, the credit of normal distribution has been given
to Gauss and is often called Gaussian distribution. Normal distribution is the maximally used
probability distribution in the theory of statistics.
A random variable X is said to fallow a normal distribution with mean  and
variance  2 , if its probability density function is
1

(x  )
2
1
for    x   ,       and   0
f ( x;  , ) 
e 2
 2
The variate x is said to be distributed normally with mean  and variance  2 and is denoted
as X ~ N ( , 2 ) .
If   0 and   1 , then
2
2
1
 x
1
f ( x) 
e 2 .
2
Here, X is said b standardised normal variate and is denoted as X ~ N (0,1) . Also pdf is
called standard normal distribution. If
X ~ N ( , 2 ) and we make a transformation,
X 
Z
, the distribution of Z is a standardised normal distribution as Z ~ N (0,1)

2
Characteristics of the normal distribution:
i)
The Normal distribution curve is bell shaped and is symmetrical about the line x  
ii) Mean, median and of mode of the normal distribution lies at same point x  
iii) The area under the normal curve within its range is always unity
iv) On either side of the line x   the frequency decreases more rapidly within the range
(    ) and get slower as we depart farther from the point  on either side. The area
under the normal curve beyond the distance  3 is only 0.27% which is very small.
Mean  Median  Mode
v)
vi) The normal curve is unimodal
Importance of the normal distribution:
i)
Most of the discrete distributions such as binomial, Poisson, etc tends to normal
distribution as n   .
ii)
Almost all sampling distributions like t ,  2 , F , etc for either large degrees of
freedom conform to normal distribution.
iii)
Many variables which are not normally distributed can be normalised through
suitable transformations.
Reference Books:
1. Probability, Random variables Stochastic processes – Papoulis.
2. Statistical Methods by S.P. Gupta – S. Chand & Sons Pub. Delhi.
3. Fundamentals of Statistics by S.S. Gupta – Himalaya Publications House.