Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
NIRMA UNIVERSITY INSTITUTE OF TECHNOLOGY DEPARTMENT OF MATHEMATICS & HUMANITIES HANDOUT PROBABILITY THEORY . Objectives: (i) (ii) (iii) (iv) To know various types of probabilities To apply them in various problems To understand theorems based on probability To implement these definitions and theorems in their field (i) Random experiment: If in each trail of an experiment conducted under identical conditions, the out come is not unique, but may be any one of the possible outcomes, the such an experiment is called random experiment. (ii) Sample Space & Events: Consider a statistical experiment, which may consist of finite or infinite number of trails each trail results into an outcome such as tossing three coins at a time. We have 8 outcomes. The set of all possible outcomes of an experiment is called sample space and is denoted by or S . At the same time, the collection of all outcomes favorable to a phenomenon or happening is called an event and is denoted be A, B, C, etc. (iii) Complementary events: An event A is said to be complementary to an event A in if A consists of all those points which are not in A . (iv) Simple or elementary events: An event having only one point is called simple or elementary event (v) Transitivity of an events: if A , B and C are three events such that A B and B C , it implies that A C . Such a property of events is called transitivity of events. (vi) Compound events: An event, which is not simple, is called a compound event. Every compound event can be uniquely represented by the union of a set of elementary events. (vii) Mutually exclusive events: Two events A and B are said to be mutually exclusive if there is no point is common between in between the points belonging to A and B . Consider the trail of tossing a coin thrice. Let the event A is that there are two heads in three tossing of a coin. A has points: HHT , HTH , THH . Again let the event B is that there are at least two tails. Event B has points: HTT , THT , TTH , TTT there is no common point amongst the events A and B . Hence, A and B are mutually exclusive events. (viii) Derived events: Two or more events joined by the conjunction ‘or’ are called derived events. For two events A and B the event A or B A B is a derived event. (ix) Intersection of events: Intersection of two events A and B lead to an event which conforms to the occurrence of A as well as B . Hence it consist of those points which are common to A and B . It is denoted as A B . (x) Impossible events: An event, which is certain to not occur, is called an impossible event. (xi) Probability (Classical or Mathematical): If a random experiment or a trial results in ‘n’ exhaustive, mutually exclusive and equally likely out comes, out of which ‘m’ are favorable to the occurrence of an event E, then the probability ‘p’ of occurrence (or happening) of E, usually denoted by P (E ) is given by: number of Favourable cases m p P( E ) Total number of exastivecases n Limitations of Classical probability: the definition of classical probability will fail on the following cases. a) If the various out comes of the random experiment are not equally likely. For example i) The probability that a ceiling fan in a room will fall is not 1 / 2 , Since the events of the fan ‘falling ‘ and ‘not falling’ though mutually exclusive and exhaustive, are not equally likely. ii) If a persons jump from the a running train, then the probability of his survival will not be 1 / 2 , since in his case the events survival & death, though mutually exclusive and exhaustive, are not equally likely. b) If the exhaustive number of out comes are infinite or unknown. Statistical probability, Probability function and conditional probabilities (a) Statistical (or Empirical) Probability: If an experiment perform repeatedly under essentially homogeneous and identical conditions, then the limiting value of the ratio of the number of times the event occurs to the number of trails, as the number of trail become indefinitely large, is called probability of happening of the event, it being assume that limit is finite and unique. Symbolically, if in N trails an event E happens M times, then the probability of happening of E , denoted by P (E ) , is given by: M P ( E ) lim . N N (b) Definition of Probability function. If P( A) is the probability function defined on a field of events if the following axioms (properties) hold. P( A) 0 , For each A , P( A) is defined real (i) Axioms of non-negative (ii) (iii) P( S ) 1 , Axioms of certainty If An is any finite or infinite sequence of disjoint events in , then n n P An P( An ) i 1 i 1 Simply -field is taken to be the collection of all subset of S . (c) Conditional Probability: Many times the information is available that an event A has occurred and one is required to find out the probability of occurrence of another event B utilizing the information about A . Such probability is known as conditional probability and is denoted by P( B A) i.e. the probability of B given A . The formula is, P( A B) . P( A) If A & B are independent then P( B | A) P( B | A) P( A B) P( A).P( B) P( B) P( A) P( A) Multiplication Theorem: For two events A and B , P( A B) P( A).P( A | B) P( A) 0 P( B).P( B | A) P( B) 0 Where P ( A | B ) & P ( B | A) represents the conditional probability Multiplication Theorem of probability to n events: For n events A1, A2 ,, An we have P( Ai A2 An ) P( A1 ).P( A2 | A1 ).P( A3 | A1 A2 ) P( An | A1 A2 An 1 ) where P( Ai | Aj Ak A1 represents the conditional probability of the event Ai given that the events A j , A k , , A1 have already happen Multiplication Theorem: For two events A and B , P( A B) P( A).P( A | B) P( A) 0 P( B).P( B | A) P( B) 0 Where P ( A | B ) & P ( B | A) represents the conditional probability Multiplication Theorem of probability to n events: For n events A1, A2 ,, An we have P( Ai A2 An ) P( A1 ).P( A2 | A1 ).P( A3 | A1 A2 ) P( An | A1 A2 An 1 ) where P( Ai | Aj Ak A1 represents the conditional probability of the event Ai given that the events A j , A k , , A1 have already happen Independent Events: Two events A and B are said to be independent if the occurrence of one does not effect the occurrence of other. Theorem: if A and B are independent events, then A and B (i) (ii) (iii) A and B A and B are also independent Pair wise independent: If A1 , A2 ,, Ak are k events, they are said to be pair wise independent iff PAi A j P( Ai ). P( A j ) i j Multiplication Theorem of probability to n independent events: For n events A1, A2 ,, An are independent iff P( Ai A2 An ) P( A1 ).P( A2 ).P( A3 ) P( An ) Bayes’ Probability: Bayes’ probability is also known as inverse probability. The problem of inverse probability arises when we have an out comes and we want to know the probability of its belonging to a specified population out of many alternative population. For example, three urns containing white (W), black (B) and red (R) balls as follows. URN I: URN II: URN III: 2W, 3B and 4R balls 3W, 1B and 2R balls 4W, 2B and 5R balls Two balls are drawn from a urn and the happen to be one white and one red balls. Now the interest lies to know the probability that the balls are drawn from URN III. Such a probability is Bayes’ probability. Bayes’ theorem: If E1 , E2 , , En are mutually disjoint events with P( Ei ) 0, (i 1,2,, n), then n for any event A , which is a subset of Ei such that P( A) 0 , we have i 1 P( Ei | A) P( Ei ).P( A | Ei ) n P( E ).P( A | E ) i 1 i , i 1,2,3, , n i (A) Random Variable: A rule that assigns a real number to each out come (Sample point) of a random experiment is called random variable (r.v.). It is governed by a function of the variable. Hence, a random variable is a real valued function X (x ) of the elements of the sample space S where x is an element S . Further, the range of the variable will be a set of real values. For example, in tossing a coin, x 1 , if the coins fall with head, and x 0 , if the coins fall with tail. (B) Type of random variables: There are two types of random variables (i) discrete random variable & (ii) continuous random variable. (C) Discrete random variable: A random variable X , which can take a finite number of values in an interval of the domain, is called discrete random variable. For example, if we toss a coin, the variable can take only two values 0 & 1 assigned to tail and head respectively i.e. 0 if x is T X ( x) 1 if x is H In rolling a die, only six values of the variable X i.e. 1, 2, 3, 4, 5 and 6 are possible. Hence, the variable X is discrete. Here the variable, X ( x) x : x 1, 2, 3, 4, 5 and 6 (D) Continuos random variable: A random variable X , which can take any values in its domain or in an interval or the union of intervals on the real line is called continuos random variable. For example the weight of middle aged people in India lying between 50 kg and 140 kg is a continuous variable. Notationally, X ( x) x : 50 x 140 Properties of random variable: The properties of random variables are: i) If X is a random variable and a, b are any two constants, then aX b is also a random variable. ii) If X is a random variable then X 2 is also a random variable. iii) If X is a random variable then 1 / X is also a random variable. iv) If X & Y are two random variables defined over a sample space, then X Y , X Y , aX , bY & aX bY , are also random variable where a, b are any two constants, except that a and b both are not zero. v) If X 1 , X 2 , , X n are n random variables then U n max( X 1 , X 2 , , X n ) and U n min( X 1 , X 2 , , X n ) are also random variable Distribution function: A function FX (x) of a random variable X for a real value x giving the probability of the event ( X x) is called a cumulative distribution function (c.d.f) or simply distribution function. Symbolically, FX ( x) P ( x) . X lies in the interval ( , x) Properties of distribution function: i) If a and b are two constant values such that a b and F is the distribution function then P(a X b) F (b) F (a) ii) If F (x) is the distribution function of a mono variate X then 0 F ( x) 1 . iii) If X Y , then F ( x) F ( y ) . iv) If F (x) is the distribution function of a mono variate X then , F () lim F ( x) 0 x F () lim F ( x) 1 x Probability mass function (Discrete density function): If X 1 , X 2 , are the variate values in the sample space S of a single dimensional variable X with probabilities of the occurrence p1 , p2 , respectively, i.e. pi P( X xi ) P( xi ) such that P( xi ) 0 i and P( xi ) 1 , all x i S P (x ) is called the probability mass (density) function. Binomial Distribution: Consider n (finite) independent Bernoulli trails with probability p (constant for each trail) of success and q 1 p of a failure. The probability of r success out of n trails is given by P ( X r) n Cr p r q n r . Physical conditions for Binomial distribution - We get the binomial distribution under the following conditions: i) Each trail results in two exhaustive and mutually exclusive and mutually disjoint out comes, termed as success and failure. ii) The number of trials “ n ” is finite. iii) The trails are independent to each other. iv) The probability of success “ p ” is constant for each trails. Important features of Binomial Distribution: i) If n 1 , the binomial distribution reduces to Bernoulli distribution. ii) Binomial distribution has two parameters n and p . iii) Mean of the binomial distribution is np and variance is npq . iv) The first four moments are 1 np , 2 npq , 3 npq(n p) and 4 npq[1 3 pq(n 2)] v) (1 2 p) 2 . The value of 2 indicate the binomial npq 1 1 distribution is positive skewed if p and symmetric if p . 2 2 The measure of Skewnes, 1 Poisson Distribution: Poisson distribution is a limiting case of Binomial distribution. If a discrete random variable X is such that the constant probability p of success for each trial is very small ( p 0) and the number of trials n is very large (n ) and np is finite, the probability of x successes is given by the probability mass function, e x P( X x) , for r 0,1,2, x! 0, other wise It is denoted by P ( x; ) Some examples of Poisson variate are: i) Number of deaths in a city due to suicides. ii) Number of defective items in a box of 1000 items. iii) Number plane accidents per week. iv) Number of mistake in typing. Normal distribution:- Normal distribution was first discovered by De-Moiver in 1733 and was also known to Laplace 1774. Later it was derived by Kark Friedrich Gauss in 1809 and used it for the study of errors in astronomy. Any how, the credit of normal distribution has been given to Gauss and is often called Gaussian distribution. Normal distribution is the maximally used probability distribution in the theory of statistics. A random variable X is said to fallow a normal distribution with mean and variance 2 , if its probability density function is 1 (x ) 2 1 for x , and 0 f ( x; , ) e 2 2 The variate x is said to be distributed normally with mean and variance 2 and is denoted as X ~ N ( , 2 ) . If 0 and 1 , then 2 2 1 x 1 f ( x) e 2 . 2 Here, X is said b standardised normal variate and is denoted as X ~ N (0,1) . Also pdf is called standard normal distribution. If X ~ N ( , 2 ) and we make a transformation, X Z , the distribution of Z is a standardised normal distribution as Z ~ N (0,1) 2 Characteristics of the normal distribution: i) The Normal distribution curve is bell shaped and is symmetrical about the line x ii) Mean, median and of mode of the normal distribution lies at same point x iii) The area under the normal curve within its range is always unity iv) On either side of the line x the frequency decreases more rapidly within the range ( ) and get slower as we depart farther from the point on either side. The area under the normal curve beyond the distance 3 is only 0.27% which is very small. Mean Median Mode v) vi) The normal curve is unimodal Importance of the normal distribution: i) Most of the discrete distributions such as binomial, Poisson, etc tends to normal distribution as n . ii) Almost all sampling distributions like t , 2 , F , etc for either large degrees of freedom conform to normal distribution. iii) Many variables which are not normally distributed can be normalised through suitable transformations. Reference Books: 1. Probability, Random variables Stochastic processes – Papoulis. 2. Statistical Methods by S.P. Gupta – S. Chand & Sons Pub. Delhi. 3. Fundamentals of Statistics by S.S. Gupta – Himalaya Publications House.