Download Word

LECTURE #2
(Refreshing Concepts on Probability)
Events and Probability
Event: possible outcome of a random experiment.
Probability (Event): (Intuitively) Ratio of number of times the Event occurred to the
number of times, N, the experiment was attempted, as N  .

If Ei, i=1,…,n all possible outcomes of an experiment:
0  P(Ei)  1
and
P [Sum(Ei)] = 1
With P denoting the probability of an event.
Note: The above definition of probability, the one that we are probably most used to, is
actually the Classical Definition arrived through deductive (intuitive) reasoning. Also
called a priori probabilities. For example: when one states that the probability of getting
a head during the tossing of a coin (a fair one of course) is one-half, he/she has arrived at
this result purely by deductive reasoning. The result does not require that any coin be
tossed (after all, it’s common sense right?). Nothing is said, however, about how one can
determine whether or not a particular coin is true/fair.
Unfortunately, there are some rather troublesome defects in the classical, or a priori
approach to defining probabilities. For example, when the total number of possible
outcomes is infinite, the deductive reasoning fails. As an example, one might seek, the
probability that a positive integer drawn at random be even. The intuitive answer (by
deductive reasoning, without doing experiments), would be one-half. If one were pressed
to justify the basis of this result, he would have to clearly stipulate his assumption that his
total set of integers of his population are ordered naturally (in sequence:
1,2,3,4,5,6,7,8,…..). However, one could just as well order the integers this way: 1,3,2;
5,7,4; 9,11,6;…..) taking the first pair of odd integers, then the first even integers, then
the second pair of odd integers, then the second odd integer and so forth. With this kind
of ordering of integers, one can easily argue that the probability of drawing an even
integer is one-third (1/3). Also, the ordering could be such that there is actually no fixed
probability value, and the probability value oscillates back and forth as the sample size
increases.
Important Note: In engineering, more so importantly in Hydrosciences, the a priori
definition of probability is often redundant or unsuitable. We are often required to
‘estimate’ probabilities based on experiments (as opposed to deductive reasoning)
1
because no universal theory on the ordering of outcomes or the size of outcomes usually
exist (e.g., consider the infinite number of storm hydrographs that have the same runoff
volume). We shall visit this issue later in the course.
Mutually Independent Events: Occurrence of any one of them bears no relation to the
occurrence of any other.
P(ABC) = P(ABC) = P(A) P(B) P(C) 
A, B, C, …: Mutually independent events.
Mutually Exclusive Events: Occurrence of one precludes the occurrence of any other.
P(ABC) = P(A+B+C+) = P(A) + P(B) + P(C) + 
A,B,C,: Mutually exclusive events.
For non-independent events, A and B, we define the conditional probability P(A/B)
(Probability that A will occur given that B has already occurred):
P(A/B) = P(A B)/P(B)

If A and B are Independent: P(A/B) = [P(A) P(B)]/P(B) = P(A)
It follows (from definition of P(A/B)):
P(A B) = P(A/B) P(B) = P(B/A) P(A)
Which gives:
P(A/B) = [ P(B/A) P(A) ] / P(B)
Simple Events: Mutually exclusive and collectively exhaustive.
If there are n possible outcomes that form simple events Ai, i=1,2,…,n given that another
event B has occurred (see Venn diagram below):
P( Ai / B) 
P( B / Ai ) P( Ai )
P( B)
Also:
P(B) = P(B/A1) P(A1) + … + P(B/An) P(An)
2
Venn Diagram: Simple events Ai and intersecting event B
Then,
P( Ai / B) 
P( B / Ai ) P( Ai )
n
BAYES'S THEOREM
 P( B / Ai ) P( Ai )
i 1
Where:
P(Ai) - Prior Probabilities
P(Ai/B) - Posterior Probabilities
P(B/Ai) - Likelihood of observing B is Ai has occurred (or it is true).
Bayes Theorem is very important:
Analysis tool that facilitates combining different kinds of information and updating
knowledge based on newly acquired information. It can be used to incorporate
additional observations to improve apriori estimates of probability of events. More
about this later!
Trivia#2
You borrow your roommate’s car and he warns you, “the gas gauge shows empty, but
there are somewhere between 3 to 4 gallons left.” Because you must drive about 40
3
miles, you ask, ‘what kind of mileage your car gets?’ The answer you get from your
roommate irritates you: ’20-25 miles per gallon.’ Now find the probability of getting back
without running out of gas, assuming you put no gas (no money of course!) in the car and
the information given by your roommate is trustworthy, despite its high degree of
uncertainty. Assume independence and for lack of better information, uniform
distributions for the gas and mileage.
Comment: Simple enough for an answer! But what if the distance you had to travel was
‘uncertain’ say ranging between 30-40 miles (after all, yahoo maps can sometimes give
you bad directions)?
Optional Reading Assignment#1
Chapter 1 and 2 of “Introduction to the Theory of Statistics (Mood)
Certainty Vs. Uncertainty
In search of natural laws that govern a phenomenon, science and engineering often faces
‘events’ that may or may not occur. For example, the event that an orbital satellite in
space is at a certain position; or the event that it will rain tomorrow in Cookeville. In any
experiment, an event that may or may not occur is called random. Natural processes in
hydrology are essentially random in nature (e.g, rainfall, runoff, evaporation, infiltration
etc.). A further aspect that may add to the randomness is the use of mathematical models
in hydroscience which are only an abstraction of reality reflecting incomplete knowledge
of the natural processes. As we may have noticed, the models we use today for predicting
hydrologic phenomenon hardly agree precisely with reality (refer to flowchart shown in
Lecture#1).
Random Variables and Probability Distributions
Random Variables: A function of the outcomes of some random experiment.
(Cumulative) Probability Distribution Function (CDF) F(x): Manner with which different
values are taken by the random variable.
F(x) = P(X  x)
Where, x: specific value, and X random variable.
Probability Density Function (PDF) (x):
x
(x) = dF(x)/dx
or F(x) =

(u) du

4

F() =

(u) du = 1

(x) is truly defined for random variables that take values over a continuous range.
If X takes a set of discrete values, xi - with nonzero probabilities pi - (x) is infinite at
these values of x. Expressed as:
(x) =

pi (x-xi)
i
(x): DIRAC DELTA function which is zero everywhere except at x=0, where it is
infinite in such a way that the integral of the function across the singularity is unity.

Important Property:
 G ( x) ( x  x 0 )dx  G ( x 0 )

Where G(x) is a finite valued and continuous function at x0.
Most of the lectures in this course will use continuous random variables.
Joint Probability Distribution Function:
Simultaneous consideration of two random variables by the joint probability distribution
function. For two random variables X and Y it is defined by:
F(x,y) = P(X  x and Y  y)
Joint Probability Density Function:
f ( x, y ) 
 2 F ( x, y )
xy
It holds that:
F(x) = F(x, )
f ( x) 

 f ( x, y)dy

For X, Y independent:
5
F(x,y) = P(Xx & Yy) = P(Xx) P(Yy) = FX(x) FY(y)
Also
(x,y) = X(x) Y(y)
Subscripts are used to emphasize that the distributions are different functions of different
random variables. In the following presentation the subscript numeral to a
distribution function will be omitted since it is implied by the arguments of the
function.
Conditional CDF: F(x/y) = P[ Xx / Yy ]
Conditional PDF:
(x/y) = dF(x/y)/dx

 f ( x / y)dx  1, and

(x,y) = (x/y) (y) = (y/x) (x)
Bayes's Theorem:
f ( x / y) 
f ( x, y )

f ( y)
f ( y / x) f ( x)

 f ( y / x) f ( x)dx

In previous equations the conditional CDFs and PDFs depend on a specific value of Y=y.
If Y is left as random then the conditional CDFs and PDFs become functions of the
random variable Y.
Variables X, Y are independent if and only if:
(x,y) = (x) (y)
Also if X and Y are independent:
(x/y) = (x)
6
CLASS DISCUSSION ON PROJECT: SHARING IDEAS
Trivia#3
To use: Apply shampoo to wet hair. Massage to lather, then rinse. Repeat.
- A typical hair-washing algorithm that fails to halt (some of you might have noticed it on
the back label of a shampoo bottle).
7