Download probability - Dei-Isep

Al-Imam Mohammad Ibn Saud University
CS433
Modeling and Simulation
Lecture 03 – Part 01
Probability Review
http://10.2.230.10:4040/akoubaa/cs433/
27 Oct 2008
Dr. Anis Koubâa
1
Goals for Today
2
 Review the fundamental concepts of
probability
 Understand the difference between
discrete and continuous random
variable
2
2
Topics
3













Sample Space
Probability Measure
Joint Probability
Conditional Probability
Bayes’ Rule
Law of Total Probability
Random Variable
Probability Distribution
Probability Mass Function (PMF)
Continuous Random Variable
Probability Density Function (PDF)
Cumulative Distribution Function (CDF)
Mean and Variance
3
3
4
Probability Review
4
Sample space
5

In probability theory, the sample space or universal sample space or event
space, often denoted S, Ω or U (for "universe"), of an experiment or random trial
is the set of all possible outcomes.

For example, if the experiment is tossing a coin, the sample space is the set {head, tail}.
For tossing a single six-sided die, the sample space is {1, 2, 3, 4, 5, 6}.

In probability theory, an event is a set of outcomes (a subset of the sample space)
to which a probability is assigned. Typically, when the sample space is finite, any
subset of the sample space is an event (i.e. all elements of the power set of the
sample space are defined as events).
5
Probability measure
6

Every event (= set of outcomes) is assigned a probability, by a function that we
call a probability measure.

The probability of every set is between 0 and 1, inclusive.

The probability of the whole set of outcomes is 1.

If A and B are two events with no common outcomes, then the probability of their
union is the sum of their probabilities.

A single card is pulled (out of 52 cards).

Possible Events:

having a red card (P=1/2);

Having a Jack (P= 1/13);
6
Joint Probability
7

The probability that event P will not happen (=event ~P will happen) is 1-prob(P).

Event Union (U = OR)

Joint Probability (A ∩ B): the probability of two events in conjunction. That is, it
Event Intersection (∩= AND)
is the probability of both events together.
p A  B   p A   p B   p A  B 

Independent Events: Two events A and B are independent if

Example
p A  B   p A   p B 

The probability to have a red card and which is a Jack card

p=p(red) * p(jack) = 1/2 * 1/13
7
Conditional Probability
8

Conditional Probability p(A|B) is the probability of some event A,
given the occurrence of some other event B.
p A | B  

p A  B 
p B 
p B  A 
P A 
If A and B are independent,
then p  A | B   p  A  and p  B | A   p  B 

If A and B are independent, the conditional probability of A, given B is simply
the individual probability of A alone; same for B given A.

p(A) is the prior probability;
p(A|B) is called a posterior probability.

Once you know B is true, the universe you care about shrinks to B.

p B | A  
8
Bayes' rule
9

We know that

Using Conditional Probability definition, we have
p A  B   p B  A 
p A | B   P B   p B | A   P A 

The Bayes rule is:
p A | B  
p B | A   p A 
p B 
9
Law of Total Probability
10

In probability theory, the law of total probability means
the prior probability of A, P(A), is equal to
the expected value of the posterior probability of A.
That is, for any random variable N,
p  A   E  p  A | N  
where p(A|N) is the conditional probability of A given N.
10
Law of Total Probability
11

Law of alternatives: The term law of total probability is sometimes
taken to mean the law of alternatives, which is a special case of the
law of total probability applying to discrete random variables.

if { Bn : n = 1, 2, 3, ... } is a finite partition of a probability space and
each set Bn is measurable, then for any event A we have
p A  
 p A  B
n

n
or, alternatively,
p A  
 p A | B
n
  p B 
n
11
Five Minutes Break
You are free to discuss with your classmates about
the previous slides, or to refresh a bit, or to ask
questions.
Administrative issues
• Groups Formation
• Choose a “class coordinator”
A random variable
13

Random Variable is also know as stochastic variable.




A random variable is not a variable. It is a function. It maps from the
sample space to the real numbers.
random variable is defined as a quantity whose values are random and
to which a probability distribution is assigned.
More formally: a random variable is a measurable function
from a sample space to the measurable space of possible values
of the variable
Example
 A coin is tossed ten times. The random variable X is the number
of tails that are noted. X can only take the values 0, 1, ..., 10,
so X is a discrete random variable.
13
Probability Distribution
14

The probability distribution of a discrete random
variable is a list of probabilities associated with each
of its possible values.

It is also sometimes called the probability function or
the probability mass function (PMF) for discrete
random variable.
14
14
Probability Mass Function (PMF)
15

Formally
 the
probability distribution or probability mass function
(PMF) of a discrete random variable X is a function that
gives the probability p(xi) that the random variable
equals xi, for each value xi:
p x i   P X  x i
 It

satisfies the following conditions:
0  p x i   1
 p x
i
i
 1
15
Continuous Random Variable
16
A continuous random variable is one which
takes an infinite number of possible values.
 Continuous random variables are usually
measurements.
 Examples include height, weight, the amount
of sugar in an orange, the time required to
run a mile.

16
Distribution function aggregates
17




For the case of continuous variables, we do not want to
ask what the probability of "1/6" is, because the answer
is always 0...
Rather, we ask what is the probability that the value is
in the interval (a,b).
So for continuous variables, we care about the derivative
of the distribution function at a point (that's the derivative
of an integral). This is called a probability density
function (PDF).
The probability that a random variable has a value in a
set A is the integral of the p.d.f. over that set A.
17
Probability Density Function (PDF)
18


The Probability Density Function (PDF) of a continuous
random variable is a function that can be integrated to obtain
the probability that the random variable takes a value in a
given interval.
More formally, the probability density function, f(x), of a
continuous random variable X is the derivative of the
cumulative distribution function F(x):
f x  

Since F(x)=P(X≤x) it follows that
d
F x 
dx
b

F b   F  a   P a  X  b   f  x   dx
a
18
Cumulative Distribution Function (CDF)
19


The cumulative distribution function (CDF) is a
function giving the probability that the random
variable X is less than or equal to x, for every value x.
Formally
 the
cumulative distribution function F(x) is defined to be:
    x  ,
F x   P X  x 
19
Cumulative Distribution Function (CDF)
20

For a discrete random variable, the cumulative
distribution function is found by summing up the
probabilities as in the example below.
    x  ,
F x   P X  x  
 P (X
 xi ) 
x i x

 p (x
i
)
x i x
For a continuous random variable, the cumulative
distribution function is the integral of its probability
density function f(x).
b

F  a   F b   P a  X  b   f  x   dx
a
20
Two Minutes Break
You are free to discuss with your classmates about
the previous slides, or to refresh a bit, or to ask
questions.
Administrative issues
• Groups Formation
• Choose a “class coordinator”
Cumulative Distribution Function (CDF)
22

Example
 Discrete
case : Suppose a random variable X has the
following probability mass function p(xi):
xi
0 1 2
3 4 5
p(xi) 1/32 5/32 10/32 10/32 5/32 1/32

The cumulative distribution function F(x) is then:
xi
0 1 2 3 4 5
F(xi) 1/32 6/32 16/32 26/32 31/32 32/32
22
Discrete Distribution Function
23
23
Mean and variance
24


The expected value (or population mean) of a random
variable indicates its average or central value.
The expected value gives a general impression of the
behavior of some random variable without giving full details
of its probability distribution (if it is discrete) or its probability
density function (if it is continuous).
Expectation of discrete random variable X
X  E  X  
n
x
i
 p x i

i 1
Expectation of continuous random variable X X  E  X  

 x  f  x  dx

24
Mean and variance
25




Example
When a die is thrown, each of the possible faces 1, 2, 3, 4, 5,
6 (the xi's) has a probability of 1/6 (the p(xi)'s) of showing.
The expected value of the face showing is therefore:
µ = E(X) = (1 x 1/6) + (2 x 1/6) + (3 x 1/6) + (4 x 1/6) + (5
x 1/6) + (6 x 1/6) = 3.5
Notice that, in this case, E(X) is 3.5, which is not a possible
value of X.
25
Mean and variance
26



The variance is a measure of the 'spread' of a distribution about
its average value. Variance is symbolized by V(X) or Var(X) or σ2.
Whereas the mean is a way to describe the location of a
distribution, the variance is a way to capture its scale or degree of
being spread out. The unit of variance is the square of the unit of
the original variable
The Variance of the random variable X is defined as:
V  X   X2  E  X  E  X



2
 
E X
2
 E X

2
where E(X) is the expected value of the random variable X.
The standard deviation is defined as the square root of the
variance, i.e.:
  2  V X s
X
X
 
26