Download Random Variables and Distribution Functions

STATISTICS
Random Variables and
Distribution Functions
Professor Ke-Sheng Cheng
Department of Bioenvironmental Systems Engineering
National Taiwan University
Definition of random variable (RV)
• For a given probability space (  ,A, P[]), a
random variable, denoted by X or X(), is a
function with domain  and counterdomain the
real line. The function X() must be such that the
set Ar, denoted by Ar   : X ()  r, belongs
to A for every real number r.
• Unlike the probability which is defined on the
event space, a random variable is defined on
the sample space.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
2
Random
experiment
Sample
space
Event
space
Probability
space
P{1 , 2 } is defined whereas X {1 , 2 } is not defined.
P X    r   P Ar   P : X ()  r
P{1 , 2}  P X  X (1 ) or X  X (2 )
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
3
Cumulative distribution function
(CDF)
• The cumulative distribution function of a
random variable X, denoted by FX () , is
defined to be
FX ( x)  P[ X  x]  P{ : X ( )  x}
x  R
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
4
• Consider the experiment of tossing two fair
coins. Let random variable X denote the
number of heads. CDF of X is
x0
 0
0.25 0  x  1

FX ( x )  
0
.
75
1

x

2


2 x
 1
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
5
FX ( x)  0.25I [ 0,1) ( x)  0.75I [1, 2) ( x)  I [ 2,  ) ( x)
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
6
Indicator function or indicator
variable
• Let  be any space with points  and A any
subset of . The indicator function of A,
denoted by I A () , is the function with domain
 and counterdomain equal to the set
consisting of the two real numbers 0 and 1
defined by
1 if   A
I A ( )  
0 if   A
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
7
Discrete random variables
• A random variable X will be defined to be discrete if
the range of X is countable.
• If X is a discrete random variable with values
x1 , x2 ,, xn ,, then the function denoted by
f X () and defined by
P[ X  x j ] if x  x j , j  1,2,, n,
f X ( x)  
0
if x  x j

is defined to be the discrete density function or
probability mass function (or simply probability
function) of X.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
8
Continuous random variables
• A random variable X will be defined to be
f X () such
continuous if there
exists
a
function
x
that FX ( x)   f X (u)du for every real number x.
• The function f X () is called the probability
density function of X.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
9
Properties of a CDF
FX ()  lim FX ( x)  0
x  
FX ()  lim FX ( x)  1
x  
FX (a)  FX (b) for a  b
FX () is continuous from the right, i.e.
lim F
0 h 0
4/30/2017
X
( x  h )  FX ( x )
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
10
Properties of a PDF
f X ( x)  0 x  R



f X ( x)  1
b
P[a  X  b]   f ( x)dx
a
For a continuous random variable X,
a
P[ X  a]   f ( x)dx  0, a  R
a
Two continuous random variables having the
same CDF can have different PDFs.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
11
Equality of random variables
• Two random variables (X and Y) can have the
same cumulative distribution function without
being the same random variable.
– X ( )  Y ( ),    . The two random variables
are equal everywhere.
• The following conditions does not imply X
and Y being equal
FX ( x)  FY ( x),   x  
f X ( x)  fY ( x),   x  
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
12
Example 1
• Determine which of the following are valid
distribution functions:
1  [e2 x / 2] x  0
FX ( x)  
2x
x0
 e /2
1 x  0
x
FX ( x)  u ( x  a)  u ( x  2a) u ( x)  
a
0 x  0
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
13
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
14
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
15
Example 2
• Determine the real constant a, for arbitrary real
constants m and 0 < b, such that
f X ( x)  ae
 x m / b
x  R
is a valid density function.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
16
• Function



f X (x) is symmetric about m.

f X ( x)dx  2 ae
( x  m ) / b
m
dx

 2ab  e  y dy  2ab  1
0
a  1/ 2b
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
17
Mixed distribution
• Not all random variables are either continuous
or discrete.
• A distribution that is a mixture of a discrete
distribution and a continuous distribution is
called a mixed distribution.
• Let X be the random variable that represents
the time delay that a motorist needs to wait
after making a required stop at a traffic stop
sign.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
18

0.4  0.6 1  e
 x 36
,
x0
P( X  0)  0.4
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
19
Censored and Truncated Data
• Y is censored when we observe X for all
observations, but we only know the true value
of Y for a restricted range of observations.
Values of Y in a certain range are reported as a
single value or there is significant clustering
around a value, say 0.
– Y is censored from below or left-censored if all
lower-than-k values of Y are recorded as k.
– Y is censored from above or right-censored if all
higher-than-k values of Y are recorded as k.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
20
•
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
21
• A truncated distribution (Y) is a conditional
distribution that results from restricting the
domain of some other probability distribution
(X).
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
22
Mixture Distribution
• A mixture distribution is the probability
distribution of a random variable that is derived
from a collection of other random variables as
follows: first, a random variable is selected by
chance from the collection according to given
probabilities of selection, and then the value of
the selected random variable is realized.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
23
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
24
Characterizing random variables
• Cumulative distribution function
• Probability density function
– Expectation (expected value)
– Variance
– Moments
– Quantile
– Median
– Mode
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
25
Expectation of a random variable
• The expectation (or mean, expected value) of
X, denoted by  X or E(X) , is defined by:
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
26
Rules for expectation
• Let X and Xi be random variables and c be any
real constant.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
27
X (t )  25 sin( t )
4/30/2017
E X (t )  ?
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
28
Variance of a random variable
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
29
•
 X  Var( X )  0
is called the standard
deviation of X.
Var[ X ]    E[ X ]  ( E[ X ])
 
2
X
 E X 
2
2
2
2
X
• Variance characterizes the dispersion of data
with respect to the mean. Thus, shifting a
density function does not change its variance.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
30
Rules for variance
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
31
• Two random variables are said to be
independent if knowledge of the value
assumed by one gives no clue to the value
assumed by the other.
• If two random variables X and Y are
independent, then E(XY)=E(X)E(Y)
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
32
Moments and central moments of a
random variable
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
33
Properties of moments
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
34
• Standardization, or normalization, does not
change the skewness and kurtosis.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
35
Quantile
• The qth quantile of a random variable X,
denoted by  q , is defined as the smallest
number  satisfying FX ( )  q .
Discrete Uniform
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
36
Median and mode
• The median of a random variable is the 0.5th
quantile, or  0.5 .
• The mode of a random variable X is defined as
the value u at which f X (u ) is the maximum
of f X () .
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
37
Note: For a positively skewed distribution, the
mean will always be the highest estimate of
central tendency and the mode will always be
the lowest estimate of central tendency
(assuming that the distribution has only one
mode). For negatively skewed distributions, the
mean will always be the lowest estimate of
central tendency and the mode will be the
highest estimate of central tendency. In any
skewed distribution (i.e., positive or negative)
the median will always fall in-between the mean
and the mode.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
38
Moment generating function
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
39
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
40
Usage of MGF
• MGF can be used to express moments in terms
of PDF parameters and such expressions can
again be used to express mean, variance,
coefficient of skewness, etc. in terms of PDF
parameters.
• Random variables of the same MGF are
associated with the same type of probability
distribution.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
41
• The moment generating function of a sum of
independent random variables is the product
of the moment generating functions of
individual random variables.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
42
Expected value of a function of a random
variable
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
43
• If Y=g(X)

E[ g ( X )]   g ( x) f X ( x)dx

 E Y    yf Y ( y )dy


Var[ X ]  E[( X   X ) ]
2

  ( x   X ) f X ( x)dx
2

4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
44
Y
Y=g(X)

E[ g ( X )]   g ( x) f X ( x)dx
y

 E Y    yf Y ( y )dy


x1
4/30/2017
x2
x3
X
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
45
Theorem
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
46
Chebyshev Inequality
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
47
• The Chebyshev inequality gives a bound,
which does not depend on the distribution of X,
for the probability of particular events
described in terms of a random variable and its
mean and variance.
4/30/2017
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
48