Download Introduction to Probability Theory

Random Variables
an important concept in probability
A random variable , X, is a numerical quantity
whose value is determined be a random
experiment
Examples
1.
2.
3.
4.
Two dice are rolled and X is the sum of the two upward
faces.
A coin is tossed n = 3 times and X is the number of times
that a head occurs.
We count the number of earthquakes, X, that occur in the
San Francisco region from 2000 A. D, to 2050A. D.
Today the TSX composite index is 11,050.00, X is the value
of the index in thirty days
Examples – R.V.’s - continued
5.
A point is selected at random from a square whose sides are
of length 1. X is the distance of the point from the lower
left hand corner.
point
X
6.
A chord is selected at random from a circle. X is the length
of the chord.
chord
X
Definition – The probability function, p(x), of
a random variable, X.
For any random variable, X, and any real
number, x, we define
p  x   P  X  x   P  X  x
where {X = x} = the set of all outcomes (event)
with X = x.
Definition – The cumulative distribution
function, F(x), of a random variable, X.
For any random variable, X, and any real
number, x, we define
F  x   P  X  x   P  X  x
where {X ≤ x} = the set of all outcomes (event)
with X ≤ x.
Examples
1.
Two dice are rolled and X is the sum of the two upward
faces. S , sample space is shown below with the value of X
for each outcome
(1,1)
2
(1,2)
3
(1,3)
4
(1,4)
5
(1,5)
6
(1,6)
7
(2,1)
3
(2,2)
4
(2,3)
5
(2,4)
6
(2,5)
7
(2,6)
8
(3,1)
4
(3,2)
5
(3,3)
6
(3,4)
7
(3,5)
8
(3,6)
9
(4,1)
5
(4,2)
6
(4,3)
7
(4,4)
8
(4,5)
9
(4,6)
10
(5,1)
6
(5,2)
7
(5,3)
8
(5,4)
9
(5,5)
10
(5,6)
11
(6,1)
7
(6,2)
8
(6,3)
9
(6,4)
10
(6,5)
11
(6,6)
12
1
p  2   P  X  2  P 1,1 
36
2
p  3  P  X  3  P 1, 2  ,  2,1 
36
3
p  4   P  X  4  P 1,3 ,  2, 2  ,  3,1 
36
4
5
6
5
4
p  5  , p  6   , p  7   , p 8  , p  9  
36
36
36
36
36
3
2
1
p 10   , p 11  , p 12  
36
36
36
and p  x   0 for all other x
Note :
 X  x  
for all other x
Graph
0.18
p(x)
0.12
0.06
0.00
2
3
4
5
6
7
8
x
9
10
11
12
The cumulative distribution function, F(x)
For any random variable, X, and any real number, x, we
define
F  x   P  X  x   P  X  x
where {X ≤ x} = the set of all outcomes (event) with X ≤ x.
Note {X ≤ x} =  if x < 2. Thus F(x) = 0.
{X ≤ x} = {(1,1)} if 2 ≤ x < 3. Thus F(x) = 1/36
{X ≤ x} = {(1,1) ,(1,2),(1,2)} if 3 ≤ x < 4. Thus F(x) = 3/36
Continuing we find
x2
 0
 1 2 x3
 36
 363 3  x  4
 6
 36 4  x  5
 10 5  x  6
36
 15
 36 6  x  7
F  x    21
 36 7  x  8
26
 36
8 x9
 30
 36 9  x  10
 33 10  x  11
 36
35
11  x  12
 36

12  x
1
1.2
1
0.8
0.6
0.4
0.2
0
0
5
10
F(x) is a step function
2.
A coin is tossed n = 3 times and X is the number of times
that a head occurs.
The sample Space S = {HHH (3), HHT (2), HTH (2), THH (2),
HTT (1), THT (1), TTH (1), TTT (0)}
for each outcome X is shown in brackets
1
p  0   P  X  0  P TTT 
8
3
p 1  P  X  1  P HTT,THT,TTH 
8
3
p  2   P  X  2  P HHT,HTH,THH 
8
1
p  3  P  X  3  P HHH 
8
p  x   P  X  x  P    0 for other x.
Graph
probability function
p(x)
0.4
0.3
0.2
0.1
0
0
1
2
x
3
Graph
Cumulative distribution function
1.2
1
F(x)
0.8
0.6
0.4
0.2
0
-1
0
1
2
x
3
4
Examples – R.V.’s - continued
5.
A point is selected at random from a square whose sides are
of length 1. X is the distance of the point from the lower
left hand corner.
point
X
6.
A chord is selected at random from a circle. X is the length
of the chord.
chord
X
Examples – R.V.’s - continued
5.
A point is selected at random from a square whose sides are
of length 1. X is the distance of the point from the lower
left hand corner.
point
X
S
An event, E, is any subset of the square, S.
P[E] = (area of E)/(Area of S) = area of E
E
The probability function
 set of all points a dist x 
p  x   P  X  x  P 
  0
 from lower left corner 
S
Thus p(x) = 0 for all values of x. The probability function for this
example is not very informative
The Cumulative distribution function
 set of all points within a

F  x   P  X  x  P 

 dist x from lower left corner 
S
x
0  x 1
x
1 x  2
x
2x
 0
  x2
 4
F  x   P  X  x  
Area A
 1

x0
0  x 1
1 x  2
2x
S
A
x
0  x 1
x
1 x  2
x
2x
Computation of Area A 1  x  2
x2 1
A

x
1
2
 2
tan    x 2  1
x
 


x 1
2
  tan 1

x2 1
1
 1 x2 1  
 2
2
2
A  2


x

x

1

x


 2
2
2





 x 2  1      x 2  x 2  1    tan 1
4

4





x2  1  x2


0


2

x


4
F  x   P  X  x  
 x 2  1     tan 1

4

1

x0
0  x 1



x2 1  x2

1 x  2
2x
1
F  x
0
-1
0
1
2
The probability density function, f(x), of a
continuous random variable
Suppose that X is a random variable.
Let f(x) denote a function define for -∞ < x < ∞ with the
following properties:
1.
f(x) ≥ 0
2.
 f  x  dx  1.


3.
b
P  a  X  b   f  x  dx.
a
Then f(x) is called the probability density function of X.
The random, X, is called continuous.
Probability density function, f(x)

 f  x  dx  1.

b
P  a  X  b   f  x  dx.
a
Cumulative distribution function, F(x)
F  x   P  X  x 
x
 f t  dt.

F  x
Thus if X is a continuous random variable with
probability density function, f(x) then the cumulative
distribution function of X is given by:
F  x   P  X  x 
x
 f t  dt.

Also because of the fundamental theorem of calculus.
F x 
dF  x 
dx
 f  x
Example
A point is selected at random from a square whose sides are of
length 1. X is the distance of the point from the lower left hand
corner.
point
X
0


2

x


4
F  x   P  X  x  
 x 2  1     tan 1


4

1

x0
0  x 1



x2 1  x2

1 x  2
2x
Now





f  x  F  x  

d  2

  x 1  
4
 dx 
x  0 or 2  x
0
x
2
 tan
0  x 1
1


 2
x 1  x 
 
2
1 x  2
Also
d  2

1
x

1


tan


dx 
4
 
1
2
 x
2

2
x
1
x

3
2
1
x
2

  2x   2 x

2 x tan


 2
x 1  x 
 
2
 1
3
2


d  1
x 1  x
tan
dx 
2
 2 x tan 1

2

x2 1
d  1
x
tan
dx 
2



x2 1 


x2  1 

Now
d
1
1
 tan  u   
du
1 u2
d  1
tan
dx 
and


1

x 1 
 1  x 2  1
2
d  1
x
tan
dx 
2



 1 2
   x 1
 2

  2x

x

x 1 
3

2
2
 x 1
2



d  2

 2
1
2
x  1    tan
x 1  x 

dx 
4
 

 x  2 x tan 1 x 2  1
2

 32
Finally


0

x

f  x  F x  
2


1
x

2
x
tan
 2
x  0 or 2  x
0  x 1

x2 1

1 x  2
Graph of f(x)
2
1.5
1
0.5
0
-1
0
1
2
Summary
Discrete random variables
For a discrete random variable X the probability
distribution is described by the probability function,
p(x), which has the following properties :
1. 0  px  1
2.
 px   1
x
3. Pa  X  b 
 p x 
a  x b
This denotes the sum over all values of x between a and b.
Graph: Discrete Random Variable
P  a  x  b 
p(x)
a
 p  x
a  x b
b
Continuous random variables
For a continuous random variable X the probability
distribution is described by the probability density
function f(x), which has the following properties :
1.
f(x) ≥ 0

2.
 f  x  dx  1.

3.
b
P  a  X  b   f  x  dx.
a
Graph: Continuous Random Variable
probability density function, f(x)

 f  x  dx  1.

b
P  a  X  b   f  x  dx.
a
A Probability distribution is similar to a distribution
of mass.
A Discrete distribution is similar to a point
distribution of mass.
Positive amounts of mass are put at discrete points.
p(x1)
p(x2)
p(x3)
p(x4)
x1
x2
x3
x4
A Continuous distribution is similar to a
continuous distribution of mass.
The total mass of 1 is spread over a continuum. The
mass assigned to any point is zero but has a non-zero
density
f(x)
The distribution function F(x)
This is defined for any random variable, X.
F(x) = P[X ≤ x]
Properties
1.
F(-∞) = 0 and F(∞) = 1.
Since {X ≤ - ∞} =  and {X ≤ ∞} = S
then F(- ∞) = 0 and F(∞) = 1.
2.
F(x) is non-decreasing (i. e. if x1 < x2 then
F(x1) ≤ F(x2) )
If x1 < x2 then {X ≤ x2} = {X ≤ x1}  {x1 < X ≤ x2}
Thus P[X ≤ x2] = P[X ≤ x1] + P[x1 < X ≤ x2]
or F(x2) = F(x1) + P[x1 < X ≤ x2]
Since P[x1 < X ≤ x2] ≥ 0 then F(x2) ≥ F(x1).
3.
F(b) – F(a) = P[a < X ≤ b].
If a < b then using the argument above
F(b) = F(a) + P[a < X ≤ b]
Thus F(b) – F(a) = P[a < X ≤ b].
4.
p(x) = P[X = x] =F(x) – F(x-)
Here
F  x    lim F  u 
ux
5. If p(x) = 0 for all x (i.e. X is continuous)
then F(x) is continuous.
A function F is continuous if
F  x    lim F  u   F  x    lim F u 
u x 
u x 
One can show that
Thus p(x) = 0 implies that F  x    F  x   F  x  
For Discrete Random Variables
F  x   P  X  x   p u 
u x
F(x) is a non-decreasing step function with
F    0 and F     1
p  x   F  x   F  x    jump in F  x  at x.
1.2
F(x)
1
0.8
0.6
0.4
p(x)
0.2
0
-1
0
1
2
3
4
For Continuous Random Variables Variables
F  x   P  X  x 
x
 f u  du

F(x) is a non-decreasing continuous function with
F    0 and F     1
f  x  F   x.
f(x) slope
F(x)
1
0
-1
0
1
x
2