Download Now we will cover discrete random variables. A random variable is a

Now we will cover discrete random variables.
A random variable is a function which maps from the sample space Ω to the set of real numbers.
Say I have a sample space of people. Say I am interested in their height in inches. Let H be the
function from the sample space to the set of real numbers. In other words, H is a random
variable.
Here is the picture:
̅ = 2.5H (i.e. the height in cm). 𝐻
̅ is a function of H. 𝐻
̅ is also a
We also wish to define 𝐻
random variable because everything in the sample space gets mapped to something.
It is a random variable which is a function of another random variable. Thus, a function of
random a variable is also random variable.
Notation:
-
Random variable X (capital letter) (i.e. X : function Ω  ℝ).
Numerical value x (lower case letter) (i.e. x ϵ ℝ).
The Probability Mass Function (PMF) tells us how likely it is that we get an outcome that leads
to a particular numerical value. We calculate the overall probability for each outcome that leads
to a particular numerical value and we generate a bar graph. The bar graph we give a name to
which is pX(x) which is the probability mass function.
The picture is as follows.
Probability mass function definitions, notations, and observations:
px(x) = P(X = x)
= P((w ϵ Ω s.t. X(w) = x))
Note that
pX(x) ≥ 0 and ∑𝑥 px(x) = 1
which means that probabilities are nonnegative and the sum of all the possible probabilities adds
to 1.
Example
X number of heads in n ( n is fixed) independent coin tosses
P(H) = p
Let n = 4, then
pX(2) = P(HHTT) + P(HTHT) + P(HTTH) + P(THHT) + P(THTH) + P(TTHH)
= 6p2(1- p)2
= (42)p2(1- p)2
In general, we have
pX(k) = (𝑛𝑘)pk(1-p)n-k
Expected Value:
Consider the following example.
Consider the following computation:
1
1
1
5
1(6) + 2(2) + 4(3) = 2
We think of this value as an average of sorts. What we did was take each value, multiply it by its
probability of occurring, and took the sum for all x.
In general, what we did was
∑ 𝑥 ∙ px(x)
𝑥
We have a name for this computation and it is called the expected value. It is denoted by E[X].
Therefore, we have the following definition.
E[X] = ∑𝑥(𝑥 ∙ px(x))
Properties of Expectations
Let X be a random variable and let Y = g(X)
We have the following picture.
Suppose we wish to compute E[Y].
We could do this by using the definition;
∑ 𝑦 ∙ py(y)
𝑦
This means that for every particular value of y, we would collect all the outcomes that lead to
that value of y and compute their probability.
It does the addition of the “y line.”
An alternative way, is the do the addition over the “x line.” Instead of using the probabilities of
y, we use the probabilities of x, multiply by g(x) instead of x, and sum for all x.
With the formula,
∑ 𝑔(𝑥) ∙ px(x)
𝑥
The picture is as follows:
We can see that we do the same arithmetic both ways. A proof will not be included here.
Properties:
If α and β are real numbers, then
E[α] = ?
Recall the definition,
E[X] = ∑𝑥(𝑥 ∙ px(x))
Here, the only value of x is α, and pX(x) = 1 (since everything in the sample space maps to α)
Here is the picture
So we have
E[α] = ∑𝑥(𝑥 ∙ px(x))
= ∑𝑥(𝛼 ∙ px(x))
= 𝛼 ∑𝑥 px(x)
=α∙1
=α
Now let’s think about
E[αX].
We recall our definition and rule for X is a random variable and Y = g(X):
E[X] = ∑𝑥(𝑥 ∙ px(x))
And
E[Y] =∑𝑥(𝑔(𝑥) ∙ px(x))
Here, g(x) = αX, so
E[αX] = ∑𝑥(𝑔(𝑥) ∙ px(x))
= ∑𝑥(𝛼𝑥 ∙ px(x))
=α ∑𝑥(𝑥 ∙ px(x))
= αE[X]
Similarly,
E[αX + β] = ∑𝑥(αx + β)px(x)
= ∑𝑥(αx)px(x) + ∑𝑥(β)px(x)
= α ∑𝑥 x ∙ px(x) + β ∑𝑥 px(x)
= α𝐸[𝑋] + β(𝑥)(1)
= α𝐸[𝑋] + β(𝑥)
Note:
E[X2] = ∑𝑥 𝑥 2pX(x)
If we think of E[X] as the average, then we define variance to be the average squared deviation
from the mean. It is denoted V(X).
Visually, what we do is take the deviation from the mean and square it. This gives more
emphasis to big distances, such as outliers. We have the following picture.
We square the distance to the mean (and do so far all points), then take the average of that. If the
variance is large, then points are distributed far away from the mean. If the variance is small,
then the likely values are around the mean.
We now derive a formula to compute the variance.
var(X) = E[|X – E[X]|2]
= E[(X – E[X])2]
= ∑𝑥(x − E[X])2pX(x)
= ∑𝑥(x2 + (E[X])2 – 2E[X]xpX(x)
= ∑𝑥 x2pX(x) + ∑𝑥(E[X])2pX(x) - ∑𝑥 2E[X]xpX(x)
= ∑𝑥 x2pX(x) + (E[X])2 ∙ ∑𝑥 𝑝X(x) - 2E[X] ∙ ∑𝑥 x ∙ pX(x)
= E[X2] + (E[X])2 ∙ (1) - 2E[X] ∙ E[X]
= E[X2] – (E[X])2
So, if we wish to compute the variance, we have the formula
var(X) = E[X2] – (E[X])2
You may be wondering why we gave a complicated definition for the variance to tell us
information about the average distance from the mean. The answer is because we had to.
Consider the following quantity which seems like it would do the job.
E[X – E[X]]
This is the average distance from the mean.
Recall that
E[αX + β] = αE[X] + β
Therefore,
E[X – E[X]] = E[X] – E[X] = 0
This tells us that the average deviation from the mean is 0. This is not a useful quantity for us.
One solution is to add absolute values to make |X - E[X]|, but it turns out to be more useful to
use the quantity
E[|X – E[X]|2]
We call this quantity the variance.
var(X) = E[|X – E[X]|2] = E[X2] – (E[X])2
A problem with the variance is that it gets the units wrong, if you want to talk about the spread of
a distribution. For example, if we start with units in meters, the variance will have units in
meters squared.
To remedy this problem, we define standard deviation (σ) to be
σX = √𝑣𝑎𝑟(𝑋)
It gives us a quantity which is in the same units of the random variable we are dealing with.
Example
Consider the following distribution
We wish to find E[X], V(X), and σ.
E[X] = ∑𝑥 𝑥 ∙ 𝑝𝑥(𝑥)
1
1
1
=1∙4+ 3∙2+ 5∙4
=3
V(X) = E[X2] – (E[X])2
Note:
E[X2] = ∑𝑥 𝑥 2 ∙ pX(x)
1
1
1
= 12 ∙ 4 + 3 2 ∙ 2 + 5 2 ∙ 4
= 11
Therefore,
var(X) = E[X2] – (E[X])2
= 11 - 32
=2
Thus,
𝜎X = √2