Download Lecture 1

Chapter 1
Introduction
Lectures 1 - 3.
In this chapter we introduce the basic notions random experiment, sample space, events and probability
of event.
By a random experiment, we mean an experiment which has multiple outcomes and one don't know in
advance which outcome is going to occur. We call this an experiment with `random' outcome. We assume
that the set of all possible outcomes of the experiment is known.
Definition 1.1. Sample space of a random experiment is the set of all possible outcomes of the random
experiment.
Example 1.0.1 Toss a coin and note down the face. This is a random experiment, since there are
multiple outcomes and outcome is not known before the toss, in other words, outcome occur randomly.
More over, the sample space is
Example 1.0.2 Toss two coins and note down the number of heads obtained. Here sample space is
.
Example 1.0.3 Pick a point `at random' from the interval
picking any point. Sample space is
. `At random' means there is no bias in
.
Definition 1.2 ( Event ) Any subset of a sample space is said to be an event.
Example 1.0.4
is an event corresponding to the sample space in Example 1.0.1.
Definition 1.3 (mutually exclusive events) Two events
are said to be mutually exclusive if
.
If
and
are mutually exclusive, then occurrence of
Note that non occurrence of
Example 1.0.5 The events
But the events
,
implies non occurrence of
need not imply occurrence of
,
, since
and vice versa.
need not be
of the sample space in Example 1.0.1 are mutually exclusive.
are not mutually exclusive.
Now we introduce the concept of probability of events (in other words probability measure). Intuitively
probability quantifies the chance of the occurrence of an event. We say that an event has occurred, if the
outcome belongs to the event. In general it is not possible to assign probabilities to all events from the
sample space. For the experiment given in Example 1.0.3, it is not possible to assign probabilities to all
. So one need to restrict to a smaller class of subsets of the sample space. For the
subsets of
random experiment given in Example 1.0.3, it turns out that one can assign probability to each interval in
as its length. Therefore, one can assign probability to any finite union of intervals in
, by
representating the finite union of intervals as a finite disjoint union of intervals. In fact one can assign
probability to any countable union interval in
by preserving the desirable property "probability of
countable disjoint union is the sum of probabilities". Also note that if one can assign probability to an
event, then one can assign probability to its compliment, since occurence of the event is same as the
non-occurance of its compliemt. Thus one seek to define probability on those class of events which
satisfies "closed under complimentation" and "closed under countable union". This leads to the following
special family of events where one can assign probabilities.
Definition 1.4 A family of subsets
following.
(i)
of a nonempty set
is said to be a
-field if it satisfies the
(ii) if
, then
(iii) if
, then
Example 1.0.6
Let
Then
i.e.,
-fields. Moreover, if
is the smallest and
Then
Let
is a
Lemma 1.0.1
is a
is the largest
be a nonempty set and
-field and is the smallest
is called the
is a
be a nonempty set. Define
are
Example 1.0.7
.
-field generated by
Let
-field of subsets of
-field of subsets of
, then
.
. Define
-field containing the set
.
.
be an index set and
be a family of
-fields. Then
-field.
Proof. Since
for all
, we have
. Now,
Similarly it follows that
Hence
is a
-field.
Example 1.0.8
is a
Let
. Then
-field and is the smallest
-field containing
This can be seen as follows. From Lemma 1.0.1,
clearly
. If
-field containing
(ii) if
, then
-field containing
is a
, then
-field. From the definition of
. Hence,
.
Definition 1.5 A family
(i)
is a
. We denote it by
of subsets of a non empty set
is said to be a field if
,
is the smallest
(iii) if
, then
Example 1.0.9 Any
.
-field is a field. In particular,
Example 1.0.10 Let
Then
are fields.
. Define
is a field but not a
-field.
Note that (i) and (ii) in the definition of field follows easily. To see (iii), for
are finite so is
(iii) follows. i.e.,
To see that
and if either
or
, if both
is finite, then
is finite. Hence
is a field.
is not a
-field, take
Now
Definition 1.6 (Probability measure)
Let
be a nonempty set and
be a
-field of subsets of
. A map
is said to be
a probability measure if P satisfies
(i)
(ii) if
are pairwise disjoint, then
Definition 1.7 (Probability space).
; where
The triplet
, a nonempty set (sample space),
,a
-field and
, a probability
measure; is called a probability space.
. Define
on
as follows.
Example 1.0.11
Let
Then
is a probability space. This probability space corresponds to the random experiment of
tossing an unbiased coin and noting the face.
Example 1.0.12
Let
Then
is a probability space.
Solution.
. Define
on
as follows.
If
(This holds since
are pairwise disjoint. Then
's are disjoint)
Therefore
Therefore properties (i) , (ii) are satisfied. Hence
Theorem 1.0.1 (Properties of probability measure)
are in
(1)
. Then
.
(2) Finite sub-additivity:
(3)Monotonicity: if
, then
(4)Boole's inequality (Countable sub-additivity):
(5)Inclusion - exclusion formula:
(6)Continuity property:
(i) For
(ii) For
Proof. Since
,
is a probability measure.
Let
a probability space and
This proves (1). Now
Therefore
since
. This proves (3).
We prove (5) by induction. For
and
(Here
Hence we have
and
Combining the above, we have
Assume that equality holds for
Consider
Therefore the result true for
. Hence by induction property (5) follows.
From property (5), we have
Hence
Thus we have (2).
Now we prove (6)(i). Set
Then
are disjoint and
(1.0.1)
Also
(1.0.2)
Using (1.0.1), we get
Now using the definition of convergence of series, one has
(1.0.3)
Hence from (1.0.3), we have
Proof of (6)(ii) is as follows.
Note that
Now using (6)(i) we have
i.e.,
Hence
From property (2), it follows that
i.e.,
Therefore
(1.0.4)
Set
Then
and are in
. Also
Hence
(1.0.5)
Here the second equality follows from the continuity property 6(i). Using (1.0.5), letting
(1.0.4), we have
in
Recall that all the examples of probability spaces we had seen till now are with sample space finite or
countable and the
-field as the power set of the sample space. Now let us look at a random experiment
with uncountable sample space and the
-field as a proper subset of the power set.
Consider the random experiment in Example 1.0.3, i.e, pick a point 'at random' from the interval
.
Since point is picked 'at random', the probability measure should satisfy the following.
(1.0.6)
The
-field we are using to define P is
is called the Borel
-field of subsets of
Our aim is to define
, the
.
.
for all elements of
Clearly
-field generated by all intervals in
, preserving (1.0.6). Set
.
Let
. then
can be represented as
where
,
Then
where
Therefore
.
, it follows from the definition of
For
Hence
Define
that
.
is a field.
on
as follows.
(1.0.7)
where
's are pair wise disjoint intervals of the form
Extension of
from
to
.
follows from the extension theorem by Caratheodary. To understand the
statement of the extension theorem, we need the following definition.
Definition 1.8 (Probability measure on a field) Let
be a nonempty set and
be a field. Then
is said to be a probability measure on
if
(i)
(ii) if
Example 1.0.13
be such that
The set function
Theorem 1.0.2 (Extension Theorem)
extension to
are pairwise disjoint and
, then
given by (1.0.7) is a probability measure on the field
A probability measure defined on a field
has a unique
.
Using Theorem 1.0.2, one can extend
defined by (1.0.7) to
see Exercise 1.6 , there exists a unique probability measure
. Since
on
preserving (1.0.6).
.