Download Introduction to Probability and Stochastic Processes with

Introduction to Probability
and Stochastic Processes
with Applications
Introduction to Probability
and Stochastic Processes
with Applications
Liliana Blanco Castaneda
National University of Colombia
Bogota, Colombia
Viswanathan Arunachalam
Universidad de los Andes
Bogota, Colombia
Delvamuthu Dharmaraja
Indian Institute of Technology Delhi
New Delhi, India
WILEY
A JOHN WILEY & SONS, INC., PUBLICATION
Copyright © 2012 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as
permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to
the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax
(978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ
07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be
suitable for your situation. You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any loss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at (800) 762-2974, outside the United States at
(317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Blanco Castaneda, Liliana.
Introduction to probability and stochastic processes with applications / Liliana Blanco Castaneda,
Viswanathan Arunachalam, Selvamuthu Dharmaraja.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-118-29440-6 (hardback)
1. Probabilities—Textbooks. 2. Stochastic processes—Textbooks. I. Arunachalam, Viswanathan,
1969- II. Dharmaraja, Selvamuthu, 1972- III. Title.
QA274.B53 2012
519.2—dc23
2012002024
Printed in the United States of America.
10 9 8 7 6 5 4 3 2 1
To Sebastian and Paula
L.B.C.
To Akshaya and Abishek
V.A.
To Kathiravan and Madhuvanth
S.D.
CONTENTS IN BRIEF
1
Basic Concepts
1
2
Random Variables and Their Distributions
3
Some Discrete Distributions
115
4
Some Continuous Distributions
145
5
Random Vectors
191
6
Conditional Expectation
265
7
Multivariate Normal Distributions
295
8
Limit Theorems
313
9
Introduction to Stochastic Processes
339
10
Introduction to Queueing Models
417
11
Stochastic Calculus
461
12
Introduction to Mathematical Finance
497
51
vii
CONTENTS
Foreword
xiii
Preface
XV
Acknowledgments
xvii
Introduction
xix
1
Basic Concepts
1.1
1.2
1.3
1.4
2
Probability Space
Laplace Probability Space
Conditional Probability and Event Independence
Geometric Probability
Exercises
1
1
14
19
35
37
Random Variables and Their Distributions
51
2.1
2.2
2.3
2.4
51
62
67
72
Definitions and Properties
Discrete Random Variables
Continuous Random Variables
Distribution of a Function of a Random Variable
X
CONTENTS
2.5
3
4
5
7
80
101
Some Discrete Distributions
115
3.1
3.2
3.3
115
123
133
138
Discrete Uniform, Binomial and Bernoulli Distributions
Hypergeometric and Poisson Distributions
Geometric and Negative Binomial Distributions
Exercises
Some Continuous Distributions
145
4.1
4.2
4.3
4.4
4.5
4.6
145
151
161
170
172
175
181
Uniform Distribution
Normal Distribution
Family of Gamma Distributions
Weibull Distribution
Beta Distribution
Other Continuous Distributions
Exercises
Random Vectors
191
5.1
5.2
5.3
5.4
5.5
191
210
217
228
5.6
6
Expected Value and Variance of a Random Variable
Exercises
Joint Distribution of Random Variables
Independent Random Variables
Distribution of Functions of a Random Vector
Covariance and Correlation Coefficient
Expected Value of a Random Vector and VarianceCovariance Matrix
Joint Probability Generating, Moment Generating and
Characteristic Functions
Exercises
235
240
251
Conditional Expectation
265
6.1
6.2
265
280
287
Conditional Distribution
Conditional Expectation Given a σ-Algebra
Exercises
Multivariate Normal Distributions
295
7.1
295
Multivariate Normal Distribution
CONTENTS
7.2
8
9
302
308
Limit Theorems
313
8.1
8.2
8.3
8.4
313
319
323
329
333
The Weak Law of Large Numbers
Convergence of Sequences of Random Variables
The Strong Law of Large Numbers
Central Limit Theorem
Exercises
Introduction to Stochastic Processes
339
9.1
9.2
340
344
353
368
371
381
389
400
406
9.3
9.4
9.5
9.6
10
Distribution of Quadratic Forms of Multivariate Normal
Vectors
Exercises
XI
Definitions and Properties
Discrete-Time Markov Chain
9.2.1
Classification of States
9.2.2
Measure of Stationary Probabilities
Continuous-Time Markov Chains
Poisson Process
Renewal Processes
Semi-Markov Process
Exercises
Introduction to Queueing Models
417
10.1
10.2
417
419
419
427
431
431
436
438
439
440
441
445
448
452
457
10.3
10.4
Introduction
Markovian Single-Server Models
10.2.1 M/M/l/oo Queueing System
10.2.2 M/M/l/N
Queueing System
Markovian MultiServer Models
10.3.1 M/M/c/oo Queueing System
10.3.2 M/M/c/c Loss System
10.3.3 M/M/c/K Finite-Capacity Queueing System
10.3.4 M/M/oo Queueing System
Non-Markovian Models
10.4.1 M/G/l Queueing System
10.4.2 GI/M/1 Queueing System
10.4.3 M/G/l/N Queueing System
10.4.4 GI/M/1/N Queueing System
Exercises
xii
CONTENTS
11
Stochastic Calculus
461
11.1
11.2
11.3
461
472
481
491
12
Martingales
Brownian Motion
Itö Calculus
Exercises
Introduction to Mathematical Finance
497
12.1
12.2
498
504
509
512
517
521
525
527
529
12.3
12.4
Financial Derivatives
Discrete-Time Models
12.2.1 The Binomial Model
12.2.2 Multi-Period Binomial Model
Continuous-Time Models
12.3.1 Black-Scholes Formula European Call Option
12.3.2 Properties of Black-Scholes Formula
Volatility
Exercises
Appendix A: Basic Concepts on Set Theory
533
Appendix B: Introduction to Combinatorics
539
Exercises
546
Appendix C: Topics on Linear Algebra
549
Appendix D: Statistical Tables
551
D.l
D.2
D.3
D.4
Binomial Probabilities
Poisson Probabilities
Standard Normal Distribution Function
Chi-Square Distribution Function
551
557
559
560
Selected Problem Solutions
563
References
577
Glossary
581
Index
585
FOREWORD
Probability theory is the fulcrum around which the present-day mathematical
modeling of random phenomena revolves. Given its broad and increasing
application in everyday life-trade, manufacturing, reliability, or even biology
and psychology, there is an ever-growing demand from researchers for strong
textbooks expounding the theory and applications of probabilistic models.
This book is sure to be invaluable to students with varying levels of skill,
as well as scholars who wish to pursue probability theory, whether pure or
applied. It contains many different ideas and answers many questions frequently asked in classrooms. The extent of the exercises and examples chosen
from a multitude of areas will be very helpful for students to understand the
practical applications of probability theory.
The authors have extensively documented the origins of probability, giving
the reader a clear idea of the needs and developments of the subject over
many centuries. They have taken care to maintain an approach that is mathematically rigorous but at the same time simplistic and thus appealing to
students.
Although a wide array of applications have been covered in various chapters, I must make particular mention of the chapters on queueing theory and
financial mathematics. While the latter is an emerging topic, there is no limit
on the applicability of queueing models to other diverse areas.
xiii
XIV
FOREWORD
In all, the present book is the result of a long and distinguished teaching
experience of probability, queueing theory, and financial mathematics, and
this book is sure to advance the readers' knowledge of this field.
Professor Alagar Rangan
Eastern Mediterranean University
North Cyprus
PREFACE
This text is designed for a first course in the theory of probability and a
subsequent course on stochastic processes or stochastic modeling for students
in science, engineering, and economics, in particular for students who wish to
specialize in probabilistic modeling. The idea of writing this book emerged
several years ago, in response to students enrolled in courses that we were
teaching who wished to refer to materials and problems covered in the lectures.
Thus the edifice and the building blocks of the book have come mainly from
our continuously updated and expanded lecture notes over several years.
The text is divided into twelve chapters supplemented by four appendices.
The first chapter presents basic concepts of probability such as probability
spaces, independent events, conditional probability, and Bayes' rule. The second chapter discusses the concepts of random variable, distribution function
of a random variable, expected value, variance, probability generating functions, moment generating functions, and characteristic functions. In the third
and fourth chapters, we present the distributions of discrete and continuous random variables, which are frequently used in the applications. The fifth
chapter is devoted to the study of random vectors and their distributions. The
sixth chapter presents the concepts of conditional probability and conditional
expectation, and an introduction to the study of the multivariate normal distribution is discussed in seventh chapter. The law of large numbers and limit
xv
XVI
PREFACE
theorems are the goals of the eighth chapter, which studies four types of convergence for sequences of random variables, establishes relationships between
them and discusses weak and strong laws of large numbers and the central
limit theorem. The ninth chapter introduces stochastic processes with discrete
and continuous-time Markov chains as the focus of study. The tenth chapter is
devoted to queueing models and their applications. In eleventh chapter eleven
we present an elementary introduction to stochastic calculus where martingales, Brownian motion, and Ito integrals are introduced. Finally, the last
chapter is devoted to the introduction of mathematical finance. In this chapter, pricing methods such as risk-neutral valuation and Black-Scholes formula
are discussed.
In the appendices, we summarize a few mathematical basics needed for
the understanding of the material presented in the book. These cover ideas
from set theory, combinatorial analysis, and linear algebra. Finally, the last
appendix contains tables of standard distributions, which are used in applications. The bibliography is given at the end of the book, though it is not a
complete list.
At the end of each chapter there is a list of exercises to facilitate understanding of the main body of each chapter, and in some cases, additional
study material. Most of the examples and exercises are classroom tested in
the courses that we taught over many years. We have also benefited from various books on probability and statistics for some of the examples and exercises
in the text. To understand this text, the reader must have solid knowledge of
differential and integral calculus and some linear algebra.
We do hope that this introductory book provides the foundation for students to learn other subjects in their careers. This book is comprehensible to
students with diverse backgrounds. It is also well balanced, with lots of motivation to learn probability and stochastic processes and their applications.
We hope that this book will serve as a valuable text for students and reference for researchers and practitioners who wish to consult probability and its
applications.
L. BLANCO, V. ARUNACHALAM, S. DHARMARAJA
Bogota, Colombia
December, 2011
ACKNOWLEDGMENTS
We are grateful to Professor Ignacio Mantilla for providing us with motivation, academic support, and advice for this book project. We are grateful
to Professor Alagar Rangan for his encouragement and careful reading of the
draft of this book and offering invaluable advice. This book has greatly benefited from his comments and suggestions. We thank Professor Diego Escobar
for his useful suggestions. Our sincere thanks to Dr. Liliana Garrido for
her careful reading as well as her suggestions. We record our appreciation
to Laura Vielma, Christian Bravo, and Hugo Ramirez for their assistance in
typing this book. We thank our students for their feedback, incisive questions
and enthusiasm, and this has served as the platform for this project. We
acknowledge National University of Colombia, Universidad de los Andes, and
Indian Institute of Technology Delhi for the institutional support.
It is a pleasure to thank our Editor, Ms. Susanne Steitz-Filler, John Wiley h
Sons, and her colleagues for providing advice and technical assistance.
Finally, last but foremost, we thank our family for their love and support.
They were instrumental in bringing this book to fruition.
L.B.C, V.A. and S.D.
xvii
INTRODUCTION
Since its origin, probability theory has been linked to games of chance. In fact
by the time of the first roman emperor, Augustus (63 B.C.-14 A.D.), random
games were fairly common and mortality tables were being made. This was
the origin of probability and statistics. Later on, these two disciplines started
drifting apart due to their different objectives but always remained closely
connected. In the sixteenth century philosophical discussions around probability were held and Italian philosopher Gerolamo Cardano (1501-1576) was
among the first to make a mathematical approach to randomness. In the seventeenth and eighteenth centuries major advances in probability theory were
made due in part to the development of infinitesimal calculus; some outstanding results from this period include: the law of large numbers due to James
Bernoulli (1654-1705), a basic limit theorem in modern probability which can
be stated as follows: if a random experiment with only two possible outcomes
(success or failure) is carried out, then, as the number of trials increases the
success ratio tends to a number between 0 and 1 (the success probability);
and the DeMoivre-Laplace theorem (1733, 1785 and 1812), which established
that for large values of n a binomial random variable with parameters n and
p has approximately the same distribution of a normal random variable with
mean np and variance np(l —p). This result was proved by DeMoivre in 1733
for the case p = | and then extended to arbitrary 0 < p < 1 by Laplace in
xix
XX
INTRODUCTION
1812. In spite of the utmost importance of the aforementioned theoretical
results, it is important to mention that by the time they were stated there
was no clarity on the basic concepts. Laplace's famous definition of probability as the quotient between cases in favor and total possible cases (under the
assumption that all results of the underlying experiment were equally probable) was already known back then. But what exactly did it mean "equally
probable"? In 1892 the German mathematician Karl Stumpf interpreted this
expression saying that different events are equally probable when there is no
knowledge whatsoever about the outcome of the particular experiment. In
contrast to this point of view, the German philosopher Johannes von Kries
(1853-1928) postulated that in order to determine equally probable events,
an objective knowledge of the experiment was needed. Thereby, if all the
information we possess is that a bowl contains black and white balls, then,
according to Strumpf, it is equally probable to draw either color on the first
attempt, while von Kries would admit this only when the number of black
and white balls is the same. It is said that Markov himself had trouble regarding this: according to Krengel (2000) in Markov's textbook (1912) the
following example can be found: "suppose that in an urn there are balls of
four different colors 1,2,3 and 4 each with unknown frequencies a, b, c and d,
then the probability of drawing a ball with color 1 equals \ since all colors
are equally probable". This shows the lack of clarity surrounding the mathematical modeling of random experiments at that time, even those with only
a finite number of possible results.
The definition of probability based on the concept of equally probable led
to certain paradoxes which were suggested by the French scientist Joseph
Bertrand (1822-1900) in his book Calcul des probabüites (published in 1889).
One of the paradoxes identified by Bertrand is the so-called paradox of the
three jewelry boxes. In this problem, it is supposed that three jewelry boxes
exist, A, B and C, each having two drawers. The first jewelry box contains
one gold coin in each of the drawers, the second jewelry box contains one silver
coin in each of the drawers and in the third one, one of the drawers contains
a gold coin and the other a silver coin. Assuming Laplace's definition of
probability, the probability of choosing the third jewelry box would be | . Let
us suppose now that a jewelry box is randomly chosen and when one of the
drawers is opened a gold coin is found. Then there are two options: either the
other drawer contains a gold coin (in which case the chosen jewelry box would
be A) or the other drawer contains a silver coin, which means the chosen
jewelry box is C. If the coin originally found is silver, there would be two
options: either the other drawer contains a gold coin, which means the chosen
jewelry box is C, or the other drawer contains a silver coin, which would mean
that the chosen jewelry box is B. Hence the probability of choosing C is \.
Bertrand found it paradoxical that opening a drawer changed the probability
of choosing jewelry box C.
The first mathematician able to solve the paradox of the three jewelry
boxes, formulated by Bertrand, was Poincare, who got the following solution
INTRODUCTION
XXI
as early as 1912. Let us assume that the drawers are labeled (in a place we
are unable to see) as a and ß and that the gold coin of jewelry box C is in
drawer a. Then the following possibilities would arise:
1. Jewelry box A, drawer a: gold coin
2. Jewelry box A, drawer ß: gold coin
3. Jewelry box B, drawer a: silver coin
4. Jewelry box B, drawer ß: silver coin
5. Jewelry box C, drawer a: gold coin
6. Jewelry box C, drawer ß: silver coin
If when opening a drawer a gold coin is found, there would be three possible
cases: 1, 2 and 5. Of those cases the only one that favors is case 5. Hence
P(C) = \.
At the beginning of the twentieth century and despite being the subject
of works by famous mathematicians such as Cardano, Fermat, Bernoulli,
Laplace, Poisson and Gauss, probability theory was not considered in the
academic field as a mathematical discipline and it was questioned whether it
was a rather empirical science. In the famous Second International Congress
of Mathematicians held in Paris in 1900, David Hubert, in his transcendental
conference of August 8, proposed as part of his sixth problem the axiomatization of the calculus of probabilities. In 1901 G. Bohlmann formulated a
first approach to the axiomatization of probability (Krengel, 2000): he defines
the probability of an event E as a nonnegative number p(E) for which the
following hold:
i) If E is the sure event, then p(E) = 1.
ii) If Ei and E2 are two events such that they happen simultaneously
with zero probability, then the probability of either E\ or E2 happening
equals ρ(£Ί) + p(E2).
By 1907 the Italian Ugo Broggi, under Hubert's direction, wrote his doctoral dissertation titled "Die Axiome der Wahrscheinlichkeitsrechnung" (The
Axioms of the Calculus of Probabilities). The definition of event is presented
loosely and it is asserted that additivity and σ-additivity are equivalent (the
proof of this false statement contains so many mistakes that it is to be assumed that Hubert did not read it carefully). However, this work can be
considered as the predecessor of Kolmogorov's.
At the International Congress of Mathematicians in Rome in 1908, Bohlmann
defined the independence of events as it is currently known and showed the
difference between this and 2 x 2 independence. It is worth noting that a
precise definition of event was still missing.
xxii
INTRODUCTION
According to Krengel (2000), in 1901 the Swedish mathematician Anders
Wiman (1865-1959) used the concept of measure in his definition of geometric
probability. In this regard, Borel in 1905 says: "When one uses the convention:
the probability of a set is proportional to its length, area or volume, then one
must be explicit and clarify that this is not a definition of probability but a
mere convention".
Thanks to the works of Frechet and Caratheodory, who "liberated" measure theory from its geometric interpretation, the path to the axiomatization
of probability as it is currently known was opened. In the famed book Grundbegriffe der Wahrscheinlichkeitsrechnung (Foundations of the Theory of Probability), first published in 1933, the Russian mathematician Andrei Nikolaevich
Kolmogorov (1903-1987) axiomatized the theory of probability by making use
of measure theory, achieving rigorous definitions of concepts such as probability space, event, random variable, independence of events, and conditional
probability, among others. While Kolmogorov's work established explicitly
the axioms and definitions of probability calculus, it furthermore laid the
ground for the theory of stochastic processes, in particular, major contributions to the development of Markov and ramification processes were made.
One of the most important results presented by Kolmogorov is the consistency theorem, which is fundamental to guarantee the existence of stochastic
processes as random elements of finite-dimensional spaces.
Probability theory is attractive not only for being a complex mathematical
theory but also for its multiple applications to other fields of scientific interest.
The wide spectrum of applications of probability ranges from physics, chemistry, genetics and ecology to communications, demographics and finance,
among others. It is worth mentioning that Danish mathematician, statistician and engineer Agner Krarup Erlang (1878-1929) for his contribution to
queueing theory.
At the beginning of the twentieth century, one of the most important scientific problems was the understanding of Brownian motion, named so after the
English botanist Robert Brown (1773-1858), who observed that pollen particles suspended in a liquid, move in a constant and irregular fashion. Brown
initially thought that the movement was due to the organic nature of pollen,
but later on he would refute this after verifying with a simple experiment that
the same behavior was observed with inorganic substances.
Since the work done by Brown and up to the end of the nineteenth century
there is no record of other investigations on Brownian motion. In 1905 in his
article "Über die von der molekularkinetischen Theorie der Warme gefordete
Bewegung von in ruhenden Flüssigkeiten suspendierten Teilchen" (On the
movement of small particles suspended in a stationary liquid demanded by
the molecular-kinetic theory of heat; (see Kahane, 1997) German theoretical
physicist Albert Einstein (1879-1955) published the main characteristics of
Brownian motion. He proved that the movement of the particle at instant t
can be modeled by means of a normal distribution and concluded that this
motion is a consequence of continuous collisions between the particle and the
INTRODUCTION
XXÜi
molecules of the liquid in which it is suspended. It is worth pointing out,
however, that Einstein himself said he did not know Brown's works (Nelson,
1967). The first mathematical research regarding Brownian motion was carried out by French mathematician Louis Bachelier (1870-1946), whose 1900
doctoral dissertation "Theorie de la speculation" (Speculation theory) suggested the Brownian motion as a model associated with speculative prices.
One of the imperfections of such a model laid in the fact that it allowed prices
to take negative values and therefore was forgotten for a long time. In 1960
the economist Samuelson (who received the Nobel Prize in Economics in 1970)
suggested the exponential of the Brownian motion to model the behavior of
prices subject to speculation.
The mathematical structure of Brownian motion, as it is known today,
is due to the famed North American mathematician Norbert Wiener (18941964). For this reason Brownian motion is also called the Wiener process.
The first articles about Brownian motion by Wiener are rather hard to follow
and only the French mathematician Paul Levy (1886-1971) was able to recognize its importance. Paul Levy notably contributed to the development of
probability by introducing the concept of the martingale, the Levy processes
among which we find the Brownian motion and the Poisson processes and the
theorem of continuity of characteristic functions. Furthermore, Levy deduced
many of the most important properties of Brownian motion. It is said (see
Gorostiza, 2001) that many times it has happened that major discoveries in
probability theory believed to be new were actually somehow contained in
Levy's works.
During the 1970s, the Black-Scholes and Merton formula, which allows the
pricing of put and call options for the European market, was written. For this
work Scholes and Merton were awarded the 1997 Nobel Prize in Economics
(Black's death in 1995 rendered him ineligible). Nevertheless, the research
carried out by Black-Scholes and Merton would have been impossible without
the previous works done by the Japanese mathematician Kiyoshi Ito (19152008), who in 1940 and 1946 published a series of articles introducing two of
the most essential notions of modern probability theory: stochastic integrals
and stochastic differential equations. These concepts have become an influential tool in many mathematical fields, e.g., the theory of partial differential
equations, as well as in applications that go beyond financial mathematics
and include theoretical physics, biology, and engineering, among others (see
Korn and Korn, 2000).
CHAPTER 1
BASIC CONCEPTS
During the early development of probability theory, the evolution was based
more on intuition rather than mathematical axioms. The axiomatic basis
for probability theory was provided by A. N. Kolmogorov in 1933 and his
approach conserved the theoretical ideas of all other approaches. This chapter
is based on the axiomatic approach and starts with this notion.
1.1
PROBABILITY SPACE
In this section we develop the notion of probability measure and present its
basic properties.
When an ordinary die is rolled once, the outcome cannot be accurately predicted; we know, however, that the set of all possible outcomes is {1,2,3,4,5,6}.
An experiment like this is called a random experiment.
Definition 1.1 (Random Experiment) An experiment is said to be random if its result cannot be determined beforehand.
It is assumed that the set of possible results of a random experiment is
known. This set is called a sample space.
Introduction to Probability and Stochastic Processes with Applications, First Edition.
1
By Liliana Blanco Castaneda, Viswanathan Arunachalam and Selvamuthu Dharmaraja
Copyright © 2012 John Wiley &; Sons, Inc.