Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
28/04/2008 Scuola di Dottorato in Scienza ed Alta Tecnologia – Dottorato in Informatica Universita’ di Torino Outline Introduction to Stochastic Processes Markov Chains Stochastic Models: Markov Chains and their Generalizations Renewal Theory Markov Renewal Processes References E. Cinlar, Introduction to Stochastic Processes, Prentice Hall, 1975; V.G. Kulkarni, Modeling and Analysis of Stochastic Systems, Chapman & Hall, 1995 Gianfranco Balbo e Andras Horvath Aprile – Maggio, 2008 Aprile - Maggio, 2008 Continuous Time Stichastic Processes Exponential Distribution (1) Exponential Distribution Poisson Process Continuous Time Markov Chains Aprile - Maggio, 2008 3 A non-negative random variable X is said to have a negative exponential distribution with parameter λ if The probability density function (pdf) of a negative-exponential random variable is The Laplace transform of the exponential random variable is Aprile - Maggio, 2008 4 Exponential Distribution (3) Exponential Distribution (2) 2 The most important property of the negative exponential distribution is the memoryless property Minimum of exponentials Given two random variables X and Y with negative exponential pdf X the new random variable Z = min(X,Y) has also an exp. pdf e- x since 0 a x Aprile - Maggio, 2008 Documento interno all'Università degli Studi di Torino 5 Aprile - Maggio, 2008 6 1 28/04/2008 Exponential Distribution (4) Exponential Distribution (5) Strong Memoryless Property Probability of First Failure Given two random variables X and Y with negative exponential pdf, it is possible to show Aprile - Maggio, 2008 7 Given two random variables X and Y with negative exponential pdf, it is possible to show Aprile - Maggio, 2008 8 Exponential Distribution (7) Exponential Distribution (6) Sum of n independent identically distributed (i.i.d.) negative exponentials Random Sum of n i.i.d. negative exponentials Given a sequence of independent random variables {Xn, n¸ 1 } all characterized by a common negative exponential pdfs with parameter λ, let N be a geometric random variable with parameter p: Given n random variables X1, X2, …, Xn with negative exponential pdfs with parameter λ, let By using the properties of Laplace transform, it is possible to show Assume that N is independent of {Xn, n¸1 } and define the following random sum of random variables From which follows By using the properties of Laplace transforms and exploiting the independence of before, it is possible to show and from which we can conclude that Z has a negative-exponential distribution with parameter λp Aprile - Maggio, 2008 9 Poisson Process (1) Aprile - Maggio, 2008 10 Poisson Process (2) A Poisson process is often used to count events occurring one at the time during a certain interval. Events of this type are assumed to be generated by an infinite set of sources acting very slowly, independently of each other and with a “complete” random behaviour. Here we define the Poisson process in terms of the probability distribution of the intervals among successive events. Recalling the distribution of a sum of k exponential i.i.d. random variables derived before, it is easy to show that When t is a nonnegative value, then In fact, Let {Xn, n¸ 1 } be a sequence of random variables representing the inter-event times Hence Sn is the time of the occurrence of the n-th event. Define From which N(t) is the number of events that have taken place during (0,t] and {N(t), t¸ 0 } is called a counting process When {Xn, n¸ 1 } is a sequence of i.i.d exponential variables with parameter λ, the counting process {N(t), t¸0 } is called a Poisson Process with parameter λ Aprile - Maggio, 2008 Documento interno all'Università degli Studi di Torino 11 Aprile - Maggio, 2008 12 2 28/04/2008 Properties of Poisson Processes Superposition and Splitting of Poisson Processes For a fixed s ¸ 0 Merging two (or more) Poisson Processes to generate a new process is called “superposition” The operation of generating two (or more) counting processes out of a single Poisson Process is called “splitting” is a Poisson Process with parameter λ, independent of {N(u), 0 · u · s }. A Poisson Process {N(t), t¸ 0 } has stationary and independent increments, i.e. Superposition Let {Ni(t), t ¸ 0 } be independent Poisson Processes of parameters {λi, 1 · i · r }. Define A stochastic process {N(t), t ¸ 0 } is a Poisson Process with parameter λ iff The process {N(t), t¸ 0 } is called the superposition of {Ni(t), t ¸ 0 } ( 1 · i · r ) {N(t), t ¸ 0 } is a Poisson Process with parameter λ = λ1 + λ2 + Λ + λr. A counting process {N(t), t ¸ 0 } is a Poisson Process with parameter λ iff N1(t) N2(t) N(t) Aprile - Maggio, 2008 13 Superposition and Splitting of Poisson Processes Aprile - Maggio, 2008 14 Other types of Poisson Processes Splitting Non-Homogeneous Poisson Processes Consider a Poisson Process {N(t), t ¸ 0 } that counts certain events that can be classified into r categories. It is possible to constructs r counting processes {Ni(t), t ¸ 0 }, i=1,2,…, r where Ni(t) is the number of events of type i that occurred up to time t. Again, we can recognize that A non-homogeneous Poisson process can be thought of as a process that counts events that occur in a non-uniform fashion so that the probability that an event takes place in an interval (t, t+h ] is given by λ(t) h + o(h) Let {N(t), t¸ 0 }be a Poisson Process of parameter λ and let {Ni(t), t¸ 0 } (i=1,2,…,r ) be generated by a Bernoulli Splitting Merchanism with parameters p1, p2, …, pr such that Compound Poisson Process Then each {Ni(t), ¸ 0} is a Poisson Process of parameter λ pi A compound Poisson Process can be constructed out of a “standard” Poisson Process N(t) o N1(t) N2(t) * o o o o * o * * o * * * o o o o * o * A compound Poisson process can be thought of as a process that counts events that are allowed to occur in “batches” of arbitrary sizes (even negative!} {N(t), t¸ 0 } of parameter λ, combined with a sequence of i.i.d random variables (independent of N(t) ) {Zn, n¸1 } such that * Aprile - Maggio, 2008 Documento interno all'Università degli Studi di Torino 15 Aprile - Maggio, 2008 16 3