Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture 4: Algorithmic Methods for G/M/1 and M/G/1 type models Dr. Ahmad Al Hanbali Department of Industrial Engineering University of Twente [email protected] 1 Lecture 4 This Lecture deals with continuous time Markov chains with infinite state space as opposed to finite space with skip-free in one direction as opposed to QBDs Lecture 3 Objective: To find equilibrium distribution of the Markov chain Lecture 4: G/M/1 and M/G/1 type models 2 Background (1): G/M/1 queue Interarrival time of jobs is arbitrary distribution, 𝐹𝐴 (𝑡) with mean 1/𝜆, and inter-arrival times are iid Jobs service times are iid exponential rvs with rate 𝜇. Interarrivals and service times are independent Service discipline can be Fisrt-In-First-Out (FIFO) Under above assumptions, the nbr of jobs in G/M/1 queue at arbitrary time 𝑡 DOES NOT form a Markov chain Let 𝑁(𝑡𝑖 ) denote the number of jobs in the queue just before the 𝑖-th arrival. The process {𝑁(𝑡𝑖 ), 𝑖 = 0,1, … } is a discretetime Markov chain Let 𝑎𝑛 denote the probability that exactly 𝑛 jobs are served during an inter-arrival time given there are at least 𝑛 jobs present at the start of inter-arrival. Then 𝑎𝑛 reads 𝑎𝑛 = ∞ 𝜇𝑡 𝑛 −𝜇𝑡 𝑒 𝑑𝐹𝐴 (𝑡) , 𝑛 0 𝑛! Lecture 4: G/M/1 and M/G/1 type models = 0,1, …, and 𝑏𝑖 = 𝑘>𝑖 𝑎𝑘 3 Background (2): G/M/1 queue 𝑎0 𝑎1 0 i-1 1 𝑏𝑖 𝑎𝑖 i i+1 𝑎2 The transition probability matrix is given 𝑏0 𝑎0 0 0 0 … 𝑏1 𝑎1 𝑎0 0 0 … 𝑏2 𝑎2 𝑎1 𝑎0 0 … 𝑃= . … 𝑎 𝑏3 3 𝑎2 𝑎1 𝑎0 𝑏4 𝑎4 𝑎3 𝑎2 𝑎1 ⋱ ⋮ ⋮ ⋮ ⋮ ⋮⋱ For stable case with 𝜆 < 𝜇 we find 𝑝𝑖 = 1 − 𝜎 𝜎 𝑖 , (Geometric distribution) where 𝜎 is the unique root in (0,1) of 𝜎 = 𝐸 𝑒 −𝜇 1−𝜎 Lecture 4: G/M/1 and M/G/1 type models 𝐴 . 4 Definition G/M/1-type processes: skip-free process to the right A 2-dimensional irreducible continuous time Markov process with states (𝑖, 𝑗), where 𝑖 = 0, … , ∞ and 𝑗 = 0, … , 𝑚 − 1 Subset of state space with common 𝑖 entry is called level 𝑖 (𝑖 > 0) and denoted 𝑙(𝑖) = {(𝑖, 0), (𝑖, 1), … , (𝑖, 𝑚 − 1)}. 𝑙(0) = {(0,0), (0,1), … , (𝑖, 𝑚0 − 1)}. This means state space is ∪𝑖≥0 𝑙(𝑖) Transition rate from (𝑖, 𝑗) to (𝑖′, 𝑗′) is equal to zero for 𝑖′ − 𝑖 ≥ 2 For 𝑖 > 0, transition rate between states in 𝑙(𝑖) and from 𝑙(𝑖) to 𝑙 𝑖 + 1 , 𝑙 𝑖 − 1 , … , 𝑙(0) are independent of 𝑖 Lecture 4: G/M/1 and M/G/1 type models 5 Skip-free to the right process Order the states lexicographically, i.e., 0,0 , … , 0, 𝑚0 , 1,0 , … , 1, 𝑚 , 2,0 , … , 2, 𝑚 , …, the generator of the skip-free process has the following form: 𝐵00 𝐵01 0 0 0 … 𝐵10 𝐵11 𝐴0 0 0 ⋱ 𝑄 = 𝐵20 𝐴2 𝐴1 𝐴0 0 ⋱ 𝐵30 𝐴3 𝐴2 𝐴1 𝐴0 ⋱ ⋮ ⋱ ⋱ ⋱ ⋱⋱ where 𝐴0 and 𝐴2 are nonnegative 𝑚-by-𝑚 matrices; 𝐴𝑖 is square matrices of size 𝑚; 𝐵00 is square matrix of size 𝑚0 ; 𝐵𝑖0 𝑚0 -by-𝑚 and 𝐵01 𝑚-by-𝑚0 nonnegative matrices. Note, (𝐵00 + 𝐵01 )𝑒 = 0, (𝐵10 + 𝐵11 + 𝐴0 )𝑒 = 0, and (𝐵0𝑖 + 𝑖 𝑙=0 𝐴𝑙 )𝑒 = 0 Lecture 4: G/M/1 and M/G/1 type models 6 Stability of G/M/1-type process Let 𝐴 = ∞ 𝑖=0 𝐴𝑖 . 𝐴 is the generator describing transitions of the M/G/1-type process between level states (i.e., in the vertical direction) Theorem: Assume the Markov chain with generator 𝐴 is irreducible with equilibrium distribution, 𝜋𝐴 = 0, 𝜋𝑒 = 1. The G/M/1-type process is ergodic if and only if ∞ 𝑖 − 1 𝐴𝑖 𝑒 (mean drift condition) 𝜋𝐴0𝑒 < 𝜋 𝑖=1 Interpretation: 𝜋𝐴0𝑒 is mean drift from level 𝑖 to 𝑖 + 1. 𝜋 ∞ 𝑖=1 𝑖 − 1 𝐴𝑖 𝑒 is the mean drift from level 𝑖 to levels smaller than 𝑖 for large 𝑖 Lecture 4: G/M/1 and M/G/1 type models 7 Equilibrium distribution of G/M/1type processes Let 𝑝𝑛 = (𝑝(𝑛, 0), . . , 𝑝(𝑛, 𝑚 − 1)) and 𝑝 = (𝑝0, 𝑝1, … ) then equilibrium equation 𝑝𝑄 = 0 reads ∞ ∞ 𝑝 = 0, 𝑝 + 𝑝 + 0𝐵01 1𝐵11 𝑖=0 𝑖 𝐵𝑖0 𝑖=2 𝑝𝑖 𝐴𝑖 = 0, ∞ 𝑖=0 𝑝𝑛+𝑖 𝐴𝑖 = 0, 𝑛 ≥ 1 Theorem: if the G/M/1-type process is ergodic the equilibrium probability distribution then reads 𝑝𝑛 = 𝑝1𝑅𝑛−1 , 𝑛 ≥ 1, where 𝑅 is the minimal nonnegative solution of the matrix equation ∞ 𝑖 𝑅 𝐴𝑖 = 0 𝑖=0 Interpretation: 𝑅 = 𝐴0𝑁 same as in QBD process Lecture 4: G/M/1 and M/G/1 type models 8 Equilibrium distribution of G/M/1type processes(cnt'd) A direct result of the previous theorem is that spectral radius of 𝑅 is < 1 and (𝐼 − 𝑅) is nonsingular Lemma: The stationary probability vectors 𝑝0 and 𝑝1 is the normalized unique solution of 𝑥0 , 𝑥1 𝐵 𝑅 = 0,0 , where 𝐵 𝑅 is the generator given by 𝐵00 𝐵01 𝐵𝑅 = . ∞ ∞ 𝑖−1 𝑖−1 𝐵𝑖0 𝐵11 + 𝑖=1 𝑅 𝐴𝑖 𝑖=1 𝑅 Normalization is done by letting 𝑥0 𝑒𝑚0 + 𝑥1 𝐼 − 𝑅 −1 𝑒𝑚 = 1. Note 𝑒𝑚0 and 𝑒𝑚 are column vectors of ones with size 𝑚0 and 𝑚, respectively Proof: follows by inserting 𝑝𝑛 = 𝑝1𝑅 𝑛−1 in the balance equations Lecture 4: G/M/1 and M/G/1 type models 9 Finding R Rearrange the equation of the rate matrix: ∞ 𝑅𝑖 𝐴𝑖 𝐴1−1 𝑅 = − 𝐴0 + 𝑖=2 Fixed point equation solved by successive substitution 𝑖 𝐴 )𝐴−1 , with 𝑅 = 0 𝑅𝑘+1 = −(𝐴0 + ∞ 𝑅 𝑖 1 0 𝑖=2 𝑘 It can be shown that 𝑅𝑘 → 𝑅 for 𝑘 → ∞ In many queueing systems for large 𝐾, 𝐴𝐾 ≈ 0. 𝑖𝐴 ≈ 𝐾 𝑖𝐴 Truncating ∞ 𝑅 𝑅 𝑖 𝑖 𝑖=2 𝑘 𝑖=2 𝑘 Lecture 4: G/M/1 and M/G/1 type models 10 Special case: GI/PH/1 queue Consider the case 𝐵01 = 𝐴0 and 𝐵11 = 𝐴1 The matrix 𝐵𝑖0 is a rank one matrix satisfying 𝐵𝑖0 = − 𝑖 𝑗=0 𝐴𝑗 𝑒𝛽. where 𝛽 is the initial state probability vector of the service time phase-type distribution Lemma: The stationary probability vector of GI/PH/1 embedded at the moment of arrivals 𝑝𝑖 = 𝑝0 𝑅𝑘 , where 𝑝0 = 𝛽 𝐼 − 𝑅 −1 𝑒 −1 𝛽. Proof: in this case 𝐵 𝑅 will be of rank one Lecture 4: G/M/1 and M/G/1 type models 11 Definition of M/G/1-type processes: skip-free process to the left A 2-dimensional irreducible continuous time Markov process with states (𝑖, 𝑗), where 𝑖 = 0, … , ∞ and 𝑗 = 0, … , 𝑚 − 1 Subset of state space with common 𝑖 entry is called level 𝑖 (𝑖 > 0) and denoted 𝑙(𝑖) = {(𝑖, 0), (𝑖, 1), … , (𝑖, 𝑚 − 1)}. 𝑙(0) = {(0,0), (0,1), … , (𝑖, 𝑚0 − 1)}. This means state space is ∪𝑖≥0 𝑙(𝑖) Transition rate from (𝑖, 𝑗) to (𝑖′, 𝑗′) is equal to zero for 𝑖 ′ − 𝑖 ≤ − 2. For 𝑖 > 0, transition rate between states in 𝑙(𝑖) and from 𝑙(𝑖) to 𝑙 𝑖 − 1 , 𝑙 𝑖 + 1 , …, are independent of 𝑖 Lecture 4: G/M/1 and M/G/1 type models 12 Skip-free to the left process Order the states lexicographically, i.e., 0,0 , … , (0, 𝑚0 − Lecture 4: G/M/1 and M/G/1 type models 13 Stability of M/G/1-type processes Let 𝐴 = ∞ 𝑖=0 𝐴𝑖 . 𝐴 is the generator describing transitions of the M/G/1-type process in the vertical direction. Theorem: Assume the Markov chain with generator 𝐴 is irreducible with equilibrium distribution, 𝑖. 𝑒. 𝜋𝐴 = 0, 𝜋𝑒 = 1, and with ∞ 𝑖=1 𝑖𝐵0𝑖 𝑒 finite. The M/G/1-type process is ergodic if and only if 𝜋𝐴0𝑒 > 𝜋 ∞ 𝑖=2 𝑖 − 1 𝐴𝑖 𝑒 (mean drift condition) Lecture 4: G/M/1 and M/G/1 type models 14 Equilibrium distribution of skip-free to the left processes Assume M/G/1-type process is ergodic. The minimal nonnegative solution 𝐺 of ∞ 𝑖 𝐴 𝐺 𝑖=0 𝑖 = 0, is then stochastic, i.e., 𝐺𝑒 = 𝑒, Interpretation: 𝑘, 𝑙 -element of 𝐺 represents probability to jump for the first time to level 𝑖 − 1 by entering state 𝑙, 𝑖 ≥ 1, given process starts in (𝑖, 𝑘) at time 0 Theorem (Matrix analytic): Assume M/G/1-type process is ergodic then equilibrium probability of 𝑙 𝑖 gives 𝑝𝑖 = − 𝑝0 𝐵𝑖 + where, 𝐵𝑖 = 𝑖−1 𝑘=1 𝑝𝑘 ∞ 𝑘, 𝐴 𝐵 𝐺 𝑖 𝑘=0 0𝑖+𝑘 Lecture 4: G/M/1 and M/G/1 type models 𝐴𝑖+1−𝑘 = 𝐴1 −1 , 𝑖 ∞ 𝑘, 𝑛 𝐴 𝐺 𝑘=0 𝑖+𝑘 = 2,3, …, = 1,2, …. 15 Finding 𝐺 In many queueing systems for large 𝐾, 𝐴𝐾 ≈ 0. Using this, 𝐴𝐾 and 𝐵𝐾 → 0, then the following recursion (backward) can used 𝐵𝑖 = 𝐵0𝑖 + 𝐵𝑖+1 𝐺 𝐴𝑖 = 𝐴𝑖 + 𝐴𝑖+1 𝐺 In this case, it is reasonable to truncate the infinite sum of 𝐺𝑙+1 at K The matrix 𝐺 can be found recursively as follows 𝑖 𝐺𝑙+1 = −𝐴1−1 𝐴0 + 𝐾 𝐴 𝐺 , 𝑙 = 1,2, … , with 𝐺0 = 0. 𝑖=2 𝑖 𝑙 It can be shown that 𝐺𝑘 → 𝐺 for 𝑘 → ∞ Lecture 4: G/M/1 and M/G/1 type models 16 Probabilities 𝑝0 and 𝑝1 Theorem: The stationary probability vectors 𝑝0 and 𝑝1 is the normalized unique solution of 𝑥0 , 𝑥1 𝐵 𝐺 = 0,0 , where 𝐵 𝐺 is the generator of an irreducible chain given by 𝐵00 𝐵𝐺 = 𝐵10 ∞ 𝑘 𝐵 𝐺 𝑘=0 01+𝑘 ∞ 𝑘 𝐴 𝐺 𝑘=0 1+𝑘 done such ∞ 𝑖=0 𝑝𝑖 𝑒 . Normalization should be =1 𝐵 𝐺 is M/G/1-type process restricted to 𝑙 0 & 𝑙(1), 𝐵 𝐺 e=0 Special case: 𝐵10 = 𝐴0 Recursion in the theorem in slide 15 will also hold from for 𝑖 = 1. 𝑘 . Moreover, 𝑝 is the normalized unique 𝐵𝐺= ∞ 𝐵 𝐺 0𝑘 𝑘=0 0 solution of 𝑥0 𝐵 𝐺 = 0 Lecture 4: G/M/1 and M/G/1 type models 17 Finding 𝑝0 : special case 𝐵10 = 𝐴0 ∞ ∞ Let 𝐵 = ∞ 𝐵 , 𝑏 = 𝑖𝐵 𝑒, 𝑎 = 0𝑖 𝑖=0 0𝑖 𝑖=1 𝑖=1 𝑖𝐴𝑖 𝑒 Assume 𝐴 = ∞ 𝑖=0 𝐴𝑖 is irreducible and let 𝜋𝐴 = 0, 𝜋𝑒 = 1 𝜇 = 𝜋𝑎 (load measure) Theorem: Assume the generator of skip-free to the left process and 𝐴 are irreducible. If the skip-free to the left process is ergodic, then 𝜇 = 𝑝0 𝜇𝑒 − 𝑏 + 𝐵 𝐴 + 𝑒𝜋 −1 𝑎 Proof: Similar Theorem 4.8 in Bini et al. (2005) Using 𝑝0 𝐵 𝐺 = 0 and additional equation in the previous Theorem, then 𝑝0 can be uniquely determined It is also possible to find 𝐸𝐿 = ∞ 𝑖=1 𝑖𝑝𝑖 𝑒 in closed form as function of 𝑝0 , 𝜋, 𝑎, 𝑏, … 18 Lecture 4: G/M/1 and M/G/1 type models 𝑥 Example: Geo /D/1 with slot reservations in care pathway patients Priority patient reserves time slot between L and H slots in future: 1. If time slot L is occupied by a regular patient, the priority patient takes the position of the regular patient, and all regular patients from position L onward are shifted to the next time slot 2. If slot L is occupied by priority patient, then new priority patient moves to the first available slot in the interval L + 1, . . . , H that is not occupied by a priority patient, and takes that position. 3. If all slots in the interval L, . . . , H are occupied by priority patients, the new priority patient is blocked 19 Lecture 4: G/M/1 and M/G/1 type models 𝑥 Example: Geo /D/1 with slot reservations in care pathway patients 𝑁2 (𝑖 ), 𝑖 ≥ 1, total number of regular patients waiting in the queue at the end of each time slot (level state) 𝑉(𝑖 ), 𝑖 ≥ 1, a state vector of length H with entries denoting which slot in the set {1, . . . , H} contains a priority patient (phase state) Two-dimensional process (𝑁2 (𝑖 ), 𝑉(𝑖 )) is discrete time Markov chain with transition probability matrix M/G/1type form Lecture 4: G/M/1 and M/G/1 type models 20 𝑥 Example: Geo /D/1 with slot reservations in care pathway patients One-step transition of the priority patients state 𝑉 𝑖 are independent of the number of regular patients present 𝑗 𝑞2 𝑞2 Let 𝑎𝑗 = 1 − and 𝐷 the one-step transition probability matrix of priority patients 𝑉 𝑖 → 𝑉 𝑖 + 1 𝐴𝑗 can be rewritten as 𝑎𝑗 𝐴𝐷, where 𝐴 is the diagonal matrix, 𝐵0 can be written as 𝑎0 𝐵𝐷, where 𝐵 is the diagonal matrix, and 𝐶𝑗 can be written as 𝑎𝑗 𝐷 We find that the equilibrium probability vector of is matrix geometric 𝜋𝑙 = −𝜋0 𝑆 𝑞2 𝐼 − 𝐴𝑆 𝑙−1 , 𝑙 = 1,2, …, where 𝑆 = 𝐶1 𝐴0 − 𝐼 + 𝑞2 𝑋 −1 and 𝑋 is minimal nonnegative solution of quadratic matrix equation Lecture 4: G/M/1 and M/G/1 type models 21 References Bini, Latouche, Meini, “Numerical Methods for Structured Markov Chains” Oxford University Press, 2005. G. Latouche and V. Ramaswami (1999), Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM. M.F. Neuts (1989), Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Dekker, INC. M.F. Neuts (1981), Matrix-geometric solutions in stochastic models. The John Hopkins University Press, Baltimore. M.E. Zonderland, R.J. Boucherie, A. Al Hanbali. Appointments in care pathways: the Geo^x/D/1 queue with slot reservations. Queueing Systems, vol. 79, issue 1 (2015). Lecture 4: G/M/1 and M/G/1 type models 22