Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Stochastic processes Definition A time-oriented, physical process that is controlled by a random mechanism A sequence of random variables [Xt], where t ∈ T is a time or sequence index State: mutually exclusive and collectively exhaustive description of attributes of the system Topics to be covered tonight … Markov process Chapman-Kolmogrov equations Markov chains Birth-death equations Queuing 1 ETM 620 - 09U Markov process Markovian property P { X j | X i } P { X j | X i , X i , X i , .. X i } t 1 t t 1 t t 1 1 t 2 2 0 t Interpretation … Examples: Arrival of customers at the bank Time required to inspect a passenger’s carry-on at the airport Etc. 2 ETM 620 - 09U Three classic examples Marketing Brand A Other Maintenance Operational Org. Maint. Depot Hospital (staffing) 3 ETM 620 - 09U Probabilities in Markov processes One-step transition probability P{Xt+1= j|Xt = i} = pij This is called stationary if P{Xt+1= j|Xt = i} = P{X1 = j|X0 = i} for t = 0, 1, 2, … Our example: Operational 4 Org. Maint. Depot ETM 620 - 09U The matrix One-step transition matrix Properties: 0 < pij < 1 From ∑Mpij = 1 0.8 0.6 0.3 To 0.2 0.3 0 0 0.1 0.7 N-step transition matrix pij(n) = P{Xt+n = j|Xt = i} = P{Xn = j|X0 = i} 5 ETM 620 - 09U Finite-state Markov chain Definition Stochastic process Finite number of states Markovian Stationary transition probabilities Initial set of probabilities Chapman-Kolmogrov equations pij(n) = l=0∑Mpil(v)plj(n-v) 6 i = 0, 1, 2, …, M j = 0, 1, 2, …, M 0<v<M ETM 620 - 09U Building the n-step transition matrix Start with 1-step probabilities Initial probability matrix 0.8 0.2 0 P 0.6 0.3 0.1 0.3 0 0.7 The 2-step transition matrix is now 7 0 . 8 0 . 20 0 . 8 0 . 20 2 P P P 0 . 6 0 . 3 0 . 1 0 . 6 0 . 3 0 . 1 0 . 300 . 7 0 . 300 . 7 ETM 620 - 09U Building the matrices in Excel Create the matrix P Copy this one or two rows below, call the copy Q Multiply P×Q using =mmult Steps: Similarly, P3=P×P×P Results from above multiplied by P And P4=P×P×P×P Etc. 8 ETM 620 - 09U Classification of states and chains First passage time Length of time (number of steps) for the process to go from state i to state j. E.g., go from Operational to Depot If fij(n) is the probability that the first passage time from i to j is n, then fij(n)=pij(n) - fij(1)*pjj(n-1 )- fij(2)*pjj(n-2) - … - fij(n-1)*pjj First return time (recurrence time) Length of time (number of steps) to return to i, i.e., fii(n) ( n ) f 1 If, n 1ii then state i is a recurrent state 9 ETM 620 - 09U Classification of states and chains (cont.) Absorbing state pii = 1 Once entered, never leave this state. Transient state ( n ) f 1 n 1ii There is some probability that the process will never return to this state. 10 ETM 620 - 09U Expected first passage time 1 p ij l j i l lj To find all expected first passage times, solve as a series of simultaneous linear equations, e.g. (for a 3×3) 11 ETM 620 - 09U States and chains Periodic state only returns to the state in τ, 2τ, 3τ, … steps, where τ>1 Accessible j is accessible from i if pij(n) > 0 for some n i and j COMMUNICATE if j is accessible from i and i is accessible from j Irreducible Only 1 class (partitioned set of the state space) exists in which all states communicate 12 ETM 620 - 09U States and chains Ergodic – a state i in a class that is Not periodic Positive recurrent (i.e., μii<∞) Steady state probabilities Irreducible, ergodic Markov chains will reach a “steady state” probability, A(n+1) = An*P Mean recurrence time μjj = 1/pj 13 ETM 620 - 09U Continuous-time Markov chains ( t ) P [ X ( t s ) j | X ( s ) i ] p ij Time is a continuous parameter State space (range of values for t) is discrete Chapman-Kolmogrov equations become pij(t)= l=0∑mpil(v) * plj(t-v) Intensity* of transition, given the state j 1 p ( t ) d jj u lim p ( t ) j jj t 0 t 0 t dt Intensity* of passage from state i to state j uij lim pij (t) d pij (t) t 0 t 0 t dt 14 ETM 620 - 09U Example: Maintenance & repair Two identical, redundant modules in a control mechanism. Failure rate: fT(t)= λe-λt t ≥0 Repair time: rT(t)= μe-μt t ≥0 The transition intensities are: u0 2 u1 u 01 2 u 12 u 02 0 u 20 0 u 10 u 21 2 u 22 2 15 ETM 620 - 09U Example (cont.) From eq. 18-17 0: __________________ 1: __________________ 2: ___________________ since p0 = p1 = p2 = 1 p0 = ________________ p1 = ________________ p2 = ________________ 16 ETM 620 - 09U Example (cont.) Let’s assume MTBF = 1/5 day λ = _________ MTTR = 1/6 day µ = _________ we can now compute the probabilities … p0 = ________________ p1 = ________________ p2 = ________________ 17 ETM 620 - 09U Birth-death processes Used in queueing theory birth = arrival to the system interarrival times are commonly assumed to be exponentially distributed λn=arrival rate given that there are n customers in the system death = departure from the system queueing system – the queue and the service facility µn= service rate given that there are n customers in the system Transition diagrams similar to what we’ve shown in our example, but with transitions 18 Transition matrix ETM 620 - 09U Transitions … Transition matrix 1 t t 0 0 ... 0 0 0 t 1 ( ) t t 0 ... 0 1 1 1 1 t 1 ( ) t t ... 0 2 2 2 2 P 0 t 1 ( ) t ... 0 2 2 2 ... 0 t ... ... 3 0 ... ... ... ... Time dependent behavior (eq. 18-23, 18-24) p ( ) p ( t ) p ( t ) p ( t ) ' p p ( t ) p ( t ) 0 0 0 1 1 ' j 19 j j j j 1 j 1 j 1 j 1 ETM 620 - 09U Steady state equations Steady-state equations (18-25) 1 p1 0 p0 0 p0 2 p2 (1 1) p1 1 p1 3 p3 (2 2) p2 j2 pj2 j pj (j1 j1) pj1 j1 pj1 j1 pj1 (j j ) pj Solving 0 p0 1 p2 1 p1 1 0 p0 2 21 p1 20 j jj1 0 pj1 p p j1 j j1j 1 0 j1j2 0 let C j jj1 1 1 and solving ,we getp 0 1 C j j 1 ETM 620 - 09U Considerations in queuing models If service and arrival rates are constant and L = expected number of customers in the queueing system Lq = expected queue length W = expected time in the system (including service time) Wq = expected waiting time in the queue (including service time) if λ is constant, then L = λW Lq = λWq System utilization coefficient (fraction of time servers are busy) ρ = λ/sµ 21 ETM 620 - 09U Basic single-server model (constant rates) Assume: s=1 unlimited potential queue length exponential interarrivals with constant λ independent exponential service times with constant µ Length of queue L 1 L 1 ( ) 2 2 q Time in system and time in queue L 1 W W ( 1 ) ( ) q q 22 ETM 620 - 09U Recall our maintenance example … MTBF = 1/5 day λ = _________ MTTR = 1/6 day µ = _________ ρ = ___________ 23 L = ___________ W = ______________ Lq = ___________ Wq = ______________ ETM 620 - 09U Other models Limited queue length Multiple servers with unlimited queue Others … 24 ETM 620 - 09U