Download Models

Output Analysis and
Experimentation for
Systems Simulation
Performance Measure
• Simulation output analysis concerns with using
simulation to estimate the quantities of interest in a
simulation model
• Quantities are referred as Performance Measure
• Examples of performance measures include
– The average system time of the first n
customers
– The long-run average system time per customer
– The first time the system breaks down
– The long-run average fraction of time the
system is down.
Types of simulation
• There are two types of simulation:
1) Transient Simulation (Short-term simulation)
2) Steady-State Simulation (Long-term
simulation)
Transient simulation is easier to study than the
steady-state simulation, because we cannot
simulate the system until infinity.
Transient Simulation
• We are interested in studying the
performance measure of the system for a
finite time horizon.
• This time could be deterministic such as:
– The performance of a system for one day,
one month etc.
• Or Stochastic (the time a certain event
occurs) such as:
– The performance of the system until the first
time it breaks down.
Transient Simulation
• Suppose we want to estimate
m= E{f(X(t)): 0 < t < T} Where
– f is a deterministic function,
– {X(t)} is a stochastic process (a sequence of
random variables),
– T is a stopping time either deterministic or
stochastic.
• Example. X(t) =N(t): the number of
customers in an M/M/1 queuing system at
time t.
Estimating Performance Measure in
the Transient Simulation
Suppose we want to estimate the average number of
customers in system; N, before it breaks down
• We simulate the system n times until it breaks
down.
• Each run, we get an estimate of the number of
customers as Ni (Note that Ni is a random
variable).
• We estimate the average number of customers in
n
1
the system by the estimator Z 
Ni

n
n i 1
Properties of estimators
1) Unbiased: E{Zn} = m
lim Z n  μ
2) Strongly consistent: n 
almost surely (i.e. with probability = 1)
3) The sequence {Zn} is asymptotically
normal:
 Zn  m 
n
  N (0,1)
   n 
The convergence here in distribution.
Confidence Interval for the mean
• The 100 (1-a) % confidence interval Ia is
an interval that satisfies: P{m  Ia} = 1 – a
• Use the asymptotic normal property to
build the confidence interval as
a
In
Z a / 2
Z a / 2 

 Z n 
, Zn 

n
n 

• Where Za/2 can be found in the standard
normal distribution table that represent
the z value where the area under the
normal distribution is 1- a/2 .
  is the standard deviation of the mean. It
can be estimated by s where:
n
1
2
s2 
(N

Z
)

i
n
n  1 i 1
• Some values for Za/2: :are as follows
• For 90% C.I, use Z.05 =1.645
• For 95% C.I, use Z.025 =1.96
Steady-State Simulation
• We are interested in studying the
performance measure of the system for a
long run period of time.
• We assume that the system will eventually
settle down (becomes stable).
Steady-State Simulation
• Two methods:
– Multiple Replications with Initial Deletion.
– Batch Means Method.
Multiple Replications with Initial Deletion
• Run the simulation for a long simulation run
• Delete initial observations (Initial Bias
Contamination)
• Evaluate the average of the remaining data as a
one sample of the mean.
• Repeat for several times to get estimate for the
mean an confidence interval.
Disadvantage of Multiple Replications
Each replication we delete some observations, so we
do loose some expenses of simulation
Batch Means Method
• Do only one very long simulation run, say on the
interval [0,T].
• Delete few contaminated observations at the
beginning [0,T0]
• Divide the remaining [T0, T] into k subintervals
[T0, T1), [T0, T2), …, [Tk-1, T], and
• Find a sample of the performance measure Zj in
each subinterval j
• Construct the 100(1-a) % confidence interval for
the data Z1 , Z2 , …, Zk