Download DRAFT Dynamic Transmission Modeling

DRAFT - Dynamic Transmission Modelling: A Report of the ISPOR-SMDM Modelling Good Research Practices Task
Force Working Group - Part 7
1
2
Introduction
The transmissible nature of communicable diseases is the critical characteristic that sets them apart from
3
those diseases more commonly modelled by health economists, such as heart disease, stroke and cancer (1, 2). If an
4
intervention reduces the number of cases in the community, then the risk to others goes down. Reduce the cases
5
enough, and the infection will be eliminated and will not return unless it is re-introduced. Even then, it will not be able
6
to spread unless there is a sufficient number of susceptible individuals, so maintaining vaccination – which reduces
7
susceptibility – at a high enough coverage (though crucially not at 100%) can permanently prevent the infection from
8
spreading (1). That is to say, there are population-level effects in addition to those that accrue to the individuals
9
reached by the programme (the sum of the parts is greater than the whole). This is not the case for non-communicable
10
diseases. If we reduce the prevalence of heart disease it makes no difference at all to the risk of heart disease in others
11
in the community. If we treated every case, we would still get new cases arising. That is, apart from the impact on the
12
quality of life of carers, there are no knock-on benefits. The overall health benefits can be simply estimated by summing
13
the individual benefits.
14
This difference is fundamental and yet often overlooked by analysts. In a recent review of cost-effectiveness
15
studies of vaccination programmes Kim and Goldie reported that only 11% of 208 studies used an approach that could
16
incorporate these indirect (as well as direct) effects (3). Others have reported similar findings for other interventions
17
against communicable diseases including mass screening and treatment programmes for Chlamydia (4). Most analysts
18
have simply adapted the same class of model used for non-communicable diseases. In doing so they are ignoring a
19
fundamental property of communicable disease control programmes – that the overall benefits are not equal to the
20
sum of the individual effects. Hence comparison across economic analyses is more difficult as results may be very
21
sensitive to the underlying model structure. Clearly then, there is a need for specific guidance in this field, over and
22
above the guidance that already exists to try to improve the generalizability and comparability of economic analyses.
23
What is a transmission dynamic model?
24
Transmission dynamic models (often shortened to just “dynamic” models) are capable of reproducing the
25
direct and indirect effects that may arise from a communicable disease control programme. They differ from other
26
(static) models used in decision sciences in that the risk of infection (sometimes referred to as the “force of infection”)
27
is a function of the number of infectious individuals (or infectious particles, such as eggs of intestinal worms) in the
28
population (or environment) at a given point in time (5). If an intervention reduces this pool of infectiousness then the
29
risk to uninfected susceptible individuals will decrease. That is, individuals who were not reached by the programme
30
can still benefit by experiencing a lower risk of infection. The models used can be deterministic or stochastic, individual
31
or population-based (see later for definitions of these terms), they may include (or be linked to) an economic and
32
health outcomes component, or may be stand-alone epidemiological analyses, they may be simple explorations of the
33
system or very detailed with large numbers of parameters. However, all these models share the same distinguishing
34
feature – that the risk of infection is dependent on the number of infectious agents at a given point in time. Other
35
aspects of these models are similar to models more commonly used in health economics and decision-sciences. For this
1
36
reason, these dynamic aspects (which arise from the communicable nature of the diseases in question) will be the focus
37
of this review. Readers are asked to refer to other papers in this series for more general advice pertaining to model-
38
based health economic analysis.
39
Basic reproduction number
40
The basic reproduction number is a fundamental metric in infectious disease epidemiology (5, 6). It is the
41
average number of secondary infections generated by a typical case in a fully susceptible population. A closely allied
42
metric is the (effective) reproduction number, which does not specify that the whole population must be susceptible.
43
The effective reproduction number (Re (t)) is simply the basic reproduction number (R0) multiplied by the fraction of the
44
population who are susceptible (s(t)) (5, 6). The reproduction number gives a measure of the ability of the disease to
45
spread in a population. A reproduction number of 1 gives a threshold for invasion of a pathogen into a population.
46
Malaria, for instance, now has a reproduction number below one in northern Europe, and although most
47
Northern Europeans are susceptible to disease, and cases are regularly introduced via travel from endemic areas,
48
malaria epidemics do not occur (7, 8). By contrast SARS had a basic reproduction number of around 3 (in healthcare
49
settings), and everyone was susceptible. That is, each case generated about 3 other cases, and each of these would be
50
expected to generate about 3 other cases, and so on, leading to an exponentially increasing epidemic (9). The basic
51
reproduction number also gives an indication of the ease of control of an infection. Returning to the malaria in northern
52
Europe example, it is obvious that there is no need for further control measures as an epidemic will not arise. SARS, on
53
the other hand, required stringent control measures for a large epidemic to be averted.
54
When is a dynamic approach appropriate?
55
Dynamic models are important in two general circumstances: (i) when an intervention impacts on the ecology
56
of a pathogen, for example by applying selection pressure resulting in “strain replacement” (10, 11), and (ii) when the
57
intervention impacts disease transmission (1, 2). A static model is acceptable if eligible target groups for intervention
58
are not epidemiologically important (e.g., evaluation of hepatitis A vaccination in travellers from low- to high-incidence
59
countries), or when effects of immunising a given group are expected to be almost entirely direct (e.g., vaccination of
60
the elderly against influenza or pneumococcal disease). Static models are also acceptable when static model
61
projections suggest that an intervention will be cost-effective, with cost-effectiveness expected to be further enhanced
62
via (excluded) dynamic effects (e.g., via prevention of secondary “add-on” cases). If such an approach is adopted, it
63
should be born in mind that undervaluing an intervention can lead to poor public health decision making, if policy
64
makers use such estimates of cost-effectiveness to decide on the optimum allocation of a limited healthcare budget.
65
Furthermore, it is important to acknowledge that not all effects associated with reduced transmission result in
66
net health and economic gains; in particular, increasing age at infection (as described below) may be associated with
67
reduced health due to the changing spectrum of illness in older individuals (12). Also, replacement effects have been
68
reported, for example, in pneumococcal disease, that may reduce health or limit health gains due to other (sub) types
69
of bacteria “substituting” those removed by vaccination. Where static models project interventions to be unattractive
70
or borderline-attractive, supplementary dynamic modelling should be undertaken to evaluate whether inclusion of
71
indirect effects of herd immunity, replacement and age shifts alter projected cost-effectiveness. Indirect intervention
2
72
effects can be incorporated using a static framework; for example, such an approach was used by European countries in
73
evaluating the economic attractiveness of pneumococcal conjugate vaccines in children; static models incorporated
74
herd-level impacts derived from US data (13, 14). However, the danger of this approach is that the level of indirect
75
protection may be very different in a different setting (with, for instance, different levels of vaccine coverage). Flow-
76
charts have been developed by the World Health Organization; these can be helpful in guiding the decision as to
77
whether or not dynamic models are appropriate (15).
78
Indirect effects of intervention programmes
79
The best-known example of economically important indirect effects is herd immunity with large-scale
80
vaccination programmes. When coverage exceeds a critical threshold (Vc) disease is eliminated, as too few susceptible
81
persons remain to ensure persistent transmission. Infectious individuals will (on average) cause less than one new
82
infection before recovering, as most contacts will be with immune individuals. As an epidemic does not occur,
83
unvaccinated individuals in a population experience a low risk of infection. For herd immunity to occur, VC has to be
84
greater than 1-1/R0 (5, 6). Successful eradication of smallpox from the world and elimination of many childhood
85
infections from countries with high infant vaccination coverage have provided “proof of concept” for this relation.
86
Indirect effects can also be observed for other large-scale population based intervention programmes against
87
communicable diseases, such as population screening (16, 17). For example population based screening for Chlamydia
88
infections clearly also has effects in age and gender groups not directly targeted for screening (16, 17). Not taking those
89
effects into account when evaluating the effectiveness of a programme may lead to overly pessimistic cost-
90
effectiveness ratios. Indirect effects also may mean that the optimal age or gender class to which interventions should
91
be targeted is not the class that experiences the greatest burden of disease, but that which contributes most to force-
92
of-infection; for example, immunizing younger individuals may be identified as the most attractive means of preventing
93
influenza-related mortality in older individuals in dynamic models, but not in static models (18). Similarly, dynamic
94
models may identify age and gender groups at less risk of sequelae as the best targets for Chlamydia screening
95
programs (16, 17).
96
As mentioned, indirect effects are not always beneficial, even when they lead to decreased incidence of
97
infection in the population. Reducing the risk of infection in susceptible individuals increases the average age at which
98
susceptible individuals become infected (5, 6), and for many communicable diseases, this increases the risk of
99
complications and mortality. Infectious diseases that may be more severe in older individuals include hepatitis A virus
100
infection, SARS, and varicella (19-21). Older age at infection may also result in higher likelihood of infection during
101
pregnancy, with devastating complications for newborns (22); several countries have seen paradoxical increases, for
102
example, in congenital rubella infections resulting from partial population coverage with rubella vaccine, with
103
concomitant increase in age at first infection (23, 24). Complex relationships may also exist between disease incidence,
104
latent infections, and immunity in older individuals for such infections as varicella; here, vaccine programs that result in
105
less “boosting” of the population as a result of infection in children may lead to surges in reactivated infection
106
(“shingles”) in older individuals (25, 26). Lastly, reduced force-of-infection may result in epidemics being more widely
107
spaced, a phenomenon which may itself have economic value, especially when future health costs and effects are
108
subjected to discounting (6). None of these phenomena is readily captured via static models.
3
109
How should uncertainty be managed?
110
Methodological uncertainty
111
Historically, most dynamic transmission modelling has been performed using system dynamics models, where
112
transition between compartments is represented by differential equations. With the increase in readily available
113
computing power, it has become possible to realize dynamic transmission models using agent-based approaches where
114
each member of a population is represented individually (27-29). Deterministic compartmental models are useful for
115
modelling average behaviour of disease epidemics in large populations. When stochastic effects (e.g., extinction of
116
disease in small populations), complex interactions between behaviour and disease, or distinctly non-random mixing
117
patterns (e.g., movement of disease on networks) are important, agent-based approaches may be preferred. The
118
choice of modelling method may influence the results, and analysts and decision-makers should be aware of the effects
119
these different choices may have (see recommendations section for further advice).
120
As with non-infectious diseases, the assumed discount rate, and the time horizon of the analysis may be
121
influential. Many control programmes against communicable diseases are preventative in nature (vaccines being the
122
classic example). They are therefore often very sensitive to these methodological choices as the up-front costs of
123
setting up a programme are usually considerable.
124
Structural uncertainty
125
Both static and dynamic models have to deal with uncertainty pertaining to exact model formulation, or
126
“structural uncertainty”. Frequently there is uncertainty related to the exact biological properties and relationships
127
that comprise a disease transmission system. For example, in modelling the transmission of human papilloma virus
128
both “susceptible-infectious-susceptible” (SIS) and “susceptible-infectious-removed/immune” (SIR) frameworks have
129
been used, because not enough information about acquisition and duration of immunity after infection with a high risk
130
HPV strain is available (30). The presence of a (controversial) short-term immune state associated with untreated
131
infection has led to the use of SIRS models for Chlamydia, and incorporation of such a state reproduces observed
132
“rebound” when screening programs are modelled (31, 32). Alternate structural assumptions may result in markedly
133
different projections of economic attractiveness.
134
Parameter uncertainty
135
Uncertainty in parameter values can be more influential in dynamic models than in static models due to non-
136
linear feedback effects leading to qualitatively different dynamic regimes. It is well known that dynamical systems may
137
display qualitatively different behaviour in different parameter regions. For example, while in some regions a stable
138
endemic equilibrium may exist, in other regions the system might have oscillatory behaviour or even chaotic behaviour.
139
It may only need a small shift in parameter values to move from one dynamic regime to another. The best-known
140
example in epidemic models is the transition from a disease free state to an endemic equilibrium when the basic
141
reproduction number crosses the threshold value one. Near this threshold, small changes in parameter values can
142
cause large changes in prevalence. Several recent models have evaluated non-linear, and “catastrophic” (for the
143
pathogen) effects of interventions for hepatitis B virus and pertussis (33, 34). This phenomenon also has implications
144
for intervention effectiveness. If an intervention is implemented in a situation near such a threshold, the indirect effects
4
145
may be very large. The same programme implemented in a different region of parameter space may result in a more or
146
less linear relationship between intervention effort and effectiveness.
147
Several challenges exist in the accurate measurement of parameters for communicable disease models. Many
148
communicable diseases of public health importance are extremely variable in their severity, and surveillance systems
149
may capture only information on infected individuals with symptoms sufficiently severe to warrant presentation for
150
medical care and diagnostic testing. As such, modelling the natural history of communicable diseases from public
151
health surveillance data is likely to result in underestimation of disease incidence, and overestimation of severity,
152
hospitalization risk, and case-fatality. For many communicable diseases, there is also a disconnect between severity of
153
illness and effective infectiousness of an individual, as more symptomatic individuals may modify their behaviour in a
154
way that reduces transmissibility. Thus transmission of infection by individuals who are minimally symptomatic may
155
represent a fundamental property of “uncontrollable” communicable diseases (35). Serological studies may be used to
156
overcome some of the challenges presented by case reporting, as antibody responses to infection provide a relatively
157
durable record of an individual’s past infection status. Seroprevalence curves can be used to estimate incidence among
158
uninfected individuals according to age, gender, and other characteristics (6, 36).
159
For most communicable diseases, transmission of infection depends not only on the infectiousness of a case,
160
but on contact patterns as well. Empirical data on contact patterns within and between age groups, derived from large
161
population-based surveys, are available for Europe, with the recently published POLYMOD study (37). Surveys of sexual
162
behaviour are also available for some populations (such as NATSAL in the UK), and are invaluable for parameterising
163
models of STI transmission although estimates may be biased by social desirability effects, and by the failure to capture
164
highly influential core groups (38).
165
Challenges in Parameter Measurement—Intervention Effectiveness
166
The impact of interventions (e.g., effectiveness of a vaccination or screening program) is often estimated from
167
quasi-experimental data derived from surveillance sources. Such data sources are subject to all of the limitations of
168
quasi-experiment. Misidentification of random variation as a true change in incidence, the natural tendency of
169
communicable diseases to evolve and oscillate as a result of population immunity and strain variation, and confounding
170
by unmeasured interventions or changes in the population may all potentially result in misidentification of an
171
intervention’s effectiveness (39). The fact that such interventions may be longstanding may make identification of pre-
172
intervention data problematic.
173
Randomized controlled trials of intervention effectiveness are preferred as a source of model effectiveness
174
parameters, but should consider the possible under- or over-estimation of intervention effectiveness commonly seen
175
with randomized trials of vaccination programs and other interventions when the disease under consideration is
176
communicable. When individual randomization is used, intervention efficacy and effectiveness will be underestimated,
177
as indirect effects will not be captured as typically neither the control nor intervention arms experience a reduced force
178
of infection, as clinical trials are usually tiny in comparison to the overall population. Additionally, it should be noted
179
that mostly models extrapolate from intermediate endpoints often measured on the level of immunity (antigens, viral
180
loads, infections) towards “hard” endpoints on mortality and serious morbidity, as randomized controlled trials often
5
181
lack the capability of signalling significant differences in hard endpoints whereas they do have the power to detect
182
differences in immune responses.
183
Identification and Synthesis of Parameter Values from Published Biomedical Literature
184
Identification of parameter values for modelling of communicable diseases should follow best practices for
185
non-communicable diseases [cite other ISPOR working paper]. It should be noted that observational studies of
186
outbreaks are more likely to be submitted for publication if they are large and/or costly; as such, estimates of
187
reproductive numbers and outbreak sizes derived from descriptive studies are likely to be higher than is the case in
188
reality. A second area of complexity relates to the likelihood that communicable diseases will have different dynamics
189
in different populations, due to heterogeneity in geography and climate, socioeconomic status, population genetics and
190
demographic structures, and the availability of control interventions. As such, methods for data synthesis across
191
multiple studies should be used with caution, and it may be prudent to use literature-derived parameter estimates to
192
construct plausible ranges or relatively flat priors, rather than parametric distributions for the purposes of stochastic
193
simulation or sensitivity analyses.
194
Calibration and Refinement of Parameter Estimates
195
Given the complexities of accurate parameter estimation, model calibration is of great importance for several
196
reasons. Well-calibrated models may force the re-estimation of uncertain or implausible parameters, and calibration
197
may be used to generate plausible parameter values when no empirical estimates are available. Furthermore, well-
198
calibrated models that reproduce observed disease incidence, trends, or natural history are extremely important for
199
the establishment of a model’s credibility with decision makers.
200
Difficulty in calibrating a disease model across multiple domains may suggest that model structures,
201
approaches or assumptions are incorrect. While frustrating for the modeller, difficulties in calibration should not be
202
glossed over or ignored, as they represent an important mechanism for quality control; furthermore, difficulty in
203
calibration may suggest that the best current understanding of the biology of a given disease is incorrect.
204
Consequently, models that are difficult to calibrate may provide important information that helps frame priorities for
205
future research.
206
Reporting results and informing health care and public health decision making
207
Numerous guidelines exist for reporting the results of economic evaluations [e.g., cite other ISPOR guidelines]. These
208
emphasize the need to present disaggregated costs (e.g., showing drug costs, hospital costs and indirect costs
209
separately) and to show the incremental health and economic impact separately before calculating an ICER. Such
210
guidelines apply equally to health-economic analyses of communicable disease control interventions, but this group
211
also suggests that analyses report the estimated change in the burden of infection as a result of the intervention, as this
212
constitutes a major motivation for the use of dynamic rather than static modelling methods. Infections can be further
213
disaggregated according to directly or indirectly prevented infections, route of transmission (e.g., sexual, vertical, by
214
vector, etc.), and population subgroups as appropriate.
6
215
Other outcomes specific to communicable disease-related models which are appropriate for reporting include
216
changes in the long-run equilibrium level (incidence or prevalence) of infection; likelihood of disease elimination; and
217
changes in the effective reproductive number of the disease.
218
Ensuring Transparency and Credibility
219
220
Transparency of the modeling process and results are necessary for peer review and knowledge translation,
221
and to make models credible to decision-makers. The following steps will serve to enhance the transparency of
222
communicable disease models: clear statements about model quality, data quality and sources, model structure and
223
validation, extensive sensitivity analyses, and clear, candid statements of limitations. Many of these are issues
224
discussed in general elsewhere. We focus below on those aspects that are specific to communicable disease models.
225
Authorities and agencies charged with the assessment of novel interventions and programs for communicable
226
diseases may overlap with those assessing interventions for non-communicable diseases. For non-communicable
227
diseases often less complex models suffice as compared to those used for communicable diseases. Both agencies
228
charged with adoption of novel health technologies, and those developing public health policy, may be unaccustomed
229
to dynamic modeling considerations (40-42). Knowledge translation and provision of educational opportunities and
230
“short-courses” for professional development by modellers to decision makers will ensure that end users have the skills
231
to understand these models. In addition to transparency of presentation, some authorities request access to the model
232
itself. Where such disclosure is justified, measures should be put in place to protect modellers’ intellectual property.
233
Finally, it should be noted that in instances where several modeling groups have evaluated similar policy questions
234
using disparate approaches, joint publication can help build confidence in the use of models as a tool for policy (see for
235
example (43)).
236
Key considerations specific to the transparent presentation of communicable disease models include the
237
provision of information on how “effective” contact rates and mixing patterns have been inferred, as estimates will
238
depend both on the data used to derive rates, and the assumed model structure, such that different estimates may be
239
obtained even using a common data set. For empirical, literature derived estimates (as in (44, 45)), there are large
240
variations in the values of these estimates. Mixing behaviour should be clarified, for example by specifying the mixing
241
matrix used.
242
When system dynamics models are used, the relevant system of differential equations should be written out
243
and included as a supplement or appendix to the original analysis. When agent-based models are used it may be
244
necessary to specify the behaviour of agents in greater detail. This will include factors such as movement of agents and
245
mixing assumptions. Descriptions of movement should address whether the model makes use of geographic zones,
246
how those zones are defined, and how agents move between zones. Descriptions of mixing behaviour should include
247
how new contacts are acquired, as well as the length of time that partnerships last (if relevant).
248
7
249
Recommendations
250
When is it Appropriate to Use a Dynamic Transmission Model?
251
Recommendation:
252
A dynamic model is needed when a modeller is trying to evaluate an intervention against an infectious disease
253
that 1) has an impact on disease transmission in the population of interest, and/or 2) alters the frequency distribution
254
of strains (e.g., genotypes or serotypes).
255
Rationale
256
Static models (models without interaction) can be used 1) if the intervention is unlikely to change the force of infection
257
(the per susceptible risk of infection) or, 2) to estimate the worst-case scenario when herd immunity or age shifts due
258
to changing force of infection cannot produce negative effects. Static models cannot adequately take into account herd
259
immunity effects nor shifts in the age distribution of infection induced by interventions. Risk of infection in susceptible
260
individuals, so-called “force of infection” is constant in time in static models, while in a dynamic model it is a function of
261
the proportion of the population that is infected (which changes over time). Hence, when the force of infection is
262
unlikely to change following an intervention (i.e. herd immunity effects are unlikely to occur) then static models can
263
adequately predict effectiveness. If intervention uptake is very low (e.g., low vaccine coverage) or is targeted at groups
264
that do not have an impact on overall transmission then static and dynamic models produce similar results (1, 46).
265
Furthermore, results are also similar for vaccines or interventions that do not prevent the circulation of the pathogen
266
since herd immunity effects are negligible
267
Dynamic transmission models should be used if an intervention is likely to change the force of infection. This can occur
268
in several ways. An intervention may decrease the proportion of the population that is susceptible (e.g., mass
269
vaccination), the contact rates between individuals (e.g., closing schools during a pandemic), duration of infectiousness
270
(e.g., antivirals), and/or the probability of transmission per act (e.g., antiretrovirals). By taking into account the changes
271
in the force of infection following an intervention, dynamic models can 1) produce non-linear dynamics, 2) predict a
272
higher number of cases prevented and 3) predict either increases or decreases in morbidity and mortality due to shifts
273
in the age at infections. Note that using static models does not always produce conservative results. For example, the
274
use of static models may overestimate the effectiveness of mass vaccination at preventing serious disease since they
275
cannot capture possible increases in morbidity due to shifts in the age at infection (1), such as those that occurred after
276
rubella vaccination in Greece (24).
277
Some diseases are caused by several types or subtypes of pathogens, and interventions can induce selective
278
pressures that cause a subset of these types or even other microbes to gain a competitive advantage (10, 47). Examples
279
are type replacement following pneumococcal and Haemophilus influenzae B conjugate vaccination (10, 48, 49) and
280
antimicrobial-resistance (47, 50). Dynamic transmission models are necessary in such instances.
281
282
Finally, decision makers are often interested in local or national elimination of an infectious disease, or
eradication (i.e., global elimination of the disease). The ability to eliminate a disease is a feature of communicable
8
283
diseases, as the non-linear (herd) effects mean that it is possible to eliminate without reaching everyone in the
284
population. Since no intervention can ever target everyone (some will always refuse, or not be reached), and no
285
intervention is ever 100% efficacious, elimination is not achievable in a static modelling framework. Dynamic models
286
must be used if elimination is a goal.
287
288
Several schemata exist for guiding the decision to use of dynamic or static models (2, 15, 46), and a number of
well-written papers demonstrate the divergence in predictions between static and dynamic models (1, 2, 16, 51).
289
290
What Type of Model to Use?
291
Recommendation:
292
The appropriate type of dynamic transmission model should be used for the analysis in question, based in part
293
on the complexity of the interactions as well as the size of the population of interest and the role of chance effects. This
294
model could be deterministic or stochastic, and population or individual-based. Justification for the model structure
295
should be given.
296
Rationale
297
There are several different types of models that can be utilized to simulate the dynamics of communicable
298
diseases. Broadly these can be classed as deterministic or stochastic, and population-based (sometimes known as
299
compartmental) or agent- or individual- (agent-) based models.
300
In deterministic models every state variable is uniquely determined by the parameter values and the previous
301
values of the state variables. Thus deterministic models will always give the same results if the model is run with the
302
same starting conditions and parameter values. The state variables in a stochastic model are not described by unique
303
values, but by probability distributions. That is, stochastic models inherently incorporate the role of chance in
304
determining the state of the system. This often occurs in small populations, or when one of the subgroups in a model is
305
small (such as at the beginning or end of an epidemic), when local extinction is likely. In these circumstances stochastic
306
models are usually more appropriate. Deterministic models approximate the average behaviour of a system and are
307
most appropriate when all population subgroups in the model are large. They are comparatively easy to fit to data and
308
may be easier to calibrate.
309
Population-based models track groups of individuals (e.g. they simulate how the number of infectious or
310
susceptible individuals changes over time). An alternative method is to model each individual, in an individual- (or
311
agent-) based model. Here, the status of each individual is explicitly tracked over time. These models treat individuals
312
as discrete entities who do not move between “compartments” but rather change their internal “state” (e.g.,
313
susceptible, infected, etc.) based on their interactions. As one of the characteristics attached to an individual is prior
314
history (e.g., number of prior infections), they are particularly useful when risk depends on past events; representation
315
of such phenomena in compartmental models requires large numbers of compartments. Furthermore, individual-
316
based models can incorporate population heterogeneity relatively easily, and also have the flexibility to assess complex
9
317
interventions. Disadvantages include slow speed, lack of analytical tractability and challenges in parameterization. They
318
are invariably stochastic in nature. Compartmental (population-based) models can be either stochastic or deterministic.
319
Methodological Uncertainty
320
Recommendation:
321
Conduct sensitivity analysis on the time horizon and discount rate.
322
Rationale
323
Although many guidelines for economic evaluation specify the use of a “lifetime” time horizon, the concept of
324
a lifetime horizon is not well defined for dynamic transmission models, as these often concern whole populations,
325
which change over time due to births, deaths and migrations. An infinite time horizon could capture all of these effects,
326
but is clearly impractical. Fixed time horizons can produce artefacts; for example, if a cohort is vaccinated just before
327
the end of the time horizon the benefits to these individuals will not be included in the cost-effectiveness estimate,
328
though the costs of vaccinating them will. Other benefits and negative outcomes may vary non-monotonically over
329
time (i.e., may not strictly increase or decrease), making projections of economic attractiveness dependent on the time
330
horizon chosen (46).
331
For high discount rates the issue of time horizon may become less important, as distant future costs, savings
332
and health gains add little to the total. In the absence of discounting results may intimately depend on the time
333
horizon. Thus, it is recommended that modellers conduct sensitivity analysis on both the length of the time horizon
334
and the discount rate (2, 46, 52).
335
Structural Uncertainty
336
Recommendation:
337
Conduct uncertainty analyses on known key structural assumptions that may have an impact on the conclusions, or
338
justify the omission of such analyses.
339
Rationale
340
Structural uncertainty refers to the impact of model choice and structure on projections of cost and effect,
341
(and thus potential policy decisions). Structural uncertainty may relate to transmission routes and important risk
342
groups (by age, gender, or risk status), behavioural assumptions about contact patterns (e.g. instantaneous
343
partnerships versus long-term partnerships, the nature of mixing between age groups, etc.), the durability of immunity
344
following infection, changes in infectiousness of hosts over time, and strain competition and replacement in the
345
pathogen. A decision not to include a specific variable or structure in the model may also have consequences for the
346
results obtained. Structural uncertainty is often ignored in model-based economic evaluations, despite evidence that it
347
can have a much greater impact on results than parameter uncertainty (1, 2, 31, 51).
10
348
Often not enough is known about the biological properties of a system for precise definition of functional
349
relationships; alternately, there may be several possible approaches to derive a model framework for a specific
350
biological process. Structural uncertainty can be extremely influential in dynamic models due to non-linear feedback
351
effects leading to qualitatively different dynamic regimes (33, 34, 51). It is particularly important, therefore to give
352
due consideration to this aspect when reporting the results of an infectious disease model.
353
When interventions are incorporated into models, choices must be made regarding the structure of
354
implementation in the model. For vaccine programs, key areas of structural uncertainty may relate to the
355
representation of actual timing of vaccine doses and the impact of boosting; several components of vaccine
356
effectiveness may be difficult to distinguish from empirical data; for example it may be difficult to distinguish vaccine
357
“take” (the probability that a vaccinated individual develops measurable immunity) from vaccine efficacy (the degree of
358
protection against infection per contact). Models of sexually transmitted infections often pose challenges related to
359
explicit structural representations of partnerships (and partner concurrency), contact tracing or partner notification,
360
and reinfection within partnerships (53).
361
Sensitivity analysis and Interpretation
362
Recommendation:
363
When conducting sensitivity analyses, consideration of important epidemic thresholds is helpful when there is a
364
possibility of the model exhibiting alternate behaviours.
365
Rationale
366
One function of a sensitivity analysis is to give some indication of the degree of influence a change in parameter values
367
may have on the outcome of interest. In non-linear dynamic transmission models the existence of regions of parameter
368
space that characterise distinct types of model behaviour (e.g., epidemic spread vs. extinction) complicates such
369
analyses. It is helpful if modellers define such behaviourally distinct regions of parameter space and explicitly state
370
whether or not the sensitivity analysis has been confined to one region or not. If the sensitivity analysis encompasses
371
more than one region, it is informative to state the probability of achieving different equilibrium states as parameter
372
values are varied.
373
Numerical Techniques
374
Recommendation:
375
For differential equation-based models, adaptive time step methods for numerical integration, that allow the degree of
376
error tolerance to be specified in advance, are preferred to those that use a fixed time step of indeterminate accuracy.
377
Rationale
378
In general, the systems of differential equations commonly used in dynamic transmission models cannot be solved
379
analytically. Thus, numerical integration techniques are used to approximate these systems. Such approximations
11
380
invariably result in error, but the magnitude of error can and must be controlled. This may be achieved by the use of
381
adaptive time step methods. If a fixed time step methodology is used that utilises too large a step, then this error may
382
be sufficiently large to influence the results of the model. By contrast, the use of very small time step to avoid such
383
error may lead to a slow and inefficient model.
384
Reporting 1: Transparency
385
Recommendation:
386
If using a differential equations model, provide the model equations. Tabulate all initial value and parameters, including
387
the mixing matrix and supply details of the type of mixing considered.
388
Rationale
389
Showing the model equations and details is required to ensure transparency and reproducibility.
390
Recommendation:
391
If using an agent-based model, thoroughly describe the rules governing the agents, the input parameter values, initial
392
conditions and all sub-models.
393
Rationale
394
Describing the rules that govern the behaviour of agents, parameter values, initial conditions and sub-models is
395
required to ensure transparency and reproducibility. In particular, describing the rules surrounding the creation and
396
dissolution of partnerships in models of sexually transmitted infections will help ensure face validity.
397
Reporting 2: Epidemic Outcomes
398
Recommendation:
399
Show the transmission dynamics over time (e.g., incidence and prevalence of infection and disease). When applicable,
400
report changes in other infection-specific outcomes such as strain replacement and the emergence of resistance to
401
antimicrobial drugs.
402
Rationale
403
Dynamic transmission models are recommended to evaluate interventions that may have an impact on disease
404
transmission or interventions that may result in changes in the frequency distribution of strains. Thus, all associated
405
outcomes should be reported to optimize comparability of model results, as well as transparency. This information may
406
also be of use to decision makers (54).
12
407
408
409
References
410
411
412
1.
Brisson M, Edmunds WJ. Economic evaluation of vaccination programs: the impact of herd-immunity. Med
Decis Making. 2003 Jan-Feb;23(1):76-82.
413
414
2.
Brisson M, Edmunds WJ. Impact of model, methodological, and parameter uncertainty in the economic
analysis of vaccination programs. Med Decis Making. 2006 Sep-Oct;26(5):434-46.
415
416
3.
Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes : a focused review of modelling
approaches. Pharmacoeconomics. 2008;26(3):191-215.
417
418
419
4.
Roberts TE, Robinson S, Barton P, Bryan S, Low N, Chlamydia Screening Studies G. Screening for Chlamydia
trachomatis: a systematic review of the economic evaluations and modelling. Sex Transm Infect. 2006 Jun;82(3):193200; discussion 1.
420
421
5.
Kretzschmar M, Wallinga J. Mathematical models in infectious disease epidemiology. In: Kramer A,
Kretzschmar M, Krickenberg K, editors. Modern Infectious Disease Epidemiology. New York: Springer; 2009. p. 209-21.
422
423
6.
Anderson RM, May RM. Infectious Diseases of Humans: Dynamics and Control. New York: Oxford University
Press; 1991.
424
425
7.
Jelinek T, TropNetEurop. Imported falciparum malaria in Europe: 2007 data from TropNetEurop. Euro Surveill.
2008 Jun 5;13(23).
426
427
8.
Lindsay SW, Hole DG, Hutchinson RA, Richards SA, Willis SG. Assessing the future threat from vivax malaria in
the United Kingdom using two markedly different modelling approaches. Malar J. 2010;9:70.
428
429
9.
Lipsitch M, Cohen T, Cooper B, Robins JM, Ma S, James L, et al. Transmission dynamics and control of severe
acute respiratory syndrome. Science. 2003 Jun 20;300(5627):1966-70.
430
431
10.
Lipsitch M. Bacterial vaccines and serotype replacement: lessons from Haemophilus influenzae and prospects
for Streptococcus pneumoniae. Emerg Infect Dis. 1999 May-Jun;5(3):336-45.
432
433
11.
Lipsitch M. Vaccination against colonizing bacteria with multiple serotypes. Proc Natl Acad Sci U S A. 1997 Jun
10;94(12):6571-6.
13
434
435
12.
Brisson M, Edmunds WJ, Gay NJ. Varicella vaccination: impact of vaccine efficacy on the epidemiology of VZV. J
Med Virol. 2003;70 Suppl 1:S31-7.
436
437
13.
Claes C, Reinert RR, von der Schulenburg JM. Cost effectiveness analysis of heptavalent pneumococcal
conjugate vaccine in Germany considering herd immunity effects. Eur J Health Econ. 2009 Feb;10(1):25-38.
438
439
440
14.
Bergman A, Hjelmgren J, Ortqvist A, Wisloff T, Kristiansen IS, Hogberg LD, et al. Cost-effectiveness analysis of a
universal vaccination programme with the 7-valent pneumococcal conjugate vaccine (PCV-7) in Sweden. Scand J Infect
Dis. 2008;40(9):721-9.
441
442
443
15.
Walker D, Beutels P, Initiative for Vaccine Research. WHO guide for standardization of economic evaluations of
immunization programmes. Available via the Internet at http://www.who.int/vaccines-documents/. Last accessed July
3, 2011. Geneva: World Health Organization2008 Contract No.: WHO/IVB/08.14.
444
445
16.
Welte R, Postma M, Leidl R, Kretzschmar M. Costs and effects of chlamydial screening: dynamic versus static
modeling. Sex Transm Dis. 2005 Aug;32(8):474-83.
446
447
17.
Fisman DN, Spain CV, Salmon ME, Goldberg M. The Philadelphia High-School STD Screening Program: key
insights from dynamic transmission modeling. Sex Transm Dis. 2008 Nov;35(11 Suppl):S61-5.
448
449
18.
Galvani AP, Reluga TC, Chapman GB. Long-standing influenza vaccination policy is in accord with individual selfinterest but not with the utilitarian optimum. Proc Natl Acad Sci U S A. 2007 Mar 27;104(13):5692-7.
450
451
19.
Brisson M, Edmunds WJ, Gay NJ, Law B, De Serres G. Modelling the impact of immunization on the
epidemiology of varicella zoster virus. Epidemiol Infect. 2000 Dec;125(3):651-69.
452
453
20.
Leung CW, Kwan YW, Ko PW, Chiu SS, Loung PY, Fong NC, et al. Severe acute respiratory syndrome among
children. Pediatrics. 2004 Jun;113(6):e535-43.
454
455
21.
Berge JJ, Drennan DP, Jacobs RJ, Jakins A, Meyerhoff AS, Stubblefield W, et al. The cost of hepatitis A infections
in American adolescents and adults in 1997. Hepatology. 2000 Feb;31(2):469-73.
456
22.
457
458
23.
Centers for Disease C, Prevention. Measles, rubella, and congenital rubella syndrome--United States and
Mexico, 1997-1999. MMWR Morb Mortal Wkly Rep. 2000 Nov 24;49(46):1048-50, 59.
459
460
24.
Panagiotopoulos T, Antoniadou I, Valassi-Adam E. Increase in congenital rubella occurrence after immunisation
in Greece: retrospective survey and systematic review. BMJ. 1999 Dec 4;319(7223):1462-7.
461
462
25.
Edmunds WJ, Brisson M. The effect of vaccination on the epidemiology of varicella zoster virus. J Infect. 2002
May;44(4):211-9.
463
464
26.
Brisson M, Edmunds WJ, Gay NJ, Miller E. Varicella vaccine and shingles. JAMA. 2002 May 1;287(17):2211;
author reply -2.
Fine JD, Arndt KA. The TORCH syndrome: a clinical review. J Am Acad Dermatol. 1985 Apr;12(4):697-706.
14
465
466
467
27.
Ajelli M, Goncalves B, Balcan D, Colizza V, Hu H, Ramasco JJ, et al. Comparing large-scale computational
approaches to epidemic modeling: agent-based versus structured metapopulation models. BMC Infect Dis.
2010;10:190.
468
469
28.
Brooks-Pollock E, Cohen T, Murray M. The impact of realistic age structure in simple models of tuberculosis
transmission. PLoS One. 2010;5(1):e8479.
470
471
29.
Perez L, Dragicevic S. An agent-based approach for modeling dynamics of contagious disease spread. Int J
Health Geogr. 2009;8:50.
472
473
474
30.
Jit M, Demarteau N, Elbasha E, Ginsberg G, Kim J, Praditsitthikorn N, et al. Human papillomavirus vaccine
introduction in low-income and middle-income countries: guidance on the use of cost-effectiveness models. BMC Med.
2011;9:54.
475
476
31.
Brunham RC, Pourbohloul B, Mak S, White R, Rekart ML. The unexpected impact of a Chlamydia trachomatis
infection control program on susceptibility to reinfection. J Infect Dis. 2005 Nov 15;192(10):1836-44.
477
478
32.
Vickers DM, Osgood ND. Current crisis or artifact of surveillance: insights into rebound chlamydia rates from
dynamic modelling. BMC Infect Dis. 2010;10:70.
479
480
33.
Medley GF, Lindop NA, Edmunds WJ, Nokes DJ. Hepatitis-B virus endemicity: heterogeneity, catastrophic
dynamics and control. Nat Med. 2001 May;7(5):619-24.
481
482
34.
van Boven M, Mooi FR, Schellekens JF, de Melker HE, Kretzschmar M. Pathogen adaptation under imperfect
vaccination: implications for pertussis. Proc Biol Sci. 2005 Aug 7;272(1572):1617-24.
483
484
35.
Fraser C, Riley S, Anderson RM, Ferguson NM. Factors that make an infectious disease outbreak controllable.
Proc Natl Acad Sci U S A. 2004 Apr 20;101(16):6146-51.
485
486
36.
Anderson RM, May RM. Age-related changes in the rate of disease transmission: implications for the design of
vaccination programmes. J Hyg (Lond). 1985 Jun;94(3):365-436.
487
488
37.
Zagheni E, Billari FC, Manfredi P, Melegaro A, Mossong J, Edmunds WJ. Using time-use data to parameterize
models for the spread of close-contact infectious diseases. Am J Epidemiol. 2008 Nov 1;168(9):1082-90.
489
490
38.
Johnson AM, Mercer CH, Erens B, Copas AJ, McManus S, Wellings K, et al. Sexual behaviour in Britain:
partnerships, practices, and HIV risk behaviours. Lancet. 2001 Dec 1;358(9296):1835-42.
491
492
39.
Halloran ME, Longini IM, Jr., Stuchiner CJ. Assessing indirect, total, and overall effects. In: Halloran ME, Longini
IM, Jr., Stuchiner CJ, editors. Design and Analysis of Vaccine Studies. New York: Springer; 2010. p. 271-312.
493
494
40.
Pandemic Influenza Outbreak Research Modelling T, Fisman D. Modelling an influenza pandemic: A guide for
the perplexed. CMAJ. 2009 Aug 4;181(3-4):171-3.
495
496
41.
Moghadas SM, Pizzi NJ, Wu J, Tamblyn SE, Fisman DN. Canada in the face of the 2009 H1N1 pandemic.
Influenza Other Respi Viruses. 2011 Mar;5(2):83-8.
15
497
498
499
42.
van Rossum TG, de Melker HE, Houweling H, Voordouw AC, Meijer CJ, Helmerhorst TJ, et al. [Vaccines against
human papillomavirus (HPV); between registration and implementation]. Ned Tijdschr Geneeskd. 2008 Apr
26;152(17):987-92.
500
501
43.
Halloran ME, Ferguson NM, Eubank S, Longini IM, Jr., Cummings DA, Lewis B, et al. Modeling targeted layered
containment of an influenza pandemic in the United States. Proc Natl Acad Sci U S A. 2008 Mar 25;105(12):4639-44.
502
503
44.
Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, et al. Social contacts and mixing patterns
relevant to the spread of infectious diseases. PLoS Med. 2008 Mar 25;5(3):e74.
504
505
506
45.
Fenton KA, Korovessis C, Johnson AM, McCadden A, McManus S, Wellings K, et al. Sexual behaviour in Britain:
reported sexually transmitted infections and prevalent genital Chlamydia trachomatis infection. Lancet. 2001 Dec
1;358(9296):1851-4.
507
508
46.
Edmunds WJ, Medley GF, Nokes DJ. Evaluating the cost-effectiveness of vaccination programmes: a dynamic
perspective. Stat Med. 1999 Dec 15;18(23):3263-82.
509
510
47.
Lipsitch M, Bergstrom CT, Levin BR. The epidemiology of antibiotic resistance in hospitals: paradoxes and
prescriptions. Proc Natl Acad Sci U S A. 2000 Feb 15;97(4):1938-43.
511
512
513
48.
Adam HJ, Richardson SE, Jamieson FB, Rawte P, Low DE, Fisman DN. Changing epidemiology of invasive
Haemophilus influenzae in Ontario, Canada: evidence for herd effects and strain replacement due to Hib vaccination.
Vaccine. 2010 May 28;28(24):4073-8.
514
515
516
49.
Hanage WP, Huang SS, Lipsitch M, Bishop CJ, Godoy D, Pelton SI, et al. Diversity and antibiotic resistance
among nonvaccine serotypes of Streptococcus pneumoniae carriage isolates in the post-heptavalent conjugate vaccine
era. J Infect Dis. 2007 Feb 1;195(3):347-52.
517
518
519
50.
Samore MH, Lipsitch M, Alder SC, Haddadin B, Stoddard G, Williamson J, et al. Mechanisms by which
antibiotics promote dissemination of resistant pneumococci in human populations. Am J Epidemiol. 2006 Jan
15;163(2):160-70.
520
521
51.
Van de Velde N, Brisson M, Boily MC. Understanding differences in predictions of HPV vaccine effectiveness: A
comparative model-based analysis. Vaccine. 2010 Jul 26;28(33):5473-84.
522
523
52.
Beutels P, Scuffham PA, MacIntyre CR. Funding of drugs: do vaccines warrant a different approach? Lancet
Infect Dis. 2008 Nov;8(11):727-33.
524
525
53.
Heijne JC, Althaus CL, Herzog SA, Kretzschmar M, Low N. The role of reinfection and partner notification in the
efficacy of Chlamydia screening programs. J Infect Dis. 2011 Feb 1;203(3):372-7.
526
527
54.
Craig BM, Brisson M, Chesson H, Giuliano AR, Jit M. Proceedings of the Modeling Evidence in HPV PreConference Workshop in Malmo, Sweden, May 9-10, 2009. Clin Ther. 2010 Aug;32(8):1546-64.
528
16