Download Exploiting Immune Response Dynamics in HIV Therapy

UNIVERSITY of CALIFORNIA
Santa Barbara
Exploiting Immune Response Dynamics in HIV Therapy
A Dissertation submitted in partial satisfaction of the
requirements for the degree
Doctor of Philosophy
in
Electrical and Computer Engineering
by
Ryan Mark Zurakowski
Committee in charge:
Professor Andrew R. Teel, Chair
Professor Joao Hespanha
Professor Mustafa Khammash
Professor Frank Doyle
Professor Duane Sears
December 2004
The dissertation of Ryan Mark Zurakowski is approved.
Professor Joao Hespanha
Professor Mustafa Khammash
Professor Frank Doyle
Professor Duane Sears
Professor Andrew R. Teel, Committee Chair
December 2004
Exploiting Immune Response Dynamics in HIV Therapy
c 2004
Copyright by
Ryan Mark Zurakowski
iii
And God said ”Let us make man in our own image, in our likeness”
-Genesis 1:26
Every study, rightly understood, is the study of God.
Soli Deo Gloria
iv
Acknowledgements
For their critical support, I would like to thank:
my parents, Mark and Therese, for their unwavering love, guidance, moral support, and prayer, and especially my dad, for giving me my first taste of grad
school at age 2.
my grandparents, for their love and prayers, and for keeping me in computers
throughout college.
Judy McAuliffe, whose excellent AP Biology class allowed me to jump right into
immunology almost ten years later.
my advisor, Andrew Teel, for refusing to let me settle into a research project I
wasn’t interested in, encouraging me to find something I was passionate about,
and supporting me even when it turned out to be very odd for an Electrical Engineer.
Sarah Holte, for her excellent SIAM conference presentation on STIs and HIV
modeling, which got me thinking.
Dominik Wodarz, for encouraging me to follow through on my initial ideas about
control and HIV modeling, and for being patient with me as I learned immunology.
Duane Sears, for teaching me immunology.
Frank Doyle, Mustafa Khammash, and Joao Hespanha, whose various forays into
the combined world of control systems and biology convinced me that such ideas
had potential.
my fellow CCEC students, for creating an excellent and dynamic working environment, and especially the MPC crowd (Michael Messina, Emre Tuna, Gene
Grimm, Chris Kellett), for patiently explaining the subtleties of MPC to me.
v
Curriculum Vitæ
Ryan Mark Zurakowski
Education
2004
Doctor of Philosophy in Electrical and Computer Engineering University of California, Santa Barbara
2000
Master of Science in Electrical and Computer Engineering
University of California, Santa Barbara
1999
Bachelor of Science in Electrical and Computer Engineering
University of California, Santa Barbara, Highest Honors
Professional Employment
2000-2004
Research Assistant, U.C. Santa Barbara
Fall 2002
Teaching Assistant, ECE 130A, U.C. Santa Barbara
Fall 2001
Grader, ECE 210A, U.C. Santa Barbara
2000
Engineering Intern, Agilent Technologies Santa Rosa, CA
1996-1999
Engineering Intern, Hewlett-Packard Santa Rosa, CA
Publications
Enhancing Immune Response to HIV Infection Using MPC-Based Treatment
Scheduling. Ryan Zurakowski and Andrew R. Teel, Proc. American Control
Conference, 2003
Utilizing Alternate Target Cells in Treating HIV Infection Through Scheduled
Treatment Interruptions. Ryan Zurakowski, Andrew R. Teel, and Dominik Wodarz, Proc. American Control Conference, 2004
vi
HIV treatment scheduling via robust nonlinear model predictive control. Ryan
Zurakowski, Michael J. Messina, Sezai E. Tuna, and Andrew R. Teel, Asian
Control Conference, 2004.
Treatment scheduling for HIV using nonlinear model predictive control. Ryan
Zurakowski, Michael J. Messina, Sezai E. Tuna and Andrew R. Teel Australian
Journal of Electrical and Electronics Engineering, 2004 (to appear).
A model predictive control based scheduling method for HIV therapy. Ryan
Zurakowski and Andrew R. Teel J. Theoretical Biology (submitted)
Honors and Awards
2000-2004
Regents’ Special Fellowship, UC Santa Barbara
1999
Eta Kappa Nu
1998
Tektronix Fellowship Award
1998
Glen Culler Scholarship
1997
Tau Beta Pi
1996-2000
Regents Scholarship, UC Santa Barbara
1996-1999
National Merit Scholarship
1996
Hewlett-Packard Employees Scholarship
1996
A.P. Scholar
Professional Societies
Society for Industrial and Applied Mathematics
Institute for Electrical and Electronics Engineering
vii
Abstract
Exploiting Immune Response Dynamics in HIV Therapy
by
Ryan Mark Zurakowski
The Human Immunodeficiency Virus (HIV) infects cells involved in the regulation of the adaptive immune response. Untreated infection usually leads to
severe immunodeficiency and death from opportunistic infections. However, recent experimental work has shown that the natural immune response to HIV
can be enhanced through the use of schedules of interrupted therapy. In some
cases, this enhancement is sufficient to induce a persistent, drug-free state in
which the virus is controlled to low levels and the patient exhibits no progressive
deterioration of the immune system.
Clinical trials attempting to use interrupted treatment schedules to exploit
this phenomenon have had mixed results. Recent modeling work that explains
the behavior shows that the immune response to HIV is governed by nonlinear
dynamics that are sensitive to small parameter variations. It is unlikely, therefore,
that a single schedule of interrupted treatment would yield consistent results
across a number of patients.
In order to successfully induce immune-mediated control of HIV infection
with a greater degree of success, it will likely be necessary to use a method of
calculating interruption schedules that will accommodate variations unique to
the patient based on measurements made during treatment. We introduce a
viii
framework based on nonlinear Model Predictive Control to determine appropriate interruption schedules for exploiting the natural immune response to HIV.
This framework incorporates models of the immune response, and uses feedback
measurements to provide robustness to the modeling and measurement errors
inherent in the application. The method is designed to be implementable in a
clinical setting.
ix
Contents
Contents
x
List of Tables
1
List of Figures
2
1 Introduction
4
1.1
Organizational Outline . . . . . . . . . . . . . . . . . . . . . . . .
6
1.1.1
Chapter 2: Biology of HIV . . . . . . . . . . . . . . . . . .
6
1.1.2
Chapter 3: Modeling HIV . . . . . . . . . . . . . . . . . .
7
1.1.3
Chapter 4: Model Predictive Control . . . . . . . . . . . .
7
1.1.4
Chapter 5: MPC-Based Treatment Scheduling . . . . . . .
8
1.1.5
Chapter 6: Future Work . . . . . . . . . . . . . . . . . . .
8
1.1.6
Chapter 7: Conclusions . . . . . . . . . . . . . . . . . . . .
9
2 Biology of HIV
2.1
2.2
10
Components of the Adaptive Immune System . . . . . . . . . . .
11
2.1.1
Antigen-Presenting Cells . . . . . . . . . . . . . . . . . . .
12
2.1.2
T Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
2.1.3
Helper-T Cells . . . . . . . . . . . . . . . . . . . . . . . . .
14
2.1.4
Cytotoxic-T Cells . . . . . . . . . . . . . . . . . . . . . . .
16
2.1.5
B Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
HIV Infection . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
2.2.1
20
HIV Structure and Function . . . . . . . . . . . . . . . . .
x
2.3
2.4
2.2.2
Target Cells . . . . . . . . . . . . . . . . . . . . . . . . . .
21
2.2.3
HIV Disease Progression . . . . . . . . . . . . . . . . . . .
21
2.2.4
Long-Term Non-Progressors . . . . . . . . . . . . . . . . .
22
HIV Therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
2.3.1
Multidrug Therapies (HAART) . . . . . . . . . . . . . . .
23
2.3.2
Nucleoside Reverse Transcriptase Inhibitors . . . . . . . .
24
2.3.3
Non-Nucleoside Reverse Transcriptase Inhibitors . . . . . .
25
2.3.4
Protease Inhibitors . . . . . . . . . . . . . . . . . . . . . .
25
2.3.5
Other Therapeutic Agents . . . . . . . . . . . . . . . . . .
26
2.3.6
Treatment Interruptions . . . . . . . . . . . . . . . . . . .
27
2.3.7
Vaccines . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
HIV Measurements . . . . . . . . . . . . . . . . . . . . . . . . . .
30
2.4.1
Antibody Tests . . . . . . . . . . . . . . . . . . . . . . . .
30
2.4.2
Viral Load Tests . . . . . . . . . . . . . . . . . . . . . . .
31
2.4.3
Flow Cytometry . . . . . . . . . . . . . . . . . . . . . . . .
31
3 Modeling HIV
33
3.1
Historical Approaches . . . . . . . . . . . . . . . . . . . . . . . . .
33
3.2
Measuring Virus Dynamics . . . . . . . . . . . . . . . . . . . . . .
34
3.3
Cytotoxic-T Cell Dynamics . . . . . . . . . . . . . . . . . . . . .
36
4 Model Predictive Control
43
4.1
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
4.2
Robust MPC without CLF terminal cost . . . . . . . . . . . . . .
45
4.2.1
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
4.2.2
Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . .
46
5 MPC-Based Treatment Scheduling
50
5.1
Control Theory and Treatment . . . . . . . . . . . . . . . . . . .
50
5.2
Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
5.3
MPC and Treatment . . . . . . . . . . . . . . . . . . . . . . . . .
59
5.4
Treatment Scheduling Algorithm . . . . . . . . . . . . . . . . . .
59
xi
5.5
5.6
5.7
Implementation and Simulation . . . . . . . . . . . . . . . . . . .
62
5.5.1
Varying the Cost Function . . . . . . . . . . . . . . . . . .
63
Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
5.6.1
Robustness: Measurement Noise . . . . . . . . . . . . . . .
65
5.6.2
Robustness: Modeling Error . . . . . . . . . . . . . . . . .
67
5.6.3
Comparison with Open-Loop . . . . . . . . . . . . . . . .
68
5.6.4
Robustness: Combined Errors . . . . . . . . . . . . . . . .
70
Additional Targets . . . . . . . . . . . . . . . . . . . . . . . . . .
72
6 Future Work
6.1
76
Model Variations . . . . . . . . . . . . . . . . . . . . . . . . . . .
76
6.1.1
Alternative Formulations . . . . . . . . . . . . . . . . . . .
77
6.1.2
Additional Therapeutic Agents . . . . . . . . . . . . . . .
78
6.2
Output Feedback Design . . . . . . . . . . . . . . . . . . . . . . .
79
6.3
Addressing Escape . . . . . . . . . . . . . . . . . . . . . . . . . .
80
7 Conclusions
83
Bibliography
86
A Selected MATLAB Code
109
A.1 optimal sequence.m . . . . . . . . . . . . . . . . . . . . . . . . . . 109
A.2 optimal sequence 2u.m . . . . . . . . . . . . . . . . . . . . . . . . 117
A.3 errorpronemc.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
A.4 mixederror.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
xii
List of Tables
5.1
5.2
Parameter Values These are the parameter values used in our
implementation of the MPC feedback algorithm. With these values, calculation of one finite-horizon optimization took approximately three minutes. . . . . . . . . . . . . . . . . . . . . . . . . .
62
Robustness Comparison We compared performance of an openloop strategy adapted from [75] with the performance of our MPCbased closed-loop algorithm at various levels of modeling error. . .
70
1
List of Figures
3.1
5.1
5.2
5.3
5.4
Two Steady States The initial condition is the same for both
figures, but in the figure on the left three weeks of treatment are
applied at the outset. A persistent CTL response develops, and
controls the viral load to low levels. In the figure on the right,
The CTL response is not established, and the virus dominates.
Parameters are as in [134]. . . . . . . . . . . . . . . . . . . . . .
39
Changed Cost Function The stage cost in the first plot has
weights with values α1 = 1, α2 = 1, α3 = 1, the second has
weights with values α1 = 100, α2 = 0.5, α3 = 0.1. Both algorithms
stabilize the desired steady state, but the second does so while
maintaining a higher average level of healthy helper-T cells, at the
cost of slower convergence and longer total anti-retroviral use. . .
64
Random Measurement Error Noise of up to 100% of the value
of each state was introduced into each measurement. The algorithm successfully stabilized the desired steady in every case,
though the state took longer to converge. The plots above show
representative outputs for up to 10%, 25%, 50%, and 100% random
measurement error respectively. . . . . . . . . . . . . . . . . . . .
66
Random Modeling Error Random variations of up to 30% of
the value of each parameter are introduced to the system, while
the algorithm uses a nominal model. The above plots are representative for, from the upper left, the cases of 5%,10%, 15%, 20%,
25%, and 30% error respectively. . . . . . . . . . . . . . . . . . . .
69
Mixed Random Error The values of the initial conditions are
varied randomly. Random variations of up to 10% of the value
of each parameter are introduced to the system as in Figure 5.3.
Random measurement noise of up to 10% of each state is added
as in Figure 5.2. The treatment scheduling algorithm successfully
stabilized the desired steady-state in every case. . . . . . . . . .
71
2
5.5
Additional, non-HIV Specific Targets These graphs show the
improved performance in the Cytotoxic Lymphocyte response to
HIV in the case where additional target cells were introduced. The
top row of plots compare responses between the case with and
without additional target cells with one-week treatment intervals
and ν = 0.3636. The bottom row of plots compare responses for
optimized combinations of fixed numbers of additional targets and
interruptions lengths (T = 0.4545, ν = 3.6364 and T = 0.5909,ν =
0, respectively). The left column compares CTL responses, the
right column compares helper-T responses. . . . . . . . . . . . .
3
75
Chapter 1
Introduction
The integration of control theory and medicine is a very natural and potentially useful idea. Many diseases, including diabetes types 1 and 2, various autoimmune disorders and chronic viral infections exhibit patterns of progression that
are fundamentally dynamic in character, and their treatment requires the use of
feedback strategies. Indeed, the successful treatment of diabetes in particular
has relied on simple feedback heuristics; however, in this and other diseases, preliminary work has shown that a more systematic, model-driven approach yields
better and more consistent results.
In this dissertation, we discuss the application of mathematical modeling
and control theory to the problem of designing therapies for the management
of HIV infection. HIV infection, and the associated disease AIDS, is a significant
and growing problem worldwide. No cure exists, and current treatments are
unsatisfactory for a number of reasons, including cost, severe side-effects, and
the eventual emergence of resistance. To combat these problems, the medical
community is actively researching both new therapeutic agents and new ways of
using existing therapeutic agents.
4
In the second category, one proposal that we found particularly interesting
was the notion that patterns of interrupted treatment with anti-viral agents could
”auto-immunize” patients, boosting their adaptive immune system’s response to
HIV to the point where it was capable of maintaining the virus at low levels without the need for continued use of the anti-viral agents. Supporting this hypothesis
were a number of anecdotes involving patients with records of poor adherence developing a non-progressive state of infection. Follow-up studies verified that this
was indeed possible, at least in patients who had begun anti-viral therapy during the acute phase of HIV infection (which likely preserved their HIV-specific
immune response). However, even among patients who began therapy early, the
success rates of the interruption schedules, which were either ad hoc open loop
schedules or simple closed-loop strategies, were never very good. Compounded
with the fact that the induced immunity was not very durable, these results have
significantly dampened enthusiasm for this technique.
Mathematical modeling of the interactions of the immune system and the
HIV virus explained the existence of treatment interruption schedules that could
induce immune-mediated control of the virus. The various outcomes were shown
to exist as a consequence of the dependence of a successful immune response on
a healthy population of helper-T cells, and the nonlinear dynamics that govern
this dependence and the loss of such cells from attack by the HIV virus. These
outcomes of these same models are quite sensitive to changes in their various
parameters; it is our hypothesis that the poor success rates seen in the various
experiments are due to a corresponding sensitivity in the actual immune response
to HIV, and that a model-based, closed-loop method of determining interruption
schedules could provide robustness to these patient variations and dramatically
improve success rates.
5
Accordingly, we have developed a Model Predictive Control (MPC) based
framework for determining appropriate schedules, and demonstrated its robustness properties through simulation. This dissertation outlines the biological,
mathematical, and control systems background necessary for the development
of this framework, introduces the framework, shows its robustness properties
through various simulations, and discusses the significance of this work and future research necessary to implement it. The organization of the dissertation is
outlined below.
1.1
Organizational Outline
1.1.1
Chapter 2: Biology of HIV
In Chapter 2 we summarize the aspects of the human immune system that are
important in HIV infection. This treatment is deliberately restricted to a highlevel, conceptual understanding of the immune system, appropriate to a modeling
and control application. In Section 2.1, we describe the various cell types involved
in an adaptive immune response, as well as their development and interactions.
In Section 2.2, we describe the structure and function of the Human Immunodeficiency virus, its target cells and their significance, the normal progression of HIV
disease, and the characteristics of HIV infection in those patients whose disease
naturally does not progress. In Section 2.3 we discuss the various approaches
to treating HIV infection, including the rationale for multi-drug treatment, the
mechanisms of action of various types of currently used therapeutic agents, a
summary of experimental therapeutic agents, the rationale for and history of the
use of treatment interruption schedules, and a brief discussion of the search for
6
preventative and therapeutic vaccines. In Section 2.4, we briefly discuss the various measurements used for diagnosing HIV infection and tracking the progression
of the disease, taking special interest in the accuracy of the various techniques
and their appropriateness for use in a feedback scheduling technique.
1.1.2
Chapter 3: Modeling HIV
In Chapter 3, we discuss the various approaches taken toward modeling the
immune system in HIV infection. In Section 3.1, we discuss various early approaches to modeling HIV infection. In Section 3.2 we discuss the pivotal modeling work done by Alan Perelson and David Ho in estimating various key rates
in HIV infection, and the importance of this work in HIV treatment. In Section 3.3, we spend considerably more time discussing the development of the
Wodarz-Nowak models of HIV infection, the various insights gained from their
use, and their applicability to the problem of designing feedback-control based
treatment schedules.
1.1.3
Chapter 4: Model Predictive Control
In Chapter 4, we introduce the concept of Model Predictive Control. We
briefly discuss various implementations of MPC. We then show the requirements
for a robust nonlinear MPC formulation that admits the use of an observer, as
developed by Grimm et al., as adapted for our uses described in Chapter 5.
7
1.1.4
Chapter 5: MPC-Based Treatment Scheduling
In Chapter 5, we develop a Model Predictive Control-based framework for
determining appropriate treatment interruption schedules for HIV therapy. In
Section 5.1, we discuss the applicability of control theory to the problem of HIV
treatment, and discuss various historical approaches. In Section 5.2, we analyze the assumptions inherent in the Wodarz-Nowak model and evaluate their
appropriateness in the context of feedback control. In Section 5.3, we discuss
the specific applicability and unique appropriateness of Model Predictive Control
theory to the problem of developing appropriate closed-loop algorithms for determining treatment interruption schedules. In Section 5.4, we show the details of
our MPC-based treatment scheduling framework as implemented in MATLAB.
In Section 5.5, we discuss implementation issues and discuss the stabilization
achieved using the feedback MPC. In Section 5.6, we discuss the robustness of
our MPC-based method to modeling and measurement errors, and compare the
experimental results with an open-loop schedule adapted from the clinical literature. In Section 5.7, we discuss a variation of the algorithm which we use to
explore the use of immuno-therapeutic drugs such as interleukin therapy, which
increase the number of HIV target cells.
1.1.5
Chapter 6: Future Work
In Chapter 6, we discuss various directions for future research, specifically
those areas that we believe will be necessary for the implementation of the framework introduced in Chapter 5. In Section 6.1, we discuss various changes in the
model of HIV infection whose study could yield useful insights. In particular, we
consider incorporating various additional states to more accurately model the im8
mune system’s behavior, and incorporating various additional controls to model
the action of additional therapeutic agents. Work investigating the use of nonspecific immune stimulants is described in detail. In Section 6.2, we discuss the
problem of adapting the framework described in Chapter 5 to using sampled measurements of only a limited subset of the states; that is, designing an appropriate
observer. We discuss a few existing methods that might be adapted for use. In
Section 6.3, we discuss the problem of the limited durability of immune-mediated
control of HIV, and discuss various possible ways to improve the durability by
preventing viral escape.
1.1.6
Chapter 7: Conclusions
In Chapter 7, we summarize our contributions and evaluate their potential.
We discuss the applicability of control theory in HIV treatment, and discuss its
potential future impact.
9
Chapter 2
Biology of HIV
Acquired Immunodeficiency Syndrome (AIDS) is a disease of the immune system, caused by the Human Immunodeficiency Virus (HIV). The human immune
system is a complex, highly regulated system whose function is to prevent disease
caused by foreign agents. The immune system is usually divided into two compartments, called innate and adaptive. The innate immune system is composed of
physiological barriers that prevent the invasion of foreign agents, and nonspecific
responses that neutralize certain classes of foreign agents. The innate immune
response is largely unaffected by HIV infection. The adaptive immune response
is mediated by a complex network of specialized cells that identify and respond
to foreign invaders. It is called adaptive due to the fact that it can respond with
great specificity to a very broad class of foreign substances, and exhibits memory,
so that subsequent re-challenge results in a powerful, immediate response. HIV
primarily infects helper-T cells, which are a key component of the adaptive immune system. Untreated HIV infection usually leads to a progressive breakdown
in the functionality of the adaptive immune system, which eventually leaves the
patient vulnerable to any number of infectious agents. Disease caused by these
10
so-called opportunistic infections eventually leads to death.
In order to understand the way in which HIV infection leads to infection, it
is necessary to understand the interactions of the components of the adaptive
immune system. In the first part of this chapter, we discuss the various components of the adaptive immune system and their interactions. We then discuss
the biology of the HIV virus and the mechanisms of its reproduction in target
cells. We then discuss the measurements used to track the course of an HIV infection. Finally, we discuss the current methods of treatment for HIV infection,
and the drugs used. Our focus will be exclusive to those parts of the immune
system that are relevant to an understanding of the progression and response
to HIV infection; as such, we deliberately neglect details that are not necessary
for an understanding of the immune response as a dynamic system. For a more
complete study of the immune system and its functioning see [45] or [76].
2.1
Components of the Adaptive Immune System
The adaptive immune system is so named because of its ability to respond to
a remarkably broad class of foreign antigens. Unlike the innate immune response,
which responds to subsequent challenges by foreign antigen with responses of similar strength to that of an initial infection, the innate immune response exhibits
“memory”; it responds much more strongly to subsequent challenges by the same
foreign antigen, resulting in what is called acquired immunity.
The two major classes of cells in the innate immune system are T-cells and
B-cells. The T-cells can be further divided into helper-T cells class I and II, and
11
cytotoxic-T cells. T-cells are characterized by the presence of the T Cell Receptor,
a highly diversified molecule that allows the T-cells to scan molecules presented on
cells for foreign origin. This provides a mechanism for surveillance of intracellular
pathogens. B-cells produce antibodies, another highly diversified molecule that
can bind to and neutralize extra-cellular pathogens. In the following sections we
will elaborate on the specific roles played by these various cell types and their
interactions.
2.1.1
Antigen-Presenting Cells
Nearly every cell in the human body displays surface proteins called Major
Histocompatability Complex class I (MHC-I). These molecules are common to all
the cells in a person, but are highly diverse in the population at large. During the
natural process of protein degradation, small peptides (protein fragments) form
complexes with MHC-I and are transported to the surface of the cell. Through
this process, called the endogenous pathway (because it presents endogenous
antigen), every cell displays a sampling of its internal proteins on its surface.
On a healthy cell, these protein fragments will all be native to the cell, but on
an infected cell, they will become dominated by fragments of proteins from the
infectious agent.
While nearly every cell displays MHC-I proteins, a similar type of protein,
the Major Histocompatability Complex class II (MHC-II) is only displayed by socalled Antigen Presenting Cells (APC). This class of cells include dendritic cells
and macrophages (B-cells also express MHC-II, but to a fundamentally different
purpose, and will be discussed separately). Like MHC-I proteins, MHC-II also
forms complexes with potentially antigenic peptides, but the source of the pep-
12
tides differ. MHC-II binds to peptides formed by the digestion of extra-cellular
agents, introduced either through endocytosis or phagocytosis. This provides a
method for the surveillance of the extra-cellular space. Also, certain APCs such
as macrophages especially seek out cells undergoing apoptosis to phagocytose. Intracellular infection, either by a virus or an intracellular bacterium, will usually
accelerate this process, resulting in a large number of infection-specific peptides
being displayed with MHC-II on the surface of the APCs.
Normally, APCs present exogenous antigen bound to MHC-II. However, if
dendritic cells are “primed” by interacting with helper-T cells that recognize
the antigen bound to the MHC-II on the cell’s surface, it also begins to display
exogenous antigen bound to MHC-I on its surface. This allows the dendritic cell to
interact with cytotoxic-T precursor cells, facilitating their expansion into mature
cytotoxic-T cells [123]. Macrophages are also activated by interaction with helperT cells, resulting in an increase in the production of MHC-II and inflammatory
cytokines, as well as an increase in phagocytic activity. It is unclear whether
macrophages also display exogenous antigen bound to MHC-I [140],[86]. While
dendritic cells are usually primed through interaction with helper-T cells, there is
significant evidence that infections that cause an inflammatory response, through
extensive tissue damage or other means, can cause dendritic cells to become
primed without interacting with helper-T cells, possible through interactions with
natural killer (NK) cells or direct antigen interactions [14],[23].
2.1.2
T Cells
T-cells, so called because they mature in the thymus, are critical cells in the
development of an adaptive immune response. T-cells are characterized by a
13
broad specificity to foreign antigen (primarily foreign peptides) and a near perfect insensitivity, or tolerance, to self. During T-cell development, a number of
random rearrangements and deletions occur in the portion of the genome responsible for creating the T-cell Receptor (TCR) protein. This results in every
immature T-cell expressing a unique TCR surface protein, providing an enormous range of specificity (rearrangements alone can account for 3 ∗ 106 unique
TCRs, other processes raise this number as high as 1013 possible unique TCR).
The immature T cells then undergo two selection processes: All cells that express
TCR that do not bind with sufficient strength to MHC molecules are killed (as
they would be incapable of recognizing MHC bound antigen), and all cells that
express TCR that bind too strongly to MHC molecules binding self-antigen are
also killed (ensuring self-tolerance in the surviving cells). The only T-cells that
survive this process are those whose TCR proteins are capable of recognizing
MHC molecules with bound peptide fragments, but do not bind strongly to any
peptide fragments occurring naturally in uninfected cells. During this process,
the lineage of the T-cells is also determined; either they become helper-T cells,
expressing the surface molecule CD4 and TCR that bind with MHC-II, or they
become cytotoxic-T cells, expressing the surface molecule CD8 and TCR that
bind to MHC-I.
2.1.3
Helper-T Cells
Helper-T cells do not directly mediate an adaptive immune response; instead,
they regulate, or “help” the development of the humoral (B-cell mediated) or
cellular (T-cell mediated) immune responses. Naive helper-T cells are long-lived
cells that remain dormant until activated through interaction with an APC displaying the MHC-II/antigen complex for which the the helper-T cell is specific.
14
When activated, the naive helper-T cell enters the cell cycle, and produces a large
number of identical clones of itself. These clones, based on co-stimulatory signals
from the APC and other cells (especially other helper-T cells), differentiate into
either helper-T 1 or helper-T 2 cells. Helper-T 1 cells are primarily involved in
promoting the cytotoxic-T cell response through “priming” of APCs displaying
the MHC-II/antigen complex specific to the helper-T cell and production of cytokines necessary for the expansion and activation of cytotoxic-T cells. Helper-T 2
cells are primarily involved in promoting the humoral immune response through
direct interaction with B-cells displaying the MHC-II/antigen complex specific
to the helper-T cell and production of cytokines that act as promoters to B-cell
clonal expansion and differentiation. An infection usually produces significant
numbers of both kinds of helper-T cells; however, certain kinds of infections will
favor one class over the other, and the dominance can switch over time.
Two signals are necessary for the activation of a naive or memory helperT cell. The first signal is provided by the complex formed by the TCR and
CD4 molecules on the T-cell and the MHC-II/antigen complex on the APC.
The second, co-stimulatory signal comes from the interaction of either CD28
or CTLA-4 (two closely related molecules expressed on the surface of helper-T
cells) with B7 molecules on the surface of the APC. The formation of the TCRCD4-MHC-II complex without the CD28-B7 co-stimulatory signal results in the
T-cell becoming anergic, permanently frozen in a resting state. This provides
an additional mechanism to prevent the development of auto-immune responses,
called peripheral tolerance. When the correct combination of signals is presented
to a helper-T cell, it becomes activated, entering the cell cycle and dividing
a number of times, producing a large number of daughter cells. All of these
daughter cells express the same TCR molecule, specific to the antigen that caused
15
activation. Cell division begins approximately 2 days after activation, and the
daughter cells will divide 2-3 times a day for up to 5 days, producing as many
as 104 daughter cells [14]. These cells either become effector cells, terminally
differentiated with a short life-span, or memory cells, similar in behavior to naive
helper-T cells, though with less stringent activation requirements [138].
2.1.4
Cytotoxic-T Cells
Cytotoxic-T cells (CTL), like helper-T cells, develop in the thymus, but after
differentiation, cytotoxic-T cells express the surface protein CD8, and their TCRCD8 complex is restricted to interactions with antigen-MHC-I complexes. Like
helper-T cells, naive cytotoxic-T cells are long-lived, and remain dormant until
interacting with an APC displaying the antigen-MHC-I complex for which the
cell’s unique TCR is specific. The co-stimulatory molecule B7 is also necessary to
activate the naive cytotoxic-T cell, and cellular anergy results when it is absent.
When the correct combination of signals is present, the naive cytotoxic-T cell becomes activated. It enters the cell cycle, producing as many as 104 daughter cells
within one week [14]. During this expansion period, signals provided by helper-T
cells condition the expanding clones to be able to revert to a functional memory
pool; however, the exact nature of this help is not known [58]. When the activated
cytotoxic-T cells interact with a cell presenting the antigen-MHC-I complex for
which the CTL is specific, it binds to the cell and kills it, either through the introduction of perforin and protease complexes, which destroy the cellular proteins,
or by the induction of programmed cell death (apoptosis) through interaction
of Fas-ligand on the surface of the CTL with Fas on the surface of the infected
cell (Fas-Fas-ligand interactions are a common mechanism for inducing apoptosis in many types of nucleated cells). As the antigenic stimulation is reduced,
16
the expanded population of CTL clones begins to reduce through apoptosis, with
roughly 10% of the clones remaining as a memory pool [125]. If helper-T cell help
was available during the expansion, these memory cells are phenotypically similar to naive CTL, capable of further rounds of clonal expansion[77]. If helper-T
cells were absent during the clonal expansion, these memory cells are incapable
of further expansion [58]. There is also evidence that the cytotoxic function of
cytotoxic-T cells that develop in the absence of helper-T cells is impaired [141],
[62].
2.1.5
B Cells
B-cells are so called because they were first characterized in birds, where they
develop in the Bursa of Fabricius. In humans, they appear to undergo maturation in the red bone marrow, so the name still fits. B cells are primarily involved
in the production of antibodies, proteins that bind with extreme specificity to a
variety of extra-cellular antigens. Like the genes that produce T-cell receptors in
T cells, the genes responsible for the production of antibodies undergo a series
of random rearrangements during the B-cells development. The total potential
diversity of antibodies is on the order of 1010 . Unlike T-cell receptors, which are
capable of binding only to peptide fragments bound to MHC molecules, antibodies are capable of binding to a large class of antibodies, including whole proteins,
polysaccharides and lipids.
During the process of maturation, B-cells producing antibody that binds to
host tissue are eliminated. Those B-cells that survive maturation enter the blood
and lymph tissues as naive B-cells. At this stage, the B-cell does not secrete
antibody, but expresses its characteristic antibody on the cell surface. When
17
an antigen binds to the B-cell’s membrane-bound antibody, the B-cell engulfs
the antigen, digests it, and processes it for presentation in MHC-II molecules on
its surface. If these MHC-II-antigen complexes are then recognized by activated
helper-T 2 cells expressing a complimentary TCR, the B-cell is activated. Certain
types of antigen, such as polysaccharides and lipids, bypass the necessity of T-cell
help for activation if they are present in sufficient concentration.
The activated B-cell undergoes a number of rounds of division, producing
a large number of daughter cells that produce antibodies with identical affinity. During this expansion, the incidence of random mutation in the antibodyproducing genes is increased. This process, called affinity maturation, results,
through selection processes, in daughter cells that produce antibodies with higher
binding affinities to the specific antigen in question. Further interactions with
helper-T cells and other surrounding tissues can also cause the B-cells to undergo
class-switching. There are five classes of antibody that B-cells can produce; IgM,
IgG, IgA, IgD, and IgE; each of which has different chemical structure in their
invariant region (the portion of the molecule that does not affect the antibodies
antigen specificity). The various antibody classes differ primarily with respect
to the tissue types where they are most effective; for example, IgA is primarily
found in mucosal secretions, IgM primarily in the blood, while IgG is found in a
variety of tissues, and even crosses the placenta, providing passive immunity to
the developing fetus.
The daughter cells either become memory cells or plasma cells. The plasma
cells are highly specialized cells with extensive internal structure that allows
them to produce incredible amounts of secreted antibody (as many as 2000 antibody molecules per cell per second). These secreted antibodies circulate in
blood and other tissue, and bind and neutralize antigen. Most plasma cells are
18
relatively short-lived, with a half-life of a few days [95], although small numbers of plasma cells are long-lived and can persist in bone marrow for years [79].
Memory B-cells, by contrast, are extremely long-lived; after an initial, apparently density-dependent contraction, numbers of antigen-specific memory B-cells
remain relatively constant for over 50 years [25]. In the absence of evidence for
memory B-cell turnover, this suggests a correspondingly long half-life. Memory B-cells, like naive B-cells, require additional T-helper 2 interactions in order
to expand and differentiate [95]. Memory B-cells, unlike naive B-cells, express
the co-stimulatory molecule B7, which allows them to activate naive or memory helper-T cells. The increased population of antigen-specific memory B-cells
(as compared to the population of naive B-cells) allows for a much stronger and
faster response to a second introduction of antigen, and is the primary factor in
acquired immunity.
2.2
HIV Infection
The Human Immunodeficiency Virus (HIV) is the causative agent of Acquired
Immunodeficiency Syndrome (AIDS), a disease that affected approximately 38
million people at the end of 2003 [137]. In most cases, untreated HIV infection
results in a slow, largely asymptomatic decline in immune function that culminates in a collapse of the hosts ability to control and eliminate infections, leading
to death from one of a number of secondary infections. It is primarily transmitted
through sexual contact and/or fluid sharing in intravenous drug use.
19
2.2.1
HIV Structure and Function
HIV is a retrovirus; its genome consists of RNA rather than DNA. A single
HIV virion consists of a lipid bilayer envelope enclosing a protein capsid, inside
of which are two identical RNA strands and the protein reverse transcriptase.
The surface of the envelope is covered with “spikes” consisting of the proteins
gp120 and gp41. During infection the gp120 binds to a CD4 molecule on the
surface of the target cell, and also to a co-receptor. This co-receptor can be the
molecule CCR5, primarily found on the surface of macrophages and helper-T
cells, or the the molecule CXCR4, primarily found on the surface of helper-T
cells. Different strains of HIV successfully bind to different co-receptors [90],
[35]. When the gp120 molecule has successfully bound to CD4 and a co-receptor,
the gp41 molecule pierces the target cell membrane, resulting in the fusion of
the viral envelope and cellular membrane. This delivers the viral capsid to the
cytoplasm, where it dissolves, revealing the viral RNA. At this point, the reverse
transcriptase protein makes a DNA copy of the viral genome, which is integrated
into the target cell’s genome using the target cell’s cellular repair mechanisms [70].
The virus then uses the host cell’s transcription machinery in order to produce
RNA transcripts, a process facilitated by the production of certain viral proteins.
The RNA transcripts are used to produce the viral proteins, many of which
are first synthesized as polyproteins. These large polypeptide precursors consist
of two or more functional proteins strung together into a larger, nonfunctional
protein. In order to assemble a functional virus, these proteins must be cleaved
into their functional parts by a protease region of one of these polyproteins. The
separated functional proteins self-assemble, and new virions bud off the host cell’s
membrane.
20
2.2.2
Target Cells
The two cell types primarily infected with HIV are macrophages and helper-T
cells, both of which express the surface protein CD4. HIV infection in macrophages is less cytotoxic than in helper-T cells, and has slower viral production
rates [133],[37]. As a consequence, these cells can serve as a reservoir for virus,
facilitating initial infection and making clearance through the use of suppressive
drugs difficult. By contrast, productively infected helper-T cells have a high
rate of viral replication, as many as 105 virions per cell [30], and infected cells
have a half-life of only 1.6 days [107]. The mechanisms by which the infected
helper-T cells die may include damage due to viral budding, increased rates of
apoptosis due to infection, syncytium formation, and killing by natural killer and
cytotoxic-T cells [70].
2.2.3
HIV Disease Progression
In the majority of cases of untreated HIV infection, the patient undergoes
a short (2-10 weeks) period of acute infection, which may be accompanied by
symptoms similar to those found in most viral infections. During this period,
there is a sharp drop in the concentration of circulating helper-T cells, and a large
spike in the level of circulating free virus (to an average of 107 /ml). During this
period, a humoral (antibody) response and a cellular (cytotoxic-T cell) response
are established [67]. After this period, the level of circulating helper-T cells
returns to near-normal (1000 cells/ml), and the viral load drops dramatically (to
an average of about 50,000/ml). During the next phase of infection, which can last
as long as 10 years, the patient remains asymptomatic, but the level of circulating
helper-T cells slowly declines. When the number of helper-T cells drops below
21
a critical threshold ( 200/ml), the patients adaptive immune system is no longer
able to control infections, and a number of so-called opportunistic infections
cause a rapid deterioration of health and a total collapse of the adaptive immune
system. The slow rate of helper-T cell decline during the long, asymptomatic
chronic infection phase once led people to believe that the virus was relatively
inactive during this period, but it is now known that vigorous viral replication
and helper-T cell turnover occurs during this time [55], [107], [106]. In fact, the
total viral production during this phase is on the order of 1010 virions per day,
and the turnover rate of helper-T cells in on the order of 2 ∗ 109 cells per day. The
dynamics of HIV infection are obviously quite fast, but the immune response is
able to maintain a near-homeostasis for a number of years.
2.2.4
Long-Term Non-Progressors
While the majority of untreated HIV-infected patients exhibit the pattern
of disease progression described above, a small number of untreated patients
show no progressive decline in helper-T cell counts, and never develop AIDS.
These patients are termed long-term non-progressors (LTNP). These patients
exhibit extremely low viral loads, frequently below the threshold for measurement. Compared to patients with progressive infections, these patients exhibit
strong HIV-specific helper-T cell responses [116],[118],[117], [44], [91]. Levels of
HIV-specific cytotoxic-T cells are similar in both cases [114], but in the case of
LTNPs, their HIV-specific cytotoxic-T cell counts are maintained at low levels
of viral load [51], [52], [85], where patients with progressive infections see corresponding decreases in the level of HIV-specific cytotoxic-T cell activity if the
viral load is suppressed by therapy [82], [88], [61], [5] (this is also seen in animal
model experiments such as [121], [59]) . Longitudinal studies of LTNPs have
22
shown that the control of the virus sometimes fails [114], probably due to mutational escape [139]. Also, even in patients with exceptionally broad specificity
in their CTL response to HIV, escape due to super-infection has been shown to
occur [2]. Long-term non-progressors are widely studied as a model for potential
therapeutic vaccines, as they apparently have naturally developed an immune
response that successfully contains the virus.
2.3
HIV Therapy
For the first ten years in which it was known, HIV infection was almost
universally fatal. Today, however, effective therapy exists that can delay the onset
of AIDS indefinitely. This is accomplished by the use of anti-retroviral drugs,
which interfere with a number of critical steps in the course of HIV replication.
In this section we will briefly cover the available treatments, and discuss adverse
side effects.
2.3.1
Multidrug Therapies (HAART)
Prior to 1995, anti-retroviral drugs were applied one or two at a time. Work
done by Dr. David Ho and colleagues [55], which we discuss in more depth in
Chapter 3, showed that the replication and mutation rates of HIV in vivo were
so high as to make the emergence of strains resistant to any one drug inevitable.
The solution was to use three or more drugs that target biochemically separate
components of the HIV replication cycle. This multi-drug therapy regime usually
consists of two Reverse Transcriptase Inhibitors and one Protease inhibitor, and
is known as Highly Active Anti-retroviral Therapy (HAART). This technique is
23
highly effective at reducing viral load and restoring immune function [46], and
its use has drastically reduced AIDS-related deaths in the United States and
other first-world nations. However, it is not without its drawbacks. HAART
is expensive, costing as much as $10,000 per patient per year, and it must be
continued for the life of the patient. Even though it can suppress the viral load
below the measurement threshold, various viral reservoirs cause re-emergence of
the virus upon cessation of therapy, even after many years of suppression [36].
Finally, the drugs used in HAART cause a number of adverse side effects in
almost all patients, ranging from the mild to life-threatening [42], [78]. While
HAART is an effective therapy that prolongs the life of HIV-infected individuals,
the associated costs keep us searching for a better solution. An excellent review
of the available drugs, their usage, and their side effects can be found in [56].
2.3.2
Nucleoside Reverse Transcriptase Inhibitors
The first class of drug developed for use in HIV infection, nucleoside reverse
transcriptase (NRTI) mimic natural nucleosides, and are introduced into the DNA
copy of the HIV RNA during the reverse transcription event of infection. However, the NRTI are nonfunctional, and their inclusion terminates the formation of
the DNA copy. Most HAART regimens include two NRTIs, which mimic different
nucleosides. The side effects associated with NRTI use seem to be related to mitochondrial toxicity [21], and can include myelotoxicity, lactic acidosis, neuropathy,
pancreatitis, lipodystrophy, fatigue, nausea, vomiting, and diarrhea. Currently
available NRTIs include Abacavir, Zidovudine, Zalcitabine, Didanosine, Stavudine, Lamivudine, and Tenofovir.
24
2.3.3
Non-Nucleoside Reverse Transcriptase Inhibitors
The non-nucleoside reverse transcriptase inhibitors (NNRTI) also block the
creation of a DNA copy of the HIV RNA, but work by binding directly to key
sites on the reverse transcriptase molecule, blocking its action. These drugs do
not work well on their own, but in conjunction with NRTIs, they increase the
effectiveness of viral suppression. They are most frequently used as an alternative to protease inhibitors in HAART. Side-effects can include hepatoxicity,
rash, dizziness, and sleepiness depending on the drug used. Currently available
NNRTIs include nevirapine, efavirenz, and delavirdine.
2.3.4
Protease Inhibitors
Protease inhibitors (PI) target the viral enzyme protease that cuts the polyproteins into their respective components. With this step of viral replication
blocked, the infected cell produces viral particles unable to infect cells. All currently available protease inhibitors can cause lipodystrophy, a severe redistribution of body-fat that can drastically change the patients appearance. Other side
effects include gastrointestinal disorders, nephrolithiasis, dry skin, severe diarrhea, and hepatoxicity. Currently available PIs include amprenavir, indinavir,
lopinavir, nelfinavir, ritonavir, and saquinavir. Most PIs are given in combination with a low dose of ritonavir, which also blocks the primary mechanism that
clears most PIs from the body, resulting in higher plasma drug levels with fewer
pills necessary.
25
2.3.5
Other Therapeutic Agents
There is a constant effort to create new anti-viral HIV drugs that combat
either the problem of multi-drug resistance (MDR) or the various side-effects.
In addition to the three existing types of medications, there are some promising
new approaches to the problem of inhibiting HIV replication, summarized in [56].
These include entry inhibitors, integrase inhibitors, and immunotherapy.
Entry inhibitors fall into three classes. Attachment inhibitors attempt to
disrupt the infection of helper-T cells by binding with the gp120 molecules on
the surface of HIV. These currently show significant promise, although resistance
may be a problem. Co-receptor antagonists attempt to disrupt the infection of
helper-T cells by blocking the co-receptor molecules on the target cells; current
studies are promising, although the shifts observed in co-receptor tropism during
normal HIV infection indicate that resistance will likely be a problem. Fusion
inhibitors bind to the gp41 protein, which becomes exposed after attachment but
before viral entry. These drugs have shown very promising results in clinical trials,
with low incidences of relatively mild side-effects, but they are large molecules
that must be given through injection or infusion, which limits their usefulness.
Integrase inhibitors attempt to disrupt the viral replication cycle by preventing the integration of the viral DNA provirus into the host DNA. These drugs
have not yet reached clinical trials.
Various attempts have been made at immunotherapy, where the patient’s own
immune response to HIV is therapeutically modified. Some of these methods
include Interleukin-2 therapy, where synthetically produced IL-2 (a cytokine that
stimulates the production of T-cells, among other effects) is injected into the
patient’s bloodstream. This has the effect of raising the levels of both helper
26
and cytotoxic T cells, but also has rather unpleasant side effects, similar to flu.
The side effects are probably due to excessive immune activation. The clinical
benefit of the increase in T-cell counts due to IL-2 therapy is also uncertain [56].
Similar approaches have also been tried with IL-10 and IL-12, without verified
benefit. Another approach uses hydroxyurea or Mycophenol, which suppress the
immune response. While treatment with these drugs does seem to reduce the viral
load (probably by reducing the number of viral target cells), they also suppress
the immune response to other infections, so their potential benefit is unclear.
Therapeutic immunizations, which attempt to boost the immune response by
injecting inactivated virus into infected patients, have also been tried, but showed
no clinical benefit.
2.3.6
Treatment Interruptions
A significant amount of effort has been put into the use of interrupted schedules of HAART. Interrupting HAART has been done for a number of reasons,
usually to manage side effects or allow treatment of a secondary infection with
which the drugs in HAART would interfere (such as hepatitis A) [89], [33]. However the case of the “Berlin patient” began a series of investigations into the use
of treatment interruptions as a way of boosting the immune response to HIV, and
potentially controlling the virus [9]. In the case of the Berlin patient, HAART was
begun during acute infection, discontinued due to poor adherence, re-initiated,
discontinued again due to hepatitis-A infection, re-initiated, then discontinued
permanently, following which there was no measurable viral rebound [73]. Follow
up with this patient showed measurable virus in the lymph nodes, but no viral
rebound for several years after discontinuation of therapy. Obviously, the treatment schedule had somehow induced an immune response capable of controlling
27
the HIV infection.
Further studies on patients who initiated HAART during acute infection,
followed by a pattern of structured treatment interruptions (STI), showed varying
degrees of success in inducing at least a transient control of viral replication in the
absence of continued therapy [75], [97], [103], [119], [115]. In all of the successful
cases, viral control was associated with increased HIV-specific helper-T responses
and strong HIV-specific CTL responses that were maintained even at low viral
load, which suggested an immune response profile similar to that seen in LTNPs.
However, a follow-up study that tracked some of those patients that successfully
controlled the virus in the absence of continued therapy showed a disappointing
lack of durability in the immune response; among 14 patients who successfully
controlled viral replication for up to 3 years following cessation of therapy, all
except one eventually experienced viral rebound [63].
Studies involving STI during chronic HIV infection were also done, but with
much less success at inducing even a transient control of infection [53], [27], [39],
[40], [100], [31], [27], [98], [120]. Although most of the patients in these studies
experienced transient increases in their HIV-specific cytotoxic-T cell responses,
this did not result in any long-term reduction in viral load. This might be explained by the fact that HIV preferentially infects HIV-specific helper-T cells [32],
and HIV-specific helper-T function may be eliminated or severely impaired early
during untreated HIV infection [4], [101], [110], [99], [64], [60], [13].
The use of STIs to induce efficient immune-mediated control of HIV is currently a matter of debate. The studies mentioned above indicate that there is
some significant success if therapy was initiated early in HIV infection, but little
to no success if therapy was initiated during chronic infection. The durability
of the induced control is disappointing, and the overall benefit of a short period
28
of immune control is questionable [63]. There is some concern that the use of
interruptions might encourage the emergence of multi-drug resistant strains of
HIV, but experimental studies show little evidence for this [120], [19], [22], [38],
[39], [93]. In order for this to be a viable treatment option, techniques that provide better success rates will need to be developed, and methods of delaying or
avoiding the mutational escape of the virus from immune control will need to be
explored.
2.3.7
Vaccines
As of now, no successful preventative vaccine exists for HIV. However, a
number of attempts have been made. An inactivated whole-virus vaccine, HIV
Immunogen, was in human phase II and II trials as a therapeutic vaccine (used on
already HIV-infected patients to boost immune responsiveness). Despite boosting
HIV-specific helper-T responses [113], this vaccine failed to show any significant
effect on disease outcome or long-term survival. New understanding of the relative importance of cellular immune responses in controlling HIV have driven the
development of most recent vaccine candidates [69]. Traditionally, this would
involve using an live attenuated vaccine; the obvious dangers associated with
infection by attenuated HIV rule this out (this is addressed mathematically in
[16]. A number of innovative ways to induce anti-HIV cellular immune responses
without this danger are in development, and are reviewed in [69]. Most of these
involve engineered virus from stock such as canarypox or adenovirus with genes
expressing certain HIV proteins recombinantly added. A separate, innovative approach involves the in vitro loading of dendritic cells with inactivated HIV, and
reintroducing the dendritic cells into the host; in animal models this has been
an effective way of generating significant cellular immune responses to an SHIV
29
strain, which resulted in controlled infection [15]. A number of these vaccine
candidates are in various stages of development; up to date information can be
found on the website of the International AIDS Vaccine Initiative (www.iavi.org).
None of the completed trials have yet shown any significant degree of protection,
though many have been shown to significantly boost anti-HIV cellular immune
responses.
2.4
HIV Measurements
In the diagnosis and treatment of HIV infection, various types of measurements are useful. In order to use feedback control derived techniques, we need
to be able to make reasonable accurate measurements of at least a subset of the
important compartments. In this section, we will briefly discuss the currently
available measurements.
2.4.1
Antibody Tests
For initial diagnosis of HIV infection, the most commonly used methods are
antibody tests. These tests scan blood samples for the presence of antibodies that
bind to HIV-specific proteins. Most commonly, an enzyme-linked immunosorbent
assay (ELISA) test is done first, and then (if positive) confirmed with a western
blot analysis. These tests are highly accurate, with a very low rate of false positives, but they only test positive after the patient has seroconverted (developed
antibodies to HIV), an event that may not happen until late in acute infection.
The common antibody tests do not measure levels of anti-HIV antibodies present;
rather, they return a present/not present type of result.
30
2.4.2
Viral Load Tests
Modern viral load tests use polymerase chain reactions to amplify the amount
of viral RNA or proviral DNA in a blood sample, allowing the number of viruses
to be measured with great accuracy to levels as small as 50 virions/ml [109]. Even
more sensitive (but significantly more expensive) approaches have been used in
[80] to measure viral loads as low as 5 virions/ml. These measurements are highly
consistent and repeatable, and are used in conjunction with T-cell counts to track
the progression of HIV disease.
2.4.3
Flow Cytometry
The numbers of various types of cells in a sample of blood serum can be very
precisely counted by a method called flow cytometry. In this method, monoclonal
antibodies with florescent tags are created that bind to various surface proteins.
The antibodies are added to the sample, and the sample is passed one cell at a
time through a space illuminated by a laser. Counts are made of the number of
cells binding the various antibodies. The cells may also be sorted at this point,
using electrostatic forces. Since the process is usually nondestructive, the cells
can later be cultured to test for various in vitro properties.
In HIV therapy, this technique is used to obtain counts of helper-T and
cytotoxic-T cells. A key determinant of the progression of HIV disease is the
reduction of helper-T cell counts; numbers of less than 200/ml are considered to
be an AIDS defining condition. In various studies, this technique has also been
used to measure the number of activated T cells of various types (see for example
[141], [17], [54]), to measure the pools of memory T-cells of various types (see for
example [5], [13]), and, in combination with an incubation with viral peptides, to
31
count the number of HIV-specific T-cells of various types (see for example [13],
[3]).
This method is extremely extensible, but it has some important drawbacks.
First, in order to measure a particular population of cells, you must know the
distinctive set of surface markers that defines it, and be able to isolate it and
create appropriate monoclonal antibodies. The more unusual the measurement
you wish to take, the more expensive the antibodies will be. Secondly, while the
counts are highly precise, the number of T cells in the blood serum is not a very
consistent or reliable measure of the total T cell count in the body; the majority of
T cells at any given time are in the various lymphatic tissues, and factors such as
the time of day, recent exercise, or meals can cause variations of hundreds of cells
per ml in the blood serum, without reflecting any real change in the total T cell
populations. These issues must be considered when using measurements made
with flow-cytometric techniques. A good overview of existing flow cytometric
systems and available monoclonal antibodies can be found in [96].
32
Chapter 3
Modeling HIV
Mathematical modeling as a tool for understanding the progression of HIV
infection has a long history. The complex and nonlinear nature of the cell-virus
and cell-cell interactions makes modeling a necessary part of understanding the
implications of various theories of pathogenesis; simple intuition often fails. In
this chapter, we present a brief review of various models that have been developed
in order to guide the treatment of HIV infection, and spend considerable time on
the Wodarz-Nowak model of infection, which we use in the scheduling algorithms
described in Chapter 5
3.1
Historical Approaches
The use of mathematical modeling to describe virus-immune system interactions has its origins in mathematical epidemiology and population dynamics.
The use of differential equations to describe the interactions of predator and prey
in ecology, or the transmission of an infection between carriers and susceptibles,
is well established. Extensions of this concept to populations of cells and viruses
33
within an organism was an obvious and natural extension. An excellent presentation of early work in this area, with applications to viral hepatitis and cancer,
can be found in [8] (a proposed feedback control algorithm for these models is
presented in [126]). This work does an excellent job of comparing models with
and without delays, compartmentalized and non-compartmentalized, and so on.
However, it quickly becomes apparent that the complexity of these models can
easily hinder their usefulness; one of the simpler models of the immune response
to a viral hepatitis infection is a 10-state nonlinear ordinary differential equation,
which is difficult to numerically simulate, much less identify.
The emergence of AIDS and the discovery of HIV led to a number of models
that explored the unique dynamics of a virus that attacked components of the
immune system itself. Of particular interest was the phenomenon unique to HIV
infection, in which the helper-T cell population declined slowly but inexorably
over a period as long as 10 years, eventually resulting in immunodeficiency. This
slow decline was unusual among viral infections, and a number of hypothesis for
the mechanism were proposed, with mathematical models developed to rigorously
explore their implications. These early models are briefly reviewed in [106]. Unfortunately, models from many contradictory hypothesis seemed to equally well
explain the available data, and the complexity of the models did not allow for
easy experimental testing of their hypotheses.
3.2
Measuring Virus Dynamics
The first real success in the mathematical modeling of HIV used an exceptionally simple model. The basic model of virus dynamics, as described in [94],
is:
34
ẋ = λ − dx − βxv
ẏ = βxv − ay
(3.1)
v̇ = ky − uv
where x is the concentration of susceptible cells (in this case, helper-T cells), y is
the concentration of infected cells, and v is the concentration of free virus. In the
absence of free virus, the susceptible cell population converges to a steady-state
governed by their rate of production λ and their rate of death dx, with a value
of λ/d. Free virus interacts with susceptible cells with a frequency proportional
to their relative concentrations, according to the mass-action principle, and successfully infects the susceptible cells at a rate βxv. Infected cells produce free
virus at a rate ky, and die at a rate ay. Free virus degrades at a rate uv. If an
anti-retroviral drug is applied that stops the infection of susceptible cells, these
equations become
ẋ = λ − dx
ẏ = −ay
(3.2)
v̇ = ky − uv
and the rate of decay of free virus approximates either a or u, whichever is slower.
Exactly this experiment was done in [55] (further reported in [107]), and estimates
were made of the turnover rates of free virus and infected cells during chronic
HIV infection, with the stunning conclusion that on the order of 109 virions
and infected helper-T cells were being created and destroyed every day. Despite
the extremely slow net loss of helper-T cells seen in chronic HIV infection, the
actual dynamics of virus production and infected cell turnover were very fast,
35
and kept in a near-homeostasis only by the rapid regeneration of helper-T cells
and the rapid destruction of infected cells. This result was quickly verified by
[127]. The implications of these results were dramatic. It had long been noticed
that the anti-retroviral drugs quickly lost effectiveness, and the reason was now
clear; the high rate of viral replication made mutational escape from single-drug
suppressive therapy inevitable. Multi-drug “cocktail” therapies, which have come
to be known as HAART, became the norm, and for the first time HIV patients
could expect to survive with the infection for many years (Dr. David Ho was also
honored with Time Magazine’s Man of the Year in 1995).
Initially, the results from this study also gave rise to the hope that long-term
treatment with highly effective combination therapy could eventually eradicate
the virus. However, a number of follow-up studies [104], [105], [36] used the same
basic technique to obtain estimates of the decay rates of other viral reservoirs; the
measured decay rates of the slower-decaying cell compartments showed that antiretroviral therapy would have to be continued for at least 60 years to eradicate
all viral reservoirs (and possibly much longer). Various experimental studies [27],
[41], [93], [136] verified that even the most effective anti-retroviral therapy fails
to eradicate the virus.
3.3
Cytotoxic-T Cell Dynamics
The model used by Perelson et al. was simple in the extreme, but also extremely useful. It is an excellent illustration of the fact that, in order to be useful,
models should be tailored to fit the application. In [134], Wodarz and Nowak introduce a model similar to that used by Perelson et al., but which included the
36
effects of the cell-mediated immune response. The model has the form
ẋ = λ − dx − βxy
ẏ = βxy − ay − p2 zy
(3.3)
ẇ = cxyw − cqyw − b2 w
ż = cqyw − hz
and the states used describe concentrations of: x, activated HIV-specific helperT cells; y, HIV-infected helper-T cells; w, cytotoxic-T cell precursors; and z,
helper-dependent cytotoxic-T cells. Activated helper-T cells (x) are generated
at a rate λ and die of apoptosis (programmed cell death) at a rate dx, which
leads to a virus-free steady-state population of λ/d. It is worth mentioning that
these numbers are high, as befits an immune response to an active infection,
with λ representing the generation of new cells due primarily to clonal expansion, rather than naive thymic emigrants, and d representing a high rate of death
due to activation-induced apoptosis. According to mass-action reasoning, healthy
helper-T cells are infected by the HIV virus at a rate proportional to the product
of their concentration and that of the free virus. The high rate of production
of free virus by infected cells together with the short half-life of HIV cause the
concentration of free virus to proportionally track the concentration of infected
cells, so there is no need for a separate virus state. The model represents the rate
of infection of healthy helper-T cells in the term βxy. The development of an
HIV-specific memory cytotoxic-T cell population is dependent on the interaction
of HIV-specific memory cytotoxic-T cells with HIV-specific helper-T cells stimulated by antigen presenting cells (APCs) displaying HIV peptides. Assuming a
relatively constant concentration of APCs, and assuming that the percentage of
APCs presenting HIV peptides is proportional to the percentage of HIV-infected
helper-T cells, the expansion of the HIV-specific memory cytotoxic-T cell popula37
tion will be proportional to the product of the concentrations of healthy helper-T
cells (x), infected helper-T cells (y), and memory cytotoxic-T cells (w), cxyw.
The differentiation of these activated memory cells into cytotoxic-T cell effectors
(z) does not require additional helper-T cell involvement and is mediated by the
interaction of memory cytotoxic-T cells and HIV-presenting APCs as described
by the term cqyw. As mentioned in Section 2.1.4, it is likely that memory CTL
(lumped here with the CTL precursors) differentiate from CTL effectors, as determined by interactions with helper-T cell during their initial activation. However,
the relatively small proportion of effector CTL that become memory CTL mean
that separating the memory-destined CTL out from the CTL effectors (as in this
model) will have no effect on the dynamics. The cytotoxic-T cell effectors (z) kill
infected helper-T cells at a rate p2 yz and undergo apoptosis at a rate hz.
The analysis of this model demonstrated an interesting fact: for a reasonable
range of parameters, this system had two stable steady-states in the forwardinvariant non-negative orthant (a complete discussion of the bifurcations of this
model can be found in [24]). One described a condition in which no lasting CTL
response was established, and the virus dominated, forming a steady-state where
viral load was limited by the availability of target cells. The second steady-state,
however, described a condition in which a persistent CTL immune response developed, controlling the viral expansion to a low level, with no significant reduction
in the steady-state value of the helper-T cells as compared to the virus-free state.
These results were interpreted as a mechanism by which HIV infection of the
same immune system could lead either to AIDS, or a CTL controlled state analogous to LTNP. Furthermore, it was shown that certain schedules of interrupted
HAART could transition the state from the region of attraction of the AIDSdescribing steady-state to the region of attraction of the LTNP-describing steady
38
state. An example of these diverging outcomes can be seen in Figure 3.1. This
model and its interpretation challenged the dominant assumption, that LTNPs
were patients with uniquely responsive immune systems, or patients infected with
uniquely disadvantaged strains of HIV. It also provided a theoretical basis for the
STI trials described in Section 2.3.6.
Figure 3.1. Two Steady States The initial condition is the same for both
figures, but in the figure on the left three weeks of treatment are applied at the
outset. A persistent CTL response develops, and controls the viral load to low
levels. In the figure on the right, The CTL response is not established, and the
virus dominates. Parameters are as in [134].
The model introduced in [134] was expanded in [133], including separate
compartments describing activated and resting helper-T cells, and infected and
healthy macrophages. In this paper, the authors showed that the conditions for
initially establishing a persistent infection with solely T-cell tropic virus were
very stringent, but that macrophage-tropic virus could easily establish infection,
providing a theoretical explanation of the relative natural immunity to HIV infection seen in CCR5-deleted patients [29]. The same model was used in [135] to
39
demonstrate that pre-existing multi-drug resistant viral strains could still result in
delayed viral rebound upon initiation of HAART if a CTL response was present,
lending further support to the work in [111] and [18], which showed mathematically that resistance was more likely to pre-exist than to develop during therapy
(these studies provided strong rationale for early introduction of therapy). A
simplified variation on the model was used in [7] to show that the slower rates
of decay seen at low levels of viral load in HAART did not necessarily represent
the clearance rate of long-lived viral reservoirs, but could also be explained by
reduced rates of killing of infected cells by CTL.
The model was further expanded in [132], exploring helper-T cell independent
mechanisms for the emergence of CTL effectors in the presence of high viral load.
The model is developed with compartments describing healthy and HIV-infected
APCs, but it is demonstrated that under reasonable assumptions, the rather
extensive model simplifies to the model used in [130]. In this follow-up paper,
Wodarz explored the implications of this extended model with respect to therapies
designed to improve the immune response to HIV infection. We will explore this
model in greater depth, as it is the model we use in Chapter 5 for our treatment
scheduling algorithms. The model is
ẋ = λ − dx − βxy
ẏ = βxy − ay − p1 z1 y − p2 z2 y
ż1 = c1 z1 y − b1 z1
(3.4)
ẇ = c2 xyw − c2 qyw − b2 w
ż2 = c2 qyw − hz2
and the states are as in equation 3.3, except that z has become z2 , and z1
represents the portion of the CTL response that is independent of helper-T cell
40
help. The steady-state behavior of this model has many possible bifurcations due
to parameter changes, which are discussed in [130]. For a reasonable range of
parameters, however, the non-negative orthant is positively invariant, and within
the non-negative orthant there are two stable steady states. The first, defined by
the equations
xa =
λc1
dc1 +b1 β
ya =
b1
c2
βxa −a
p1
z1a =
(3.5)
wa = 0
z2a = 0,
exists and is stable when β >
c1 [c2 b1 (λ−qd)−b2 c1 d]
.
b1 (c2 b1 q+b2 c1 )
This steady-state represents a
condition where no persistent memory-CTL response has been established, and a
viral set-point is established that is limited by the availability of target cells and
the action of helper-T cell independent CTL functions. This would correspond
to a patient in the chronic infection stage, progressing toward AIDS. The second
stable steady state, defined by the equations
xo =
yo =
λ
d+βyo
√
[c2 (λ−dq)−b2 β]−
[c2 (λ−dq)−b2 β]2 −4βc2 qdb2
2βc2 q
(3.6)
z1o = 0
wo =
hz2o
c2 qyo
z2o =
yo c2 (βq−a)+b2 β
,
c2 p2 yo
exists and is stable when the quantity [c2 (λ−dq)−b2 β]2 −4βc2 qdb2 is not negative.
If the infectivity of the virus is small such that β <
41
c1 [c2 b1 (λ−qd)−b2 c1 d]
,
b1 (c2 b1 q+b2 c1 )
this steady
state is always established (this describes the system while HAART is being
applied with sufficiently high effectiveness).
This model more accurately reflects the observed differences between progressive and non-progressive HIV infections as described in Section 2.2.4, yet
still retains the bistable nature of Equation 3.3, with its implications for the
possibility of designing treatment interruption schedules that would induce longterm immune-mediated control of the virus. The model and its implications were
tested in experiments using anti-retroviral therapy interruptions during acute infection of Simian Immunodeficiency Virus (SIV) in Macaques (an animal model
of HIV infection). The results, reported in [131], [71], [74], [72], showed consistent success in inducing immune-mediated control of the SIV infected, with good
resistance to homologous and heterologous re-challenge, that verified the basic
predictions of the model (strong, highly proliferative SIV-specific CTL responses
that persisted in the absence of measurable viral load). This experimental verification gives us confidence in the appropriateness of this model for our purposes
in Chapter 5 of designing a model-based algorithm for determining appropriate
treatment interruption schedules.
42
Chapter 4
Model Predictive Control
4.1
Background
Model Predictive Control (MPC) is a feedback control technique in which the
control is determined by solving an optimal control problem over a finite horizon
starting from the current operating condition. The optimization problem may
have state and terminal constraints, and may involve a constrained control set.
The resulting finite horizon optimal control is applied for a short time (a fraction
of the finite horizon used in the optimal control), after which a new finite horizon
optimal control is calculated from the new operating condition. This technique
has also been called Receding-Horizon Optimal Control (RHOC). It is very popular in chemical engineering applications because of its natural formulation of
control laws from control objectives. Its main drawback is computational cost,
which puts lower limits on sampling times.
Most formulations of MPC guarantee closed-loop stability in one of two ways:
either they use terminal constraints, which force the control to drive the state
43
into an appropriate constraint set (usually the desired steady-state) within the
optimization horizon, or they use a local control Lyapunov function as a terminal
cost, that, under appropriate restrictions on stage costs, and given a sufficiently
long optimization horizon, can guarantee robust semi-global stability. These two
approaches are reviewed in [81].
The first class of formulations suffers from a significant problem: as reported in
[49], [50], the closed loop systems generated by MPC methods employing terminal
constraints can be globally asymptotically stable but, for too short of a horizon
length, have zero robustness to measurement error. That is, arbitrarily small
disturbances can be found that result in either lack of convergence or instability.
While the stability given by the second class of methods employing a CLF
terminal cost function to ensure stability can be robust, it also suffers from certain
limitations. First, it requires the computation of an appropriate local CLF for
the desired steady-state, a potentially difficult task if the linearization of the
system about the steady-state is not stabilizable. Second, and more importantly
for applications in medicine such as ours, the requirement of a CLF terminal cost
is an unacceptable restriction on the formulation. It involves penalizing terms
without any performance justification in order to ensure stability.
The authors of [48] and [47] present a formulation of MPC that avoids these
limitations, allowing unconstrained MPC with no terminal cost or constraint set,
and semi-definite stage cost. We discuss these results in the next section.
44
4.2
Robust MPC without CLF terminal cost
In this section, we review the MPC formulation of [48] and summarize its
results. Although the results in [48] are quite general and allow for a broad class
of terminal cost functions, we adapt the results for our uses, which do not use a
terminal cost function.
4.2.1
Results
The results in [48] are for discrete-time (possibly) nonlinear systems of the
form
Xk+1 = f (Xk , uk ).
(4.1)
The system is assumed to be time invariant. A solution of the system at time k
from initial condition X under control input sequence u is written as
φk (X, u).
(4.2)
For a horizon length N we use a cost function of the form
JN (X, u) =
N
−1
X
l (φi (X, u), ui )
(4.3)
i=0
where l is a continuous positive semi-definite stage cost (the results in [48] allow
for the use of a terminal cost function, but we do not use this, and omit it for
simplicity). The value function
VN (X) = inf JN (X, u)
u
(4.4)
is a function of the state that takes the optimal value of the cost function on the
allowable set of controls. Note that if the set of allowable controls U is compact, as
45
it is in our application, the infimum is always achieved for some control sequence
u whose components lie in U, and the infimum equals the minimum. We define
the feedback control law κN (X) to be the first element of the control sequence u
satisfying Equation 4.4, yielding a closed-loop system of the form
Xk+1 = f (Xk , κN (Xk )).
(4.5)
The results in [48] state that for sufficiently large horizon length N the desired
steady-state (or, more generally, attractor) is globally asymptotically stable if
the system and stage cost satisfy certain assumptions detailed in Section 4.2.2.
Moreover, a continuous Lyapunov function can be constructed that proves that
this stability is robust to small state and parameter variations.
4.2.2
Assumptions
In this section we detail the assumptions that must be met in order to ensure
closed-loop robust global asymptotic stability. We also discuss how our implementation meets these requirements, or, when this is impossible to prove directly,
what inferences can be made. The results in [48] are for unconstrained systems;
work in [47] extends the results to allow for state constraints. Both papers allow for the use of generalized terminal cost functions; however, we omit this for
simplicity.
The first assumption required by [48] is that the stage cost be continuous.
Our stage cost (Equation 5.5) is continuous.
The second required assumption is that either U is compact or the optimal
controls satisfy a boundedness condition, namely that for each compact set C,
η ∈ R, N , there exists µ > 0 such that all components of u satisfying JN (X, u) ≤
46
η for X ∈ C are upper bounded in norm by µ. In our application, U := {0, 1},
which is obviously bounded.
The third required assumption is that the stage cost satisfy a detectability
requirement as follows: Let σ : Rn → R≥0 satisfy ασ (|X|A ) ≤ σ(X) ≤ ασ (|X|A )
for some ασ , ασ ∈ K∞ , where A is the desired steady-state or, more generally, the
desired attractor (by the notation of [48], σ is a proper indicator function for A),
and K∞ is the class of functions from R≥0 → R≥0 which are continuous, strictly
increasing, zero at zero, and unbounded. For some σ, αW ∈ K∞ , γW ∈ K∞ ,
αW ∈ G (where G is the class of functions g : R≥0 → R≥0 that are continuous
and nondecreasing), there exists a continuous function W : Rn → R≥0 such that
for all X ∈ Rn , u ∈ U
W (X) ≤ αW (σ(X)),
(4.6)
W (f (X, u) − W (X) ≤ −αW (σ(Xi )) + γW (l (X, u)).
Proving that this third assumption holds in our application is difficult without
forming an explicit discrete-time representation of our sampled continuous-time
model. We know, however, that the condition holds for l as described in Equation 5.5 for at least some neighborhood of the desired steady-state described in
Equation 3.6. This is because the steady-state is locally exponentially stable (a
property that is mirrored in the sampled discrete-time system), and therefore
admits a quadratic local Lyapunov function. Taking such a function as both W
and σ, and letting γW and αW be constants easily verifies this. We are led to
believe that the property holds globally by the fact that, in the continuous-time
system, l → 0 =⇒ X → Xo , which is a similar detectability condition, but we
cannot prove this without explicitly defining the discrete-time system.
The fourth required assumption is that there exists an function α ∈ K∞
upper-bounding the value function such that Vi (X) ≤ α(σ(X)) for all X ∈ Rn ,
47
i ∈ Z.
The authors of [48] show a number of ways to prove that this assumption
holds. In our case, however, it is easier to give an informal proof. In the case
where u = 1, there exists one globally asymptotically stable steady-state for the
system described in Equation 3.4. From a neighborhood of this steady-state,
there exist known finite-length admissible controls that drive the system to the
region of attraction of the desired control-free steady-state. Therefore, a globally
asymptotically stabilizing control strategy is: if the state is outside the region of
attraction of the desired control-free steady-state, let u = 1 for a time sufficiently
long for the state to converge to a neighborhood of the globally asymptotically
stable steady-state, and then apply the known admissible control to drive the
state to the region of attraction for the desired steady-state. Our stage cost is
such that we can say that l (X, u) ≤ K1 |X − Xo |2 + K2 |u|2 for all X, u where Xo
is the value of the desired steady-state described by Equation 3.6. From points
within the region of attraction of this steady-state, an admissible control is u = 0,
Pi−1
P
2
and Vi (X) = i−1
k=0 K1 |Xk − Xo | . This is bounded in i since
k=0 l (Xk , 0) ≤
the point Xo is locally exponentially stable, so the condition is satisfied locally.
From any point outside the region of attraction of Xo , we can drive the system to
the region of attraction of Xo in finite time using the algorithm described above.
Since l is bounded in the state and control, this means that the condition is
satisfied globally. This is sufficient to prove that our system satisfies assumption
4.
The authors of [48] develop an algorithm for constructing a continuous Lyapunov function for the closed-loop system 4.5 using these assumptions. This
algorithm includes a method for calculating a value for N large enough to guarantee robust asymptotic stability, but this requires an explicit formulation of the
48
discrete-time system; for our purposes, we determined an appropriate horizon
length through simulation.
49
Chapter 5
MPC-Based Treatment
Scheduling
While the work in [134], [130] postulated the existence of treatment schedules
that would induce a CTL-mediated long-term control of HIV infection, they
did not provide any method for determining such schedules. In this chapter we
present an algorithmic method for determining treatment schedules in a clinically
implementable closed-loop fashion that provides good robustness to measurement
noise and modeling error. We briefly review the existing control-based approaches
to HIV treatment, discuss the suitability of MPC based methods, introduce our
algorithm, and demonstrate its robustness properties through simulation.
5.1
Control Theory and Treatment
Using control theory to guide treatment of HIV is not a new idea. In [129], the
authors propose a therapy scheduling algorithm that uses open-loop optimal control to choose combinations of treatments that have the greatest inhibitory effect,
50
based on measurements of existing viral strains in the patient. The authors of
[65] apply finite horizon open-loop optimal control to a model of HIV chemotherapy developed in [66]. The authors of [20] develop a continuous-time feedback
strategy based on the model developed in [28]. More recently, the authors of [26]
apply open-loop optimal control to a simplified version of the model introduced
in [134], and the authors of [24] propose the use of an innovative control strategy that would induce CTL-mediated control of the virus through the gradual
reduction of the treatment dosage. The authors of [122] use a variation of the
technique we propose here that explores the effects of allowing a finer sampling
of treatment efficacy.
The use of control theory to plan HIV therapy is a natural outgrowth of the
mathematical modeling of the immune system. The question of what therapy
is “best” given a certain set of assumptions is naturally formalized through the
use of open-loop optimal control. The robustness to measurement and modeling
error afforded by feedback strategies, such as those proposed in [20] and [122]
is also naturally desirable, especially when pursuing objectives that are sensitive
to these errors. The schedules that induce CTL-mediated immune control of
infection as proposed in [134] can be complex and non-intuitive, and very sensitive to changes in parameters and measurement noise; calculating appropriate
interruption schedules in this case almost requires the use of feedback control.
However, actually implementing these strategies is often quite difficult. The
authors of [65], [20], [26], and [24] all assume they can continuously vary the level
of suppression delivered in the therapy. In reality, this is very difficult to achieve;
HAART drugs are taken orally in fixed doses, and the dynamics of drug uptake
and clearance make delivering anything but a steady dose impossible to do with
any accuracy. Also, applying suppressive therapy at less than complete levels
51
is dangerous. While the increased replication rate in the absence of suppression
increases the mutation rate, and consequently the rate at which drug-resistant
mutants are generated, the competitive advantage of the wild-type causes it to
out-compete the resistant mutants and minimizes the chance of persistent colonies
emerging. In the case of full suppression, the resistant strains have a strong competitive advantage, but the low replication rate ensures a low mutation rate and
a small probability of generating resistant mutants. Partial suppression allows
larger numbers of resistant mutants to be generated in an environment where
they enjoy a competitive advantage. The danger of this can be seen in studies that showed a significant increase in the emergence of resistance in patients
previously subjected to mono- or dual-therapy as compared to treatment-naive
patients when started HAART ([108], [87]). Mono- and dual-therapies differ
in two ways from HAART; the suppression of the virus is less complete, and
the probability of resistant mutants is higher, since fewer nucleotide changes are
needed to confer resistance. Partially suppressive HAART would mimic at least
one of these factors, and should be avoided.
These factors restrict the feasible use of HAART to treatment interruption
schedules; on-off patterns of treatment that don’t switch too quickly. This represents a barrier to the use of feedback control, as there are few strategies for
designing feedback controllers for nonlinear systems with discontinuous feedback.
Also, most of the strategies above assume full-state measurements; however, as
we discussed in Section 2.4, the availability, cost, and accuracy of measurements
varies significantly over the relevant states. An implementable control strategy
would have to admit the use of a limited set of available measurements, preferably
solely from viral load, which is both accurate and inexpensive.
52
5.2
Model Evaluation
The development of appropriate models for use in control is an interesting
problem. We must balance the accuracy of the model with its complexity, and
keep in mind the measurement requirements for effective feedback. In this section,
we evaluate the appropriateness of the Wodarz-Nowak model (Equation 3.4) for
use in our MPC-based feedback control strategy.
In choosing an appropriate model for use in feedback control, we must always
balance the need for accuracy with the associated complexity of the model. In
MPC, a model that is inappropriately complex becomes a problem for two reasons. The first is computation time; complex models require increased amounts
of time to compute the optimal control sequences. The second is accuracy; highly
complex, nonlinear models may cause error propagations in numerical simulators
that can compromise the accuracy of the optimal control calculations. Conversely,
a model that has been over-reduced can interfere with the feedback control computations by introducing errors too large to be compensated for by the feedback.
The challenge is to find an appropriate level of model reduction.
The Wodarz-Nowak model is a reduced-order version of a more complex
model. Obviously, biological systems that are only indirectly related are not
modeled, as this would unnecessarily complicate the model. In [132], the authors
take into account the interaction of APCs with helper-T cells and cytotoxic-T
precursors, using separate compartments for each species and their complexes.
Three major assumptions allow this model to be drastically reduced: that the
dynamics of the complexes are fast, that the number of antigen-presenting cells is
small, and that the dynamics of free virus are fast compared to those of infected
cells. Under these assumptions, it can be shown that the virus level will track
53
proportionally the level of infected cells with relatively fast dynamics and that
the concentration of helper-T activated antigen-presenting cells quickly reaches
a quasi-equilibrium value of
a1 xy
1+a2 x(1+a3 w)+a4 x
(5.1)
where x is the concentration of virus-specific helper-T cells, y is the concentration
of virus-infected cells, and w is the concentration of virus-specific CTL. These
assumptions are unquestionably valid. Under the further assumption that the
concentration of available virus-specific help is small, Equation 5.1 reduces to
c2 qxy which is the form used in [134], [132]. This additional assumption, that
the available help is small, is very likely good during primary infection before
the HIV-specific helper-T cells have had a chance to expand, which is the region
of interest for our application. If it is assumed that the expansion of memory
CTL is dependent on interaction with helper-T activated APCs, as suggested
by much of the experimental evidence cited in Section 2.1.4, then the expansion
of memory CTL is given by the term c2 qxyw as in Equation 3.4. As discussed
in Section 2.1.4, the best current research indicates that helper-dependent CTL
first differentiate into effector cells, with a small subset of those effector cells,
predetermined during their initial expansion, reverting to memory cells. Directly modeling this would require us to use two seperate states to keep track
of which effector cells were destined to become memory cells. Since the number
of memory-destined effector cells is small compared to the total number, their
effector function is insignificant, and the approximate model used in [134], [130],
which has memory cells developing first and then differentiating into effectors, is
simpler and functionally identical.
54
There are two possible pathways for helper-T-independent CTL expansion;
however, under appropriate assumptions, the expansion terms due to both these
mechanisms have the same form. The first possible pathway is the so-called
classical pathway, in which helper-T secreted cytokines directly activate CTL
upon exposure to antigen. While not strictly helper-independent (this response
does not exist if help is truly absent), it is not dependent on HIV-specific helper-T
cells (with high virus levels, inflammation effects cause non-specific helper-T cells
to secrete cytokines), and, at high virus loads, the effect of helper-T concentration
vanishes from the equations. In [132], the authors consider the interactions of
helper-T cells, CTL, antigen, and cytokines, with separate states to represent each
species and their complexes. If we assume that the dynamics of complexes are fast
and that the dynamics of the cytokines are fast, then the cytokine concentration
reaches a quasi-steady-state value of
a5 xy
,
1+a6 xy
(5.2)
which reduces to a constant at high viral loads, leaving a proliferation term for
this mechanism of helper-independent CTL expansion of c1 yz1 , which is the form
used in [134] and [130].
The other pathway for helper-independent CTL expansion is through the
helper-independent activation of APCs; this can happen with viruses such as
HIV that are capable of infecting certain professional APCs, which will cause
HIV antigen to be displayed in MHC-I complexes on the APC surface without
the normal requirement for helper-T activation. The need for helper-T activation can also be bypassed if the infection is causing inflammation, which results in the release of cytokines that activate APCs without the need for help.
In both of these cases, following assumptions identical to those for the helper55
dependent activation of APCs, the quasi-steady-state concentration of APCs activated without help is proportional to the viral load, and the proliferation of
helper-independent CTL takes the form c1 yz1 . We can therefore lump both
of these helper-independent mechanisms into a single term. Both the helperdependent and helper-independent CTL responses likely develop from the same
pool of naive CTL precursors; modeling them as developing independently from
each other is an acceptable simplification if we assume that the HIV-specific CTL
are not the limiting species in the expansion, an assumption that is likely to be
true if we do not model the very early stages of infection, before the initial HIVspecific CTL expansion occurs. This begins before seroconversion, so it is very
unlikely that the model will need to account for this region.
The terms relating to the expansion of virus-specific helper-T cells can be
reduced in a similar fashion; however, the expansion of of helper-T cells does
not require the APCs to be activated, so the number of APCs that stimulate
helper-T expansion can be assumed to be proportional to the number of infected
cells. This gives us an expansion term for the helper-T cells of a7 + a8 xy, which
represents a constant rate of new thymic derivatives plus the rate of clonal expansion. There is, very likely, a natural density-dependent decay term representing
activation-induced apoptosis, which would give the helper-T expansion dynamics a traditional s-curve shape, but we can correctly neglect this term since the
region of interest is where a complete helper-T response is not being established.
The complete dynamics of the specific helper-T population is then
ẋ = a11 + a12 xy − a13 x − a14 xy,
(5.3)
since, in the case of HIV infection, the helper-T cells are the target for the HIV
virus. In the case of untreated HIV infection, the rate of infection of HIV-specific
56
helper-T cells is likely to overwhelm the HIV-specific expansion, and this equation reduces to ẋ = λ − dx − βxy, which is the functional form used in [134],
[130]. Technically, this should be accompanied by a corresponding increase in the
coefficient of the expansion term βxy for the infected cell compartment, but this
is a simple scaling issue and will not affect the dynamics of the system. However,
the modeling of HAART as a simple suppression of this term is incorrect, as
efficient suppression will result in the emergence of a positive antigen-dependent
expansion term a12 xy. At equilibrium induced by efficient suppression, this term
can be neglected, as the amount of presented antigen becomes vanishingly small,
but in the intermediate period during interruptions in HAART, this term may
be significant. If the parameters of the approximate model are matched against
measured data, the error introduced by this approximation will be partially reduced by an increase in the value of the parameter λ, representing a linearization
about some intermediate operating condition, and our feedback control will likely
compensate for any remaining introduced error.
The modeling work in [134], [130], [132] has good justification for the appropriateness of the model reductions it makes, but has one major oversight.
The models are ordinary differential equations, and as such, they neglect delays. There is a delay of 1-3 days between the activation of a T-cell and the
beginning of expansion, a delay of approximately one day between a target cell’s
infection by HIV and the beginning of the production of new virions, and multiple unknown other delays associated with the cellular signaling and activation
mechanisms that mediate the various reactions. The aggregate effects of these
delays is difficult to determine; using a different model, the authors of [12] show
that a better fit to measured data can be made with a model that incorporates
these delays in a stochastic fashion. However, the goal of the model in question
57
is prediction, not feedback control. All of the delays that could be considered
are significantly smaller than the sampling period of one week that we use; however, they are not really small enough to say that their effects will be negligible;
not incorporating the delays will introduce error into the model, which may be
small enough to be compensated for by the robust stabilization afforded by our
feedback control algorithm. Delays could be incorporated into the model used
in the feedback algorithm, but this would increase complexity. Also, the results
that insure stability for feedback MPC do not explicitly cover the case of systems
with delays. To rule out the possibility of introducing destabilizing effects, the
theoretical background would need to be expanded.
Perhaps the strongest recommendation of the model Equation 3.4 for use
in control is the congruence between the schedules recommended by the model
and those schedules that have succeeded in inducing immune-mediated control in
experiment. Many of the schedules that yielded success in the experiments described in [75], [73], that resulted in some success in experiment are also indicated
by the model. Conversely, the long interruptions used in [103], [119] resulted in
much lower rates of success, and are not indicated by the model. In conclusion,
the assumptions used to develop the model reduction presented in [130], [132]
are well-founded, and appropriate for the region of interest for MPC-based feedback control. Certain assumptions, especially the neglect of delays, will introduce
modeling error. This error may be small enough to be compensated for by the
robust stabilization of the feedback control, but only experiment can verify this.
58
5.3
MPC and Treatment
In order to develop treatment schedules that exploit the possibility of immunemediated control in the face of the difficulties discussed above, we need a feedback
control strategy for the nonlinear, continuous-time model 3.4 that is compatible
with using sampled-data measurements of only part of the state, using sampleand-hold controls that take values from a discrete set. The MPC algorithms
described in Chapter 4 are uniquely well-suited to this task. MPC strategies
readily accommodate the restricted control set we must use. The use of stage
costs to determine strategies flows readily from medical notions of treatment
objectives and systemic cost. Moreover, the problems with computational cost
that are normally associated with MPC methods are irrelevant in our application;
we implement a sampling time of one week, compatible with weekly clinical visits,
which is sufficient time to carry out quite computationally intensive algorithms.
5.4
Treatment Scheduling Algorithm
On the time scales for which we are implementing our algorithms, the actions
of the various drug types in HAART can be aggregated into a simple reduction
in the infection of new target cells. This can be represented in the model 3.4
by introducing a coefficient to the term representing viral infection. The model
59
becomes
ẋ = λ − dx − β(1 − ηu)xy
ẏ = β(1 − ηu)xy − ay − p1 z1 y − p2 z2 y
(5.4)
ż1 = c1 z1 y − b1 z1
ẇ = c2 xyw − c2 qyw − b2 w
ż2 = c2 qyw − hz2
where u represents the application (u = 1) or absence (u = 0) of HAART and
0 < η < 1 represents the effectiveness of the therapy. The use of MPC requires a
discretized version of this model. It does not, however, need to be explicit, and
we obtain the discretization implicitly through numerical simulation.
In [142], [143] we used this extended version of the model to implement an
MPC-based feedback treatment scheduling algorithm of the type described in
Chapter 4. The goal of HAART is to simultaneously achieve suppression of the
virus and maintenance of healthy helper-T cells. This would be represented in
a cost function of the form l (Xi , ui ) = −α1 (xi )2 + α2 (yi )2 . The objective of our
treatment scheduling is to drive the patient to a state in which the immune system
will suppress the virus without continued treatment. Using the cost function
indicated by standard HAART objectives will not achieve this; since the steadystate value of healthy helper-T cells is higher and the steady-state value of infected
helper-T cells is lower for full HAART suppression than for immune-mediated
control, an MPC algorithm with this stage cost will always choose to use fully
suppressive HAART. In order to achieve immune-mediated suppression of the
virus, we must include in the stage cost a term which encourages the growth
of the helper-dependent CTL compartment, either w or z2 (this satisfies the
conditions for robust stability discussed in Chapter 4). We choose to use the
term w, and penalize its distance from the desired steady-state value, because
60
it is more natural. We could choose to include a term which penalizes the level
of infected cells y, however this is somewhat redundant with encouraging the
growth of x and experiment has shown that this formulation requires the use of
longer horizon lengths in order to ensure stability. Including a penalty on drug
use u allows us to account for the systemic cost associated with HAART, and
also seems to allow the use of shorter horizon times. All these considerations
yield a stage cost of the form
l (Xi , ui ) = α1 (xi − xo )2 + α2 (wi − wo )2 + α3 |ui |
(5.5)
where αj are positive weighting constants and xo , wo are the steady-state values
of their respective states at the desired equilibrium 3.6. It is easy to see that this
system satisfies all the requirements for closed-loop, robust asymptotic stability
as described in Chapter 4 except for condition 4.6. Analytically verifying that l
satisfies condition 4.6 is difficult without explicitly defining the discretized version of the system. However, the local asymptotic stability of Xo guarantees that
this condition is satisfied at least locally, and the analogous detectability properties of |x|A through l on the continuous-time system encourage us to believe that
these properties are satisfied globally. The robust performance exhibited in simulation (as detailed in Section 5.5) further verifies this. The work in [48] describes
an analytical method of determining a horizon length N which is sufficiently long
to ensure stability; however, among other things, it requires an explicit characterization of the the discretization of our model. Instead, we determined an
appropriate horizon length through experiment. For most of our work, N = 6
was used; longer horizon lengths seemed to have no effect on the closed-loop, but
significantly affected computation time.
61
5.5
Implementation and Simulation
We implemented the algorithm in MATLAB, using parameter values adapted
from [130], which are listed in Table 5.1. Example code can be seen in the
appendices. The nonlinearities and diverse time-scales in our model forced us to
use a stiff solver, small maximum step-sizes, and explicit Jacobian calculations.
Even so, initial conditions far away from the normal region of operation, or those
with very small state values, would cause the solver to fail to converge. The region
of initial conditions for which we could successfully calculate optimal controls
could be increased by decreasing the maximum step-size used by the solver, with
a corresponding cost in computation time.
Table 5.1. Parameter Values These are the parameter values used in our
implementation of the MPC feedback algorithm. With these values, calculation
of one finite-horizon optimization took approximately three minutes.
λ
1
d β
0.1 1
η
N
0.9799 6
a p1 p2
0.2 1 1
α1 α2 α3
1
1
1
c1
c2
b1
b2
q
0.03 0.06 0.1 0.01 0.5
MaxStep Evaluation Time
0.1
174 s
Simulations from a large number of initial conditions from the positive orthant
verified that the algorithm successfully globally stabilizes the desired steady-state
(initial conditions which cause the numerical integrator to fail were still stabilized
if we reduced the maximum integrator step size). Furthermore, as we discuss in
Section 5.6, the closed-loop exhibited impressive levels of robustness to state and
measurement error.
62
5.5.1
Varying the Cost Function
MPC-based methods allow great flexibility in the formulation of the cost functions, allowing fine-tuning of the performance of the system and incorporation
of various additional criteria. As a simple method of demonstrating this, we adjusted the weightings of the elements of the stage cost. When we increased the
weight on the term penalizing decreased helper-T concentration while decreasing
the weight on the terms penalizing excess drug usage and rewarding CTL memory
growth, the algorithm returned a schedule that converged to the desired equilibrium more slowly, using more anti-retroviral therapy overall, but did so while
maintaining a higher average healthy helper-T cell concentration. A comparison
of the performance of the adjusted cost function and the nominal cost function
can be seen in Figure 5.1. It is important to emphasize that the cost functions
cannot be changed arbitrarily; the variations must satisfy the conditions outlined
in Chapter 4. Furthermore, changing the cost functions also affects the sensitivity
of the algorithm to error; for random error of up to 30 % of each parameter, the
system with cost function as in Figure 5.1 exhibited only a 47 % rate of success
in stabilizing the desired steady-state, compared to over 90 % when using the
nominal stage cost. We do, however, have great flexibility in determining our
cost functions.
5.6
Robustness
We implemented the algorithm in MATLAB. In this section, we show simulation results that illustrate the algorithm’s performance over a variety of conditions. For simplicity and readability, we plot only the healthy helper-T cells, viral
63
Figure 5.1. Changed Cost Function The stage cost in the first plot has weights
with values α1 = 1, α2 = 1, α3 = 1, the second has weights with values α1 = 100,
α2 = 0.5, α3 = 0.1. Both algorithms stabilize the desired steady state, but the
second does so while maintaining a higher average level of healthy helper-T cells,
at the cost of slower convergence and longer total anti-retroviral use.
load, CTL memory and control states (x,y,w,and u). In every case, z2 tracked
w, and z1 rapidly approaches zero as immune control is established, so this is a
sufficient sampling to understand the results. x, y, and w are plotted as solid,
dashed, and dotted lines respectively, and u is plotted as a shaded area. The algorithm provided robust stability in simulation from every initial condition. We
show in Sections 5.6.1-5.6.4 that the method successfully stabilizes the desired
steady-state despite state measurement error and modeling error. Many of these
experimental robustness results were first presented in [143].
64
5.6.1
Robustness: Measurement Noise
The main benefits to using a closed-loop control method are disturbance rejection and robustness. Our MPC-based method grants us a certain degree of
robustness to measurement and modeling errors, but the degree of this robustness
can not be directly calculated, and we must explore it through simulation.
We use a Monte-Carlo-like approach with randomly generated noise signals. A
standard approach would be to apply Gaussian white noise to the measurements,
but this would admit negative-valued states, which would violate the assumptions of the model. Instead, we introduced flat-random, magnitude-limited noise
as a percentage of the actual state values. This preserves the assumptions of
the model, has zero mean, and is unbiased. We introduced into the state measurement a random noise signal, which would add or subtract from each state
as much as 10%, 25%, 50%, or 100% of the actual state value, and used this
signal with error to compute our feedback controls. The algorithm succeeded
in stabilizing the desired steady state for each of 100 simulations at each of the
error magnitudes. At higher error magnitudes, the error caused the treatment
scheduling algorithm to take significantly longer to stabilize the system, using
many unnecessary treatment interruptions, but in every case the controller eventually induced a successful immune response. A sampling of these results can be
seen in Figure 5.2. All simulations are from the common starting condition of
x = 10, y = 0.1, z1 = 0.1, w = 0.1, z2 = 0.1. A common starting condition is
used for easy comparison. The stable steady-state in the case where u = 1 is a
natural candidate for a common starting condition, as it is the likely condition
of a patient who has been on therapy for some time, but it artificially eliminates
some the the model dynamics by eliminating the influence of the state z1 , which
65
has the effect of making it easier for the algorithm to stabilize the desired steadystate. Our common steady-state is close to the natural, full-therapy steady-state,
but avoids these problems.
Figure 5.2. Random Measurement Error Noise of up to 100% of the value
of each state was introduced into each measurement. The algorithm successfully
stabilized the desired steady in every case, though the state took longer to converge. The plots above show representative outputs for up to 10%, 25%, 50%,
and 100% random measurement error respectively.
It is not unusual for feedback-controlled systems to display impressive ro66
bustness to flat random (as shown here) or Gaussian random measurement noise.
These types of noise do not change the average value of the measured signal, and,
so long as the controller is not too aggressive, the disturbances introduced by the
faulty measurements tend to decay. The absolute level of robustness to measurement noise, if we consider the possibility of colored noise and pathological cases,
is likely much lower. However, flat random or Gaussian random error is likely
the best model for measurements of the type used in this application, so we are
likely to see this level of robustness in practice. This type of robustness will also
allow us to successfully use an observer, as discussed in Section 6.2.
5.6.2
Robustness: Modeling Error
Implementing this algorithm requires that we first identify the parameters
in the model, finding estimates of such parameters as the rate of generation of
helper-T cells, the rate of death of infected cells, and the effectiveness of HAART
at suppressing viral replication, among others. However, it is likely that the
values of these parameters will change from patient to patient, and drift over
time, introducing modeling error.
To explore the robustness of our technique to errors in these estimates, we introduced a random variation into every parameter in the model. The scheduling
algorithm continues to use the nominal, but now incorrect values to calculate its
schedules. As in the case for our study of robustness to measurement error, we use
magnitude-limited flat-random noise, but here our rationale is different. The numerical simulators seem to be sensitive to the parameter values, and models with
outlying parameter values seemed to cause non-convergence. This could be solved
by decreasing the maximum step size, but only at the cost of increased computa-
67
tion time. Since running 100 simulations of this sort already takes roughly three
weeks on a single computer, we used flat-random noise to allow us to control our
computer usage, decreasing the step size only when necessary. In addition to
these practical considerations, flat-random noise is also unbiased, and allows us
to make easy comparisons with the open-loop case.
We ran at least 100 simulations each with this error randomly distributed at
up to 5%, 10%, 15%, 20%, 25%, and 30% of each parameter value, allowing the
algorithm up to two years to successfully stabilize the desired steady-state. These
simulations were carried out from the same common initial condition described in
Section 5.6.1. For up to 15% error the scheduling algorithm induced a successful
immune response every time. When we allowed up to 20% error, the scheduling
algorithm failed to induce a successful immune response one time out of 140. The
number of errors increased to two out of 100 at 25% error, and twelve out of 129
at 30% error. A representative sampling of these results can be seen in Figure 5.3,
and the results are summarized in Table 5.2. This is a worst-case scenario, as it is
unlikely that we would be equally uncertain about every parameter. If we knew
which parameters were the sources of uncertainty, we could likely show a much
greater robustness. Nonetheless, the degree of robustness we do see certainly
validates the use of feedback in treatment scheduling.
5.6.3
Comparison with Open-Loop
The interruption schedules used in the studies mentioned in Section 2.3.6
use either open-loop or simple closed-loop strategies (such as interrupt, wait
for rebound, re-initiate therapy). One open-loop strategy, from [75], involved a
sequence of interruptions with the pattern of one week off, three weeks on therapy.
68
Figure 5.3. Random Modeling Error Random variations of up to 30% of the
value of each parameter are introduced to the system, while the algorithm uses a
nominal model. The above plots are representative for, from the upper left, the
cases of 5%,10%, 15%, 20%, 25%, and 30% error respectively.
This strategy had a success rate of roughly 50%, and interestingly enough also
works from the full-therapy steady-state for our model with nominal parameters.
69
A slightly modified version of this open-loop schedule, which begins with a threeweek therapy interval instead of an interruption, also works from our common
initial condition described in Section 5.6.1. The rate of success can be explained
as being due to parameter variation between patients, and we can compare the
rate of success with an open-loop strategy with that of our closed-loop strategy
at various levels of modeling error. The results of these experiments can be seen
in Table 5.2. As we can see, 30% parameter variation is sufficient to explain the
failure rate seen in practice, but the closed-loop strategy still works for better
than 90% of the cases. This is a strong recommendation for the use of closed-loop
feedback in practice.
Table 5.2. Robustness Comparison We compared performance of an open-loop
strategy adapted from [75] with the performance of our MPC-based closed-loop
algorithm at various levels of modeling error.
% Error Open Loop
5%
52.6 %
10 %
52.2 %
15 %
53 %
20 %
53 %
25 %
52.2 %
30 %
49.5 %
5.6.4
# of samples
500
500
500
500
500
500
Closed Loop # of samples
100 %
100
100 %
100
100 %
115
99.4 %
140
98 %
100
90.7 %
129
Robustness: Combined Errors
For ease of comparison, we analyzed the scheduling algorithm’s robustness
to modeling and measurement error in isolation, starting from a common initial
condition. However, the treatment scheduling algorithm is also robust to these
errors when they occur simultaneously. To demonstrate this, we ran 100 simulations, starting from random initial conditions, in which we introduced random
70
errors into both the model parameters and the state measurements of up to 10%
of their respective values. A sampling of the results can be seen in Figure 5.4.
In every case, the treatment scheduling algorithm induced a successful immune
response.
Figure 5.4. Mixed Random Error The values of the initial conditions are
varied randomly. Random variations of up to 10% of the value of each parameter
are introduced to the system as in Figure 5.3. Random measurement noise of
up to 10% of each state is added as in Figure 5.2. The treatment scheduling
algorithm successfully stabilized the desired steady-state in every case.
71
The robust performance of the treatment scheduling algorithm under model
and measurement uncertainty is very encouraging. It provides strong motivation
for the use of MPC-based treatment scheduling in patients who begin therapy
during acute infection, as it has the potential to raise the success rate significantly
over the open-loop strategies which have been used previously.
5.7
Additional Targets
One significant advantage of the MPC-based treatment scheduling framework
we have developed is its ability to easily adapt to changes in the model and the
inputs. This is especially useful in evaluating the potential use of additional
therapeutic agents in the context of inducing immune control of HIV using interruptions of HAART. A number of studies have suggested that immune activation,
whether HIV-specific or non-specific, contributes directly to disease progression,
and that consequently, immune responses to HIV should be suppressed [43], [57],
[102], [112]. In the absence of an effective immune response to HIV, suppression of peripheral helper-T cell activation makes perfect sense. Suppression of
HIV replication by anti-retroviral drugs is not directly dependent on the level of
viremia, and increased helper-T cell activation can only lead to higher viremia
and quicker depletion of the helper-T population. Higher levels of viremia also
increase the risk of emergence of anti-retroviral resistant mutant strains. However, in the case where immune-mediated control is considered, intuition does not
lead to an easy conclusion. Immune-mediated control is dependent on the level
of viremia. Additional targets increase both the rate of new infections and the
antigenic stimulus which in turn increases the rate at which the immune response
grows. The closed-loop control may be helped or hindered by the effects of adding
72
additional non HIV-specific activated helper-T cells as targets for viral infection.
In [1], a variant of the model introduced in [134] was used to evaluate the
use of vaccines that increase the HIV-specific helper-T response, such as the HIV
Immunogen vaccine discussed in Section 2.3.7. In this paper, they showed that,
while increasing the HIV-specific helper-T response was helpful in the context of
a successful CTL-mediated immune response, in the absence of such a response
it actually accelerated the progression of the disease. In [144], we addressed the
possibility of adding additional, non-HIV specific helper-T cells (which could be
accomplished through the use of Interleukin therapies discussed in Section 2.3.5).
We introduce an additional control variable u2 to the model, which becomes
ẋ = λ − dx − β(1 − ηu1 )xy
ẏ = β(1 − ηu1 )(x + νu2 )y − ay − p1 z1 y − p2 z2 y
(5.6)
ż1 = c1 z1 y − b1 z1
ẇ = c2 xyw − c2 qyw − b2 w
ż2 = c2 qyw − hz2 .
This term represents the activation of helper-T cells that are not involved
in the immune response against HIV, but do serve as targets for the virus. In
this formulation ν is a positive constant that scales this effect. Using this simple
approach, we solved a variation of the MPC algorithm discussed in Section 5.4
in order to determine if, and under what conditions, such a generalized helperT activation could ever be useful. We showed that, while additional targets
accelerated disease progression in the absence of CTL control, in the context of
treatment interruptions, they could actually decrease the time needed to establish
an effective CTL-mediated immune control of the virus. A summary of these
results can be seen in Figure 5.5.
73
This result, which is rather counter-intuitive, demonstrates the usefulness of
re-evaluating previously discarded therapeutic agents in the context of MPCbased treatment scheduling. The Interleukin therapies and the HIV Immunogen
vaccines failed in their original goals, but they did allow for modulation of certain key immunological states and parameters, and therefore represent additional
potential control inputs for use in therapy. As shown here, the MPC framework
we have developed is a powerful tool for investigating potential alternative uses
for these agents. The same approach could be beneficially applied to the various
vaccine strategies discussed in Section 2.3.7, that may be put to beneficial use
whether or not they succeed as preventative vaccines.
74
1
1
With additional targets
Without additional targets
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
Healthy Helper T
anti−HIV CTLp
With additional targets
Without additional targets
0.5
0.4
0.5
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
50
100
150
200
250
300
0
350
0
50
100
150
days
200
250
1
With additional targets
Without additional targets
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
Healthy Helper T
anti−HIV CTLp
350
1
With additional targets
Without additional targets
0.5
0.4
0.5
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
300
days
0
50
100
150
200
250
300
0
350
days
0
50
100
150
200
250
300
350
days
Figure 5.5. Additional, non-HIV Specific Targets These graphs show the
improved performance in the Cytotoxic Lymphocyte response to HIV in the case
where additional target cells were introduced. The top row of plots compare responses between the case with and without additional target cells with one-week
treatment intervals and ν = 0.3636. The bottom row of plots compare responses
for optimized combinations of fixed numbers of additional targets and interruptions lengths (T = 0.4545, ν = 3.6364 and T = 0.5909,ν = 0, respectively).
The left column compares CTL responses, the right column compares helper-T
responses.
75
Chapter 6
Future Work
We have introduced a framework for developing schedules that exploit the
immune response dynamics in HIV therapy, but a lot more work will be necessary
to implement it. In this section, we discuss various areas of future research we see
as natural and beneficial extensions to what we have done. In particular, we look
at methods of updating and adapting the model to better fit the application,
the design of an appropriate observer for output feedback, and addressing the
problem of viral escape from immune control.
6.1
Model Variations
The design of an appropriate model for use in control is something of an art;
a balance must be made between the complexity and accuracy of the model. In
this section, we discuss the various potential changes that could be made to the
model used in our treatment scheduling algorithm, looking first at alternative
formulations for the model dynamics and compartments, then at the available
controls.
76
6.1.1
Alternative Formulations
The model introduced in [130] is well recommended for use in MPC-based
strategies, for the reasons discussed in Sections 3.3 and 5.2. However, there are
changes in the model that should be considered. The dynamics of resting and
activated T-cells were considered in [1], and a compartmentalized consideration of
these differences is presented in [10]; these dynamics may be important, especially
in validating the model against measured data, as the long-term effects of HIV
infection seem to be governed by activation events.
The incorporation of delays into HIV modeling is discussed in [92], [12], [34],
and good evidence is presented that incorporating the known delays in the system
(such as the approximately one-day delay between infection of a helper-T cell
and the beginning of viral production) results in a much better match between
modeled and measured dynamics. Whether or not this is necessary in our case
is unknown, as our time scales are significantly larger than the delays, and our
algorithm is robust to small errors, but the possibility should still be evaluated.
The emergence of drug-resistant viral strains is always of concern in HIV
therapy; recent results in [124] show that viral evolution continues behind the
blood-brain barrier even during highly effective suppressive therapy. The emergence of drug resistance is modeled in various ways in [128], [130], [10], [11]. The
incorporation of resistance into our models would be interesting for three reasons.
First, a model that estimates the relative risk of emergence of drug-resistant viral strains could be incorporated into our cost functions, allowing us to choose
schedules that minimize the risk. Second, a model incorporating a resistant strain
in competition with a wild-type strain would let us investigate the possibility of
inducing immune control of both strains, and would let us describe conditions
77
under which this might be possible. Third, it may also be possible to incorporate
viral resistance into an observer design (as discussed in Section 6.2), which might
allow us to detect the presence of a resistant strain early on, and switch the drug
profile accordingly.
6.1.2
Additional Therapeutic Agents
There are a number of therapies discussed in Section 2.3 which have been
discarded because they did not have the desired results in the context of therapy
for suppressing viral load. However, as we showed in Section 5.7, the fact that
these therapies failed in their original purpose does not mean that they cannot
be useful in the context of interruption schedules for inducing persistent immune
control of HIV. The Interleukin therapies and the HIV Immunogen vaccines failed
in their original goals, but they did allow for modulation of certain key immunological states and parameters, and therefore represent additional potential control
inputs for use in therapy. Likewise, many other discarded therapies may be of
use to improve the performance of an STI-based therapy regime. As shown in
Section 5.7, the MPC framework we have developed is a powerful tool for investigating potential alternative uses for these agents. The same approach could be
beneficially applied to the various vaccine strategies discussed in Section 2.3.7,
that may be put to beneficial use whether or not they succeed as preventative
vaccines. Also, additional therapies could be of use in reducing the viral load setpoint induced by immune-mediated control, which could help solve the problem
of limited durability described in Section 6.3.
78
6.2
Output Feedback Design
In our development of the MPC scheduling method in Chapter 5, we neglected
the problem of output feedback as applied to the HIV treatment problem. However, as we discussed above, we are limited in which states we can measure. Also,
measurement of certain states involves problems of cost and unavoidable measurement error. For these reasons, implementation of this technique will require
the use of an observer.
The design and use of an observer in an MPC-based controller is not trivial nor
well-established. The work in [83] insures that the specific MPC implementation
which we are using allows the use of any observer that asymptotically approaches
the true value of the state, but does not show how to create such an observer. It
is easy to see, by carrying out the higher derivatives of y, the the system defined
in equation 3.4 is observable in principle from the state y for at least a subset
of the possible states and parameters, but this is for the continuous-time system; whether this observability could be exploited using sampled measurements
is uncertain. Also, even for continuous-time nonlinear systems the theory of observers is not well established, especially for systems with multiple steady-states.
One potential solution, proposed in [68], uses multiple extended Kalman filters
centered around each steady-state. Another approach, detailed in [84], uses a
MPC-type algorithm to obtain estimates; instead of optimizing over a control
space on a forward-looking horizon time, it optimizes over an estimate space over
a backward-looking sampling time. This method may be more appropriate for
use in an MPC setting, but it will need to be adapted for discrete-time implementations; the development in [84] is for continuous-time systems. Very recently,
the authors of [6] address the problem of designing an observer for a sampled
79
continuous-time system without an explicit discrete-time characterization. This
has obvious applicability to our case. The use of output feedback is a necessary
step toward the implementation of our scheduling algorithm, but considerable
theoretical development will be necessary to create an appropriate observer.
6.3
Addressing Escape
As discussed in sections 2.2.4 and 2.3.6, the excitement about the possibility of
immune-mediated control of HIV, either naturally in Long-term non-progressors
or induced through scheduled treatment interruptions, is muted somewhat by the
evidence that the virus tends to eventually escape immune control. Even if this
escape is unavoidable, the drug holiday afforded by an episode of induced immune
control is superior to other drug holidays in that the disease does not progress
during this time. There are also several possible avenues of research that could
improve the durability of immune control, which we discuss in this section.
The escape of the virus from immune control is almost certainly due to mutation, where the mutant virus expands more quickly than the immune system
can adapt, not unlike primary infection. Perhaps the simplest approach to improved durability is frequent monitoring of the patient during episodes of immunemediated control. This way, an escape strain of the virus could be detected early,
before it compromises the helper-T pool, and a second series of interruptions
could be undertaken in order to generate an effective immune response against
the new strain. The dynamics of this second set of interruptions would be slightly
more complex, as we would need to improve the response to the new strain of
the virus without compromising the control of the existing strains. Future work
should be undertaken to determine under what conditions this would be possible.
80
The other potential solution is to prevent the emergence of an escape strain
in the first place. Mutational escape is driven by two factors; viral replication
rate and the presence of selective pressure. We cannot reduce the selective pressure without losing immune control, so our best option in order to prevent the
escape of the virus from immune control is to reduce the viral replication rate.
It is possible that the escape mutants are developing during a transient phase,
before the system has converged to its final steady-state values. If this is the case,
we can reduce the transient period by adjusting our cost functions to penalize
more heavily the time spent away from the equilibrium. However, it may also
be the case that the viral load at the steady-state is simply too high to prevent
escape. If this is the case, we need to examine ways to reduce the steady-state
viral load. This could be accomplished by introducing a low-dose anti-retroviral
regimen once effective immune control is established. Presumably, the lower dose
would result in lower incidences of adverse side effects, and the combined effect
of immune control and anti-retroviral therapy would keep the replication rate of
the virus low enough to prevent mutational escape. However, this approach is
limited; while this results in a transient reduction of viral load to very low levels,
the immune response begins to decay in the absence of antigenic stimulation, and
the viral set point will ultimately converge to some value between the set point
established by immune control in the absence of anti-retroviral therapy and the
set point established by HAART alone. Furthermore, too strong an application
of anti-retroviral therapy for too long will destroy the stability of the immunemediated control; removal of the anti-retroviral therapy will result in convergence
to the steady-state described by equation 3.5 instead of the steady-state described
by equation 3.6. Another possibility would be to introduce therapeutic immunizations, which would increase the antigenic stimulus without increasing the
81
amount of infectious virus. Investigations into the effects of current therapeutic
immunization methods and modeling work to accommodate these options will
allow us to explore these options.
82
Chapter 7
Conclusions
The human immune response to infection is a complex and highly adaptive
system. HIV infection affects one of the primary regulatory cells of the immune
response, and the resulting disease progression is highly nonlinear and often counterintuitive. Although HIV disease takes years to develop, experiment coupled
with mathematical modeling has shown that the underlying dynamics are much
faster.
Although most cases of untreated HIV infection result in progressive immunodeficiency, the study of the few patients whose disease does not progress has
yielded many interesting insights. In particular, it has been shown that long-term
non-progressors are characterized by a highly proliferative, persistent population
of HIV-specific cytotoxic-T cells when compared to patients with progressive infections. This is likely related to the lack of HIV-specific helper-T cell help, which
is necessary for the successful development of cytotoxic-T memory cells. Supporting this hypothesis is the success seen in trials that used interrupted treatment
schedules to attempt to boost the immune responses of HIV patients during acute
infection, and the relative lack of success seen in similar trials during chronic in83
fection. However, the overall low success rates in these trials have led to a waning
enthusiasm with respect to STIs.
The modeling work of Wodarz and Nowak simultaneously explains the existence of the possibility of STI-induced immune control of HIV and the low rates
of success in the trials attempting to induce it. The nonlinear equations governing
the evolution of the immune response to HIV show that the long-term outcome of
infection can fundamentally change based on intervention in early infection, but
these same equations are sensitive to parameter variations, which are inevitable.
It is unlikely that an open-loop treatment schedule exists which will successfully
induce immune control in a large percentage of patients.
We have addressed this problem by introducing an MPC-based framework for
calculating treatment schedules. The model-based feedback scheme gives impressive robustness to model and state disturbances, which could result in a much
higher success rate in inducing immune control if implemented. The particular MPC approach we adapted to our purpose is well-suited to the application.
The freedom with regard to the optimization formulation allows us to use cost
functions easily understood as clinical objectives. Its sample-and-hold nature is
easily implementable as weekly clinical visits, and the fact that it admits the
use of an observer will allow us to use those measurements that are accurate
and cost-effective. For these reasons, we believe that this is the first feedback
control-based approach that has more than theoretical significance, as this is the
first framework that could conceivably be implemented.
We have implemented this framework in MATLAB and explored the robustness properties through simulations. The algorithm showed impressive robustness
to both state and measurement errors, leading us to believe that it will perform
well across a variety of patients. We also adapted the model of infection and
84
the framework in order to explore the use of therapies that increase the level
of activation of helper-T cells, providing additional, non-specific targets to the
virus. The results emphasized the counterintuitive nature of the dynamics, and
the ease with which the changes were made underscore the flexibility of the the
framework.
While the results so far are very promising for the future of MPC-based treatment scheduling, a great deal of work remains to be done. This will demand a
great deal of collaboration across disciplines. We have sketched a few avenues
of research we expect to be worthwhile, which we are pursuing together with
colleagues in immunology.
85
Bibliography
[1] H.K. Altes, D. Wodarz, and V.A.A. Jansen. The dual role of CD4 T helper
cells in the infection dynamics of HIV and their importance for vaccination.
J Theor Biol, 214(4):633–46, Feb 2002.
[2] M. Altfeld, T.M. Allen, X.G. Yu, M.N. Johnston, D. Agrawal, B.T. Korber,
D.C. Montefiori, D.H. O’Connor, B.T. Davis, P.K. Lee, E.L. Maier, J. Harlow, P.J.R. Goulder, C. Brander, E.S. Rosenberg, and B.D. Walker. HIV-1
superinfection despite broad CD8+ T-cell responses containing replication
of the primary virus. Nature, 420(6914):434–9, Nov 2002.
[3] M. Altfeld, J. van Lunzen, N. Frahm, X.G. Yu, C. Schneider, R.L. Eldridge,
M.E. Feeney, D. Meyer-Olson, H. Stellbrink, and B.D. Walker. Expansion
of pre-existing, lymph node-localized CD8+ T cells during supervised treatment interruptions in chronic HIV-1 infection. J Clin Invest, 109(6):837–43,
Mar 2002.
[4] M. Altfeld, E.S. Rosenberg, R. Shankarappa, J.S. Mukherjee, F.M. Hecht,
R.L. Eldridge, M.M. Addo, S.H. Poon, M.N. Phillips, G.K. Robbins, P.E.
Sax, S. Boswell, J.O. Kahn, C. Brander, P.J. Goulder, J.A. Levy, J.I.
Mullins, and B.D. Walker. Cellular immune responses and viral diversity
86
in individuals treated during acute and early HIV-1 infection. J Exp Med,
193(2):169–80, Jan 2001.
[5] V. Appay, L. Papagno, C.A. Spina, P. Hansasuta, A. King, L. Jones, G.S.
Ogg, S. Little, A.J. McMichael, D.D. Richman, and S.L. Rowland-Jones.
Dynamics of T cell responses in HIV infection. J Immunol, 168(7):3660–6,
Apr 2002.
[6] M. Arcak and D. Nesic. A framework for nonlinear sampled-data observer
design via approximate discrete-time models and emulation. Automatica,
40(11):1931–1938, November 2004.
[7] R.A. Arnaout, M.A. Nowak, and D. Wodarz. HIV-1 dynamics revisited:
biphasic decay by cytotoxic T lymphocyte killing? Proc R Soc Lond B Biol
Sci, 267(1450):1347–54, Jul 2000.
[8] A. Asachenkov, G. Marchuk, R. Mohler, and S. Zuev. Disease Dynamics.
Systems and Control: Foundations and Applications. Birkhauser, Boston,
1994.
[9] B. Autran and G. Carcelain. AIDS. Boosting immunity to HIV–can the
virus help? Science, 290(5493):946–9, Nov 2000.
[10] S.H. Bajaria, G. Webb, M. Cloyd, and D. Kirschner. Dynamics of naive
and memory CD4+ T lymphocytes in HIV-1 disease progression. J Acquir
Immune Defic Syndr, 30(1):41–58, May 2002.
[11] S.H. Bajaria, G. Webb, and D.E. Kirschner. Predicting differential responses to structured treatment interruptions during HAART. Bull Math
Biol, 66(5):1093–118, Sep 2004.
87
[12] H.T. Banks, D.M. Bortz, and S.E. Holte. Incorporation of variability into
the modeling of viral delays in HIV infection dynamics. Math Biosci,
183(1):63–91, May 2003.
[13] M.R. Betts, D.R. Ambrozak, D.C. Douek, S. Bonhoeffer, J.M. Brenchley,
J.P. Casazza, R.A. Koup, and L.J. Picker. Analysis of total human immunodeficiency virus (HIV)-specific CD4(+) and CD8(+) T-cell responses:
relationship to viral load in untreated HIV infection. J Virol, 75(24):11983–
91, Dec 2001.
[14] M.J. Bevan. Helping the CD8(+) T-cell response. Nat Rev Immunol,
4(8):595–602, Aug 2004.
[15] N. Bhardwaj and B.D. Walker. Immunotherapy for AIDS virus infections:
cautious optimism for cell-based vaccine. Nat Med, 9(1):13–4, Jan 2003.
[16] S.M. Blower, K. Koelle, D.E. Kirschner, and J. Mills. Live attenuated HIV
vaccines: predicting the tradeoff between efficacy and safety. Proc Natl
Acad Sci U S A, 98(6):3618–23, Mar 2001.
[17] M. Bofill, A. Mocroft, M. Lipman, E. Medina, N.J. Borthwick, C.A.
Sabin, A. Timms, M. Winter, L. Baptista, M.A. Johnson, C.A. Lee,
A.N. Phillips, and G. Janossy. Increased numbers of primed activated
CD8+CD38+CD45RO+ T cells predict the decline of CD4+ T cells in
HIV-1-infected patients. AIDS, 10(8):827–34, Jul 1996.
[18] S. Bonhoeffer and M.A. Nowak. Pre-existence and emergence of drug resistance in HIV-1 infection. Proc R Soc Lond B Biol Sci, 264(1382):631–7,
May 1997.
88
[19] S. Bonhoeffer, M. Rembiszewski, G.M. Ortiz, and D.F. Nixon. Risks and
benefits of structured antiretroviral drug therapy interruptions in HIV-1
infection. AIDS, 14(15):2313–22, Oct 2000.
[20] M.E. Brandt and G. Chen. Feedback control of a biodynamical model of
HIV-1. IEEE Trans Biomed Eng, 48(7):754–9, Jul 2001.
[21] K. Brinkman and T. Kakuda. Mitochondrial toxicity of nucleoside analogue
reverse transcriptase inhibitors: a looming obstacle for long-term antiretroviral therapy? Curr Opin Infect Dis, 13(1):5–11, Feb 2000.
[22] R.P. Bucy and J.M. Kilby. Perspectives on inducing efficient immune control of HIV-1 replication–a new goal for HIV therapeutics? AIDS, 15 Suppl
2:S36–42, Feb 2001.
[23] R.M. Buller, K.L. Holmes, A. Hgin, T.N. Frederickson, and H.C. Morse.
Induction of cytotoxic T-cell responses in vivo in the absence of CD4 helper
cells. Nature, 328(6125):77–9, 1987.
[24] H.J. Chang, H. Shim, and J.H. Seo. Control of immune response of hiv
infection model by gradual reduction of dose. In Proceedings of the 43rd
IEEE Conference on Decision and Control, December 2004 (to appear).
[25] S. Crotty, P. Felgner, H. Davies, J. Glidewell, L. Villarreal, and R. Ahmed.
Cutting edge: long-term B cell memory in humans after smallpox vaccination. J Immunol, 171(10):4969–73, Nov 2003.
[26] R.V. Culshaw, S. Ruan, and R.J. Spiteri. Optimal HIV treatment by maximising immune response. J Math Biol, 48(5):545–62, May 2004.
[27] R.T. Davey, N. Bhat, C. Yoder, T.W. Chun, J.A. Metcalf, R. Dewar,
V. Natarajan, R.A. Lempicki, J.W. Adelsberger, K.D. Miller, J.A. Kovacs,
89
M.A. Polis, R.E. Walker, J. Falloon, H. Masur, D. Gee, M. Baseler, D.S.
Dimitrov, A.S. Fauci, and H.C. Lane. HIV-1 and T cell dynamics after
interruption of highly active antiretroviral therapy (HAART) in patients
with a history of sustained viral suppression. Proc Natl Acad Sci U S A,
96(26):15109–14, Dec 1999.
[28] F.M. de Souza. Modeling the dynamics of HIV-1 and CD4 and CD8 lymphocytes. IEEE Eng Med Biol Mag, 18(1):21–4, 1999.
[29] M. Dean, M. Carrington, C. Winkler, G.A. Huttley, M.W. Smith, R. Allikmets, J.J. Goedert, S.P. Buchbinder, E. Vittinghoff, E. Gomperts, S. Donfield, D. Vlahov, R. Kaslow, A. Saah, C. Rinaldo, R. Detels, and S.J.
O’Brien. Genetic restriction of HIV-1 infection and progression to AIDS
by a deletion allele of the CKR5 structural gene. Hemophilia Growth
and Development Study, Multicenter AIDS Cohort Study, Multicenter
Hemophilia Cohort Study, San Francisco City Cohort, ALIVE Study. Science, 273(5283):1856–62, Sep 1996.
[30] D.S. Dimitrov and M.A. Martin. HIV results in the frame. CD4+ cell
turnover. Nature, 375(6528):194–5; author reply 198, May 1995.
[31] G. D’Offizi, C. Montesano, C. Agrati, C. Gioia, M. Amicosante, S. Topino, P. Narciso, L.P. Pucillo, G. Ippolito, and F. Poccia. Expansion of
pre-terminally differentiated CD8 T cells in chronic HIV-positive patients
presenting a rapid viral rebound during structured treatment interruption.
AIDS, 16(18):2431–8, Dec 2002.
[32] D.C. Douek, J.M. Brenchley, M.R. Betts, D.R. Ambrozak, B.J. Hill,
Y. Okamoto, J.P. Casazza, J. Kuruppu, K. Kunstman, S. Wolinsky,
90
Z. Grossman, M. Dybul, A. Oxenius, D.A. Price, M. Connors, and R.A.
Koup.
HIV preferentially infects HIV-specific CD4+ T cells.
Nature,
417(6884):95–8, May 2002.
[33] M. Dybul, T.W. Chun, C. Yoder, B. Hidalgo, M. Belson, K. Hertogs,
B. Larder, R.L. Dewar, C.H. Fox, C.W. Hallahan, J.S. Justement, S.A.
Migueles, J.A. Metcalf, R.T. Davey, M. Daucher, P. Pandya, M. Baseler,
D.J. Ward, and A.S. Fauci. Short-cycle structured intermittent treatment
of chronic HIV infection with highly active antiretroviral therapy: effects
on virologic, immunologic, and toxicity parameters. Proc Natl Acad Sci U
S A, 98(26):15161–6, Dec 2001.
[34] P. Essunger and A.S. Perelson. Modeling HIV infection of CD4+ T-cell
subpopulations. J Theor Biol, 170(4):367–91, Oct 1994.
[35] A.S. Fauci and G. Pantaleo, editors. Immunopathogenesis of HIV Infection.
Springer, New York, 1997.
[36] D. Finzi, J. Blankson, J.D. Siliciano, J.B. Margolick, K. Chadwick, T. Pierson, K. Smith, J. Lisziewicz, F. Lori, C. Flexner, T.C. Quinn, R.E. Chaisson, E. Rosenberg, B. Walker, S. Gange, J. Gallant, and R.F. Siliciano.
Latent infection of CD4+ T cells provides a mechanism for lifelong persistence of HIV-1, even in patients on effective combination therapy. Nat
Med, 5(5):512–7, May 1999.
[37] R.A. Fouchier,
L. Meyaard,
M. Brouwer,
E. Hovenkamp,
and
H. Schuitemaker. Broader tropism and higher cytopathicity for CD4+ T
cells of a syncytium-inducing compared to a non-syncytium-inducing HIV-1
isolate as a mechanism for accelerated CD4+ T cell decline in vivo. Virology, 219(1):87–95, May 1996.
91
[38] S.D.W. Frost. Dynamics and evolution of HIV-1 during structured treatment interruptions. AIDS Rev, 4(3):119–27, 2002.
[39] S.D.W. Frost, J. Martinez-Picado, L. Ruiz, B. Clotet, and A.J.L. Brown.
Viral dynamics during structured treatment interruptions of chronic human
immunodeficiency virus type 1 infection. J Virol, 76(3):968–79, Feb 2002.
[40] F. Garcia, M. Plana, G.M. Ortiz, S. Bonhoeffer, A. Soriano, C. Vidal,
A. Cruceta, M. Arnedo, C. Gil, G. Pantaleo, T. Pumarola, T. Gallart, D.F.
Nixon, J.M. Mir, and J.M. Gatell. The virological and immunological consequences of structured treatment interruptions in chronic HIV-1 infection.
AIDS, 15(9):F29–40, Jun 2001.
[41] F. Garcia, M. Plana, C. Vidal, A. Cruceta, W.A. O’Brien, G. Pantaleo,
T. Pumarola, T. Gallart, J.M. Mir, and J.M. Gatell. Dynamics of viral load
rebound and immunological changes after stopping effective antiretroviral
therapy. AIDS, 13(11):F79–86, Jul 1999.
[42] T. Gegeny. Surviving with HIV in the HAART era: emerging challenges.
Res Initiat Treat Action, 6(3):12–4, Sep 2000.
[43] J.V. Giorgi, L.E. Hultin, J.A. McKeating, T.D. Johnson, B. Owens, L.P.
Jacobson, R. Shih, J. Lewis, D.J. Wiley, J.P. Phair, S.M. Wolinsky, and
R. Detels. Shorter survival in advanced human immunodeficiency virus
type 1 infection is more closely associated with T lymphocyte activation
than with plasma virus burden or virus chemokine coreceptor usage. J
Infect Dis, 179(4):859–70, Apr 1999.
[44] S.E. Gloster, P. Newton, D. Cornforth, J.D. Lifson, I. Williams, G.M. Shaw,
and P. Borrow. Association of strong virus-specific CD4 T cell responses
92
with efficient natural control of primary HIV-1 infection. AIDS, 18(5):749–
55, Mar 2004.
[45] R.A. Goldsby, T.J. Kindt, and B.A. Osborne. Kuby immunology. W.H.
Freeman and Co., 4 edition, 2000.
[46] C.M. Gray, J. Lawrence, E.A. Ranheim, M. Vierra, M. Zupancic, M. Winters, J. Altman, J. Montoya, A. Zolopa, J. Schapiro, A.T. Haase, and T.C.
Merigan. Highly active antiretroviral therapy results in HIV type 1 suppression in lymph nodes, increased pools of naive T cells, decreased pools
of activated T cells, and diminished frequencies of peripheral activated HIV
type 1-specific CD8+ T cells. AIDS Res Hum Retroviruses, 16(14):1357–69,
Sep 2000.
[47] G. Grimm, M. J. Messina, A. R. Teel, and S. E. Tuna. Nominally robust
model predictive control with state constraints. In Proceedings of 42nd
IEEE Conference on Decision and Control, volume 2, page 1413?1418, December 2003.
[48] G. Grimm, M.J. Messina, A.R. Teel, and S.E. Tuna. Model predictive control when a local control lyapunov function is not available. In Proceedings
of the 2003 American Control Conference, volume 5, pages 4125–4130, June
2003.
[49] G. Grimm, M.J. Messina, S.E. Tuna, and A.R. Teel. Examples of zero robustness in constrained model predictive control. In 42nd IEEE Conference
on Decision and Control, volume 4, pages 3724–3729, 2003.
[50] G. Grimm, M.J. Messina, S.E. Tuna, and A.R. Teel. Examples when nonlin-
93
ear model predictive control is nonrobust. Automatica, 40(10):1729–1738,
October 2004.
[51] T. Harrer, E. Harrer, S.A. Kalams, P. Barbosa, A. Trocha, R.P. Johnson,
T. Elbeik, M.B. Feinberg, S.P. Buchbinder, and B.D. Walker. Cytotoxic T
lymphocytes in asymptomatic long-term nonprogressing HIV-1 infection.
Breadth and specificity of the response and relation to in vivo viral quasispecies in a person with prolonged infection and low viral load. J Immunol,
156(7):2616–23, Apr 1996.
[52] T. Harrer, E. Harrer, S.A. Kalams, T. Elbeik, S.I. Staprans, M.B. Feinberg,
Y. Cao, D.D. Ho, T. Yilma, A.M. Caliendo, R.P. Johnson, S.P. Buchbinder,
and B.D. Walker. Strong cytotoxic T cell and weak neutralizing antibody
responses in a subset of persons with stable nonprogressing HIV type 1
infection. AIDS Res Hum Retroviruses, 12(7):585–92, May 1996.
[53] P.A. Haslett, D.F. Nixon, Z. Shen, M. Larsson, W.I. Cox, R. Manandhar,
S.M. Donahoe, and G. Kaplan. Strong human immunodeficiency virus
(HIV)-specific CD4+ T cell responses in a cohort of chronically infected
patients are associated with interruptions in anti-HIV chemotherapy. J
Infect Dis, 181(4):1264–72, Apr 2000.
[54] M.D. Hazenberg, S.A. Otto, B.H.B. van Benthem, M.T.L. Roos, R.A.
Coutinho, J.M.A. Lange, D. Hamann, M. Prins, and F. Miedema. Persistent immune activation in HIV-1 infection is associated with progression
to AIDS. AIDS, 17(13):1881–8, Sep 2003.
[55] D.D. Ho, A.U. Neumann, A.S. Perelson, W. Chen, J.M. Leonard, and
M. Markowitz. Rapid turnover of plasma virions and CD4 lymphocytes
in HIV-1 infection. Nature, 373(6510):123–6, Jan 1995.
94
[56] C. Hoffmann and B.S. Kamps, editors. HIV Medicine. Flying Publisher,
2003.
[57] P.W. Hunt, J.N. Martin, E. Sinclair, B. Bredt, E. Hagos, H. Lampiris,
and S.G. Deeks. T cell activation is associated with lower CD4+ T cell
gains in human immunodeficiency virus-infected patients with sustained
viral suppression during antiretroviral therapy. J Infect Dis, 187(10):1534–
43, May 2003.
[58] E.M. Janssen, E.E. Lemmens, T. Wolfe, U. Christen, M.G. von Herrath,
and S.P. Schoenberger. CD4+ T cells are required for secondary expansion
and memory in CD8+ T lymphocytes. Nature, 421(6925):852–6, Feb 2003.
[59] X. Jin, D.E. Bauer, S.E. Tuttleton, S. Lewin, A. Gettie, J. Blanchard, C.E.
Irwin, J.T. Safrit, J. Mittler, L. Weinberger, L.G. Kostrikis, L. Zhang, A.S.
Perelson, and D.D. Ho. Dramatic rise in plasma viremia after CD8(+) T
cell depletion in simian immunodeficiency virus-infected macaques. J Exp
Med, 189(6):991–8, Mar 1999.
[60] S.A. Kalams, S.P. Buchbinder, E.S. Rosenberg, J.M. Billingsley, D.S. Colbert, N.G. Jones, A.K. Shea, A.K. Trocha, and B.D. Walker. Association
between virus-specific cytotoxic T-lymphocyte and helper responses in human immunodeficiency virus type 1 infection. J Virol, 73(8):6715–20, Aug
1999.
[61] S.A. Kalams, P.J. Goulder, A.K. Shea, N.G. Jones, A.K. Trocha, G.S.
Ogg, and B.D. Walker. Levels of human immunodeficiency virus type 1specific cytotoxic T-lymphocyte effector and memory responses decline after
suppression of viremia with highly active antiretroviral therapy. J Virol,
73(8):6721–8, Aug 1999.
95
[62] S.A. Kalams and B.D. Walker. The critical need for CD4 help in maintaining effective cytotoxic T lymphocyte responses. J Exp Med, 188(12):2199–
204, Dec 1998.
[63] D. Kaufmann, M. Lichterfeld, M. Altfeld, T. Allen, M. Johnston, P. Lee,
B. Wagner, E. Kalife, D. Strick, E. Rosenberg, and B. D. Walker. Limited
durability of immune control following treated acute hiv infection. In The
11th Conference on Retroviruses and Opportunistic Infections, San Francisco, CA, February 2004.
[64] D.E. Kaufmann and E.S. Rosenberg. The value of preserving HIV-specific
immune responses. J HIV Ther, 8(1):19–25, Feb 2003.
[65] D. Kirschner, S. Lenhart, and S. Serbin. Optimal control of the chemotherapy of HIV. J Math Biol, 35(7):775–92, Aug 1997.
[66] D. Kirschner and G.F. Webb. A mathematical model of combined drug
therapy of HIV infection. Journal of Theoretical Medicine, 1:25–34, 1997.
[67] R.A. Koup, J.T. Safrit, Y. Cao, C.A. Andrews, G. McLeod, W. Borkowsky,
C. Farthing, and D.D. Ho. Temporal association of cellular immune responses with the initial control of viremia in primary human immunodeficiency virus type 1 syndrome. J Virol, 68(7):4650–5, Jul 1994.
[68] A.J. Krener and A. Duarte. A hybrid computational approach to nonlinear
estimation. In Proceedings of the 35th Conference on Decision and Control,
pages 1815–1819, 1996.
[69] Norman L. Letvin. Progress Toward an HIV Vaccine. Annu Rev Med, Aug
2004.
96
[70] A.M.L. Lever, editor. The Molecular Biology of HIV/AIDS. Molecular
Medical Science Series. John Wiley and Sons, Ltd., 1996.
[71] J.D. Lifson, J.L. Rossio, R. Arnaout, L. Li, T.L. Parks, D.K. Schneider,
R.F. Kiser, V.J. Coalter, G. Walsh, R.J. Imming, B. Fisher, B.M. Flynn,
N. Bischofberger, M. Piatak, V.M. Hirsch, M.A. Nowak, and D. Wodarz.
Containment of simian immunodeficiency virus infection: cellular immune
responses and protection from rechallenge following transient postinoculation antiretroviral treatment. J Virol, 74(6):2584–93, Mar 2000.
[72] J.D. Lifson, J.L. Rossio, M. Piatak, T. Parks, L. Li, R. Kiser, V. Coalter, B. Fisher, B.M. Flynn, S. Czajak, V.M. Hirsch, K.A. Reimann, J.E.
Schmitz, J. Ghrayeb, N. Bischofberger, M.A. Nowak, R.C. Desrosiers, and
D. Wodarz. Role of CD8(+) lymphocytes in control of simian immunodeficiency virus infection and resistance to rechallenge after transient early
antiretroviral treatment. J Virol, 75(21):10187–99, Nov 2001.
[73] J. Lisziewicz, E. Rosenberg, J. Lieberman, H. Jessen, L. Lopalco, R. Siliciano, B. Walker, and F. Lori. Control of HIV despite the discontinuation
of antiretroviral therapy. N Engl J Med, 340(21):1683–4, May 1999.
[74] F. Lori, M.G. Lewis, J. Xu, G. Varga, D.E. Zinn, C. Crabbs, W. Wagner,
J. Greenhouse, P. Silvera, J. Yalley-Ogunro, C. Tinelli, and J. Lisziewicz.
Control of SIV rebound through structured treatment interruptions during
early infection. Science, 290(5496):1591–3, Nov 2000.
[75] F Lori, R Maserati, A Foli, E Seminari, J Timpone, and J Lisziewicz.
Structured treatment interruptions to control HIV-1 infection. Lancet,
355(9200):287–8, Jan 2000.
97
[76] P.M Lydyard, A. Whelan, and M.W. Fanger. Immunology. BIOS Scientific
Publishers, 2004.
[77] L.T. Madakamutil, U. Christen, C.J. Lena, Y. Wang-Zhu, A. Attinger,
M. Sundarrajan, W. Ellmeier, M.G. von Herrath, P. Jensen, D.R. Littman,
and H. Cheroutre. CD8alphaalpha-mediated survival and differentiation of
CD8 memory T cell precursors. Science, 304(5670):590–3, Apr 2004.
[78] C. Manegold, C. Hannoun, A. Wywiol, M. Dietrich, S. Polywka, C.B. Chiwakata, and S. Gnther. Reactivation of hepatitis B virus replication accompanied by acute hepatitis in patients receiving highly active antiretroviral
therapy. Clin Infect Dis, 32(1):144–8, Jan 2001.
[79] R.A. Manz, M. Lhning, G. Cassese, A. Thiel, and A. Radbruch. Survival of
long-lived plasma cells is independent of antigen. Int Immunol, 10(11):1703–
11, Nov 1998.
[80] M. Di Mascio, R.M. Ribeiro, M. Markowitz, D.D. Ho, and A.S. Perelson. Modeling the long-term control of viremia in HIV-1 infected patients
treated with antiretroviral therapy. Math Biosci, 188:47–62, 2004.
[81] D. Mayne, J.B. Rawlings, C.V. Rao, and P. Scokaert. Constrained model
predictive control: Stability and optimality. Automatica, 36:789–814, 2000.
[82] A.J. McMichael and S.L. Rowland-Jones. Cellular immune responses to
HIV. Nature, 410(6831):980–7, Apr 2001.
[83] M.J. Messina, S.E. Tuna, and A.R. Teel. Robust stabilization of discretetime nonlinear systems by certainty equivalence output feedback with applications to model predictive control. In Proceedings of the 2004 American
Control Conference, June 2004.
98
[84] H. Michalska and D.Q. Mayne. Moving horizon observers and observerbased control. IEEE Transactions on Automatic Control, 40(6):995–1006,
June 1995.
[85] S.A. Migueles, A.C. Laborico, W.L. Shupert, M.S. Sabbaghian, R. Rabin,
C.W. Hallahan, D. Van Baarle, S. Kostense, F. Miedema, M. McLaughlin,
L. Ehler, J. Metcalf, S. Liu, and M. Connors. HIV-specific CD8+ T cell
proliferation is coupled to perforin expression and is maintained in nonprogressors. Nat Immunol, 3(11):1061–8, Nov 2002.
[86] D.A. Mitchell, S.K. Nair, and E. Gilboa. Dendritic cell/macrophage precursors capture exogenous antigen for MHC class I presentation by dendritic
cells. Eur J Immunol, 28(6):1923–33, Jun 1998.
[87] A. Mocroft, L. Ruiz, P. Reiss, B. Ledergerber, C. Katlama, A. Lazzarin,
F.D. Goebel, A.N. Phillips, B. Clotet, and J.D. Lundgren. Virological
rebound after suppression on highly active antiretroviral therapy. AIDS,
17(12):1741–51, Aug 2003.
[88] L. Mollet, T.S. Li, A. Samri, C. Tournay, R. Tubiana, V. Calvez, P. Debr,
C. Katlama, and B. Autran. Dynamics of HIV-specific CD8+ T lymphocytes with changes in viral load.The RESTIM and COMET Study Groups.
J Immunol, 165(3):1692–704, Aug 2000.
[89] L.J. Montaner. Structured treatment interruptions to control HIV-1 and
limit drug exposure. Trends Immunol, 22(2):92–6, Feb 2001.
[90] J.P. Moore. Coreceptors: implications for HIV pathogenesis and therapy.
Science, 276(5309):51–2, Apr 1997.
99
[91] R.B. Moss, F.C. Jensen, and D.J. Carlo. Insights into HIV-specific immune
function: implications for therapy and prevention in the new millennium.
Clin Immunol, 95(2):79–84, May 2000.
[92] P.W. Nelson and A.S. Perelson. Mathematical analysis of delay differential
equation models of HIV-1 infection. Math Biosci, 179(1):73–94, 2002.
[93] A.U. Neumann, R. Tubiana, V. Calvez, C. Robert, T.S. Li, H. Agut, B. Autran, and C. Katlama. HIV-1 rebound during interruption of highly active
antiretroviral therapy has no deleterious effect on reinitiated treatment.
Comet Study Group. AIDS, 13(6):677–83, Apr 1999.
[94] M.A. Nowak and R.M. May. Virus Dynamics: Mathematical Principles of
Immunology and Virology. Oxford University Press, New York, 2000.
[95] A.F. Ochsenbein, D.D. Pinschewer, S. Sierro, E. Horvath, H. Hengartner,
and R.M. Zinkernagel. Protective long-term antibody memory by antigendriven and T help-dependent differentiation of long-lived memory B cells to
short-lived plasma cells independent of secondary lymphoid organs. Proc
Natl Acad Sci U S A, 97(24):13263–8, Nov 2000.
[96] M.G. Omerod, editor. Flow cytometry: a practical approach. Oxford University Press, 3 edition, 2000.
[97] G.M. Ortiz, D.F. Nixon, A. Trkola, J. Binley, X. Jin, S. Bonhoeffer,
P.J. Kuebler, S.M. Donahoe, M.A. Demoitie, W.M. Kakimoto, T. Ketas,
B. Clas, J.J. Heymann, L. Zhang, Y. Cao, A. Hurley, J.P. Moore, D.D.
Ho, and M. Markowitz. HIV-1-specific immune responses in subjects who
temporarily contain virus replication after discontinuation of highly active
antiretroviral therapy. J Clin Invest, 104(6):R13–8, Sep 1999.
100
[98] G.M. Ortiz, M. Wellons, J. Brancato, H.T. Vo, R.L. Zinn, D.E. Clarkson,
K. Van Loon, S. Bonhoeffer, G.D. Miralles, D. Montefiori, J.A. Bartlett,
and D.F. Nixon. Structured antiretroviral treatment interruptions in chronically HIV-1-infected subjects. Proc Natl Acad Sci U S A, 98(23):13288–93,
Nov 2001.
[99] M.A. Ostrowski, J.X. Gu, C. Kovacs, J. Freedman, M.A. Luscher, and K.S.
MacDonald. Quantitative and qualitative assessment of human immunodeficiency virus type 1 (HIV-1)-specific CD4+ T cell immunity to gag in
HIV-1-infected individuals with differential disease progression: reciprocal
interferon-gamma and interleukin-10 responses. J Infect Dis, 184(10):1268–
78, Nov 2001.
[100] A. Oxenius, D.A. Price, H.F. Gnthard, S.J. Dawson, C. Fagard, L. Perrin, M. Fischer, R. Weber, M. Plana, F. Garca, B. Hirschel, A. McLean,
and R.E. Phillips. Stimulation of HIV-specific cellular immunity by structured treatment interruption fails to enhance viral control in chronic HIV
infection. Proc Natl Acad Sci U S A, 99(21):13747–52, Oct 2002.
[101] B.E. Palmer, E. Boritz, N. Blyveis, and C.C. Wilson. Discordance between frequency of human immunodeficiency virus type 1 (HIV-1)-specific
gamma interferon-producing CD4(+) T cells and HIV-1-specific lymphoproliferation in HIV-1-infected subjects with active viral replication. J Virol, 76(12):5925–36, Jun 2002.
[102] Papagno, Spina, Marchant, Salio, Rufer, Little, Dong, Chesney, Waters,
Easterbrook, Dunbar, Shepherd, Cerundolo, Emery, Griffiths, Conlon,
McMichael, Richman, Rowland-Jones, and Appay. Immune Activation
101
and CD8(+) T-Cell Differentiation towards Senescence in HIV-1 Infection.
PLoS Biol, 2(2):E20, Feb 2004.
[103] E. Papasavvas, G.M. Ortiz, R. Gross, J. Sun, E.C. Moore, J.J. Heymann,
M. Moonis, J.K. Sandberg, L.A. Drohan, B. Gallagher, J. Shull, D.F. Nixon,
J.R. Kostman, and L.J. Montaner. Enhancement of human immunodeficiency virus type 1-specific CD4 and CD8 T cell responses in chronically infected persons after temporary treatment interruption. J Infect Dis,
182(3):766–75, Sep 2000.
[104] A.S. Perelson, P. Essunger, Y. Cao, M. Vesanen, A. Hurley, K. Saksela,
M. Markowitz, and D.D. Ho. Decay characteristics of HIV-1-infected compartments during combination therapy. Nature, 387(6629):188–91, May
1997.
[105] A.S. Perelson, P. Essunger, and D.D. Ho. Dynamics of HIV-1 and CD4+
lymphocytes in vivo. AIDS, 11 Suppl A:S17–24, 1997.
[106] A.S. Perelson and P.W. Nelson. Mathematical analysis of hiv-1 dynamics
in vivo. SIAM Review, 41(1):3–44, 1999.
[107] A.S. Perelson, A.U. Neumann, M. Markowitz, J.M. Leonard, and D.D. Ho.
HIV-1 dynamics in vivo: virion clearance rate, infected cell life-span, and
viral generation time. Science, 271(5255):1582–6, Mar 1996.
[108] A.N. Phillips, S. Staszewski, F. Lampe, M.S. Youle, S. Klauke, M. Bickel,
C.A. Sabin, H.W. Doerr, M.A. Johnson, C. Loveday, and V. Miller. Human immunodeficiency virus rebound after suppression to <400 copies/mL
during initial highly active antiretroviral therapy regimens, according to
102
prior nucleoside experience and duration of suppression.
J Infect Dis,
186(8):1086–91, Oct 2002.
[109] M. Piatak, M.S. Saag, L.C. Yang, S.J. Clark, J.C. Kappes, K.C. Luk, B.H.
Hahn, G.M. Shaw, and J.D. Lifson. Determination of plasma viral load
in HIV-1 infection by quantitative competitive polymerase chain reaction.
AIDS, 7 Suppl 2:S65–71, Nov 1993.
[110] C.J. Pitcher, C. Quittner, D.M. Peterson, M. Connors, R.A. Koup, V.C.
Maino, and L.J. Picker. HIV-1-specific CD4+ T cells are detectable in
most individuals with active HIV-1 infection, but decline with prolonged
viral suppression. Nat Med, 5(5):518–25, May 1999.
[111] R.M. Ribeiro and S. Bonhoeffer. Production of resistant HIV mutants
during antiretroviral therapy. Proc Natl Acad Sci U S A, 97(14):7681–6,
Jul 2000.
[112] R.M. Ribeiro, H. Mohri, D.D. Ho, and A.S. Perelson. In vivo dynamics of T
cell activation, proliferation, and death in HIV-1 infection: why are CD4+
but not CD8+ T cells depleted? Proc Natl Acad Sci U S A, 99(24):15572–7,
Nov 2002.
[113] Gregory K Robbins, Marylyn M Addo, Hien Troung, Almas Rathod, Kathy
Habeeb, Benjamin Davis, Howard Heller, Nesli Basgoz, Bruce D Walker,
and Eric S Rosenberg. Augmentation of HIV-1-specific T helper cell responses in chronic HIV-1 infection by therapeutic immunization. AIDS,
17(8):1121–6, May 2003.
[114] B. Rodes, C. Toro, E. Paxinos, E. Poveda, M. Martinez-Padial, J.M. Benito,
V. Jimenez, T. Wrin, S. Bassani, and V. Soriano. Differences in disease
103
progression in a cohort of long-term non-progressors after more than 16
years of HIV-1 infection. AIDS, 18(8):1109–16, May 2004.
[115] E.S. Rosenberg, M. Altfeld, S.H. Poon, M.N. Phillips, B.M. Wilkes, R.L.
Eldridge, G.K. Robbins, R.T. D’Aquila, P.J. Goulder, and B.D. Walker.
Immune control of HIV-1 after early treatment of acute infection. Nature,
407(6803):523–6, Sep 2000.
[116] E.S. Rosenberg, J.M. Billingsley, A.M. Caliendo, S.L. Boswell, P.E. Sax,
S.A. Kalams, and B.D. Walker. Vigorous HIV-1-specific CD4+ T cell responses associated with control of viremia. Science, 278(5342):1447–50,
Nov 1997.
[117] E.S. Rosenberg, L. LaRosa, T. Flynn, G. Robbins, and B.D. Walker. Characterization of HIV-1-specific T-helper cells in acute and chronic infection.
Immunol Lett, 66(1-3):89–93, Mar 1999.
[118] E.S. Rosenberg and B.D. Walker. HIV type 1-specific helper T cells: a
critical host defense. AIDS Res Hum Retroviruses, 14 Suppl 2:S143–7, Jun
1998.
[119] L. Ruiz, G. Carcelain, J. Martnez-Picado, S. Frost, S. Marfil, R. Paredes,
J. Romeu, E. Ferrer, K. Morales-Lopetegi, B. Autran, and B. Clotet. HIV
dynamics and T-cell immunity after three structured treatment interruptions in chronic HIV-1 infection. AIDS, 15(9):F19–27, Jun 2001.
[120] L. Ruiz, J. Martinez-Picado, J. Romeu, R. Paredes, M.K. Zayat, S. Marfil,
E. Negredo, G. Sirera, C. Tural, and B. Clotet. Structured treatment interruption in chronically HIV-1 infected patients after long-term viral suppression. AIDS, 14(4):397–403, Mar 2000.
104
[121] J.E. Schmitz, M.J. Kuroda, S. Santra, V.G. Sasseville, M.A. Simon,
M.A. Lifton, P. Racz, K. Tenner-Racz, M. Dalesandro, B.J. Scallon,
J. Ghrayeb, M.A. Forman, D.C. Montefiori, E.P. Rieber, N.L. Letvin, and
K.A. Reimann. Control of viremia in simian immunodeficiency virus infection by CD8+ lymphocytes. Science, 283(5403):857–60, Feb 1999.
[122] H. Shim, S. Han, C.C. Chung, S.W. Nam, and J.H. Seo. Optimal scheduling
of drug treatment for hiv infection: Continuous dose control and receding
horizon control. International Journal of Control, Automation, and Systems, 1(3):282–288, September 2003.
[123] L.J. Sigal, S. Crotty, R. Andino, and K.L. Rock. Cytotoxic T-cell immunity to virus-infected non-haematopoietic cells requires presentation of
exogenous antigen. Nature, 398(6722):77–80, Mar 1999.
[124] T.K. Smit, B.J. Brew, W. Tourtellotte, S. Morgello, B.B. Gelman, and N.K.
Saksena. Independent Evolution of Human Immunodeficiency Virus (HIV)
Drug Resistance Mutations in Diverse Areas of the Brain in HIV-Infected
Patients, with and without Dementia, on Antiretroviral Treatment. J Virol,
78(18):10133–48, Sep 2004.
[125] J. Sprent and C.D. Surh. T cell memory. Annu Rev Immunol, 20:551–79,
2002.
[126] R.F. Stengel, R.M. Ghigliazza, and N.V. Kulkarni. Optimal enhancement
of immune response. Bioinformatics, 18(9):1227–35, Sep 2002.
[127] X. Wei, S.K. Ghosh, M.E. Taylor, V.A. Johnson, E.A. Emini, P. Deutsch,
J.D. Lifson, S. Bonhoeffer, M.A. Nowak, and B.H. Hahn. Viral dynamics in
105
human immunodeficiency virus type 1 infection. Nature, 373(6510):117–22,
Jan 1995.
[128] L.M. Wein, R.M. D’Amato, and A.S. Perelson. Mathematical analysis of
antiretroviral therapy aimed at HIV-1 eradication or maintenance of low
viral loads. J Theor Biol, 192(1):81–98, May 1998.
[129] L.M. Wein, S.A. Zenios, and M.A. Nowak. Dynamic multidrug therapies
for HIV: a control theoretic approach. J Theor Biol, 185(1):15–29, Mar
1997.
[130] D. Wodarz. Helper-dependent vs. helper-independent CTL responses in
HIV infection: implications for drug therapy and resistance. J Theor Biol,
213(3):447–59, Dec 2001.
[131] D. Wodarz, R.A. Arnaout, M.A. Nowak, and J.D. Lifson. Transient antiretroviral treatment during acute simian immunodeficiency virus infection
facilitates long-term control of the virus. Philos Trans R Soc Lond B Biol
Sci, 355(1400):1021–9, Aug 2000.
[132] D. Wodarz and V.A. Jansen. The role of T cell help for anti-viral CTL
responses. J Theor Biol, 211(4):419–32, Aug 2001.
[133] D. Wodarz, A.L. Lloyd, V.A. Jansen, and M.A. Nowak. Dynamics of
macrophage and T cell infection by HIV. J Theor Biol, 196(1):101–13,
Jan 1999.
[134] D. Wodarz and M.A. Nowak.
Specific therapy regimes could lead to
long-term immunological control of HIV.
96(25):14464–9, Dec 1999.
106
Proc Natl Acad Sci U S A,
[135] D. Wodarz and M.A. Nowak. HIV therapy: managing resistance. Proc Natl
Acad Sci U S A, 97(15):8193–5, Jul 2000.
[136] J.K. Wong, M. Hezareh, H.F. Gnthard, D.V. Havlir, C.C. Ignacio, C.A.
Spina, and D.D. Richman. Recovery of replication-competent HIV despite
prolonged suppression of plasma viremia. Science, 278(5341):1291–5, Nov
1997.
[137] S. Woods, editor. 2004 Report on the Global AIDS Epidemic. Joint United
Nations Programme on HIV/AIDS, 2004.
[138] C. Wu, J.R. Kirman, M.J. Rotte, D.F. Davey, S.P. Perfetto, E.G. Rhee,
B.L. Freidag, B.J. Hill, D.C. Douek, and R.A. Seder. Distinct lineages of
T(H)1 cells have differential capacities for memory cell generation in vivo.
Nat Immunol, 3(9):852–8, Sep 2002.
[139] O.O. Yang, P.T.N. Sarkis, A. Ali, J.D. Harlow, C. Brander, S.A. Kalams,
and B.D. Walker. Determinant of HIV-1 mutational escape from cytotoxic
T lymphocytes. J Exp Med, 197(10):1365–75, May 2003.
[140] J.W. Yewdell, C.C. Norbury, and J.R. Bennink. Mechanisms of exogenous
antigen presentation by MHC class I molecules in vitro and in vivo: implications for generating CD8+ T cell responses to infectious agents, tumors,
transplants, and vaccines. Adv Immunol, 73:1–77, 1999.
[141] A.J. Zajac, J.N. Blattman, K. Murali-Krishna, D.J. Sourdive, M. Suresh,
J.D. Altman, and R. Ahmed. Viral immune evasion due to persistence of
activated T cells without effector function. J Exp Med, 188(12):2205–13,
Dec 1998.
107
[142] R. Zurakowski and A.R. Teel. Enhancing immune response to HIV infection using MPC-based treatment scheduling. In Proceedings of the 2003
American Control Conference, volume 2, pages 1182 – 1187, June 2003.
[143] R. Zurakowski and A.R. Teel. A model predictive control based scheduling
method for HIV therapy. Journal of Theoretical Biology, 2004 (submitted).
[144] R. Zurakowski, D. Wodarz, and A.R. Teel. Utilizing alternate target cells
in treating HIV infection through scheduled treatment interruptions. In
Proceedings of the 2004 American Control Conference, June 2004.
108
Appendix A
Selected MATLAB Code
We present the MATLAB code for certain key programs. The program optimal sequence.m solves the length N optimization problem from a given point.
The program optimal sequence 2u.m solves the same problem, but allows for
the introduction of additional target cells. The program errorprone.m simulates the behavior of a system with parameter values which differ from nominal,
returning one week’s simulation results. The program mixederror.m uses the
programs optimal sequence.m and errorprone.m in order to test the robustness of our MPC-based treatment scheduling algorithm on a system with up to
10% variation in the values of the parameters and 10% measurement error.
A.1
optimal sequence.m
This program returns the arg min of the cost function, that is, the length N
optimal control sequence.
function [seq,out] = f(Ne,Me,init);
109
% Global function parameter definitions
global T lambda lambda_z1 lambda_w lambda_z2 d betawt betawot;
global a p1 p2 c1 b1 c2 b2 q h N M costold;
T=7;
lambda=1;
lambda_z1=0;
lambda_w=0.001;
lambda_z2=0.001;
d=0.1;
betawt=0.0201; % eta times beta
betawot=1;
% Beta with no treatment.
a=0.2;
p1=1;
p2=1;
c1=0.03;
b1=0.1;
c2=0.06;
b2=0.01;
q=0.5;
h=0.1;
N = Ne;
M = Me;
costold=1e500;
% End global function parameter definitions
[seq,
seq =
out =
clear
clear
clear
out]=find_best_sequence(init);
seq;
out;
N M init lambda lambda_z1 lambda_w lambda_z2 d;
betawt betawot a p1 p2 c1 b1 c2 b2 q h Ne Me;
global;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%SUBPROGRAMS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% BEGIN SUBPROGRAM SEQUENCE_FINAL_STATE
% This is the subprogram that calculates the cost function
% For each sequence. To save time, it skips the rest of
110
% the sequence if the cost is already above the current
% best. I can’t remember where the name came from.
function [vout,stagecost]=sequence_final_state(vin,state,xo);
global costold;
cost1 = 0;
n = max(size(vin));
for i=1:n
control=str2num(vin(i));
if cost1 < costold
state = solve1e(control,state);
cost = stage_cost(state,control,xo);
end
if cost1 > costold
disp(’skipped’, vin, i)
end
cost1 =cost1+cost;
end
vout = state;
stagecost = cost1;
clear i n state1 cost1 vin state xo cost vinvin;
% END SUBPROGRAM SEQUENCE_FINAL_STATE
% BEGIN SUBPROGRAM TERMINAL_COST
% Implements the terminal cost. Not actually
% used, but kept around just in case
function cost = terminal_cost(P,xc,xo);
%cost = (xc-xo)*P*(xc-xo)’;
cost = 0;
clear P xc xo
% END SUBPROGRAM TERMINAL_COST
% BEGIN SUBPROGRAM STAGE_COST
% Implements the stage cost.
function cost = stage_cost(xc,input,xo);
cost = 1000.*(xc(1)-xo(1))^2+(xc(4)-xo(4))^2+input;
clear xc input xo
%END SUBPROGRAM STAGE_COST
%
%
%
%
%
BEGIN SUBPROGRAM FIND_LINEARIZATION
This program funds the location of the optimal
steady-state X_o and returns a matrix A which is
a legacy bit from when we were using terminal
costs.
111
function [A,xo] = find_linearization(dummy);
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h;
x2=(c2*(lambda+d*q)-b2*betawot+ ...
sqrt((c2*(lambda+d*q)-b2*betawot)^2- ...
4*c2^2*lambda*q*d))/(2*c2*d);
y2=b2/(c2*(x2-q));
z22=(betawot*x2-a)/p2;
w2=h*z22/(c2*q*y2);
z12=0;
xo=[x2 y2 z12 w2 z22];
A=[-d-betawot*y2 -betawot*x2 0 0 0;
betawot*y2 betawot*x2-a-p2*z22 -p1*y2 0 -p2*y2;
0 c1*z12 c1*y2-b1 0 0;
c2*y2*w2 c2*x2*w2-c2*q*w2 0 c2*x2*y2-c2*q*y2-b2 0;
0 c2*q*w2 0 c2*q*y2 -h];
clear dummy x2 y2 z22 w2 z12;
% END SUBPROGRAM FIND_LINEARIZATION
% BEGIN SUBPROGRAM FIND_BEST_SEQUENCE
% This program runs through every possible
% control sequence and finds the one that
% minimizes the cost function
function [seq,out]=find_best_sequence(state);
global lambda lambda_z1 lambda_w lambda_z2 d ...
betawt betawot a p1 p2 c1 b1 c2 b2 q h N M costold;
mat = diag([1e1 1e0 1e-3 1e-2 1e-3]); %PA +A’P = -mat
[A,xo] = find_linearization(1);
NN=2^N;
costold = 1e500;
bestout =state;
for i=1:NN
thing1=dec2bin(NN - i,N);
[vout,stagecost]=sequence_final_state([thing1 ...
zeros(1,M)],state,xo);
xc= vout;
costnew = stagecost;
if costnew < costold
costold = costnew;
bestyet=thing1;
bestout=xc;
end
112
end
seq = bestyet;
out=bestout;
clear i k N thing1 thing2 cost costold costnew
clear bestout bestyet NN xc P A xo mat
% END FIND_BEST_SEQUENCE
% BEGIN SUBPROGRAM SOLVE1E
% This chooses the treatment or no treatment option and
% calculates one evolution
% over T seconds from the input initial conditions.
function [yout]=solve1e(u,v);
global T;
if u==0
OPTIONS=odeset(’MaxStep’,0.1,’Jacobian’,...
@jacobwot,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wot,[0 0.5*T T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
if u==1
OPTIONS=odeset(’MaxStep’,0.1,’Jacobian’,...
@jacobwit,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wt,[0 0.5*T T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
clear u v OPTIONS T
% END SUBPROGRAM SOLVE1E
% BEGIN SUPROGRAM WT
% This describes the treatment transfer function
function [vout] = wt(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator
%This inherits the model parameter definitions;
global lambda lambda_z1 lambda_w lambda_z2 d betawt betawot;
global a p1 p2 c1 b1 c2 b2 q h;
% End Model Parameter Definitions
% Here we get the states from the function input
x=v(1);
113
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak model
xp = lambda-d*x-betawt*x*y;
yp = betawt*x*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz-Nowak
vout =[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM WT
% BEGIN SUBPROGRAM WOT
% This describes the no-treatment transfer function
function [vout] = wot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d betawt betawot;
global a p1 p2 c1 b1 c2 b2 q h;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak model
xp = lambda-d*x-betawot*x*y;
yp = betawot*x*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
114
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz Nowak
vout=[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM NOTREATMENT
% BEGIN SUBPROGRAM JACOBWOT
% This describes the no-treatment transfer function jacobian
function [jout] = jacobwot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d betawt betawot;
global a p1 p2 c1 b1 c2 b2 q h T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawot*y;
jout(1,2)=-betawot*x;
jout(2,1)=betawot*y;
jout(2,2)=betawot*x-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
%END SUBPROGRAM JACOBWOT
115
% BEGIN SUBPROGRAM JACOBWIT
% This describes the treatment transfer function jacobian
function [jout] = jacobwit(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d betawt betawot;
global a p1 p2 c1 b1 c2 b2 q h T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawt*y;
jout(1,2)=-betawt*x;
jout(2,1)=betawt*y;
jout(2,2)=betawt*x-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
%END SUBPROGRAM JACOBWIT
116
A.2
optimal sequence 2u.m
This program finds the optimal sequence pair for application of HAART and
additional targets.
function [seq,out,out1] = f(Ne,Me,init);
% Global function parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h N M;
global mzeros costold alt T;
lambda=1;
lambda_z1=0;
lambda_w=0.001;
lambda_z2=0.001;
d=0.1;
betawt=0.0201; % eta times beta
betawot=1;
% Beta with no treatment.
a=0.2;
p1=1;
p2=1;
c1=0.03;
b1=0.1;
c2=0.06;
b2=0.01;
q=0.5;
h=0.1;
N = Ne; % Treatment Length
M = Me; % A run w/o treatment at the end.
for f=1:M
mzeros=strcat(mzeros,’0’);
end
costold=1e500;
%alt=3.6364; %Optimal for T=0.4545
%alt=0; %
alt=0.3636; %Optimal for T=7
%T = 0.4545; % Optimal for alt=3.6364
%T=0.5909; % Optimal for alt=0
T=7;
% End global function parameter definitions
[seq, out]=find_best_sequence(init);
seq = seq;
117
out = out;
out1 = solve1e(str2num(seq(1,1)),str2num(seq(2,1)),init);
clear N M init lambda lambda_z1 lambda_w lambda_z2 d
clear betawt betawot a p1 p2 c1 b1 c2 b2 q h Ne Me
clear global
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%SUBPROGRAMS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% BEGIN SUBPROGRAM SEQUENCE_FINAL_STATE
function [vout,stagecost]=sequence_final_state(vin,vin2,state,xo);
global costold M;
cost1 = 0;
n = max(size(vin));
for i=1:n
control=str2num(vin(i));
control2=str2num(vin2(i));
if cost1 < costold
state = solve1e(control,control2,state);
cost = stage_cost(state,control,control2,xo);
end
if cost1 > costold
disp(’skipped’, vin, i)
end
cost1 =cost1+cost;
end
vout = state;
stagecost = cost1;
clear i n state1 cost1 vin state xo cost vinvin;
% END SUBPROGRAM SEQUENCE_FINAL_STATE
% BEGIN SUBPROGRAM TERMINAL_COST
function cost = terminal_cost(P,xc,xo);
cost = 0;
clear P xc xo
% END SUBPROGRAM TERMINAL_COST
% BEGIN SUBPROGRAM STAGE_COST
function cost = stage_cost(xc,input,input2,xo);
cost = (xc(1)-xo(1))^2+(xc(4)-xo(4))^2+input+input2;
118
clear xc input xo
%end
% BEGIN SUBPROGRAM FIND_LINEARIZATION
function [A,xo] = find_linearization(dummy);
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h;
x2=(c2*(lambda+d*q)-b2*betawot+sqrt((c2*(lambda+...
d*q)-b2*betawot)^2-4*c2^2*lambda*q*d))/(2*c2*d);
y2=b2/(c2*(x2-q));
z22=(betawot*x2-a)/p2;
w2=h*z22/(c2*q*y2);
z12=0;
xo=[x2 y2 z12 w2 z22];
A=[-d-betawot*y2 -betawot*x2 0 0 0;...
betawot*y2 betawot*x2-a-p2*z22 -p1*y2 0 -p2*y2;...
0 c1*z12 c1*y2-b1 0 0;...
c2*y2*w2 c2*x2*w2-c2*q*w2 0 c2*x2*y2-c2*q*y2-b2 0;...
0 c2*q*w2 0 c2*q*y2 -h];
clear dummy x2 y2 z22 w2 z1
% END SUBPROGRAM FIND_LINEARIZATION
% BEGIN SUBPROGRAM FIND_BEST_SEQUENCE
function [seq,out]=find_best_sequence(state);
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h N M;
global mzeros costold;
[A,xo] = find_linearization(1);
NN=2^N;
costold = 1e500;
bestout =state;
bestyet =dec2bin(0,N);
for i=1:NN
thing1=dec2bin(NN - i,N);
for j=1:NN
thing2=dec2bin(NN - j,N);
[vout,stagecost]=sequence_final_state(strcat(thing1,...
mzeros),strcat(thing2,mzeros),state,xo);
xc= vout;
costnew = stagecost;
if costnew < costold
costold = costnew;
bestyet=[thing1;thing2];
119
bestout=xc;
end
end
end
seq = bestyet;
out=bestout;
clear i k N thing1 thing2 cost costold costnew
clear bestout bestyet NN xc P A xo mat
%end
% END FIND_BEST_SEQUENCE
% BEGIN SUBPROGRAM SOLVE1E
% This chooses the treatment or no treatment
% option and calculates one evolution
% over T days from the input initial conditions.
function [yout]=solve1e(u,u2,v);
global T;
if u==0 & u2==0
OPTIONS=odeset(’MaxStep’,1,’Jacobian’,...
@jacobwotwot,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wotwot,[0 0.1 T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
if u==0 & u2==1
OPTIONS=odeset(’MaxStep’,1,’Jacobian’,...
@jacobwotwit,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wotwt,[0 0.1 T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
if u==1 & u2==0
OPTIONS=odeset(’MaxStep’,1,’Jacobian’,...
@jacobwitwot,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wtwot,[0 0.1 T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
if u==1 & u2==1
OPTIONS=odeset(’MaxStep’,1,’Jacobian’,...
@jacobwitwit,’AbsTol’,1e-6,’RelTol’,1e-6);
[t,y]=ode23s(@wtwt,[0 0.1 T],v,OPTIONS);
120
yout=y(3,:);
clear y,t;
end
clear u v OPTIONS T
% END SUBPROGRAM SOLVE1E
% BEGIN SUPROGRAM WTWT
% This describes the treatment-treatment transfer function
function [vout] = wtwt(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator
%This inherits the model parameter definitions;
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h alt;
% End Model Parameter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak
xp = lambda-d*x-betawt*x*y;
yp = betawt*x*y+alt*betawt*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz-Nowak
vout =[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM WTWT
% BEGIN SUPROGRAM WTWOT
% This describes the treatment-notreament transfer function
121
function [vout] = wtwot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator
%This inherits the model parameter definitions;
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h alt;
% End Model Parameter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak model
xp = lambda-d*x-betawt*x*y;
yp = betawt*x*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz-Nowak
vout =[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM WTWOT
%
%
%
%
BEGIN SUBPROGRAM WOTWT
This describes the transfer function
without HAART
and with additional targets
function [vout] = wotwt(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h alt;
% End Model Paramter Definitions
% Here we get the states from the function input
122
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak model
xp = lambda-d*x-betawot*x*y;
yp = betawot*x*y+alt*betawot*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz Nowak
vout=[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM WOTWT
% BEGIN SUBPROGRAM WOTWOT
% This describes the no-treatment transfer function
function [vout] = wotwot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2;
global d betawt betawot a p1 p2 c1 b1 c2 b2 q h alt;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak model
xp = lambda-d*x-betawot*x*y;
yp = betawot*x*y-a*y-p1*y*z1-p2*y*z2;
z1p = c1*z1*y-b1*z1+lambda_z1;
123
wp = c2*x*y*w-c2*q*y*w-b2*w+lambda_w;
z2p = c2*q*y*w-h*z2+lambda_z2;
% End Wodarz Nowak
vout=[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t
%end
% END SUBPROGRAM WOTWOT
%BEGIN SUBPROGRAM JACOBWOTWOT
% This describes the treatment transfer function jacobian
% For the case without HAART
% and without additional targets
function [jout] = jacobwotwot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2;
global d betawt betawot a p1 p2 c1 b1 c2 b2 q h T alt;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawot*y;
jout(1,2)=-betawot*x;
jout(2,1)=betawot*y;
jout(2,2)=betawot*x-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
124
%END SUBPROGRAM JACOBWOTWOT
%BEGIN SUBPROGRAM JACOBWOTWIT
% This describes the treatment transfer function jacobian
% For the case without HAART
% and with additional targets
function [jout] = jacobwotwit(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h T alt;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawot*y;
jout(1,2)=-betawot*x;
jout(2,1)=betawot*y;
jout(2,2)=betawot*x+betawot*alt-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
%END SUBPROGRAM JACOBWOTWIT
% BEGIN SUBPROGRAM JACOBWITWIT
125
% This describes the treatment transfer function jacobian
% For the case with both HAART
% and additional targets
function [jout] = jacobwitwit(t,v); % Note that t is never used;
% It just satisfies the format for the simulator.
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h T alt;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawt*y;
jout(1,2)=-betawt*x;
jout(2,1)=betawt*y;
jout(2,2)=betawt*x+betawt*alt-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
%END SUBPROGRAM JACOBWITWIT
%BEGIN SUBPROGRAM JACOBWITWOT
% This describes the treatment transfer function jacobian
% For the case with HAART
% and without additional targets
function [jout] = jacobwitwot(t,v);
% Note that t is never used;
% It just satisfies the format for the simulator.
126
% This inherits the model parameter definitions
global lambda lambda_z1 lambda_w lambda_z2 d;
global betawt betawot a p1 p2 c1 b1 c2 b2 q h T alt;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
jout = zeros(5);
jout(1,1)=-d-betawt*y;
jout(1,2)=-betawt*x;
jout(2,1)=betawt*y;
jout(2,2)=betawt*x-a-p1*z1-p2*z2;
jout(2,3)=-p1*y;
jout(2,5)=-p2*y;
jout(3,2)=c1*z1;
jout(3,3)=c1*y-b1;
jout(4,1)=c2*y*w;
jout(4,2)=c2*x*w-c2*q*w;
jout(4,4)=c2*x*y-c2*q*y-b2;
jout(5,2)=c2*q*w;
jout(5,4)=c2*q*y;
jout(5,5)=-h;
%END SUBPROGRAM JACOBWITWOT
A.3
errorpronemc.m
This program calculates one week’s evolution of the system with actual parameter
values, importing a random seed for consistency.
function [yout] = errorpronemc(u,v,randsed,percent)
global lambda lambdareal lambda_z1 lambda_z1real ...
lambda_w lambda_wreal lambda_z2 lambda_z2real ...
d dreal betawt betawtreal betawot betawotreal ...
a areal p1 p1real p2 p2real c1 c1real b1 ...
127
b1real c2 c2real b2 b2real q qreal h hreal T;
randseed=zeros(16,1);
randseed=randsed;
u=str2num(u);
T=7;
%percent=0.5;
lambda=1;
lambdareal=lambda+lambda*percent*(randseed(1));
lambda_z1=0;
lambda_z1real=lambda_z1+lambda_z1*percent*(randseed(2));
lambda_w=0.001;
lambda_wreal=lambda_w+lambda_w*percent*(randseed(3));
lambda_z2=0.001;
lambda_z2real=lambda_z2+lambda_z2*percent*(randseed(4));
d=0.1;
dreal=d+d*percent*(randseed(5));
betawt=0.0201; % The effective beta with treatment applied
betawtreal=betawt+betawt*percent*(randseed(6));
betawot=1;
% Beta with no treatment.
betawotreal=betawot+betawot*percent*(randseed(7));
a=0.2;
areal=a+a*percent*(randseed(8));
p1=1;
p1real=p1+p1*percent*(randseed(9));
p2=1;
p2real=p2+p2*percent*(randseed(10));
c1=0.03;
c1real=c1+c1*percent*(randseed(11));
b1=0.1;
b1real=b1+b1*percent*(randseed(12));
c2=0.06;
c2real=c2+c2*percent*(randseed(13));
b2=0.01;
b2real=b2+b2*percent*(randseed(14));
q=0.5;
qreal=q+q*percent*(randseed(15));
h=0.1;
hreal=h+h*percent*(randseed(16));
% End global function parameter definitions
if u==0
OPTIONS=odeset(’MaxStep’,0.1,’Jacobian’,...
128
@jacobwot,’RelTol’,1e-6,’AbsTol’,1e-6);
[t,y]=ode23s(@wot,[0 0.5*T T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
if u==1
OPTIONS=odeset(’MaxStep’,0.1,’Jacobian’,...
@jacobwit,’RelTol’,1e-6,’AbsTol’,1e-6);
[t,y]=ode23s(@wit,[0 0.5*T T],v,OPTIONS);
yout=y(3,:);
clear y,t;
end
clear u v OPTIONS T
% BEGIN SUBPROGRAM WOT
% This describes the no-treatment transfer function
function [vout] = wot(t,v);
% Note that t is never used; It just satisfies the
% format for the simulator.
% This inherits the model parameter definitions
global lambda lambdareal lambda_z1 lambda_z1real ...
lambda_w lambda_wreal lambda_z2 lambda_z2real ...
d dreal betawt betawtreal betawot betawotreal ...
a areal p1 p1real p2 p2real c1 c1real ...
b1 b1real c2 c2real b2 b2real q qreal h hreal T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak
% model from "Helper-dependant vs. ..."
xp = lambdareal-dreal*x-betawotreal*x*y;
yp = betawotreal*x*y-areal*y-p1real*y*z1-p2real*y*z2;
z1p = c1real*z1*y-b1real*z1+lambda_z1real;
wp = c2real*x*y*w-c2real*q*y*w-b2*w+lambda_wreal;
z2p = c2real*q*y*w-hreal*z2+lambda_z2real;
129
% End Wodarz Nowak
vout=[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t;
%end
% END SUBPROGRAM NOTREATMENT
% BEGIN SUBPROGRAM JACOBWOT
% This describes the no-treatment transfer function jacobian
function [jout] = jacobwot(t,v);
% Note that t is never used; It just satisfies the
% format for the simulator.
% This inherits the model parameter definitions
global lambda lambdareal lambda_z1 lambda_z1real;
global lambda_w lambda_wreal lambda_z2 lambda_z2real;
global d dreal betawt betawtreal betawot betawotreal a areal;
global p1 p1real p2 p2real c1 c1real b1 b1real;
global c2 c2real b2 b2real q qreal h hreal T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
xp = lambdareal-dreal*x-betawotreal*x*y;
yp = betawotreal*x*y-areal*y-p1real*y*z1-p2real*y*z2;
z1p = c1real*z1*y-b1real*z1+lambda_z1real;
wp = c2real*x*y*w-c2real*q*y*w-b2*w+lambda_wreal;
z2p = c2real*q*y*w-hreal*z2+lambda_z2real;
jout = zeros(5);
jout(1,1)=-dreal-betawotreal*y;
jout(1,2)=-betawotreal*x;
jout(2,1)=betawotreal*y;
jout(2,2)=betawotreal*x-areal-p1real*z1-p2real*z2;
jout(2,3)=-p1real*y;
jout(2,5)=-p2real*y;
jout(3,2)=c1real*z1;
jout(3,3)=c1real*y-b1real;
jout(4,1)=c2real*y*w;
130
jout(4,2)=c2real*x*w-c2real*q*w;
jout(4,4)=c2real*x*y-c2real*q*y-b2;
jout(5,2)=c2real*q*w;
jout(5,4)=c2real*q*y;
jout(5,5)=-hreal;
%END SUBPROGRAM JACOBWOT
% BEGIN SUBPROGRAM JACOBWIT
% This describes the treatment transfer function jacobian
function [jout] = jacobwit(t,v);
% Note that t is never used; It just satisfies
% the format for the simulator.
% This inherits the model parameter definitions
global lambda lambdareal lambda_z1 lambda_z1real;
global lambda_w lambda_wreal lambda_z2 lambda_z2real;
global d dreal betawt betawtreal betawot betawotreal;
global a areal p1 p1real p2 p2real c1 c1real b1;
global b1real c2 c2real b2 b2real q qreal h hreal T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
xp = lambdareal-dreal*x-betawtreal*x*y;
yp = betawtreal*x*y-areal*y-p1real*y*z1-p2real*y*z2;
z1p = c1real*z1*y-b1real*z1+lambda_z1real;
wp = c2real*x*y*w-c2real*q*y*w-b2*w+lambda_wreal;
z2p = c2real*q*y*w-hreal*z2+lambda_z2real;
jout = zeros(5);
jout(1,1)=-dreal-betawtreal*y;
jout(1,2)=-betawtreal*x;
jout(2,1)=betawtreal*y;
jout(2,2)=betawtreal*x-areal-p1real*z1-p2real*z2;
jout(2,3)=-p1real*y;
jout(2,5)=-p2real*y;
jout(3,2)=c1real*z1;
131
jout(3,3)=c1real*y-b1real;
jout(4,1)=c2real*y*w;
jout(4,2)=c2real*x*w-c2real*q*w;
jout(4,4)=c2real*x*y-c2real*q*y-b2;
jout(5,2)=c2real*q*w;
jout(5,4)=c2real*q*y;
jout(5,5)=-hreal;
%END SUBPROGRAM JACOBWIT
% BEGIN SUBPROGRAM WIT
% This describes the no-treatment transfer function
function [vout] = wit(t,v);
% Note that t is never used; It just satisfies
% the format for the simulator.
% This inherits the model parameter definitions
global lambda lambdareal lambda_z1 lambda_z1real;
global lambda_w lambda_wreal lambda_z2 lambda_z2real;
global d dreal betawt betawtreal betawot betawotreal a areal;
global p1 p1real p2 p2real c1 c1real b1 b1real c2 c2real;
global b2 b2real q qreal h hreal T;
% End Model Paramter Definitions
% Here we get the states from the function input
x=v(1);
y=v(2);
z1=v(3);
w=v(4);
z2=v(5);
% End function input
% This is the differential equation Wodarz-Nowak
% model from "Helper-dependant vs. ..."
xp = lambdareal-dreal*x-betawtreal*x*y;
yp = betawtreal*x*y-areal*y-p1real*y*z1-p2real*y*z2;
z1p = c1real*z1*y-b1real*z1+lambda_z1real;
wp = c2real*x*y*w-c2real*q*y*w-b2*w+lambda_wreal;
z2p = c2real*q*y*w-hreal*z2+lambda_z2real;
% End Wodarz Nowak
vout=[xp;yp;z1p;wp;z2p];
clear xp yp z1p wp z2p x y z2 w z2 v t;
132
%end
% END SUBPROGRAM TREATMENT
A.4
mixederror.m
This program runs 100 simulations with the actual system modified randomly by
up to 10% of the value of each parameter and random measurement noise of up to 10%
of the value of each state added at each sampling instant, keeping a running tally of
the number of times the system fails to stabilize the desired steady-state.
q=0;
for index=1:100
incon = [10 0.1 0.1 0.1 0.1];
start = incon;
seqloop = [];
seq = ’1’;
seq1 = ’1’;
i=0;
rand(’state’,sum(100*clock));
randseed=rand(16,1)-0.5;
percent = 0.2;
percent2 = 0.2;
while str2num(seq1) ~= 0
rand(’state’,sum(100*clock));
error = percent2.*(rand(5,1)-0.5);
start1 =start;
[seq,out]=optimal_sequence(6,0,start1+error.*start1);
[out2] = errorpronemc(seq(1),start,randseed,percent);
start = out2;
%
disp(seq(1));
seqloop = [seqloop seq(1)];
i=i+1;
if out2(4) > 10
seq1 = ’0’;
end
if i>103
seq1 = ’0’;
q=q+1;
133
end
end
disp([index q])
end
134