Download Pattern recognition by primary and secondary response of an

Theory Biosci. (2001) 120: 93±106
Ó Urban & Fischer Verlag
http://www.urbanfischer.de/journals/theorybiosc
Pattern recognition by primary and secondary
response of an Artificial Immune System
F. Castiglione1, S. Motta2 and G. Nicosia2
1
Center for Applied Computer Science (ZPR/ZAIK), University of Cologne, Cologne, Germany
Department of Computer Science, University of Rome ªLa Sapienzaº, Roma, Italy
Department of Mathematics and Computer Science, Catania, Italy
2
Address for correspondence: Filippo Castiglione, Center for Applied Computer
Science (ZPR/ZAIK), University of Cologne, Weyertal 80, D-50931 KoÈln, Germany,
phone: [+ 49] (221) 470 60 26, fax: [+ 49] (221) 470 51 60, e-mail: [email protected]
Received: January 2, 2001; accepted: February 28, 2001
Key words: Immune system, artificial immune system, pattern recognition, Kullback
entropy, bit-string model, immune response, hypermutation
Summary: In this paper we show how an Artificial Immune System can be used to
study pattern recognition processes and learning. In particular we show the ability of
the model to discover and maintain coverage of the diverse patterns through mechanism of evolution and mutation.
1 Introduction
The pattern recognition ability of the Immune System (IS) is a key feature
in the defense of the host organism. The IS has to assure recognition of
each potentially dangerous molecule or substance (herein generically called
antigen) that can infect the host organism. It must first recognize it as
ªdangerousº or extraneous and then mount a response to eliminate it. This
task is accomplished by the complex machinery made by cellular interactions and molecular productions.
During the last years a number of studies in the field of Immunological
Computation have been published, which attempt to use methods and concepts of the computer science to design immunity-based system applications in science and engineering [1]. Artificial Immune systems are adaptive
systems in which learning takes place by evolutionary mechanisms similar
to biological evolution. The study of such systems is tied to the study of
1431-7613/01/120/02±93/$ 15.00/0
94
F. Castiglione, S. Motta and G. Nicosia
natural systems in that one wants, first, to understand the dynamics of
such complex behaviour when they face particular computational problems that are normally solved by conventional specialized algorithms and,
second, one wish to develop new techniques that mimic the natural systems under study to catch their ability to solve problem otherwise difficult
to be solved by conventional methods. The present paper was inspired,
among the other, by the computer experiments performed by Forrest et al.
(1993) [2]. In particular we focus our attention on the recognition ability
of the IS and present results of similar experiments using a different model. In [2] a Genetic Algorithm ([3]) is used to model the immune system.
Our approach, instead, uses an artificial immune system known as the Celada-Seiden model (CS-model, [4, 5]) to explore the pattern recognition
feature. The CS-model is a quite rich model that incorporates many properties of natural immune system, including humoral and cellular response,
recognition, memory, self-nonself discrimination, thymus selection, extended repertoire, hypermutation, and self-adaptation.
To our knowledge this is the first attempt to use such sophisticated model
to study the pattern recognition problem. On the other hand, numerical simulations lead to non trivial consideration on the dynamics of the immune
response under multiple antigen attack.
The paper is organized as follows: in section 2 we briefly describe the
model of the immunesystem used and in 3 we list the properties of our artificial immune system; in section 4 we show how it accomplishes the pattern recognition task and in section 5 we conduce various experiments. In
particular in 5.3 we make use of the mutation to obtain generalization
over the pattern space. In section 6 we show the entropy function associated to the quantity of information the systems discover during the recognition process. In 7 we discuss the problem of the recognition capacity
and finally in 8 we highlight the future direction.
2 Immune system modeling
The immune system opposes two different responses against pathogenic
entities: the humoral, mediated by antibodies, and the cellular, mediated
by cells. Like the nervous system the IS performs pattern recognition tasks
and retains memory of the antigens to which it has been exposed.
The research on the IS's dynamics in the last two decades has produced a
certain number of mathematical and computational models. The different
approaches include differential-equation based models [6, 7, 8], cellular
automata models [9], classifier systems [10] and genetic algorithms [2].
Among these models the Artificial Immune System (AIS) seems best suited to handle the great complexity of the reality. One of the richest models
is the aforementioned CS-model. It has a computational counterpart called
Pattern recognition by primary and secondary response of an AIS
95
IMMSIM and more recent parallel development in C/PVM [11] able to
perform larger simulations. In the following we will briefly review the CSmodel leaving the reader to the bibliography for further references [4, 11,
12, 13].
The model includes: antigens (Ag), B lymphocytes (B), plasma B cells
(PLB), antigen processing cells (APC), T-helper lymphocytes (Th), immunecomplexes (IC) and antibodies (Ab). The Ag is the target of the immune
response. Th and B lymphocytes are responsible for the discrimination of
the self-nonself, while the plasma B cells produce antibodies able to label
the antigens to be eliminated by the APCs. AP cells represent the wide
class of macrophages. Their function is to present the phagocytated antigens to T helper cells for activation. The immune-complexes are Ab-Ag
ties ready to be phagocytated by the macrophages.
Beyond the aforementioned entities the model includes the T killer cells
(Tk), the Epithelial cells (Ep) or generic virus-target cells, and various lymphokines such as the interferon-c (IFN). They are necessary to simulate
the cellular response. We will not mention this any further because in the
following experiments all the interactions driving the T killer activation
have been turned off so to consider only the humoral response.
The CS-model defines precise interaction rules of two types: specific interactions that occur when some entities bind each other through receptors
and non-specific interactions that occur when two entities interact without
any recognition process. In the first case, to avoid the great complexity of
the chemical interactions between receptors, the bindings are mimicked as
stochastic events. The probability that two receptors interact is determined
by a bit to bit matching over the bit-strings representing them [10]. Thus,
both cell receptors and molecules are modeled as binary strings of lengtht l.
The binding between two strings occurs with a certain probability which is
function of the match m computed as the Hamming distance of the two bitstrings. The probability to interact, called the affinity function, is a simple
truncated exponential function with threshold mc (l/2 < mc £ l), v(m) =
a(l±m)/(l±mc) for m ³ mc and zero for m < mc. In the case mc = l we set
v(l) = 1. The parameter a Î (0, 1) determines the sharpness of the affinity.
The body is represented by a regular triangular (six neighbours) two-dimensional grid. Each site may contains many different entities (whose
number changes in time) that interact in loci and diffuse to adjacent sites
according to a random rule. Each time step some cells die by natural death
while new cells are produced by the bone marrow. In the model the birthdeath rate is set to zero so that in absence of antigen's stimulation the population is found in a steady state.
An important feature is the self-nonself recognition capacity through the
thymus selection of naive T lymphocytes [14]. In few words, in the thymus, each new T cell undergoes a selection on the basis of the recognition
of self molecules. Those cells that recognize the self are eliminated.
96
F. Castiglione, S. Motta and G. Nicosia
Lastly, the model comprises the hypermutation phenomena observed during
the immune responses. This consists in the fact that the DNA portion coding for the antibodies is subjected to mutation during the proliferation of
the B lymphocytes. This furnish the system the ability to generate diversity.
The mechanism of mutation of the cell receptor represented by a binary
string is modeled by a Binomial process with parameters pm and l.
The model is based on the theory of the clonal selection first stated by Burnet and Ledeberg in 1959 [15]. This theory suggests that among all the possible cells with different receptors circulating in the host organism, only
those who are actually able to recognize the antigen will start to proliferate
by duplication (cloning). The increase of those population and the production of cells with longer expected life-time assures the organism a higher
specific responsiveness to that antigenic pattern, establishing a defense over
time. In particular, on recognition, B and T memory cells are produced.
Plasma B cells, deriving from stimulated B lymphocytes, are in charge in
the production of antibodies targeting the antigen. This mechanism is usually observed looking at the population of lymphocytes in two subsequent
antigenic infections. The first exposition to the antigen triggers the primary
response; in this phase the pattern is recognized and the memory is developed. During the secondary response, that occurs when the same antigen is
encountered again, a rapid and more abundant production of antibody is
observed, resulting from the stimulation of the cells already specialized and
present as memory cells. The secondary response can be elicited from any
antigen which is similar, although not identical, to the original one which
established the memory. This is known as cross-reactivity.
3 Properties of the AIS
The IS is interesting because it exhibits many properties that we would
like incorporate into artificial systems: the IS is diverse, distributed, error
tolerant, dynamic, self-monitoring, and adaptable [16]. These properties
give the IS certain key characteristic that most artificial systems today
lack: robustness, adaptivity, autonomy, flexibility and scalability. This
work focuses on biological principles, architecture, and algorithms extracted from immunology and applied to design of an algorithm for pattern recognition. Robustness is a consequence of the fact that the IS is diverse, distributed, dynamic and error tolerant. Diversity improves
robustness on both a population and individual level, for example, different people are vulnerable to different infections. The IS is distributed in a
robust fashion: its many components interact locally to provide global
protection, so there is no central control and hence no single point of failure. The IS is dynamic in that individual components are continually created, destroyed, and circulated throughout the body, which increases the
Pattern recognition by primary and secondary response of an AIS
97
temporal and spatial diversity of the IS. Finally, the IS is robust to errors
(error tolerant) because the effect of any single IS action is small, so a few
mistakes in classification and response are not catastrophic.
The IS is adaptable in that it can learn to recognize and respond to new infections and retain a memory of those infections to facilitate future responses. This adaptivity is made possible by the dynamic functioning of the
IS, which enables the IS to discard components that are useless or dangerous and to improve on existing components through mechanism of evolution and mutation. Adaptations remain in force for periods of time up to
the lifetime of an organism. This immunological memory allows the IS to
more rapidly the second time around to pathogens similar to ones it has encountered in the past. Immune memory implements signature-based detection, by allowing the IS to monitor for characteristics of known pathogens.
The IS is autonomons in that there is no outside control required. The IS
is an integrated part of the body, and hence the same mechanisms that
monitor and protect the rest of the body also monitor and protect the IS.
Furthermore, the distributed, decentralized, nature of the IS contributes to
its autonomous nature, not only is there no outside control, but there is
no way of imposing outside control or even inside, centralized control.
The IS is flexible in the allocation of resources for the protection of the
body. When a serious infection threatens the body, the IS draws upon
more resources, generating more IS components, and at other times, the IS
uses fewer resources. For example, a stressed person is more vulnerable to
illness, and an ill person will generally be better off resting (and thus freeing more resources for the IS). Furthermore, the IS is flexible in where it
directs its resources: the more a particular pathogen type is responsible for
damage, the more likely it is that the IS will tailor its response to that
pathogen type.
Viewed from the perspective of distributed processing, the IS is scalable:
communication and interaction between all IS components is localized, so
there is little overhead when increasing the number of components. There
may be architectural constraints on this scalability, but at the level of individual components, it is expected that resource expenditure would scale
linearly with the number of components.
These principles can be regarded as general guidelines for design which
can, in general, be achieved by using algorithms or mechanisms copied directly from immunology.
4 Recognition ability
This section describes how the presented model was designed to solve pattern recognition problems. The recognition ability is a key feature that
emerges from the cooperation of cells and molecules with different roles.
98
F. Castiglione, S. Motta and G. Nicosia
From the interplay of receptor-recognition, cell-interaction, clone expansion and mutation, the system is able to ªlearnº the foreign pattern.
The pattern is represented by the antigen that, as the cell receptors, is
modeled as a bit string. Once it is ªinjectedº into the lattice-grid the recognition process starts. The system needs few time steps to mount the
immune response. If the recognition takes place then we observe an exponential proliferation of the clone of B cells whose receptor has recognized
the antigen. Other entities proliferate as well but we take the proliferating
clones of B cells as reference of the recognition, i. e., in this observable we
will identify the signal that the system has recognized the pattern. In the
following experiments the bit string length l is set to 12 giving
212 = 4096 possible patterns. Each cell of the initial population is equipped
with a receptor that is randomly generated. The cells are placed at random
on a lattice-grid of 256 sites. For the sake of simplicity we also set the expected mean life of the memory cells to infinity to guarantee a time-unlimited recognition capacity. The parameter a of the affinity function v(m)
mentioned in the previous section is set to 0.1 while the threshold mc = l.
We start presenting 500 copies of the pattern 111111111111 at the initial
time step and 1500 cells for each of the B, T and APC types.
After few steps the clonal growth of memory cells starts. This corresponds
to the beginning of the learning phase. During the clonal expansion the
plasma B cells are generated. These produce antibodies matching the antigen so that the 500 patterns are eliminated within about 50 time steps (figure 1). The same pattern in the same quantity is injected again after
100 time steps to elicit a secondary response and to measure the effectiveness of the recognition. Once the antigens are depleted the number of
memory cells stops to grow. This means that the recognition is consolidated and the information on the presented pattern has reached its maximum value. During the second response the antigen is eliminated in much
shorter time (ten times faster, see figure 1).
In summary we distinguish between primary and secondary response giving to the last the meaning of having recognized the pattern to which the
system was previously trained. A stylized algorithm of the system is given
below.
Input (pattern);
init (populations);
for (t = 0; t ¬ steps; t++)
{
-<
interact (); /* cells interact */
proliferate (); /* clone expansion */
if (pm ! = 0) {
hypermutation (); /* mutation */
}
age (); /* death */
Pattern recognition by primary and secondary response of an AIS
99
Fig. 1. Recognition of one pattern (populations versus time steps).
bone-marrow (); /* new cells */
diffuse ();
}
Output (B population).
The input is the antigen pattern presented in a certain number of copies.
After random initialization of cell populations, the loop iterates the interactions among entities, the proliferation of stimulated cells (eventually mutating), the death of entities due to aging (the memory cells as well as the
antigens never die because we set their mean life to infinity) and the birth
of new cells from the bone marrow. The output is basically the dimension
of the population of those clones that have recognized the antigen.
If mc = l the recognition of a single string is given by the complementarybit map, i. e., bitwise not. In the other cases, mc < l, the recognition is performed with a certain tolerance because we allow l ± mc bits not to be necessarily complementary. However, this simplistic view does not represent
a strong limitation because in general one can give whatever meaning to
the bit string representing the pattern and use much more complicated
matching function than the simple Hamming distance, e. g., any map f : {0,
1}l ´ {0, 1}l ® {0, l} could determine a different recognition system. The
evolutionary ªengineº below remains the same.
100
F. Castiglione, S. Motta and G. Nicosia
5 Experiments
In the following we will test some features of the pattern recognition: the
ability to recognize multiple patterns, the response to different variable
size patterns, and the coverage of the whole repertoire. For this last we
will make use of the mutation to fill the potential repertoire {0, 1}l.
5.1 Learning multiple patterns
The goal of this experiment is to see if the system can recognize different
patterns presented simultaneously.
The number of initial cells per type is 2000. The threshold mc = l so that
only perfect matching strings can proliferate. The system can either recognize a pattern showing a clonal growth of B cells or not recognizing it just
because it is not able to ªlearnº the pattern within 200 time steps (the time
needed to learn a certain pattern depends on the initial population, see section 5). The patterns are recognized at different time steps due to the stochasticity of the interactions. Within 60 time steps they are all recognized
and before step 90 they are eliminated. At that moment the memory ceases
to develop.
Fig. 2. Ag and B clones population versus time steps. In the upper plot the population of the
recognizing clones of B cells. In the lower plot the antigen population. All the patterns are
recognized. The remaining clones (not shown) do not grow to higher value as the elicited
ones.
Pattern recognition by primary and secondary response of an AIS
101
In figure 2 one can see that all five patterns presented at the start are being
learned and memorized.
5.2 Learning mixture of two patterns
In this experiment we want to test the dependency of the recognition on
the amount of patterns presented.
Consider the following pattern's population: 1000 (66%) type
111111111111 and 500 (33%) type 111111000000. N = 1500 and mc = l. In
figure 3 it is shown the population of B cells that recognize the pattern.
The B cell receptor 000000000000 grows to higher values than the other,
that is, the response as expected is proportional to the presented pattern.
5.3 The problem of coverage
In this experiment we want to solve the problem of coverage, i. e., we
want to see if the model is able to evolve a set of bit strings that have the
property of repertoire completeness.
The generalization issue arises when there are not enough individuals in
the population to assign one individual (or subpopulation) to each task,
niche, or pattern. In the immune system this means that the system does
not have the capacity to produce one antibody that is a perfect match for
Fig. 3. Antigen (down) and B cell clones (up) populations versus time steps. The response is
proportional to the amount of each of the two patterns presented.
102
F. Castiglione, S. Motta and G. Nicosia
each presented antigen. The mechanism used by the IS to solve this resource constraint is to first evolve individuals that can cover more than
one antigen and then discover better matching clones via mutation.
In order to test it we set mc to allow also imperfect matches. We set
mc = 10 and the initial population per cell type to 1500. To better evidence
the effect of the mutation, we allow the system to produce perfect matches
only by mutation, disabling the production of higher match by the bone
marrow, i. e., we introduce a hole in the repertoire. Thus, for a complete
recognition, the system has to produce the perfect match by mutating the
two-bit mismatch first and the one-bit mismatch after. This process requires time and, more important, requires a large number of training patterns. In fact, as can be observed in figure 4, twenty injections of the same
antigen are necessary before the zero mismatch reaches the higher population value. The dynamics proceeds as follows: during the first 20 time
steps the antigen is not eliminated because the immune response has not
fully started yet. After 35 time steps the antibodies are being produced
(not shown) and the first three antigens' injections have been eliminated.
Each new injection provokes a burst of matching clones. The low match
cells have already developed the memory so that they quickly eliminate
the antigens and proliferate. During the proliferation the mutation turns
new receptors eventually producing a better one. The final maturation is
Fig. 4. Antigen (down) and B cell match (up) populations versus time steps. We need a large
number of presentations before the system generalizes. In the lower plot we simulate twenty
ªpresentationsº of the same antigen.
Pattern recognition by primary and secondary response of an AIS
103
due to the evolutionary pressure of the affinity function (the probability
to interact v(m)) favoring the higher match in the competition for the limited resource represented by the antigen. The dashed line in the upper plot
of figure 4 corresponds to the minimum match population, the continuous
line corresponds to all the B cells with mismatch one and the thick line is
the perfect match population. In the inset it is shown the first 100 time
step with the curves in logarithmic scale. It is evident that only the minimum match is present at the start (because of the artificial hole in the repertoire) and better matches are created successively through mutation.
6 The learning cascade
In the same figure one can see how the recognition process proceeds from
low to high match as if the information on the pattern is discovered in discrete steps in a monotonic way. To investigate it, an entropy function associated to the quantity of information the system has discovered during the
response [13] has been defined. The information gain or Kullback relative
entropy [17] has been shown to behave like a proper entropy function at
least during the recognition process [13]. Calling f (t)
m the fraction of B population at time t which matches m bits of the antigen, the information
Fig. 5. Log-log plot of G(s) during the 200 time steps in which the system learns to recognize
the presented pattern. The final state is associated with a higher information. After the
response is activated G(s) increases monotonically as a proper entropy function.
104
F. Castiglione, S. Motta and G. Nicosia
P
gain is defined as G…s† ˆ m fm…t† log …fm…t‡† =fm…t† . It represents the information cost needed to bring the system from low-affinity state at time t to
high-affinity state at time t + s. In other words the gain is the amount of
information the system has already learned from the presented pattern.
Figure 5 shows G(s) for the cases mc = 10 previously discussed (see figure 4) and a similar case with mc = 11 (not shown). Once the learning process starts, the gain increases monotonically reaching a final steady state.
Each pattern presentation brings G(s) to a new state increasing the information on the pattern. This process has been called ªlearning cascadeº [13]
and has been investigated under continuous antigenic stimulation.
7 Recognition capacity
It is unquestionable that for such a model the number of possible patterns
that can be recognized (the recognition capacity) depends on a high degree
on the initial number of entities we displace on the lattice-grid. To understand to what extent this can invalidate the recognition ability we performed a set of experiments in which the number of presented patterns
along with the initial number of cells is varied. Moreover, to understand if
the mutation process can represent a solution to this problem we performed the same experiments turning the mutation on.
For these experiments we used mc = l and 150 time steps. The mutation
per-bit rate was pm = 0.02 giving a probability to mutate a string in one or
more bits equal to 0.215. The initial population is varied from 125 to 4000
while we present from 20 up to 100 different patterns.
The initial population N determines the average number of cells having a
match with the antigen equal to m. In fact, by the binomial distribution
we insert on average N(lm)/2l cells with match m. This determines the expressed initial repertoire and the chance that the antigen cannot be recognized immediately.
Figure 6 shows the recognition rate corresponding to the percentage of recognized patterns. Smaller recognition capacity is associated to small initial population. This fact can be explained calculating the number of randomly chosen cell-receptors that we need to draw to get one perfect
match. This number is a random variable drawn from a geometric distribution with parameter p = 2±l. The average value is (1 ± p)/p, that in case
l = 12 gives 4095 cells to be created before to get a perfect match. Thus, if
we start with N B cells and we set the number of new cells produced by
the bone marrow in each time step proportional to N (i. e., mN), we have
N(1 + mt) cells created within t steps. It follows that on average we have to
wait t ~ (4095 ± N)/mN time steps to have a perfect match. On the other
hand, if the mutation is turned on, the system can generate diversity with
a higher probability.
Pattern recognition by primary and secondary response of an AIS
105
Fig. 6. Recognition rate for different initial populations and number of different patterns presented. One data points is relative to the same system configuration but with the mutation on
(pm = 0.02).
Figure 6 shows a recognition capacity of about 100 percent in both situations when the initial population is sufficiently large (~ 3500). On the contrary, when N is small (< 1500) and thus the repertoire is not complete, the
mutation guarantees a higher recognition ability. As final remark it is
worth to mention that the recognition capacity trivially increases when mc
< l as the initial number of recognizing cells is larger due to the binomial
distribution of the initial repertoire of cell receptors.
8 Conclusions
In this paper we have studied the pattern recognition ability of an artificial
immune system. We have shown that it is able to recognize multiple presented patterns and to cover a large repertoire thanks to the mutation feature. We have worked on a relative small number of possible patterns
(212). Nevertheless this limit can be extended using parallel computing. In
[11] it is shown that the number of pattern can be extended to 224, useful
to study other immunological features.
The bottleneck seems to be the initial number of cells represented in the
system. In fact to assure a large coverage of the pattern space this number
should be equivalent to the cardinality of the pattern space. The mutation
facility breaks this limit assuring a higher recognition rate augmenting the
ability to search for higher matches in the pattern space. The threshold
106
F. Castiglione, S. Motta and G. Nicosia
parameter mc on the affinity function can be used to tune the tolerance in
the recognition process. The larger mc is, the more precise is the pattern
matching. With smaller mc and mutation turned on, one can tune the recognition precision varying the number of presentation of the same pattern. Lastly, it has been discussed that a certain degree of customization is
possible by modifying the matching function, by modeling a multi-string
antigen and giving complex rules for the behaviour and the interactions
among the entities. In this respect it seems that the immunological pattern
recognition paradigm is suitable for problems in which the recognition
must undergo complex sub-matching steps and where the final response
must be elicited by majority rules in a competition landscape.
References
[1] Dasgupta, D. eds. (1998) Artificial Immune Systems and their Applications, Springer-Verlag.
[2] Forrest, S.; Javornik, B., Smith, R. E. and Perelson, A. S. (1993) Evolutionary Computation 1 (3): 191.
[3] Holland, J. H. (1992) Adaptation in natural and artificial systems (2nd ed.) Cambridge,
MA MIT Press, (1st ed., 1975).
[4] Seiden, P. E. and Celada, F. (1992) J. Theor. Biol. 158: 329.
[5] Celada, F. and Seiden, P.E. (1992) Immunology Today 13: 56.
[6] Perelson, A. S. (1989) Immunological Review 110: 5.
[7] Perelson, A. S.; Weisbuch, G. and Coutinho, A. (1992) Theoretical and Experimental insights into immunology, Springer-Verlag, NY.
[8] Bersini, H. and Varela, F. J. (1990) in the Proceedings of the First Conference on Parallel
Problem Solving from Nature, 343.
[9] Stauffer, D. and Pandey, R. (1992) Computers in Physics 64 (4): 404.
[10] Farmer, J. D.; Packard, N. H. and Perelson, A. S. (1986) Physica D 22: 187.
[11] Bernaschi, M. and Castiglione, F. (2000) submitted to Computers in Biology and Medicine (2000).
[12] Castiglione, F.; Mannella, G.; Motta, S. and Nicosia, G. (1999) Int. J. of Modern Phys. C
10 (4): 677.
[13] Succi, S., Castiglione, F. and Bernaschi, M. (1997) Phys. Rev. Lett. 79: 4493.
[14] Morpurgo, D., SerentaÁ, R., Seiden, P. E. and Celada, F. (1995) Int. Immunol. 7 (4): 505.
[15] Burnet, F. M. (1959) The Clonal Selection Theory of Acquired Immunity, Cambridge
Univ. Press.
[16] Forrest, S. and Hofmeyr, S. A. (2000) Immunology as Information Processing in Segel, L. A. and Cohen, I. R. (eds.), Design Principles for Immune System & Other Distributed Autonomous Systems, Oxford Univ. Press.
[17] Ellis, R. S. (1995) Entropy, Large Deviations and Statistical Mechanics, Springer-Verlag,
New York.