Download THE EVOLUTION OF INDIRECT RECIPROCITY Robert

Social Networks
North-Holland
11 (1989) 213-236
THE EVOLUTION
Robert
BOYD
OF INDIRECT
213
RECIPROCITY
*
University of California
Peter J. RICHERSON
**
University of California
Human societies are based on cooperation
among large numbers of genetically unrelated individuals. This behavior
is puzzling from an evolutionary
perspective.
Because cooperators
are
unrelated
it cannot
be the result of kin selection, and the large scale seems to preclude
explanations
based on direct reciprocity.
Alexander (1987) has proposed that large-scale cooperation among humans can be understood
as resulting from networks of “indirect”
reciprocity.
For
example, individual A may help individual B even though A receives no direct reciprocal benefit.
Instead, B might help C who helps D who finally returns the help indirectly
to A. Here we
describe a simple mathematical
model of the evolution of indirect reciprocity.
Analysis of this
model suggests that indirect reciprocity is unlikely to be important
unless interacting
groups are
fairly small.
1. Introduction
While human societies are unusual for their scale, complexity
and
extent of cooperation,
they are not unique. Sociality of any kind is rare
in nature, and most of the animal societies that do exist are quite
limited in size and complexity. However, there are a number of eusocial
taxa whose societies are characterized
by large groups, substantial
division of labor and extensive cooperation.
Eusocial behavior characterizes insect species like bees, ants and termites (Wilson 1971) and
the naked mole rat, a subterranean
African rodent (Jarvis 1981).
Multicellular plants and many forms of multicellular
invertebrates
can
also be thought of as eusocial societies made up of individual cells
(Buss 1987).
* Department
of Anthropology,
University of California,
Los Angeles, CA 90024, U.S.A.
* * Institute of Ecology, University of California,
Davis, CA 95616, U.S.A.
0378-8733/89/$3.50
0 1989. Elsevier Science Publishers
B.V. (North-Holland)
214
R. Boyd and P.J. Richerson
/ Evolution of indirect reciprocity
Human behavior is unique in that cooperation
and division of labor
take place in societies composed of large numbers of unrelated individuals. In other eusocial species, societies are made up of close genetic
relatives. The cells in a multicellular organism are typically members of
a clone, and the individuals in insect and naked mole rat colonies are
siblings. According to contemporary
evolutionary
theory, cooperative
behavior can only be favored by selection when social groups are
formed so that cooperators
are more likely to interact with other
cooperators
than with non-cooperators
(Hamilton
1975; Brown et al.
1982; Nunney 1985). It is widely agreed that kinship is the most likely
source of such non-random
social interaction. Human society is thus an
unusual and interesting special case of the evolution of cooperation.
Several authors have conjectured that human eusociality is based on
reciprocal cooperation
(Trivers 1971; Wilson 1975; Alexander 1987). It
is unclear, however, whether natural selection will favor reciprocal
cooperation
in sizable groups. Several related analyses (Boorman and
Levitt 1980; Axelrod 1980, 1984; Axelrod and Hamilton 1981; Brown
et al. 1982; Aoki 1984; Peck and Feldman 1985; Sugden 1986; Boyd
and Lorberbaum
1987) suggest that cooperation
can arise via reciprocity when pairs of individuals interact repeatedly. These results suggest
that the evolutionary
equilibrium
in this setting is likely to be a
contingent strategy with the general form of tit-for-tat:
“cooperate
the
first time you interact with another individual, but continue to cooperate only if the other individual also cooperates.”
Recently, we (Boyd
and Richerson, 1988) have analyzed a model in which larger groups of
individuals interact repeatedly
in a potentially
cooperative
situation.
This analysis suggests that the conditions under which reciprocity
can
evolve become extremely restrictive as group size increases above a
handful of individuals. The model is, however, simplified in a variety of
ways. In particular, it does not allow networks of cooperative
relationships within larger groups. Richard Alexander (1987) has argued that
such networks, which he calls networks of “indirect”
reciprocity,
are
crucial for understanding
human sociality. However, since there has
been no explicit theoretical
treatment
of the evolution
of indirect
reciprocity, it is unclear whether Alexander’s argument is cogent.
The goal of this paper is to clarify this issue by extending existing
theory of the evolution of reciprocity to explicitly include the possibility of indirect reciprocity.
We review models of the evolution
of
reciprocity, and present a model of the evolution of indirect reciprocity.
R. Boyd and P.J. Richerson / Evolution of indirect reciprocity
215
An analysis of this model suggests that the conditions necessary for the
evolution of reciprocity become restrictive as group size increases.
2. Models of the evolution of reciprocal cooperation
For the most part, evolutionary
models of cooperation
have been
developed by biologists interested in explaining cooperative
behavior
among non-human animals (see Wade 1978; Uyenoyama
and Feldman
1980; Michod 1982; Wilson 1980, for reviews). This work assumes that
individual differences in social behavior, including the strategies that
govern individual behavior in potentially reciprocal social interactions,
are affected by heritable genetic differences. They further assume that
the outcome of potentially
cooperative
social interactions
affect an
individual’s reproductive
success. Successful behavioral strategies will
thus increase in the population through natural selection. The question
then is: under what conditions will natural selection favor behavioral
strategies that lead to cooperation
in large groups? The answer to this
question should illuminate contemporary
human cooperation
to the
extent that evolved propensities shape human behavior.
If behavioral strategies are transmitted
culturally instead of genetically, evolutionary
models also provide insight into the conditions
under which cooperative behavior will arise in contemporary
societies.
Some authors (Axelrod 1984; Brown et al. 1982; Maynard Smith 1982;
Pulliam 1982; Boyd and Richerson 1982, 1985) have constructed
models, formally quite similar to the genetic ones, which assume that
behavioral
strategies are transmitted
from one individual to another
culturally, by teaching, imitation, or some other form of social learning.
These models assume that the probability that a strategy is transmitted
culturally is proportional
to the average payoff associated with that
strategy. There are many plausible ways that this can occur. For
example, it may be that people tend to imitate wealthy or otherwise
successful individuals.
The kinds of strategies that can evolve with
cultural transmission
often differ from those that can evolve with
genetic transmission because of differences in the properties of the two
kinds of inheritance
systems (e.g. Boyd and Richerson,
1982). (For
general discussions of the relationship
between genetic and cultural
evolution, see Cavalli-Sforza
and Feldman 1981, Lumsden and Wilson
1981, and Boyd and Richerson 1985).
216
R. Boyd and P.J. RIcherson / Evolution of indirect reciprocity
Table 1
Each player has the choice of two strategies, C for cooperate and D for defect. The pairs of entries
in the table are the payoffs for players 1 and 2, respectively. associated with each combination
of
strategies. In the case of Prisoners dilemma it is assumed that T > R > P > S. and 2R > S+T
Player 2
Player 1
C
D
C
D
R, R
T, S
S, T
P, P
Models designed to elucidate the evolution
of reciprocity
among
pairs of individuals
share many common features. Each assumes a
population of individuals. Pairs of individuals sampled from this population interact a number of times. During each interaction
individuals
may either cooperate (C) or defect (0). Table 1 gives the incremental
effect of each interaction on the fitness of the members of a pair. This
pattern of fitness “payoffs” defines a single period prisoner’s dilemma;
it means that cooperative
behavior is altruistic in the sense that it
reduces the fitness of the individual performing the cooperative behavior, but increases fitness of the other individual in the pair (Axelrod
and Hamilton 1981; Boyd 1988). By assumption,
each individual
is
characterized
by an inherited
strategy that determines
how it will
behave. Strategies may be fixed rules, -like unconditional
defection
(“always defect”), or contingent ones like tit-for-tat (“cooperate
during
the first interaction;
subsequently do whatever the other individual did
last time”). The pair’s two strategies determine the effect of the entire
sequence of interactions on each pair member’s fitness.
Analysis of such models suggests that lengthy paired interactions
are
likely to lead to the evolution of reciprocity. First, reciprocating
strategies, like tit-for-tat,
leading to mutual cooperation
are successful if
pairs of individuals
are likely to interact
many times. Second, a
population
in which unconditional
defection
is common
can resist
invasion by cooperative
strategies under a wide range of conditions.
However, there seem to be a variety of plausible mechanisms that allow
reciprocating
strategies to increase when rare. Axelrod and Hamilton
(1981) and Axelrod (1984) have shown that a very small degree of
assortative
group formation,
when coupled with the possibility
of
prolonged reciprocity, allows strategies like tit-for-tat to invade non-co-
R. Boyd and P.J. Richerson / Evolution of indirect reciprocity
217
operative populations.
Other mechanisms have been suggested by Peck
and Feldman (1985) and Boyd and Lorberbaum
(1987).
In our previous (Boyd and Richerson 1988) model of the evolution
of reciprocal cooperation
in sizeable groups, groups of n individuals
were sampled from a larger population,
and then allowed to interact
repeatedly in an n-person prisoner’s dilemma. This assumption means
that cooperation
is costly to the individual, but beneficial to the group
as a whole. An analysis of this model suggests that increasing the size
of interacting
social groups reduces the likelihood that selection will
favor reciprocating
strategies. As in the two-individual
case, if groups
persist long enough, both non-cooperative
and reciprocal behavior are
favored by selection when common. For large groups, however, the
conditions
under which reciprocity
can increase when rare become
extremely restrictive. This conclusion
should be viewed with caution
because the n-person prisoner’s dilemma captures only one of many
kinds of cooperative social behavior in sizable groups. It might be that
reciprocity could evolve under other circumstances.
Alexander (1985, 1987) imagines that individual A may help individual B even though A receives no direct reciprocal benefit. Instead, B
might help C who helps D who finally returns the help to A. He calls
this kind of interaction indirect reciprocity. As in the two-person case,
cooperation
is enforced by the threat of punishment.
Alexander envisions several possibilities. A may continue to help B only if A continues
to be helped by D. Or, alternatively,
A may continue to help B only if
B continues to help C. Or, perhaps both. In all these cases (and others
outlined by Alexander), contingent behavior is based on local information. An individual knows what happens to individuals with which it
interacts, but not about behavior along the entire chain of indirect
reciprocity. Our aim is to explore whether Alexander’s conjecture that
indirect reciprocity can evolve by natural selection is cogent.
In the following sections we present two models of the evolution of
indirect reciprocity.
These models differ in the amount of knowledge
available to individuals. In the first model, individuals only know what
is done to themselves, not whether the potential recipients of their own
cooperation
are helping others. This model has very similar properties
to our n-person model (Boyd and Richerson 1988). As groups become
large, the conditions
under which reciprocity
can evolve become extremely restrictive. In the second model, individuals can condition their
cooperation
on the behavior of the potential recipient-if
the recipient
R. Boyd and P.J. Richerson
218
/ Evolution
of indirect reciprocity
helps others, I will help him. This model is intermediate
between the
two-person and n-person models. Increasing group size does reduce the
possibility of cooperation,
but not so severely as in the previous case.
3. Model assumptions
We believe that the following model captures many of the important
features of Alexander’s proposal. Suppose that groups of size n are
sampled from a large population,
and interact repeatedly.
The probability that the group persists for one more interaction
is w and thus
that it persists t or more interactions,
w’. During each interaction
an
individual can either cooperate
(C) or defect (0). The incremental
effect of a single interaction on the fitness of an individual depends on
that individual’s behavior and the behavior of one other individual,
who we will refer to as the “upstream”
individual, as shown in Table 2.
We further assume that the same individual is upstream throughout the
life of the group. Thus, it is as if individuals were arranged in a ring.
An individual’s behavior affects the fitness of the individual
downstream; an individual’s
fitness is affected by the behavior
of the
upstream individual (recall the aphorism “what goes around comes
around”). An individual’s fitness is the sum of the incremental
fitness
effects from all interactions.
Each individual is characterized
by an inherited “strategy”
which
specifies whether it will choose cooperation
or defection during any
time period based on the history of the behavior of its neighbors up to
that point. We consider
three strategies:
unconditional
defection
tit-for-tat” (UTFT) and “downstream
tit-for-tat”
(ALLD), “upstream
Table 2
Behavior
Behavior of
focal individual
Where
b > c > 0.
Cooperate
Defect
of upstream
individual
Cooperate
Defect
b-c
b
-c
0
R. Boyd and P.J. Richerson / Evolution of indirect reciprocity
(DTF’T).
These strategies
ALLD:
UTFT:
Always defect.
Cooperate
on the first move and then cooperate
on each
subsequent
move if the individual upstream cooperated
on
the previous interaction. Individuals using upstream tit-for-tat
are nice to others if third parties are nice to them.
Cooperate
on the first move and then cooperate
on each
subsequent move if the individual downstream cooperated
on
the previous interaction.
Individuals
using downstream
titfor-tat are nice to people who are themselves nice to others.
DTFT:
are defined
219
as follows:
UTFT and DTFT are the simplest generalizations
of tit-for-tat to the
case of indirect reciprocity. Both strategies specify contingent altruism
based on local information.
UTFT is easier to implement,
requiring
only that individuals know what has happened to them, while individuals using DTFT must know what others have done to third parties.
Both UTFT and DTFT become identical to ordinary tit-for-tat when
n = 2.
After all social interactions are completed, individuals in the population reproduce. The probability
of reproduction
is determined
by the
results of social behavior. Thus, the representation
of a particular
strategy in the next generation is a monotonically
increasing function
of the average payoff received by individuals
playing that strategy
during the previous period. (As argued by Brown et al. 1982, this
assumption is consistent with haploid genetic inheritance
of strategies
and some simple forms of cultural transmission.)
We then ask, which
strategies or combinations
of strategies can persist? In what follows we
will first consider populations
in which upstream tit-for-tat competes
with unconditional
defection;
then we will consider populations
in
which downstream tit-for-tat competes with unconditional
defection.
4. Upstream tit-for-tat
Consider a population in which only ALLD and UTFT are present. To
determine the evolutionary
history of such a population
we must first
calculate the expected fitness of individuals
who use each strategy.
220
R. Boyd and P.J. Richerson
/ Euolution of indirect reciproat~
Recall that groups can be thought of as organized into rings of social
interaction-an
individual can be helped by the individual immediately
upstream, who can in turn be helped by the next individual upstream.
Then define u as the number of consecutive individuals upstream from
a focal individual who use the strategy UTFT. If, for example, the
individual immediately upstream is ALLD, then u = 0. If the individual upstream is UTFT, but the next individual upstream is ALLD, then
u = 1. If all n individuals in the group are UTFT, then u = n - 1.
In a population
in which only ALLD and UTFT are present, the
expected fitness of an ALLD individual given that there are u UTFT
individuals directly upstream, V( ALLD 1u), is calculated in the following way:
V(ALLD~u)=(1+w+w2+
=
0-q
(1-w)
...
fd-‘)b
for all 24.
ALLD individuals never cooperate;
they receive a benefit of h each
time the upstream individual cooperates. If there is an ALLD individual upstream, the focal ALLD individual receives nothing. If there is
an UTFT individual upstream, the focal individual receives b for at
most u interactions.
During the first interaction
all UTFT individuals
cooperate.
A second interaction
takes place with probability
W. If a
second interaction
does occur, any UTFT individuals
who had an
ALLD individual immediately
upstream defect. On the third interacwith u = 0 and 1 are
tion with probability
w2, UTFT individuals
defecting, and so on. Each ALLD individual begins a chain of defections that move downstream. Thus, u gives the number of interactions
before this chain reaches the focal individual,
assuming the game
persists that many turns.
The expected payoff of an UTFT individual is calculated in a similar
way. There are two differences.
First, each period that the UTFT
individual cooperates it suffers a cost c. Second, if u = n - 1, the group
is made up entirely of UTFT individuals,
and cooperation
will continue as long the group persists. This leads to the following expression
for the expected fitness of an UTFT individual given that there are u
R. Boyd and P.J. Richerson
UTFT
individuals
/ Ewlution
of indirect reciprocity
221
upstream:
(l-w”+yC
for u<n-1,
(1-W)
(2)
for u=n-1.
After the episode of social behavior
that generates
these payoffs,
individuals in the population
reproduce.
We assume that individual
fitness is the sum of a baseline fitness W, and the payoff resulting from
social interaction.
We assume that W, is much larger than the payoff
from social interactions
for all strategies and group compositions.
The
kinds of
expected fitness of UTFT averaged over all the different
groups, W( UTFT), is given by:
n-l
W(UTFT)=
c m(.IUTFT){W,+
u=o
The term in braces is
group in which there
This term is multiplied
itself in such a group,
groups. Similarly, the
W( ALLD), is
V(UTFT( u)}.
(3)
the expected fitness of a UTFT individual in a
are u UTFT individuals immediately
upstream.
by the probability that a UTFT individual finds
m( u ) UTFT), and is summed over all possible
expected fitness of an unconditional
defector,
n-l
c
W(ALLD)=
m(u(ALLD){Wo+
V(ALLDJu)},
(4)
u=O
where m( u 1ALLD) is the probability
that an ALLD individual finds
itself in a group in which there are u UTFT individuals immediately
upstream.
If the frequency of UTFT in the population before social interaction
is p, then the frequency before social interaction in the next generation,
p’, is:
[~(UTFT)P'=P+P(l-PI
W(ALLD)]
w
3
222
R. Boyd and P.J. Richerson / Ervlution of indirect reclprocrt,:
where
W=pW(UTFT) + (1
-p)W(ALLD).
To determine the long-run evolutionary
outcomes, we determine
the
frequencies
of UTFT that represent
equilibria
(denoted
j?) of the
recursion (5). Stable equilibria tell us where the population
can go in
the long run, and unstable equilibria tell us about the range of initial
conditions
that will evolve toward different
stable equilibria.
For
example, in many cases the only stable equilibria will be a population
composed of only ALLD individuals, or a population made up of only
UTFT individuals. Then if the unstable equilibrium frequency is near
the stable equilibrium composed of pure ALLD, it is easy to imagine
that even chance events will carry the population
to a value of p that
exceeds the unstable equilibrium point, in which case the population
will evolve off toward the other stable equilibrium at which only UTFT
individuals are present. Here we say that the “domain of attraction”
of
the pure cooperative
equilibrium
is large, or, more informally,
that
cooperation
is likely to evolve. In contrast, if the unstable equilibrium
is near the all-cooperation
equilibrium, then it is much less likely that
chance events will displace a population
sufficiently
far from the
non-cooperative
equilibrium to approach the unstable equilibrium. The
domain of attraction of pure cooperation
is small in this case.
First consider the case in which groups are formed by randomly
sampling individuals from the population.
Here, the evolutionary
dynamics of the population are particularly simple. If wn-lb < c, then the
only stable equilibrium
is a population
composed entirely of ALLD
individuals
(3 = 0). Moreover,
the population
will reach this equilibrium from any initial frequency of reciprocators.
In this situation the
long-term benefits of indefinite cooperation
in a group of n UTFT
individuals are less than the short-term benefits achieved by being a
defector in a group of n - 1 UTFT individuals. If ~“~ib > c, there are
potentially benefits to cooperation
that exceed the costs. Now there are
two stable equilibria,
j = O-a
population
made up of all ALLD
individuals-and
$ = l-a
population made up of all UTFT individuals. There is also a single unstable internal equilibrium,
$,. If the initial
eventually
frequency of UTFT is less than I;,, then the population
consists of all ALLD individuals. If the initial frequency of UTFT is
is eventually
composed
entirely of
greater than I;,, the population
R. Boyd and P.J. Richerson
Thrrhold
frequmcy
223
/ Eaolution of indirect reciprociiy
for
fu to incrrrw
r
0.8
Thrsshol d , ,
Frequency .
8.4
-b/c
- 1.4
-b/c
= 2.0
--b/c
= 4.0
... b/c = 8.0
1
:
a
‘.
OF:::::!:::!::!:
2
4
c
m (0 12 14 12 1) 22 22 24 22 22 m
Croup Size
Thr&o\d
frequency
32
(n)
for Tu to increase
c --w
... ” =
- 0.9
0.99
0.9999
0.999
b
2
4
2
2
(0 12 14 16 (2 P
Group Size
tz 24 22 22 22 1
InI
Fig. 1. Threshold frequency of UTFT (3,) necessary
of varying b/c with w = 0.99. (b) Effect of varying
for CJTFT to increase
w with h/c = 2.
in frequency.
(a) Effect
224
R. Boyd and P.J. Richerson
/ Evolution of indirect recrprocity
UTFT individuals. The larger is 3; the less likely it is that a population
will escape from the ALLD equilibrium and evolve cooperation.
(For
mathematical
details see the Appendix.)
We have not been able to derive an analytical expression for j,.
However, it is easy to determine these values numerically. As shown in
Figure 1, numerical calculations suggest that increasing w and b/c and
decreasing n cause ji to decrease. Relatively small groups, long periods
of interaction
and high benefit-cost
ratios cause j?, to be small and
thus the domain of attraction of the all-UTFT equilibrium to be quite
large. However, for larger groups the domain of attraction
of ALLD
remains quite large even when groups interact a hundred thousand
times.
Next we consider the case in which groups are formed assortatively
so that like types are more likely to interact than chance alone would
dictate. Such non-random
interaction
plays an important
role in existing models of evolution of reciprocity.
Reciprocating
strategies like
tit-for-tat cannot increase when rare when individuals interact at random. However, if individuals interact a large number of times (w + l),
even a small amount of assortment will allow reciprocating
strategies to
increase (Axelrod and Hamilton
1981). Boyd and Richerson
(1988)
have shown that as groups become larger the conditions under which
reciprocating
strategies can increase rapidly become extremely restrictive even when assortment
is allowed. To determine
the effect of
assortative social interaction on the evolution of indirect reciprocity, we
assume that the conditional probability
that any other randomly chosen individual in a group is UTFT given that the focal individual is
UTFT is given by:
Prob(UTFT)UTFT)=r+(l-r)p;
(6)
similarly,
Prob(ALLD)ALLD)=r+(l-t-)(1-p),
(7)
where 0 < r < 1. This model is meant to capture the general notion of
assortative social interaction in a mathematically
tractable form. There
is a chance Y of drawing an individual with an identical strategy to the
focal individual and a chance 1 - Y of picking an individual at random
from the population
(who of course will also be identical to the focal
individual with probability
p). While this model is not precisely con-
R. Boyd and P. J. Richerson
/ Evolution of indirect reciprocity
225
Threshold value of r for 7u to incrraee
‘1
,~____
. . . . . . .
.
__a_-__
____-__-
______
-
______
.._..........................~..”.................
a
2
4
2
m
10 12 14 12 12 20 22 24 1
Group Size
Thrkold
22 m
s
InI
value of r for Tu to increase
‘1
0.9
0.a
0.7
--*
= 0.9
--I
-
0.99
-
0.999
=
0.9999
0.2
Thresttol d
Value of r
I
-II
...
”
b
2
4
6
2
t0
12 14 16 0
Group Size
22 Zz 24 22 22 m
(nl
Fig. 2. Degree of assortment (r) necessary for UTFT to increase when rare. (a) Effect of varying
b/c with w = 0.99. (b) Effect of varying w with b/c = 2. Note that the asymptotic
value of r is,
in all cases, the value predicted by kin selection alone.
226
R. Bqyd and P.J. Richerson / Eoolu~wn
of indirect reciprouly
sistent with all genetic models, it is robust enough to determine
the
conditions under which a reciprocating
strategy can invade a population in which all defection is common. With these assumptions
it is
shown in the Appendix that UTFT can increase when rare if:
rb - c
inclusive fitness
effect
+
r)(rw)“-‘(b - WC) > o
(1 -
(l-w)
reciprocity
.
(8)
effect
Selection can favor cooperative
behavior
when there is assortative
social interaction even with no possibility of reciprocity, because cooperators are more likely than defectors to benefit from the cooperation.
The first term on the left-hand side of (8) represents
this inclusive
fitness effect (Hamilton
1975). This term indicates that even if w is
zero UTFT can increase as long as the inclusive fitness of CJTFT
individuals
is higher than that of unconditional
defectors.
In the
present context, the most interesting cases are ones in which the first
term is negative, meaning that cooperation
could not be favored
without reciprocity. The second term on the left-hand side of (8) gives
the effect of reciprocity when reciprocators
are rare. Notice that this
term is proportional
to ( TW)~~ I. This means that the reciprocity effect
decreases exponentially
as group size increases. As is shown in Figure
2, the result is that when groups are small and persist a long time,
UTFT can increase with quite small amounts of assortment.
When
groups are larger, reciprocity has almost no effect. Even in groups of 16
with very long expected numbers of interactions,
the values of Y are
essentially those predicted by the calculus of inclusive fitness alone.
Either unconditional
cooperation
is favored or no cooperation
is
favored.
5. Downstream tit-for-tat
Next consider a population
in which only ALLD and DTFT are
present. Once again, the first step is to calculate the expected fitness of
individuals who use each strategy. Let d be the number of consecutive
DTFT individuals downstream from a focal individual. In a population
in which only ALLD and DTFT are present, a DTFT individual’s
R. Boyd and P.J. Richerson / Evolution
expected fitness depends
upstream:
((1-M:
on d and whether
d+2)&
(1-
(1 -
V( DTFT 1d) = { _ (1 - &+I)
(l-w)
there is a DTFT
(1 -MJd+l)(.
4
c
b-c
4
227
of indirectreciprocity
individual
for d c n - 2
and
DTFT upstream
for d < n - 1
and
A LLD upstream
for d= n - 1.
j l-w
(9)
In contrast, the expected fitness of an ALLD individual
on the strategy of the individual immediately upstream:
V( ALL 1d) =
b
if upstream
individual
is DTFT
i 0
if upstream
individual
is ALLD.
depends
only
00)
The expected fitnesses of the two strategies, W( ALLD) and W( DTFT),
are calculated by averaging over all possible group compositions
(see
the Appendix). This leads to a recursion for the frequency of DTFT in
the population that is closely analogous to equation (5), which can be
used to find equilibria and determine their stability.
When groups are formed randomly,
the conditions
under which
DTFT can evolve become more restrictive
as group size increases.
However, the effect of group size is less pronounced
than in the case of
UTFT. If wb < c, then the only stable equilibrium
is a population
composed entirely of ALLD individuals (jj = 0), and the population
will reach this equilibrium from any initial frequency of reciprocators.
If wb > c, there are two stable equilibria, j? = 0, and jj = l-a
population made up of all DTFT individuals. Notice that these conditions are
independent
of n, unlike the UTFT case. Any time reciprocity
can
persist in interacting
pairs, indirect reciprocity
based on DTFT can
also persist in larger groups. When a population composed of all DTFT
individuals
is evolutionarily
stable, there is also a single unstable
internal equilibrium,
fi,. Any population
with an initial frequency of
DTFT greater than 9,. will eventually be composed
entirely of DTFT
228
R. Boyd and P.J. Richerson / Euolurion
Threshold
frequency
of indirect reciprocity
for Td to increree
l0.)[email protected]..
.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .._............
a
2
4
C
8
10 12 14 16 12 20 22 24 t
Group Size
Threshold
frequency
S
S
I
(nl
for 7d to increase
b
2
4
C
2
10 12 14 II 12 a0 22 24 1
Group Size
Fig. 3. Threshold
Effect of varying
28 0
P
(nl
frequency of DTFT (p,) necessary for DTFT to increase
b/c with w = 0.99. (b) Effect of varying w with b/c = 2.
in frequency.
(a)
R. Boyd and P.J. Richerson / Evolution of Indirect recrprocity
229
individuals. As before, the larger is j?,, the smaller is the domain of
attraction
of the cooperative
equilibrium,
jj = 1. Figure 3 displays
values of EC that were obtained numerically.
These values are always
lower than the corresponding
value for UTFT shown in Figure 1, and
the difference is larger for larger groups. This means that DTFT has a
larger domain of attraction than does UTFT. Moreover, the effect of
large groups is qualitatively
different in the two cases. As n increases,
the domain of attraction
of UTFT shrinks toward zero. When n is
large enough (n > log,( c/b) - l), UTFT ceases to be evolutionarily
stable, and ALLD is the only ESS. In contrast, as n becomes large in
the case at hand, the domain of attraction
of DTFT approaches
an
asymptotic
value, PC + c/wb. However, the conditions
under which
DTFT can increase are still more restrictive in large groups than among
pairs. When n = 2, the domain of attraction of ALLD shrinks to zero
as w + 1. This means that if pairwise interactions
persist long enough,
even very small initial numbers of reciprocators
will allow reciprocity
to be favored by selection. In contrast, in larger groups the domain of
attraction of UTFT shrinks toward an asymptote that is guaranteed to
be greater than zero (the asymptotic value of j,. is less than c/b).
Next we consider the case in which groups are formed assortatively
so that like types are more likely to interact than chance alone would
dictate. Using the same model of non-random
group formation
as in
the previous section, it can be shown that DTFT can increase when
rare if:
rb - c
inclusive fitness
effect
+
(‘1’-;;)
((nv)n-l(wb-c)-w(wrb-c))
reciprocity
>O.
effect
(11)
As before, the first term on the left-hand side of (11) represents the
effect of kin selection (Hamilton 1975). Even if w is zero, DTFT can
increase as long as the inclusive fitness of DTFT individuals is higher
than that of unconditional
defectors. The second term on the left-hand
side of (11) gives the effect of reciprocity when reciprocators
are rare.
Notice that here there is some effect of reciprocity even if group sizes
are large. As is shown in Figure 4, the result is that when groups are
small and persist a long time, DTFT can increase with quite small
230
R. Boyd and P.J. Richerson
Threshold value
/ Evolution of indirect reciprocity
of r for Td to increase
t-
a
03..
0.8..
0.7..
-b/c
- 1.4
-b/c
- 2.0
-b/c
- 4.0
0.8..
Thresh01 d
Value of r””
... b/c - 9.0
2
4
C
I
10 12 14 12 l# 22 22 24 ZL 22 0
Group Size M
Threshold
value
of r for Td to increase
b
0.)[email protected]..
--*
- 0.9
OS.-
-u=o.99
$yea,“:d,0.5..
--*=0.999
0.4..
2
.. . y = 0.9999
4
C
0
10 12 14 12 l@ 20 22 24 22 22 ID
Group
Size
(n)
Fig. 4. Degree of assortment (r) necessary for DTFT to increase when rare. (a) Effect of varying
b/c with w = 0.99. (b) Effect of varying w with b/c = 2.
R. Boyd and P.J. Richer-son / Evolution of indirect reciprocity
231
amounts of assortment. When groups are larger, reciprocity continues
to have some effect. However, unless benefit-cost
ratios are quite high,
it requires a sizable degree of assortment
for reciprocity
to increase
when rare.
6. Discussion
Our analysis suggests three conclusions about the evolution of indirect
reciprocity:
(1) At best, indirect reciprocity is less likely to evolve than pairwise
reciprocity. The evolutionary
dynamics UTFT are quite similar to the
n-person generalizations
of tit-for-tat analyzed by Boyd and Richerson
(1988). As groups become large the conditions under which UTFT can
increase when rare become extremely
restrictive.
The evolution
of
DTFT is not nearly so strongly affected by increasing group size. The
evolution of DTFT is evolutionarily
stable whenever pairwise reciprocity is stable. However, as groups become large the threshold frequency
necessary for DTFT to be favored increases. While this threshold does
not approach 1, it can reach substantial values. Similarly, the amount
of assortment necessary for both UTFT and DTFT to increase when
rare also increases with group size. Moreover, the threshold degree of
assortment can be substantial even if w is quite close to 1.
(2) The fact that DTFT can evolve under a much wider range of
conditions than UTFT suggests that strategies which are based on the
principle “be nice to people who are nice to others” will be more
successful than strategies based on the principle: “be nice if others are
nice to you”. Many authors have explained the evolution of direct
reciprocity
in pairs as a result of return benefit (e.g. Irons 1979).
Selection favors reciprocators
because they benefit from their cooperative behavior in the long run. An alternative
explanation
is that
reciprocal strategies such as tit-for-tat are favored because they lead to
assortative interaction
of cooperators
(Michod and Sanderson
1985).
Even if individuals
are paired at random,
the fact that tit-for-tat
individuals convert to defection if they experience
acts of defection
from others causes a non-random
distribution
of cooperative behavior.
Indirect reciprocity is interesting in this regard because it separates the
two processes. Our results suggest that return benefit is much less
important than assortative interaction.
In the long run, it seems to be
better to concentrate
on identifying and withholding cooperation
from
232
R. Boyd and P.J. Richerson
/ Evolution
of indirect reciprocity
non-cooperators
than to worry about whether you are a recipient of
cooperation
or defection.
(3) The evolutionary
importance
of indirect reciprocity
depends
critically on the information
available to individuals. Individuals need
only know what happens to them in order to use UTFT. In contrast,
individuals using DTFT need to know what the downstream individual
did to someone else. In real-world situations, identifying
behavior as
cooperation
or defection may be difficult or even impossible.
One’s
sample of others’ behavior may often be small and biased (e.g. people
may be on their best behavior around rewarding others). In general one
would expect that individuals
would have much better information
about what others did to them than they would have about interactions
with third parties.
Because this model is simplified in a variety of ways, these results
should be regarded with some caution. We believe that the following
simplifications
are likely to be most important.
Real social networks are made up of many interconnected
loops of
varying lengths. Individuals are at the center of a web of potentially
cooperative interactions. Closer others will participate in multiple short
loops tending toward high w, while more distant individuals
will
participate
in large loops with smaller w. Our intuition, schooled by
these models, is that indirect reciprocity is only likely to be effective for
relatively small, close, long-lasting loops. However, since the models
only apply to single loops in isolation, there may be something about
the linking of many loops that permits the extension
of indirect
reciprocity to the large-group situation envisioned by Alexander.
We have assumed that behavior is error-free. Individuals who mean
to cooperate never defect by mistake, and defectors never mistakenly
cooperate.
In the pairwise repeated prisoner’s dilemma, errors have
little effect on the qualitative properties of the model if tit-for-tat
is
slightly modified (Sugden 1986); Boyd (1989) has labeled the modified
tit-for-tat.
Suppose that two contrite
tit-for-tat
strategy “contrite”
players interact and one of them errs. On the next interaction, the other
individual defects. On the second interaction
after the error, the individual making the error does not respond to this punishment
by
defecting, but instead continues to cooperate.
This act of contrition
allows reciprocal cooperation
to persist as long as errors are not too
common. This form of contrition is not possible in the case of indirect
reciprocity
unless each individual is aware of the behavior of every
R. Boyd and P.J. Rxherson
/ Evolution of indirect reciprocity
233
other individual in the group. This fact suggests that indirect reciprocity may be much more vulnerable to errors than is reciprocity
among
pairs.
Cultural transmission
makes possible mechanisms
that are potentially more effective than genetic transmission
in creating assortment
(Boyd and Richerson
1985: Chapters
7 and 8) and large chance
changes in trait frequencies (Cavalli-Sforza
and Feldman 1981: 204).
Thus, although kin selection may exhibit only a modest synergy with
reciprocity
in the models analyzed here, the group selection made
possible by some forms of cultural transmission
may make achieving
the threshold-unstable
equilibria for reciprocal
strategies easier, and
their subsequent spread more effective, than is the case for genetically
transmitted strategies.
References
Alexander, R.D.
1985 “A biological interpretation
of moral systems.” Zygon 20: 3-20.
Alexander, R.D.
1987 The Bio/ogv of Moral Systems. New York: Aldine De Gruyter.
Aoki, K.
1984 “A quantitative
genetic model of reciprocal
altruism:
A condition
for kin or group
selection to prevail.” Proceedings ofthe National Academy of Sciences USA 80: 4065-4068.
Axelrod, R.
1980 “Effective choice in the prisoner’s dilemma.” Journal of Conflict Resolution 24: 3-20.
Axelrod, R.
1984 The Evolution of Cooperation. New York: Basic Books.
Axelrod, R. and W.D. Hamilton
1981 “The evolution of cooperation.”
Science 211: 1390-1396.
Boorman, S. and P. Levitt
1980 The Genetrcs of Altruism. New York: Academic Press.
Boyd, R.
1988 “Is the repeated
prisoner’s
dilemma
game a good model of reciprocal
altruism?”
Ethology and Sociobiology 9: 211-221.
Boyd, R.
1989 “Mistakes allow evolutionary
stability in the repeated prisoner’s dilemma game.” Journal of Theoretical Biology 136: 47-56.
Boyd, R. and Lorberbaum,
J.
1987 “No pure strategy is evolutionarily
stable in the repeated prisoner’s dilemma game.”
Nature 327: 58-59.
Boyd, R. and P.J. Richerson
1982 “Cultural
transmission
and the evolution of cooperative
behavior.”
Human Ecology 10:
325-351.
Boyd, R. and P.J. Richerson
1985 Culture and the Evolutionary Process. Chicago: University of Chicago Press.
Boyd, R. and P.J. Richerson
1988 “The evolution of reciprocity
in sizable groups.” Journal of Theoretlcnl Biology 132:
337-356.
234
R. Boyd and P.J. Richerson
/ Euolutron of mdirect reciprocit.y
Brown, J.S., M.J. Sanderson and R.E. Michod
Journul of Theoretrcul Biology YY:
1982 “Evolution
of social behavior by reciprocation.”
319-339.
Buss, L.
1987 The Evolution of Individuality. Princeton:
Princeton University Press.
Cavalli-Sforza,
L.L. and M.W. Feldman
1981 Culrural Transmissron and Evolufion. Princeton:
Princeton University Press.
Hamilton, W.D.
1975 “Innate social aptitudes of man: An approach
from evolutionary
genetics.” In R. Fox
(ed.). Biosocial Anthropologv, pp. 135-232. London: Malaby.
Irons, W.
1979 “Natural
selection, adaptation,
and human social behavior.”
In N. Chagnon
and W.
Irons (eds.), Evolutionary Biology and Human Social Behavior, pp. 4-39. North Scituate,
MA: Duxbury.
Jarvis, J.U.M.
1981 “Eusociality
in a mammal: Cooperative
breeding in naked mole rat colonies.” Soewe
212: 571-573.
Lumsden, C.J. and E.O. Wilson
1981 Genes, Mind, and C&we. Cambridge,
MA: Harvard University Press.
Maynard Smith. J.
1982 Evolurionary and the Theory of Games. London: Cambridge
University Press.
Michod. R.E.
1982 “The theory of kin selection.” Annual Review’ of Ecology and Svstemutics 13: 23-56.
Michod, R.E. and M.J. Sanderson
1985 “Behavioural
structure
and the evolution of cooperation.”
In P.J. Greenwood,
P.H.
Harvey and M. Slatkin (eds.), Eoolution: Essays in honor of John Mavnurd Smrth, pp.
95-106. Cambridge,
UK: Cambridge
University Press.
Nunney, L.
1985 “Group
selection, altruism.
and structured
deme models.” American Naruralist 126:
212-230.
Peck, J. and M.W. Feldman
1985 “The evolution of helping behavior in large, randomly
mixed populations.”
Amerrcan
Naturalist 127: 209-221.
Pulliam, H.R.
1982 “A social learning model of conflict and cooperation
in human societies.”
Human
Ecolog): IO: 353-363.
Sugden. R.
1986 The Economrcs of Rights, Co-operation and Welfare. Oxford: Basil Blackwell.
Trivers, R.
1971 “The evolution of reciprocal altruism.” Quarterly Reuiew of Bloloa 46: 35-57.
Uyenoyama,
M. and M.W. Feldman
1980 “Theories
of kin and group selection: A population
genetics perspective.”
Theoreticcrl
Population Bmlogy 17: 380-414.
Wade. M.J.
1978 “A critical review of group selection models.” Quarterly Reurew of Biologv 53: 101-114.
Wilson, D.S.
1980 The Natural Selection of Populations and Communities.
Menlo Park. CA: Benjamin/
Cummings.
Wilson, E.O.
1971 The Insect Societies. Cambridge,
MA: Harvard University Press.
Wilson, E.O.
1975 Sociobiology: The New Synthesis. Cambridge,
MA: Belknap/Harvard
Universtty Press.
R. Boyd and P.J. Richerson
/ Evolution of indirect reciprocity
235
Appendix
Results for section 4
When groups are formed at random, the equilibrium
behavior of a
population in which UTFT and ALLD are present depends on whether
the inequality w “-lb > c is satisfied. If it is, then both 13= 0 and 1 are
stable equilibria, and there is a single unstable equilibrium. When it is
not satisfied, j? = 0 is stable, 3 = 1 is unstable and there are no internal
equilibria.
It follows from (5) that 0 and 1 are always equilibria. Let D(p) =
W( UTFT) - W( ALLD).
Then at any interior equilibrium
D = 0, and
UTFT will increase whenever D > 0. Substituting
(1) and (2) into (3)
and (4) and using the fact that when groups are formed at random
m( U( UTFT) = m( u 1A LLD) = p”( 1 - p) yields the following expression for D(p):
n-2
D(p)
=
-c
Performing
C
wn-‘b
$‘(I-~)++$
+/-’
the summation
and simplifying
u=o
1 It;,
D(P) = ___
( wP)"-l
+ c1 _ wJcl _ wpj
- c
l-w
[b(l -
.
(Al)
yields
wp) - (1 -P)wc].
642)
D(0) = - c, and 3 = 0 is always stable. D(1) > 0 as long as
n-1b > c, and thus 8 = 1 is stable if that condition
is satisfied. The
Thus
W
derivative of D(p)
with respect to p has at most one root in the
interval (0, 1). Thus, if both fixed equilibria are stable, D has a single
root in the interval (0, l), 3,. If only fi = 0 is stable (so that wn-‘b < c),
then the derivative of D evaluated at p = 1 is positive, and thus D has
no roots in the interval (0, 1).
When groups are formed assortatively
according to (6) and (7) and
the frequency of UTFT is very low, the expected fitness of ALLD
individuals is approximately
W, since virtually all ALLD individuals
will find themselves in groups consisting of n ALLD individuals. The
236
R. Boyd and P.J. Richerson
expected
/ Eoolution
fitness of UTFT individuals
UTFT will increase if W(UTFT)
summation in (A3) and simplifying
of indirect
reciprocity
is approximately
given by:
- W( ALLD) > 0. Performing
yields (8) in the text.
the
Results for section 5
When groups are formed
given by:
at random
n-3
W(DTFT)
W(ALLD)
G(p)
= W, +p
-p)
and W( AUD)
-
1-W
(A51
W(ALLD).
yields the following
G(p)=pwb-c+
41 -P>
(1
are
(1 - Wd)h - (1 - Wd+‘)C
= w, +pb.
= W(DTFT)
simplifying
c pd(l
d=O
W( DTFT)
_
w)
Performing
the sums
expression for G:
(wP)“-‘wJ
- 4.
in (A4)
and
@6)
G(0) = -c, thus I; = 0 is a stable equilibrium. G(1) = wb - c, and thus
j? = 1 is stable only if wb > c. Let the second term on the right-hand
side of (A6) be H(p). When wb < c, H is negative for all values of p,
and thus there are no internal equilibria. When wb > c, H is a positive
function with a single maximum
which is concave for arguments
greater than the argument that maximizes
N. Thus there is only a
single unstable equilibrium in the interval (0, 1).
When groups are formed assortatively
and DTFT is very rare,
for W( DTFT) W( ALLD)
is approximately
W,. The expression
W(ALLD)
given in the text can be found by substituting
Y for p in
(A4) and performing the summation, and then subtracting
W,.