Download MAB Paper2 EvolutionaryGameTheory

Evolutionary Game Theory:
An Analogical Aide for Understanding
By
Melissa Beretta
10.27.2014
Game Theory and Democracy
2
Introduction
John Maynard Smith did not enjoy his education. He felt hindered and dismayed with the
lack of science education available to him as a student. John Maynard Smith did not believe in
God. Early on, around the age of 15, he accepted a life of atheism due to his profound
understanding of Charles Darwin’s theory of evolution. John Maynard Smith did not grow up
believing he would modernize science through an interdisciplinary connection combining two
fascinating fields. Yet, he did (“John”).
In 1982, John Maynard Smith revolutionized the way the world understands and studies
evolution. With the publication of his book Evolution and the Theory of Games, Maynard Smith
challenged the scientific community to step beyond the Darwinian model and dig for deeper
comprehension of natural selection through mathematics (“John”). More specifically, he applied
the game theory, a concept then only applied to trivial games, economics, and business, to the
theory of evolution. He provided the framework for an analogy so vast and comprehensive that
many researchers and scholars have accepted it as a manner of dissecting the intricacies of
natural selection (“John”). Maynard Smith’s insights have spawned mountains of research
throughout the scientific community surrounding a topic that is now referred to as evolutionary
game theory.
Evolutionary Game Theory as an Analogy
Evolutionary game theory is defined, as its name suggests, as the manner in which we
apply classical game theory to examine the evolution of species and their behaviors. In the game
theoretical lens, “individual players make decisions, and the payoff to each player depends on the
decisions made by all” (Easley). When applied to evolution, it is crucial to remember that
individual organisms are usually not making conscious decisions about the strategies they
3
employ, though the actions of each organism will affect the expected payoff of those they
interact with. By applying game theory to evolution, researchers emphasize the importance of the
interactions between species and between organisms, not simply the appearance of certain
genes/strategies.
There are extremely important differences between evolutionary game theory and
classical game theory that I will point out now. First off, classical game theory is based on the
concept of players being able to choose their optimal strategy. In evolutionary game theory, this
choice is very often not present, and in our discussion of the connection, it is necessary to
understand that organisms are “genetically hard-wired” to play a certain strategy, or as we will
discuss soon, possess a certain gene (Easley). In other words, “in evolutionary game theory, we
don’t think of individuals engaged in complex deductive reasoning while they try to decide what
to do. Instead, strategies are selected over time by the inexorable process of evolution” (Phelps).
Secondly, in classical game theory, there are many markers of what constitutes a good payoff, or
what constitutes success. This may be a personal feeling of value, monetary success, getting
revenge, etc. However, “in evolutionary game theory, the payoffs are expressed in terms of
individual fitness, which is defined as the reproductive success of one’s offspring but is often
estimated by individual weight gain or reproductive state” (Riechert). These differences are vital
in understanding and applying evolutionary game theory.
Due to these distinctions, game theory is merely an analogy for evolution as a whole. The
lack of choice in determining strategies lead to this interesting manner of exploring why certain
behaviors and traits are passed on from generation to generation and chosen for by natural
selection, while other seemingly beneficial traits may fail to increase fitness. The analogy begins
with the important parallel between strategies and genes. In the classical model, each game
4
requires players to pick a strategy that they think will allow them the greatest payoff. In the
evolutionary model, organisms and species are born with certain genes (big beak vs. small beak,
tall vs. short, aggressive vs. not aggressive), and these genes act as strategies that they play
against each other naturally (Easley). Mutations of any kind or the entrance of invasive species
will lead to the presence of new and different strategies (Phelps, Easley).
When two species or two organisms with different strategies interact, often in efforts to
procure certain resources or territory (Phelps, Riechert, Turner), the analogy causes us to look at
their contact as a type of game. Both are playing their strategies, or showing their genes and
behaviors, and the outcome of this interaction leads to either a winner and a loser or a tie. The
importance of this interaction in determining which strategies will win, or which genes will
survive in a population, prompts researchers to look at payoff matrices and calculate the
expected payoff for each gene in the scheme of all possible interactions (Easley). This will soon
be discussed in further detail.
But, if evolution is a game, what constitutes winning? As previously mentioned, “the
fitness of an individual is a measure of how likely that individual is to be able to reproduce and
pass on their genes” (Phelps). Therefore, the payoff correlates, in this analogy, to the fitness of
the organism, and it’s “overall fitness will be equal to the average fitness it experiences from
each of its many pairwise interactions with others, and this overall fitness determines its
reproductive success,” which we could describe as “winning the game” (Easley). Having more
offspring with the same genes that survive themselves to reproduce and make the population
carrying that “strategy” bigger is, in the game theoretical analogy, the way to win the game.
In this paper, I will look deeper into evolutionary game theory and the ways in which it
the classical model applies to and deviates from Darwin’s theory of natural selection and
5
evolution. In the next paragraph, I will look at payoff matrices as a manner of calculating fitness
and organizing the outcomes of various interactions between organisms. Next, I will define what
it means to be an evolutionary stable strategy, and discuss what occurs when a stable strategy
does not exist. I then plan to delve into various specific examples that offer insight into the
application of evolutionary game theory. Through a discussion of prisoner’s dilemma, I will
mention how scientists have seen connections to beetles, viruses, trees, and plant root systems
and how these parallels drawn allow a bigger picture understanding of the game theoretical
analogy. Finally, with a conclusion focused on the areas in which this analogy does not follow
through, I will explain how Nash equilibrium and the Hawk and Dove model are only somewhat
applicable to evolution in its purest form. Now, I begin with developing a deeper understanding
of the payoff matrix and how we will use this to understand evolutionary interactions.
Determining Stable Strategies
In classical game theory, payoff matrices are often used to determine the outcome of an
election or a game. Game theorists will use simplified versions of more complex games in order
to understand which decisions and strategies are best, and payoff matrices can help in their
evaluation. As we will discuss later on, with situations involving prisoner’s dilemma, payoff
matrices create a superior visual schema for understanding and quantifying dominant strategies.
In evolutionary game theory, payoff matrices are valuable as they offer a more concrete
manner of describing the benefits and costs of having certain genes. The payoff, as noted earlier,
is the fitness of the organism; this, however, causes problems for evolutionary game theorists, as
many times the fitness gained by the species is not quantifiable or measurable in a way that
really indicates the interaction’s results. Nevertheless, payoff matrices have offered a way of
evaluating behaviors and genes in terms of how they will fare in evolution, predicting which will
6
prosper and which will die out (Easley). In other words, they “[weigh] the costs and benefits in
terms of fitness associated with different strategies and [predict] the evolutionary fate of the
different types” (Turner).
These tools are most useful in determining the presence of evolutionary stable strategies
(Easley), a topic we will discuss next. First, however, we must establish two important
assumptions we will make in our initial discussion, as many evolutionary game theorists make as
well. We begin by working with “two-player two-strategy games” that are symmetric (Easley).
This simplicity and symmetry allow game theorists to use payoff matrices and classical game
theoretic models in order to predict the outcomes of more complex biological and environmental
happenings. The two players of our games will be two different organisms, and they will have
one of two different strategies, or possess one of two different genes.
Game theory and payoff matrices specifically are used to identify what are called
evolutionarily stable strategies. According to David Easley and Jon Kleinberg of Cambridge
University, “we say that a given strategy is evolutionarily stable if, when the whole population is
using this strategy, any small group of invaders using a different strategy will eventually die off
over multiple generations” (Easley). Therefore, like in classical game theory, the objective of
evolutionary game theory is to seek out and predict which strategies, or genes and behaviors, will
survive above all others. These strategies “can’t be invaded by any other strategy” (Phelps).
There are times, however, when a population cannot develop one stable strategy, and it is
necessary for the organisms to reach a balance in which both genes, or behaviors, are present.
This, as thoughtfully detailed in Steve Phelps and Michael Wooldridge’s piece on the Hawk and
Dove model, leads to the need for an evolutionarily mixed strategy, a topic that we will cover
7
later on (Phelps). For now, we will look at a variety of examples in which evolutionarily stable
strategies triumph over other competitors.
Before we begin, I remind you that the game theoretic analogy for exploring evolution is
just that: an analogy. Remember, “evolutionary stability…supposes no intelligence or
coordination on the part of the players” (Easley). It will be assumed that no organism we deem as
showing an evolutionarily stable strategy or gene is choosing consciously to show that gene.
They are genetically set to play the moves they are hard-wired to play.
Application to Evolutionary Games
Easley and Kleinberg use a “two-player, two-strategy” competition for resources in their
book in order to demonstrate evolutionary game theory in action. In this game, two types of
beetles are set to face each other in an environment where their interactions will determine who
gets food and is, therefore, able to survive and reproduce. Some beetles are small, and others are
large, and these two body sizes represent the strategies that each organism will play. The goal of
examining this interaction is to determine the payoff of having either of these body sizes and
whether one will be deemed evolutionarily stable by triumphing over the other over time enough
to wipe it out of the population (Easley).
The game is played when two beetles interact. The following describes the possible
payoff to each beetle:
“…when beetles of the same size compete they get equal shares of the food…when a
large beetle competes with a small beetle, the alrge beetle gets the majority of the
food…large beetles experience less of a fitness benefit from a given quantity of food,
since some of it is diverted into maintaining their expensive metabolism…” (Easley).
Therefore, the payoff matrix looks like this:
8
Researchers use this matrix to determine which of the two strategies, if any, are
evolutionarily stable. To do this, they utilize the following generalized matrix, again taken from
Easley and Kleinberg’s book:
Through the use of various inequalities, scientists determine, “In a two-player, two-strategy
symmetric game, S is evolutionarily stable precisely when either (i) a > c, or (ii) a = c and b > d”
(Easley). Therefore, when looking back at the matrix for the beetle example, the small trait is
actually not evolutionarily stable, while, it turns out, the large trait is stable. In the Body-Size
Game, “the population of large beetles resists the invasion of small beetles, and so Large is
evolutionarily stable” (Easley).
To Body-Size Game with the beetles fully, it is important to notice that while exhibiting
the large body gene is advantageous and obviously the dominant strategy by game theoretic
definition, when both beetles are large, their expected payoff is less than if both are small.
Therefore, a population of all small beetles will provide a larger payoff than a population of all
large beetles. Here “evolution by natural selection is causing the fitness of the organisms to
decrease over time” (Easley). The matrix structure provides an undeniable and important link to
9
Prisoner’s Dilemma, a concept that is often seen both in real-life scenarios and in natural
selection.
Prisoner’s dilemma is a method of decision making through which players act in their
own best interest and end up worse off than if they had collaborated. More specifically, “the
typical prisoner's dilemma is set up in such a way that both parties choose to protect themselves
at the expense of the other participant. As a result… both participants find themselves in a worse
state than if they had cooperated with each other in the decision-making process” (“Prisoner’s”).
This paradox appears constantly in everyday life. Common applications include prisoners in
interrogation and students cheating on tests. In both scenarios, the dominant strategy for each
player is to tattle on their cohort, yet when they both play this strategy, they end up worse off
than if they had cooperated toward a mutually average goal (“Prisoner’s”).
When we look at the payoff matrix for the Body-Size Game, the prisoner’s dilemma is
obvious. If beetles were able to maintain a single gene population where all organisms had the
small body size, their fitness overall would be greater than in an all-large population. However,
as is always important to remember in evolutionary game theory, beetles cannot choose whether
to be small or large (Easley). They cannot look at this payoff matrix and determine that they
would be better off in the long run if the species of beetles collaborated. Instead, once a small
large-body population infiltrates the small-body population, through mutation or physical
territory invasion, natural selection will pick the large gene over time and wipe out the small
gene. Therefore, “starting from a population of small beetles, evolution by natural selection is
causing the fitness of the organisms to decrease over time” (Easley). This goes against some of
Darwin’s theory of evolution as it contradicts the hope that over time the population’s fitness
will increase due to the selection of adaptive traits. Through the game theoretical lens,
10
interactions between animals, when experiencing this prisoner’s dilemma type situation, can
appear less advantageous for the species than previously believed.
The application of prisoner’s dilemma to evolution does not stop with beetles.
Researchers have found that certain viruses, which are not commonly considered to be living
organisms, show a sort of prisoner’s dilemma and game theory when interacting. In one specific
case, researcher Paul E. Turner studied the interaction between cooperators and cheaters in a
population of viruses. Viruses infect host cells, like those in humans or other living beings, and
“when more than one virus infects a cell, the metabolic products are freely accessible to any of
the co-infecting viruses” (Turner). If both types of viruses provide beneficial resources, they both
benefit. However, some viruses act more as “parasites on parasites,” and they steal proteins and
capsids to help their own survival without sharing any in return (Turner). Turner’s study involves
Phage Φ6, the cooperator, and a slightly altered version known as Phage ΦH2. According to
Easley’s description of the research, “ΦH2 is able to take advantage of chemical products
produced by Φ6, which gives ΦH2 a fitness advantage when it is in the presence of Φ6”
(Easley).
This dynamic leads to a prisoner’s dilemma. Only ΦH2 can be evolutionarily stable, as it
dominates in a population of Φ6; however, a population of all Φ6 phages is better off fitnesswise than a population of all ΦH2 (Easley). When describing prisoner’s dilemma as a whole,
Turner says, “It explains how a potential, but uncertain, reward can drive individuals to behave
in a way that is collectively irrational. When both prisoners follow their individual interests, both
lose” (Turner). However, these viruses are not driven to behave in a cheater or cooperator mode.
They are genetically disposed to be either Φ6 or ΦH2. In the end, natural selection chooses the
ΦH2 “strategy” as evolutionarily stable. Therefore, “cheaters can successfully displace
11
cooperators, while simultaneously lowering the average fitness of the population” (Turner).
This conclusion is somewhat contrary to Darwin’s own statements about evolution. He
says that natural selection “steers the population to become better adapted to its environment
over time” (Turner), but apparently, this is not always the case—especially in cooperator/cheater
situations. In the case of Turner’s phages, when the ΦH2 population was uninhabited by Φ6
phages, the ΦH2 phages died off quickly because they had no way of developing the resources
necessary for survival. They were so unfit on their own that they could not survive (Turner). This
problem will lead to a discussion of evolutionarily mixed strategies, which will be discussed later
in the paper.
Other applications of the prisoner’s dilemma have been found in trees and other types of
plants. For trees, height is a factor that plays into survival due to the importance of sunlight for
growth and reproduction. Though the gene varies and all trees are different heights, for the
purpose of game theoretical connections, the two-strategy game is played between short and tall
trees. The game is described like this: “If two neighboring trees both grow short, then they share
the sunlight equally. They also share the sunlight equally if they both grow tall, but in this case
their payoffs are each lower because they have to invest a lot of resources in achieving the
additional height” (Easley). In the end, being tall is the evolutionarily stable strategy for trees
because a small tree population cannot invade one of tall trees. However, all trees would be more
fit if they all had the short gene versus the tall gene. Again, this is a case where prisoner’s
dilemma connects perfectly, allowing us to understand why natural selection chooses the
dominant strategy when that leads to a lesser outcome.
Studies of plant root systems show similar interactions between two different strategies:
Conserve and Explore. When two plants are rooted in the same soil, they interact. They can
12
either conserve, by taking up only their own share of the soil and gaining half of the resources, or
they can explore, venture over into the soil of the other plant and attempt to gain more resources
(Easley). Exploring takes more energy from the plant, yet it is the evolutionarily stable strategy,
because a plant that conserves will lose resources to any plant that explores. Therefore,
prisoner’s dilemma is evident in that “all plants are better off in a population where everyone
plays Conserve, but only Explore is evolutionarily stable” (Easley). Natural selection chooses
explorers over conservers, but in the end, the population is less fit than if they had all conserved.
Overall, as we’ve seen, prisoner’s dilemma, a trademark of classical game theory, is
widely applicable to evolution and natural selection. Though players are not making conscious
choices about which gene they get to express, as a result of natural selection, “both participants
find themselves in a worse state than if they had cooperated with each other in the decisionmaking process” (“Prisoner’s”). Unfortunately, organisms and viruses do not get the opportunity
to choose whether to cooperate with one another. They cannot collude. They simply let evolution
take its course and their natural interactions decide how they will fare in the long run.
Other topics is Evolutionary Game Theory
Another topic of classical game theory worth mentioning here is Nash equilibrium. In
situations where there is Nash equilibrium, “the optimal outcome of a game is one where no
player has an incentive to deviate from his or her chosen strategy after considering an opponent's
choice” (“Nash”) This means that the strategy chosen by each player is his or her best choice no
matter what the other person plays. Easley points out that in evolutionary game theory, “it is
possible to have a game where (S, S) is a Nash equilibrium, but S is not evolutionarily stable”
(Easley). This lack of one evolutionarily stable strategy leads to a modification of the game
theory typically used to observe natural selection. Here, we assume that there can be a mixed
13
strategy, or a balance between two genes that allows the genes or behaviors to coexist at an
optimal level, in which the combination of the two is actually the equilibrium point.
When stable strategies are not available for a population of organisms, natural selection
may find a mixed strategy to select for that allows organisms to successfully maintain a that gene
or behavior over any others that invade. There are a select number of situations, such as, most
notably, the Hawk-Dove model, in which “the individuals or the population as a whole must
display a mixture of the two behaviors in order to have any chance of being stable against
invasion by other forms of behavior” (Easley).
In the Hawk-Dove game, which Steve Phelps and Michael Woodbridge describe in their
paper Game Theory and Evolution, there are two strategies/behaviors available. One can be a
Hawk, aggressive and confrontational, or a Dove, passive and non-confrontational. These are
“innate predispositions” (Phelps), and therefore, not chosen consciously by the animal. Again,
winning the game means passing on genes to the next generation through reproduction, and to do
this, they must “[obtain] a particular resource from the environment” (Phelps). When Hawks (the
aggressors) and Doves (the pacifists) interact, Hawks most to all of the resources. When two
organisms of the same disposition meet, they will split the resources; however, two aggressors
meeting does not yield as much food per animal as two pacifists meeting, as their payoff is offset
by a higher energy expenditure due to the more confrontational interaction (Phelps).
However, consider that the organisms are now able to discern and choose between the
aggressive behavior and the passive behavior. While in non-thinking organisms, like bacteria or
fungi or lower-level life forms, this is not plausible, in higher-order animals and animals that are
able to manipulate their own behavior, this is possible. If these organisms are able to choose,
they can mix these two strategies in an optimal manner. According to Easley, “the kind of mixed
14
equilibrium that we see…in the Hawk-Dove Game is typical of biological situations in which
organisms must break the symmetry between two distinct behaviors, when consistently adopting
just one of these behaviors is evolutionarily unstable” (Easley). The problem with this model is
that it assumes this level of rationality and intelligence across individuals and populations that
may not be rational or intelligent. Therefore, many attempt to steer away from the application of
the Hawk-Dove model, or at least the application of mixed strategies to this game, in order to
maintain the analogical component of evolutionary game theory.
Conclusion
Some people remain skeptical of the way in which game theory is applied to evolution.
Many researchers walk a thin line between assuming thought and purposeful action on behalf of
the viruses, plants, or animals that they study, even though the assumption in evolutionary game
theory is that these organisms are predisposed to play certain strategies through genetics. Some
scientists are uncomfortable using the analogy of game theory as it places too much control in
the hands of individuals or populations that we have no basis to believe can assert it.
Nevertheless, I believe, as do many researchers, that the application of classical game
theory to evolution is a useful manner of understanding how natural selection determines which
genes will survive in a population. This mathematical method of prediction compares genes and
behaviors with their expected payoffs in a manner that is easy to comprehend and apply to a
variety of situations.
In the future, evolutionary game theory could be used to determine which human traits
and behaviors are more likely to survive on through evolution. The Cooperator vs. Cheater lens
is already in use as researchers attempt to determine whether we could end up with a population
15
of selfish or of cooperating individuals (Phelps). The game theoretic analogy can help predict
this unknowable future.
16
Summary
In this paper, I explore evolutionary game theory, a concept that applies classical game
theoretical models to Darwin’s theories of evolution and natural selection. Proposed by John
Maynard Smith in the 1980s, this method of understanding biological and environmental
interactions has benefitted many researchers and scientists in exploring why genes and behaviors
persist and survive throughout generations. While there are important distinctions between the
two theories, many find it an interesting and helpful in exploring natural evolutionary processes.
The essay begins with a brief discussion of evolutionary game theory and how it relates
to classical game theory in the analogical sense. In classical game theory, games are played
amongst players, each of which playing their own strategies. In evolutionary game theory, these
players are organisms, big or small, and their strategies are genes or behaviors they express. A
crucial difference between the two comes from the lack of rational thought assumed on the
players of the evolutionary game of survival and reproduction. Organisms are genetically
predisposed toward certain traits that affect their interactions with others, and game theoretical
models are used to understand these interactions and how they’re affected by natural selection.
Next, the discussion moves to evaluating payoff matrices as a manner of determining
which strategies are evolutionarily stable. We define stable strategies as those that, when invaded
by other pure strategies, will not be overtaken or outlived—they will successfully rid their
population of the invasive gene or behavior and return to normal over the course of generations.
Identifying stable strategies allows researchers to make predictions about which genes will
survive in a population based on the payoff from playing each strategy.
To relate these theories to a specific case, I look at the Large vs. Small Beetle game, and
evaluate how each strategy leads to a certain playoff and a different likelihood of survival. This
leads me toward the discussion of prisoner’s dilemma, a classical game theoretical model
demonstrating how playing dominant strategy can lead to lower fitness over time. Sometimes,
though Darwin predicted that fitness of a population of organisms would increase over time, a
trait is selected that causes organisms to be less fit or less able to survive and reproduce. Other
examples mentioned in this piece that apply the prisoner’s dilemma to evolution are height of
trees, plant root systems, and cheating viruses. These each demonstrate evolutionary cases in
which traits were evolutionarily stable without increasing the overall fitness of the population.
Briefly, we discuss the cases when stable strategies do not occur. In the Hawk-Dove
model, proposals suggest that when organisms are able to combine behaviors, such as being
aggressive or passive, they may find a mixed strategy that is more stable than a pure one.
However, this often assumes rational thought on behalf of non-thinking organisms, which is a
dangerous line to cross.
Overall, this paper works simply to explore the concept of evolutionary game theory and
the ways in which it does and does not work to explain natural selection. There are a variety of
ways in which researchers have attempted to use game theoretical models to help the world
understand evolution, and there are still many possibilities for further application.
17
Works Cited
Easley, David, and Jon Kleinberg. "Chapter 7: Evolutionary Game Theory." Networks, Crowds,
and Markets: Reasoning about a Highly Connected World. New York: Cambridge UP,
2010. 209-27. Print.
"John Maynard Smith." New World Encyclopedia. N.p., n.d. Web. 22 Oct. 2014.
<https://www.newworldencyclopedia.org/entry/John_Maynard_Smith>.
"Nash Equilibrium." Investopedia. N.p., n.d. Web. 21 Oct. 2014.
<http://www.investopedia.com/terms/n/nash-equilibrium.asp>.
Phelps, Steve, and Michael Wooldridge. "Game Theory and Evolution." IEEE Intelligent
Systems 28.4 (2013): 76-81. Web. 22 Oct. 2014.
"Prisoner's Dilemma." Investopedia. N.p., n.d. Web. 20 Oct. 2014.
<http://www.investopedia.com/terms/p/prisoners-dilemma.asp>.
Riechert, Susan E. "Spider Fights as a Test of Evolutionary Game Theory." American Scientist
74.6 (1986): 604-10. JSTOR. Web. 22 Oct. 2014.
Turner, Paul E. "Cheating Viruses and Game Theory." American Scientist 93.5 (2005): 428-35.
JSTOR. Web. 22 Oct. 2014.