Download BIO INSPIRED COMPUTING In

BIO INSPIRED COMPUTING
SEMINAR REPORT
Submitted by
ULLAS MOHAN
In partial fulfillment of the award of the degree
Of
BACHELOR OF TECHNOLOGY
In
COMPUTER SCIENCE AND ENGINEERING
SCHOOL OF ENGINEERING
COCHIN UNIVERSITY OF SCIENCE & TECHNOLOGY
COCHIN-682 022
JULY 2008
DIVISION OF COMPUTER ENGINEERING
SCHOOL OF ENGINEERING
COCHIN UNIVERSITY OF SCIENCE & TECHNOLOGY,
COCHIN-682 022
Certificate
Certified that this is a bonafide record of the Seminar Entitled
“BIO-INSPIRED COMPUTING”
Done by the following Student
Ullas Mohan
Of the VIIth semester,Computer Science and Engineering in the year 2008 in
partial fulfillment of the requirements to the award of Degree Of Bachelor Of
Technology in Computer Science and Engineering of Cochin University of Science
and Technology
Mr Pramod Pavithran
Dr David Peter.S
Seminar Guide
Head of Division
Date
ACKNOWLEDGEMENT
At the outset, I thank the Lord Almighty for the grace, strength
and hope to make my endeavor a success.
I also express my gratitude to Dr. David Peter S, Head of the
Division and my Seminar Guide for providing me with adequate facilities,
ways and means by which I was able to complete this seminar. I express my
sincere gratitude to him for his constant support and valuable suggestions
without which the successful completion of this seminar would not have
been possible.
I thank Mr. Pramod Pavithran , my seminar guide for his
boundless cooperation and helps extended for this seminar. I express my
immense pleasure and thankfulness to all the teachers and staff of the
Division of Computer Engineering, CUSAT for their cooperation and
support.
Last but not the least, I thank all others, and especially my
classmates and my family members who in one way or another helped me in
the successful completion of this work
ULLAS MOHAN
ABSTRACT
Biologically
inspired
(often
hyphenated
as
biologically-inspired)
computing (also bio-inspired computing) is a field of study that loosely
knits together subfields related to the topics of connectionism, social
behaviour and emergence. It is often closely related to the field of artificial
intelligence, as many of its pursuits can be linked to machine learning. It
relies heavily on the fields of biology, computer science and mathematics.
Briefly put, it is the use of computers to model nature, and simultaneously
the study of nature to improve the usage of computers. Biologically inspired
computing is a major subset of natural computation.
Some areas of study encompassed under the canon of biologically inspired
computing, and their biological counterparts:

genetic algorithms ↔ evolution

emergent systems ↔ ants, termites, bees, wasps

artificial immune systems ↔ immune system
The way in which bio-inspired computing differs from traditional artificial
intelligence (AI) is in how it takes a more evolutionary approach to learning,
as opposed to the what could be described as 'creationist' methods used in
traditional AI. In traditional AI, intelligence is often programmed from
above: the programmer is the creator, and makes something and imbues it
with its intelligence. Bio-inspired computing, on the other hand, takes a
more bottom-up, decentralised approach; bio-inspired techniques often
involve the method of specifying a set of simple rules, a set of simple
organisms which adhere to those rules, and a method of iteratively applying
those rules. After several generations of rule application it is usually the case
that some forms of complex behaviour arise. Complexity gets built upon
complexity until the end result is something markedly complex, and quite
often completely counterintuitive from what the original rules would be
expected to produce.
TABLE OF CONTENTS
CHAPTER NO
TITLE
LIST OF FIGURES
PAGE NO:
iii
1.
INTRODUCTION
1
2.
EVOLUTION ALGORITHMS
2
2.1 Components of Algorithm
5
3.
2.1.1 Representation
6
2.1.2 Evaluation Function
7
2.1.3 Population
8
2.1.4 Parent Selection Mechanism
8
2.1.5 Mutation
9
2.1.6 Recombination
10
2.1.7 Replacement
11
2.1.8 Initialization
12
2.1.9 Termination
12
SWARM INTELLIGENCE
14
3.1 Introduction
14
3.2 Techniques
15
3.2.1 Ant Colony Optimization
15
3.2.2 Particle Swarm Optimization
15
3.2.3 Stochastic Diffusion Search
16
3.3 Applications
16
3.4 Ant Colony Optimization
17
3.5 Particle Swarm Optimization
19
4.
ARTIFICIAL IMMUNE SYSTEM
4.1 Immune Algorithms
23
24
5.
CONCLUSION
28
6.
REFERENCES
29
LIST OF FIGURES
NO:
NAME
PAGE
1.
Examples Of Swarm Intelligence
14
2.
Example for ant colony optimization
18
3.
Particle Swarm Optimization
22
-iii-
Bio Inspired Computing
1.INTRODUCTION
Biologically inspired computing (also bio-inspired computing) is a field of study that
loosely knits together subfields related to the topics of connectionism, social behavior
and emergence. It is often closely related to the field of artificial intelligence, as many of
its pursuits can be linked to machine learning. It relies heavily on the fields of biology,
computer science and mathematics. Biologically inspired computing is a major subset of
natural computation.
The field of biocomputation has a twofold definition: the use of
biology or biological processes as metaphor, inspiration, or enabler in developing new
computing technologies and new areas of computer science; and conversely, the use of
information science concepts and tools to explore biology from a different theoretical
perspective. In addition to its potential applications, such as DNA computation,
nanofabrication, storage devices, sensing, and health care, biocomputation also has
implications for basic scientific research. It can provide biologists, for example, with an
IT-oriented paradigm for looking at how cells compute or process information, or help
computer scientists construct algorithms based on natural systems, such as evolutionary
and genetic algorithms. Biocomputing has the potential to be a very powerful tool.
Division Of computer science, SOE, CUSAT
1
Bio Inspired Computing
2. EVOLUTION ALGORITHMS
As the history of the field suggests there are many different variants of
Evolutionary Algorithms. The common underlying idea behind all these techniques is the
same given a population of individuals the environmental pressure causes natural
selection (survival of the fittest) and this causes a rise in the fitness of the population.
Given a quality function to be maximised we can randomly create a set of candidate
solutions, i.e., elements of the function’s domain, and apply the quality function as an
abstract fitness measure the higher the better. Based on this fitness, some of the better
candidates are chosen to seed the next generation by applying recombination and/or
mutation to them. Recombination is an operator applied to two or more selected
candidates (the so-called parents) and results one or more new candidates (the
children).Mutation is applied to one candidate and results in one new candidate.
Executing recombination and mutation leads to a set of new candidates (the offspring)
that compete – based on their fitness (and possibly age)– with the old ones for a place in
the next generation. This process can be iterated until a candidate with sufficient quality
(a solution) is found or a previously set computational limit is reached.
In artificial intelligence, an evolutionary algorithm (EA) is a subset of
evolutionary computation, a generic population-based metaheuristic optimization
algorithm. An EA uses some mechanisms inspired by biological evolution: reproduction,
mutation, recombination, and selection. Candidate solutions to the optimization problem
play the role of individuals in a population, and the cost function determines the
environment within which the solutions "live". Evolution of the population then takes
place after the repeated application of the above operators. Artificial evolution (AE)
describes a process involving individual evolutionary algorithms; EAs are individual
components that participate in an AE.
Evolutionary algorithms consistently perform well approximating solutions to all
types of problems because they do not make any assumption about the underlying fitness
landscape.
Division Of computer science, SOE, CUSAT
2
Bio Inspired Computing
Apart from their use as mathematical optimizers, evolutionary computation and
algorithms have also been used as an experimental framework within which to validate
theories about biological evolution and natural selection, particularly through work in the
field of artificial life. Techniques from evolutionary algorithms applied to the modelling
of biological evolution are generally limited to explorations of microevolutionary
processes, however some computer simulations, such as Tierra and Avida, attempt to
model macroevolutionary dynamics.
A limitation of evolutionary algorithms is their lack of a clear genotype-phenotype
distinction. In nature, the fertilized egg cell undergoes a complex process known as
embryogenesis to become a mature phenotype. This indirect encoding is believed to
make the genetic search more robust (i.e. reduce the probability of fatal mutations), and
also may improve the evolvability of the organism.
Genetic algorithm - This is the most popular type of EA. One seeks the solution of a
problem in the form of strings of numbers (traditionally binary, although the best
representations are usually those that reflect something about the problem being solved these are not normally binary), virtually always applying recombination operators in
addition to selection and mutation. This type of EA is often used in optimization
problems.
Genetic programming - Here the solutions are in the form of computer programs, and
their fitness is determined by their ability to solve a computational problem.
Evolutionary programming - Like genetic programming, only the structure of the
program is fixed and its numerical parameters are allowed to evolve;
Evolution strategy - Works with vectors of real numbers as representations of solutions,
and typically uses self-adaptive mutation rates;
Basically,there are two fundamental forces that form the basis of evolutionary
systems:
• Variation operators (recombination and mutation) create the necessary diversity and
thereby facilitate novelty, while
• selection acts as a force pushing quality.
Division Of computer science, SOE, CUSAT
3
Bio Inspired Computing
The combined application of variation and selection generally leads to improving fitness
values in consecutive populations. It is easy (although somewhat misleading) to see such
a process as if the evolution is optimising, or atleast “approximising”, by approaching
optimal values closer and closer over its course. Alternatively, evolution it is often seen
as a process of adaptation. From this perspective, the fitness is not seen as an objective
function to be optimised, but as an expression of environmental requirements. Matching
these requirements more closely implies an increased viability, reflected in a higher
number of offspring. The evolutionary process makes the population adapt to the
environment better and better.
Many components of such an evolutionary process are stochastic. During selection
fitter individuals have a higher chance to be selected than less fit ones, but typically even
the weak individuals have a chance to become a parent or to survive.For recombination
of individuals the choice of which pieces will be recombined is random. Similarly for
mutation, the pieces that will be mutated within a candidate solution, and the new pieces
replacing them, are chosen randomly. The general scheme of an Evolutionary Algorithm
can is given in a pseudo-code fashion;
BEGIN
INITIALISE population with random candidate solutions;
EVALUATE each candidate;
REPEAT UNTIL ( TERMINATION CONDITION is satisfied ) DO
1 SELECT parents;
2 RECOMBINE pairs of parents;
3 MUTATE the resulting offspring;
4 EVALUATE new candidates;
5 SELECT individuals for the next generation;
END
The representation of a candidate solution is often used to characterise different
streams. Typically, the candidates are represented by (i.e., the data structure encoding a
solution has the form of) strings over a finite alphabet in Genetic Algorithms (GA), real-
Division Of computer science, SOE, CUSAT
4
Bio Inspired Computing
valued vectors in Evolution Strategies (ES), finite state machines in classical
Evolutionary Programming (EP) and trees in Genetic Programming (GP).
Technically, a given representation might be preferable over others if it
matches the given problem better, that is, it makes the encoding of candidate solutions
easier or more natural. For instance, for solving a satisfiability problem the
straightforward choice is to use bit-strings of length n, where n is the number of logical
variables, hence the appropriate EA would be a Genetic Algorithm. For evolving a
computer program that can play checkers trees are well-suited (namely, the parse trees of
the syntactic expressions forming the programs), thus a GP approach is likely. It is
important to note that the recombination and mutation operators working on candidates
must match the given representation. Thus for instance in GP the recombination operator
works on trees, while in GAs it operates on strings. As opposed to variation
operators,selection takes only the fitness information into account, hence it works
independently from the actual representation. Differences in the commonly applied
selection mechanisms in each stream are therefore rather a tradition than a technical
necessity.
2.1 COMPONENTS OF ALGORITHM
EAs have a number of components, procedures or operators that must be specified in
order to define a particular EA. The most important components are:
•
representation (definition of individuals)
•
evaluation function (or fitness function)
•
population
•
parent selection mechanism
•
variation operators, recombination and mutation
•
survivor selection mechanism (replacement)
Each of these components must be specified in order to define a particular EA.
Furthermore, to obtain a running algorithm the initialisation procedure and a termination
condition must be also defined.
Division Of computer science, SOE, CUSAT
5
Bio Inspired Computing
2.1.1 Representation (Definition of Individuals)
The first step in defining an EA is to link the“real world” to the “EA world”, that is to set
up a bridge between the original problem context and the problem solving space where
evolution will take place. Objects forming possible solutions within the original problem
context are referred to as phenotypes, their encoding, the individuals within the EA, are
called genotypes. The first design step is commonly called representation, as it amounts
to specifying a mapping from the phenotypes onto a set of genotypes that are said to
represent these phenotypes. For instance, given an optimisation problem on integers, the
given set of integers would form the set of phenotypes. Then one could decide to
represent them by their binary code, hence 18 would be seen as a phenotype and 10010 as
a genotype representing it. It is important to understand that the phenotype space can be
very different from the genotype space, and that the whole evolutionary search takes
place in the genotype space. A solution – a good phenotype – is obtained by decoding the
best genotype after termination. To this end, it should hold that the (optimal) solution to
the problem at hand – a phenotype – is represented in the given genotype space. The
common EC terminology uses many synonyms for naming the elements of these two
spaces. On the side of the original problem context, candidate solution, phenotype, and
individual are used to denote points of the space of possible solutions. This space itself is
commonly called the phenotype space. On the side of the EA, genotype, chromosome,
and again individual can be used for points in the space where the evolutionary search
will actually take place. This space is often termed the genotype space. Also for the
elements of individuals there are many synonymous terms. A place-holder is commonly
called a variable, a locus (plural: loci), a position, or in a biology oriented terminology a
gene. An object on such a place can be called a value or an allele.
It should be noted that the word “representation” is used in two slightly
different ways. Sometimes it stands for the mapping from the phenotype to the genotype
space. In this sense it is synonymous with encoding, e.g., one could mention binary
Division Of computer science, SOE, CUSAT
6
Bio Inspired Computing
representation or binary encoding of candidate solutions. The inverse mapping from
genotypes to phenotypes is usually called decoding and it is required that the
representation be invertible: to each genotype there has to be at most one corresponding
phenotype. The word representation can also be used in a slightly different sense, where
the emphasis is not on the mapping itself, but on the “data structure” of the genotype
space. This interpretation is behind speaking about mutation operators for binary
representation, for instance
.
2.1.2 Evaluation Function (Fitness Function)
The role of the evaluation function is to represent the requirements to
adapt to. It forms the basis for selection, and thereby it facilitates improvements. More
accurately, it defines what improvement means. From the problem solving perspective, it
represents the task to solve in the evolutionary context. Technically, it is a function or
procedure that assigns a quality measure to genotypes. Typically, this function is
composed from a quality measure in the phenotype space and the inverse representation.
To remain with the above example, if we were to maximise x2 on integers, the fitness of
the genotype 10010 could be defined as the square of its corresponding phenotype: 182 =
324.The evaluation function is commonly called the fitness function in EC.This might
cause a counterintuitive terminology if the original problem requires minimisation for
fitness is usually associated with maximisation. Mathematically, however, it is trivial to
change minimisation into maximisation and vice versa.
Quite often, the original problem to be solved by an EA is an optimization
problem In this case the name objective function is often used in the original problem
context and the evaluation (fitness) function can be identical to, or a simple
transformation of, the given objective function.
Division Of computer science, SOE, CUSAT
7
Bio Inspired Computing
2.1.3 Population
The role of the population is to hold (the representation of) possible solutions. A
population is a multiset1 of genotypes. The population forms the unit of evolution.
Individuals are static objects not changing or adapting, it is the population that does.
Given a representation, defining a population can be as simple as specifying how many
individuals are in it, that is, setting the population size. In some sophisticated EAs a
population has an additional spatial structure, with a distance measure or a
neighbourhood relation. In such cases the additional structure has to be defined as well to
fully specify a population.
As opposed to variation operators that act on the one or two parent individuals, the
selection operators (parent selection and survivor selection) work at population level. In
general, they take the whole current population into account and choices are always made
relative to what we have. For instance, the best individual of the given population is
chosen to seed the next generation, or the worst individual of the given population is
chosen to be replaced by a new one. In almost all EA applications the population size is
constant, not changing during the evolutionary search.
The diversity of a population is a measure of the number of different solutions present.
No single measure for diversity exists, typically people might refer to the number of
different fitness values present, the number of different phenotypes present, or the
number of different genotypes. Other statistical measures, such as entropy, are also used.
Note that only one fitness value does not necessarily imply only one phenotype is present,
and in turn only one phenotype does not necessarily imply only one genotype. The
reverse is however not true: one genotype implies only one phenotype and fitness value.
2.1.4 Parent Selection Mechanism
The role of parent selection or mating selection is to distinguish among individuals based
on their quality, in particular, to allow the better individuals to become parents of the next
generation. An individual is a parent if it has been selected to undergo variation in order
Division Of computer science, SOE, CUSAT
8
Bio Inspired Computing
to create offspring. Together with the survivor selection mechanism, parent selection is
responsible for pushing quality improvements. In EC, parent selection is typically
probabilistic. Thus, high quality individuals get a higher chance to become parents than
those with low quality. Nevertheless, low quality individuals are often given a small,
but positive chance, otherwise the whole search could become too greedy and get stuck in
a local optimum.The role of variation operators is to create new individuals from old
ones. In the corresponding phenotype space this amounts to generating new candidate
solutions. From the generate-and-test search perspective, variation operators perform the
“generate” step. Variation operators in EC are divided into two types based on their
activity
2.1.5 Mutation
A unary3 variation operator is commonly called mutation. It is applied to one genotype
and delivers a (slightly) modified mutant, the child or offspring of it. A mutation operator
is always stochastic: its output – the child – depends on the outcomes of a series of
random choices. It should be noted that an arbitrary unary operator is not necessarily seen
as mutation. A problem specific heuristic operator acting on one individual could be
termed as mutation for being unary. However, in general mutation is supposed to cause a
random, unbiased change. For this reason it might be more appropriate not to call
heuristic unary operators mutation. The role of mutation in EC is different in various ECdialects, for instance in Genetic Programming it is often not used at all, in Genetic
Algorithms it has traditionally been seen as a background operator to fill the gene pool
with “fresh blood”, while in Evolutionary Pro- gramming it is the one and only variation
operator doing the whole search work.
It is worth noting that variation operators form the evolutionary implementation of the
elementary steps within the search space. Generating a child amounts to stepping to a
new point in this space. From this perspective, mutation has a theoretical role too: it can
guarantee that the space is connected. This is important since theorems stating that an EA
will (given sufficient time) discover the global optimum of a given problem often rely on
Division Of computer science, SOE, CUSAT
9
Bio Inspired Computing
the property that each genotype representing a possible solution can be reached by the
variation operators . The simplest way to satisfy this condition is to allow the mutation
operator to “jump” everywhere, for example, by allowing that any allele can be mutated
into any other allele with a non-zero probability. However it should also be noted that
many researchers feel these proofs

The arity of an operator is the number of objects that it takes as inputs

An operator is unary if it applies to one object as input.
Usually these will consist of using a pseudo-random number generator to generate
a series of values from some given probability distribution. We will refer to these
as “random drawings” have limited practical importance, and many implementations of
EAs do not in fact possess this property.
2.1.6 Recombination
A binary variation operator5 is called recombination or crossover. As the names indicate
such an operator merges information from two parent genotypes into one or two offspring
genotypes. Similarly to mutation, recombination is a stochastic operator: the choice of
what parts of each parent are combined, and the way these parts are combined, depend on
random drawings. Again, the role of recombination is different in EC dialects: in Genetic
Programming it is often the only variation operator, in Genetic Algorithms it is seen as the
main search operator, and in Evolutionary Programming it is never used. Recombination
operators with a higher arity (using more than two parents) are mathematically possible
and easy to implement, but have no biological equivalent. Perhaps this is why they are not
commonly used, although several studies indicate that they have positive effects on the
evolution.
The principal behind recombination is simple – that by mating two individuals with
different but desirable features, we can produce an offspring which combines both of those
features. This principal has a strong supporting case– it is one which has been successfully
applied for millennia by breeders ofplants and livestock, to produce species which give
Division Of computer science, SOE, CUSAT
10
Bio Inspired Computing
higher yields or have other desirable features. Evolutionary Algorithms create a number of
offspring by random recombination, accept that some will have undesirable combinations
of traits, most may be no better or worse than their parents, and hope that some have
improved characteristics. Although the biology of the planet earth, (where with a very few
exceptions lower organisms reproduce asexually, and higher organisms reproduce
sexually suggests that recombination is the superior form of reproduction, recombination
operators in EAsare usually applied probabilistically, that is, with an existing chance of
notbeing performed.It is important to note that variation operators are representation
dependent. That is, for different representations different variation operators have to
defined. For example, if genotypes are bit-strings, then inverting a 0 to a 1 (1 to a 0) can
be used as a mutation operator. However, if we represent possible solutions by tree-like
structures another mutation operator is required.
2.1.7 Survivor Selection Mechanism (Replacement)
The role of survivor selection or environmental selection is to distinguish among
individuals based on their quality. In that it is similar to parent selection, but it is used in
a different stage of the evolutionary cycle. The survivor selection mechanism is called
after having having created the offspring of the selected parents. In EC the population
size is (almost always) constant, thus a choice has to to be made on which individuals
will be allowed in the next generation. This decision is usually based on their fitness
values, favouring those with higher quality, although the concept of age is also frequently
used. As opposed to parent selection which is typically stochastic, survivor selection is
often deterministic, for instance ranking the unified multiset of parents and offspring and
selecting the top segment (fitness biased), or selecting only from the offspring (agebiased). Survivor selection is also often called replacement or replacement strategy. In
many cases the two terms can be used interchangeably. The choice between the two is
thus often arbitrary. A good reason to use the name survivor selection is to keep
terminology consistent. A preference for using replacement can be motivated by the
skewed proportion of the number of individuals in the population and the number of
Division Of computer science, SOE, CUSAT
11
Bio Inspired Computing
newly created children. In particular, if the number of children is very small with respect
to the population size, e.g., 2 children and a population of 100. In this case, the survivor
selection step is as simple as to chose the two old individuals that are to be deleted to
make place for the new ones. In other words, it is more efficient to declare that everybody
survives unless deleted, and to choose whom to replace. If the proportion is not skewed
like this, e.g., 500 children made from a population of 100, then this is not an option, so
using the term survivor selection is appropriate.
2.1.8 Initialisation
Initialisation is kept simple in most EA applications: The first population is seeded by
randomly generated individuals. In principle, problem specific heuristics can be used in
this step aiming at an initial population with higher fitness. Whether this is worth the
extra computational effort or lot is very much depending on the application at hand.
There are, however, some general observations concerning this issue based on the socalled anytime behaviour of EAs.
2.3.9 Termination Condition
As for a suitable termination condition we can distinguish two cases. If the problem has a
known optimal fitness level, probably coming from a known optimum of the given
objective function, then reaching this level (perhaps only with a given precision > 0)
should be used as stopping condition. However, EAs are stochastic and mostly there are
no guarantees to reach an optimum, hence this condition might never get satisfied and the
algorithm may never stop. This requires that this condition is extended with one that
certainly stops the algorithm. Commonly used options for this purpose are the following:
1. the maximally allowed CPU time elapses;
2. the total number of fitness evaluations reaches a given limit;
3. for a given period of time (i.e, for a number of generations or fitness
evaluations), the fitness improvement remains under a threshold value;
4. the population diversity drops under a given threshold.
Division Of computer science, SOE, CUSAT
12
Bio Inspired Computing
The actual termination criterion in such cases is a disjunction: optimum value hit
or condition x satisfied. If the problem does not have a known optimum, then we need no
disjunction, simply a condition from the above list or a similar one that is guaranteed to
stop the algorithm
Division Of computer science, SOE, CUSAT
13
Bio Inspired Computing
3.SWARM INTELLIGENCE
3.1 INTRODUCTION
Swarm intelligence (SI) is artificial intelligence based on the collective behavior of
decentralized, self-organized systems. The expression was introduced by Gerardo Beni
and Jing Wang in 1989, in the context of cellular robotic systems.
SI systems are typically made up of a population of simple agents interacting locally with
one another and with their environment. The agents follow very simple rules, and
although there is no centralized control structure dictating how individual agents should
behave, local interactions between such agents lead to the emergence of complex global
behavior. Natural examples of SI include ant colonies, bird flocking, animal herding,
bacterial growth, and fish schooling. The application of swarm principles to robots is
called swarm robotics, while 'swarm intelligence' refers to the more general set of
algorithms.
Figure 3.1 Examples of Swarm Intelligence
Swarm intelligence is the emergent collective intelligence of groups of simple
autonomous agents. Here, an autonomous agent is a subsystem that interacts with its
environment, which probably consists of other agents, but acts relatively independently
from all other agents. The autonomous agent does not follow commands from a leader, or
Division Of computer science, SOE, CUSAT
14
Bio Inspired Computing
some global plan . For example, for a bird to participate in a flock, it only adjusts its
movements to coordinate with the movements of its fock mates, typically its neighbors
that are close to it in the fock. A bird in a flock simply tries to stay close to its neighbors,
but avoid collisions with them. Each bird does not take commands from any leader bird
since there is no lead bird. Any bird can be in the front, center and back of the swarm.
Swarm behavior helps birds take advantage of several things including protection from
predators (especially for birds in the middle of the flock), and searching for food
(essentially each bird is exploiting the eyes of every other bird).
3.2 TECHNIQUES
Some of the techniques in swarm intelligence are illustrated below:
3.2.1 ANT COLONY OPTIMIZATION
Ant colony optimization is a class of optimization algorithms modeled on the actions of
an ant colony. Artificial 'ants' - simulation agents - locate optimal solutions by moving
through a parameter space representing all possible solutions. Real ants lay down
pheromones directing each other to resources while exploring their environment. The
simulated 'ants' similarly record their positions and the quality of their solutions, so that
in later simulation iterations more ants locate better solutions. One variation on this
approach is the bees algorithm, which is more analogous to the foraging patterns of the
honey bee.
3.2.2 PARTICLE SWARM OPTIMIZATION
Particle swarm optimization or PSO is a global optimization algorithm for dealing with
problems in which a best solution can be represented as a point or surface in an ndimensional space. Hypotheses are plotted in this space and seeded with an initial
velocity, as well as a communication channel between the particles[3][4]. Particles then
move through the solution space, and are evaluated according to some fitness criterion
after each timestep. Over time, particles are accelerated towards those particles within
their communication grouping which have better fitness values. The main advantage of
Division Of computer science, SOE, CUSAT
15
Bio Inspired Computing
such an approach over other global minimization strategies such as simulated annealing is
that the large number of members that make up the particle swarm make the technique
impressively resilient to the problem of local minima.
3.2.3 STOCHASTIC DIFFUSION SEARCH
Stochastic Diffusion Search or SDS is an agent based on probabilistic global
search and optimization technique best suited to problems where the objective function
can be decomposed into multiple independent partial-functions. Each agent maintains a
hypothesis which is iteratively tested by evaluating a randomly selected partial objective
function parameterised by the agent's current hypothesis. In the standard version of SDS
such partial function evaluations are binary resulting in each agent becoming active or
inactive. Information on hypotheses is diffused across the population via inter-agent
communication. Unlike the stigmergic communication used in ACO, in SDS agents
communicate hypotheses via a one-to-one communication strategy analogous to the
tandem running procedure observed in some species of ant. A positive feedback
mechanism ensures that, over time, a population of agents stabilise around the global-best
solution. SDS is both an efficient and robust search and optimisation algorithm, which
has been extensively mathematically described.
3.3 APPLICATIONS
Swarm Intelligence-based techniques can be used in a number of
applications. The U.S. military is investigating swarm techniques for controlling
unmanned vehicles. The European Space Agency is thinking about an orbital swarm for
self assembly and interferometry. NASA is investigating the use of swarm technology for
planetary mapping. A 1992 paper by M. Anthony Lewis and George A. Bekey[5]
discusses the possibility of using swarm intelligence to control nanobots within the body
for the purpose of killing cancer tumors. Artists are using swarm technology as a means
of creating complex interactive systems or simulating crowds. Tim Burton's Batman
Returns was the first movie to make use of swarm technology for rendering, realistically
depicting the movements of a group of penguins using the Boids system. The Lord of the
Division Of computer science, SOE, CUSAT
16
Bio Inspired Computing
Rings film trilogy made use of similar technology, known as Massive, during battle
scenes. Swarm technology is particularly attractive because it is cheap, robust, and
simple.
The inherent intelligence of swarms has inspired many social and political philosophers,
in that the collective movements of an aggregate often derive from independent decision
making on the part of a single individual. A common example is how the unaided
decision of a person in a crowd to start clapping will often encourage others to follow
suit, culminating in widespread applause. Such knowledge, an individualist advocate
might argue, should encourage individual decision making (however mundane) as an
effective tool in bringing about widespread social change.
The use of Swarm Intelligence in Telecommunication Networks has also been
researched, in the form of Ant Based Routing. This was pioneered separately by Dorigo
et al and Hewlett Packard in the mid-1990s, with a number of variations since. Basically
this uses a probabilistic routing table rewarding/reinforcing the route successfully
traversed by each "ant" (a small control packet) which flood the network. Reinforcement
of the route in the forwards, reverse direction and both simultaneously have been
researched: backwards reinforcement requires a symmetric network and couples the two
directions together; forwards reinforcement rewards a route before the outcome is known
(but then you pay for the cinema before you know how good the film is). As the system
behaves stochastically and is therefore lacking repeatability, there are large hurdles to
commercial deployment.
3.4 ANT COLONY OPTIMIZATION
The ant colony optimization algorithm (ACO), introduced by Marco
Dorigo in 1992 in his PhD thesis, is a probabilistic technique for solving computational
problems which can be reduced to finding good paths through graphs. They are inspired
by the behavior of ants in finding paths from the colony to food.In the real world, ants
(initially) wander randomly, and upon finding food return to their colony while laying
down pheromone trails. If other ants find such a path, they are likely not to keep traveling
Division Of computer science, SOE, CUSAT
17
Bio Inspired Computing
at random, but to instead follow the trail, returning and reinforcing it if they eventually
find food.
Over time, however, the pheromone trail starts to evaporate, thus reducing its
attractive strength. The more time it takes for an ant to travel down the path and back
again, more time the pheromones have to evaporate. A short path, by comparison, gets
marched over faster, and thus the pheromone density remains high as it is laid on the path
as fast as it can evaporate. Pheromone evaporation has also the advantage of avoiding the
convergence to a locally optimal solution. If there were no evaporation at all, the paths
chosen by the first ants would tend to be excessively attractive to the following ones. In
that case, the exploration of the solution space would be constrained.Thus, when one ant
finds a good (i.e. short) path from the colony to a food source, other ants are more likely
to follow that path, and positive feedback eventually leads all the ants following a single
path. The idea of the ant colony algorithm is to mimic this behavior with "simulated ants"
walking around the graph representing the problem to solve.
Figure 3.2 Example for Ant colony optimization
Ant colony optimization algorithms have been used to produce near-optimal
solutions to the traveling salesman problem. They have an advantage over simulated
Division Of computer science, SOE, CUSAT
18
Bio Inspired Computing
annealing and genetic algorithm approaches when the graph may change dynamically;
the ant colony algorithm can be run continuously and adapt to changes in real time. This
is of interest in network routing and urban transportation systems.
ACO algorithms are usually applied to discrete (combinatorial) optimization
problems. Assuming that the problem to be optimized can be represented by a graph, the
general ACO algorithm can be described as given below:.
1. Initialization: assign the same initial pheromone value to each edge of the graph, and
randomly place an ant in a location of the search space.
2. Population loop: for each ant, do:
2.1 Probabilistic transition rule: according to a given probabilistic transition rule, move
ant over the space so that a solution to the problem is built.
2.2 Goodness evaluation: evaluate the goodness of the solution obtained by this ant.
2.3 Pheromone updating: update the pheromone level of each edge by reinforcing good
solutions.Reduce the pheromone level of each edge (evaporation).
3. Cycle: repeat Step 2 until a given convergence criterion is met.
3.5 PARTICLE SWARM OPTIMIZATION
The particle swarm optimization algorithm was introduced to study social
and cognitive behavior, but it has been largely applied as a problem-solving technique in
engineering and computer science. There are two main versions of the PSO algorithm: a
binary and a real-valued version. With the exception of the representation, the two
versions of the algorithm are very much the same, thus only the real-valued (most
popular) version will be discussed here.
The particle swarm approach assumes a population of individuals
represented as binary strings or real-valued vectors – particles, which suffer an iterative
procedure of adaptation to their environment. It also assumes that these individuals are
social, what implies that they are capable of interacting with other individuals within a
given neighborhood.
There are two main types of information available to each
individual of the population. The first is their own past experiences, and the second is the
Division Of computer science, SOE, CUSAT
19
Bio Inspired Computing
knowledge about how individuals around them have performed. The authors likened
these two types of information to the individual learning and cultural transmission.
Individuals tend to be influenced by its success along its past history and also by the
success of any individual in its neighborhood, i.e., with which it interacts. To these
‘schemes of interactions’ between individuals, the authors termed sociometric principles.
Individuals can interact with each other in a number of ways. The simplest form is a
binary interaction, where the individual interacts with its two nearest neighbors. Any
number k of nearest neighbors can be used. If the number of nearest neighbors is less
than the total number of individuals in the population, then this sociometric principle is
called lbest, else (if k = N) it is called gbest. Conceptually, gbest connects all the
individuals together, what means that its social interaction is maximal. In contrast, lbest
results in a local neighborhood for the individual. The authors claim that the binary
particle swarm algorithm can be interpreted as a qualitative or quantitative social
optimization algorithm, while the real valued (continuous) version of PSO is a truly
numeric optimization algorithm. In the latter version, the PSO searches for optima in Rn,
where n is the dimension of the search space. The continuous version of PSO assumes
individuals as points in a space, and the change over time is represented as movements of
the points, now defined as particles. A psychological system is viewed as an information
processing function, and each coordinate of a particle in the search space corresponds to a
psychological measure. Forgetting and learning are viewed respectively, as an increase or
a decrease in the value of a given coordinate.
Assume that the position of a particle i is given by xi and its velocity by vi.
The velocity is a vector of numbers that are added to the position coordinates of the
particle in order to move it throughout the search space along the iterations (t is the time
index): xi(t) = xi(t−1) + vi(t) (1) The social-psychological theory used as inspiration to
envelop the PSO algorithm suggests that individuals oving along a sociocognitive space
should be influenced by their own previous behavior and by the successes of its
neighbors. It is important to note that neighborhood is related to the topologic space that
defines the sociometric structure of the population, not to the distance between
individuals in the parameter space. In both versions (binary and continuous) of the
Division Of computer science, SOE, CUSAT
20
Bio Inspired Computing
algorithm, a neighborhood is defined for each individual based on its position in the
topological population array. The population array is usually implemented as a ring
structure, with the last member being a neighbor of the first one. As the particles are
moving in the space, the direction of movement is a function of its current position and
velocity, the location of the individual’s current best success pi, and the best position
found by any member of the neighborhood pg: xi(t) = f(xi(t−1), vi(t−1), pi, pg) The
change vi in the trajectory of a particle is a function of the difference between the
individual’s previous best and current positions, and the difference between the
neighborhood’s best and the current individual’s position. The formula for changing
velocity assumes continuous variables:
vi(t) = vi(t−1) + ϕ1(pi − xi(t−1)) + ϕ2(pg − xi(t−1)) (3)
In order to avoid that this system explodes when the particles’ oscillations become too
large, the velocity of the particles is damped by a factor Vmax:
if vid > Vmax, then vid = Vmax. (4)
if vid < −Vmax, then vid = −Vmax. (5)
The general PSO algorithm is summarized below:.
1. Initialization: randomly initialize a population of particles.
2. Population loop: for each particle, do:
2.1 Goodness evaluation and update: evaluate the goodness’ of the particle. If its
goodness is greater than its best goodness so far, then this particle becomes the best
particle found so far.
2.2 Neighborhood evaluation: if the goodness of this particle is the best among all its
neighbors, then this particle becomes the best particle of the whole neighborhood.
2.3 Determine vi: apply equation (3).
2.4 Particle update: apply the updating rule given by equation (1).
3. Cycle: repeat Step 2 until a given convergence criterion is met
Division Of computer science, SOE, CUSAT
21
Bio Inspired Computing
Figure 3.3 Particle Swarm Optimization
Division Of computer science, SOE, CUSAT
22
Bio Inspired Computing
4.ARTIFICIAL IMMUNE SYSTEM
Artificial immune systems (AIS) have been defined as adaptive systems
inspired by the immune system and applied to problem solving. In a simplified form, to
design an AIS it is necessary to choose an appropriate shape-space for the components of
the system, one or more affinity measure(s), and an immune algorithm. The shape-space
is a formalism used to create abstract (‘artificial’) representations for the components of
the immune system. The ‘shape’ of an immune cell or molecule corresponds to all the
features required to quantify interactions between the cell or molecule and the
environment, and also with other elements of the system.
There are four main types of shape-spaces: Euclidean or realvalued, Hamming, Integer, and Symbolic. In Euclidean shape-spaces the elements of the
system are represented as real-valued vectors. In Hamming shapespaces the elements of
the system are attribute strings built out of a finite alphabet. In Integer shape-spaces, cells
and molecules are represented as integer values. (Note that Integer shape-spaces are a
particular case of Hamming shape-spaces.) Symbolic shape-spaces use different types of
attributes to represent a single element, for example, an integer value and a string such as
‘color’. There are two main types of interactions that can be performed by an element of
an artificial immune system. One is the interaction with the environment. For example,
an AIS can be used as a pattern recognition tool, thus the ‘artificial immune cells’ are
used to recognize a set (or sets) of ‘artificial antigens’ (patterns). In this case, the degree
of recognition, known as the affinity between the immune cell and the antigen, is
measured via a function that quantifies the strength of the match between the two. If we
assume that two cells interact to the extent their ‘shapes’ are similar, then a similarity
measure can be used according to the shape-space adopted. As an example, assume an
immune cell with the following shape Ab = [1,0,0,0,1] in a binary Hamming shape-space,
and an antigen with the following shape Ag = [0,1,1,1,1]. If affinity is directly
proportional to their similarity, then the expression L − Hamming distance is a suitable
measure to quantify immune recognition, where L is the length of the string. Their
Division Of computer science, SOE, CUSAT
23
Bio Inspired Computing
affinity in this case is Aff = 5 − 4 = 1. There are several types of affinity measures, which
usually vary according to the shape-space adopted, which by itself is usually defined by
the problem in hand. The last ‘building block’ of artificial immune systems corresponds
to the immune algorithms. There are a number of different algorithms that can be applied
to many domains, from data analysis to autonomous navigation. These immune
algorithms were inspired by works on theoretical immunology and several processes that
occur within the immune system. They can be classified as population-based and
network-based immune algorithms. In population-based algorithms, the elements of the
system are not connected with each other, meaning that they only interact directly with
the environment.
Interactions with other elements of the system can only be
performed indirectly, via, for example, a reproductive operator. In network-based AIS by
contrast, some (or all) elements of the system are interconnected. This way, there are two
levels of interaction within this system: with the environment and with other elements in
the system.
4.1. IMMUNE ALGORITHMS
In order not to overload the text with descriptions of several immune
algorithms, the focus will be given to two population-based AIS (negative and clonal
selection algorithms), and to two types of network-based AIS (continuous and discrete
immune networks). The main role played by the immune system is to protect our
organisms against infectious diseases (caused by viruses, bacteria, etc.), and to eliminate
debris and malfunctioning cells. To perform these functions, the immune system has to
be able to distinguish between our own cells (known as self) and those elements that do
not belong to the organism itself (known as nonself). One of the processes by which the
immune system differentiates self from nonself (self/nonself discrimination) is termed
negative selection. This gave rise to the negative selection algorithm. After distinguishing
between self and nonself, the immune system has to perform an immune response in
order to eliminate the nonself substances. Clonal selection is the name given to a theory
Division Of computer science, SOE, CUSAT
24
Bio Inspired Computing
that explains how the immune cells and molecules cope with invading nonself elements,
known as antigens.
Assuming that the set of self elements is known,the
standard negative selection algorithm aims at generating another set of immune cells,
known as detectors, that recognizes any cell (pattern) but those belonging to the self set.
The algorithm is summarized below
1. Initialization: randomly generate a number of candidate detectors (attribute strings).
2. Censoring: while a set of detectors of a given size has not yet been produced, do:
2.1 Affinity evaluation: determine the affinity between every self and a candidate
detector.
2.2 Selection: if the candidate detector recognizes any element of the self this candidate
is eliminated. Else, place this candidate detector in the detector set.
3. Monitoring: after the set of detectors has been generated, monitor a new set of self for
any variation. It means that if any element of the detector set matches an element of the
new self-set, then a nonself element was detected.
The theory known as clonal selection is used to
explain how the immune system ‘fights’ against an antigen. When a bacterium invades
our organism, it starts multiplying and damaging our cells. One form the immune system
found to cope with this replicating antigen was by replicating the immune cells successful
in recognizing and fighting against this disease-causing element. Those cells capable of
recognizing the antigen reproduce themselves asexually in a way proportional to heir
degree of recognition: the better the antigenic ecognition, the higher the number of
offspring (clones) generated. During the process of cell division (reproduction),
individual cells suffer a mutation that allows them to become more adapted to (increase
affinity with) the antigen recognized: the higher the affinity of the parent cell, the lower
the mutation they suffer. Assuming in this case a set of antigens to be recognized, a basic
clonal selection algorithm, named CLONALG, works as shown below:
1. Initialization: randomly initialize a repertoire (population) of attribute strings (immune
cells).
2. Population loop: for each antigen, do:
2.1 Selection: select those cells whose affinities with the antigen are greater.
Division Of computer science, SOE, CUSAT
25
Bio Inspired Computing
2.2 Reproduction and genetic variation: generate copies of the immune cells: the better
each cell recognizes the antigen, the more copies are produced. Mutate (perform
variations) in each cell inversely proportional to their affinity: the higher the affinity, the
smaller the mutation rate.
2.3 Affinity evaluation: evaluate the affinity of each mutated cell with the antigen.
3. Cycle: repeat Step 2 until a given convergence criterion is met.
According to the immune network theory, immune cells have portions of
their receptor molecules that can be recognized by other immune cells in a way similar to
the recognition of an invading antigen. This results in a network of communication
(recognition) between immune cells. When an immune cell recognizes an antigen or
another immune cell, it is stimulated. On the other hand, when an immune cell is
recognized by another immune cell, it is suppressed. The sum of the stimulation and
suppression received by the network cells, plus the stimulation by the recognition of an
antigen corresponds to the stimulation level S of a cell, as described by Equation 6:S =
Nst − Nsup + As, (6)where Nst corresponds to the network stimulation, Nsup is the
network suppression, and As corresponds to the antigenic stimulation.
The general immune network algorithm is given below:
1. Initialization: initialize a network of immune cells.
2. Population loop: for each antigen, do:
2.1 Antigenic recognition: match network cells against the antigen.
2.2 Network interactions: match network cells against network cells.
2.3 Metadynamics: introduce new cells into the network and eliminate useless ones
(based on a certain criterion).
2.4 Stimulation level: evaluate the stimulation level of each network cell taking into
account the results of the previous steps (Equation (6)).
2.5 Network dynamics: update the network’s structure and free parameters according to
the stimulation level of individual cells.
3. Cycle: repeat Step 2 until a given convergence criterion is met.
The stimulation level of an immune cell is going to determine its probability of
reproduction and genetic variation. There are two main types of immune network models:
Division Of computer science, SOE, CUSAT
26
Bio Inspired Computing
continuous and discrete. Continuous immune networks have their repertoires of cells
governed by one or more sets of ordinary differential equations (ODE), each of which
corresponds to the variation in number and affinity of a given immune cell. Discrete
immune networks are governed by a difference equation that is used in an iterative
procedure of adaptation controlling the number and affinities of individual cells.
Division Of computer science, SOE, CUSAT
27
Bio Inspired Computing
CONCLUSION
We can conclude that even the simplest of these natural intelligences can
achieve complex information processing and computational tasks that current artificial
intelligences find very challenging. We have seen the extent to which biological systems
such as neurons, brains, insects, and insect colonies have influenced the design of
artificial intelligences and our understanding of what intelligence actually is. In addition
we learnt how biological processes such as evolution and co evolution can be harnessed
to automatically produce solutions to challenging problems in AI as well as more
generally in computer science, and how biological phenomena such as flocking can
inspire algorithms in computer science.
Evolutionary algorithms have been successfully applied to a variety of optimization
problems such as wire routing, scheduling, traveling salesman, image processing,
engineering design, parameter fitting, computer game playing, knapsack problems, and
transportation problems. Ant colony algorithms help find solutions to routing problems.
They give more flexible adaptation to failures and network congestion and use only local
knowledge for routing and avoid costly communication of state to all network nodes.
Applications of artificial immune systems include computer security, monitoring network
services, distinguishing between spam n non spams, detecting viruses and malicious self
propagating codes.
Therefore Biologically inspired computing could allow the creation of new machines
with promising characteristics such as fault-tolerance, self-replication or cloning,
reproduction, evolution, adaptation and learning, and growth
Division Of computer science, SOE, CUSAT
28
Bio Inspired Computing
REFERENCES
1.www.cs.comp/lecture_slides.pdf
2.www.wikipedia.org
3.www.tech4all.com
4.www.biologicalcomputing.com
5. informatics.indiana.edu/rocha/i-bic/
Division Of computer science, SOE, CUSAT
29