Download Biologically inspired computing

Biologically inspired computing
How useful bugs can solve computing problems
Thiemo Krink
EVALife Group, Dept. of Computer Science,
University of Aarhus, Denmark
Introduction
Challenges in contemporary information technology
computational hard problems
data analysis, scheduling, chip layout, etc.
security in distributed systems
pervasive computing, internet, banking, etc.
intelligent machines
autonomous robots, expert systems, etc.
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Biologically-inspired computing
An important idea
to take inspiration from biology for new IT solutions
Examples
Artificial neural networks
Evolutionary algorithms
Swarm intelligence
Self-assembling robots
DNA-based computing
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Neural networks
Properties of the brain
holistic
parallel
associative
learning
redundancy
self-organisation
dendrites
cell body
axon
The basic unit - the neurone
Vertical cut through
the neocortex of a cat
Functional parts of
the human brain
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolution
Why is evolution interesting for IT?
convincing problem solutions in nature!
problem solving without knowledge
adaptation to changing problems
parallel information processing
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
The idea
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
describe a problem by a set of parameters
interpret the parameters as artificial genes
consider the genes as blueprints of individuals
apply evolution to the individuals
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Problem
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
quality measure: f (x1, x2, x3, x4)
problem parameters: x1, x2, x3, x4
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Problem
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
quality measure: f (x1, x2, x3, x4)
problem parameters: x1, x2, x3, x4
artificial genes:
1 0 1 0 0 0 1 1
0 0 1
0
representation of one solution
fitness: f (genes)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Problem
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
quality measure: f (x1, x2, x3, x4)
Individual
problem parameters: x1, x2, x3, x4
artificial genes:
1 0 1 0 0 0 1 1
0 0 1
0
representation of one solution
fitness: f (genes)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Initialisation
Fitness
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
Population
Results
Evaluation
Mutation
Selection
Recombination
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Initialisation
Fitness
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
Population
Results
Evaluation
Mutation
Selection
Recombination
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Initialisation
Fitness
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
Population
Results
Evaluation
Mutation
Selection
Recombination
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Initialisation
Fitness
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
Population
Results
Evaluation
Mutation
Selection
Recombination
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation
Initialisation
Fitness
(Fogel, 1966)
(Rechenberg, 1973)
(Holland, 1975)
Population
Results
Evaluation
Mutation
Selection
Recombination
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
A case study
(Rechenberg, 1973)
The task: Design a bent tube with a maximum flow
Goal: gas flow f (x1, x2, …, x6) = fmax
x3
x4
x5
x6
f
x2
x1
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
A case study
(Rechenberg, 1973)
The task: Design a bent tube with a maximum flow
Goal: gas flow f (x1, x2, …, x6) = fmax
x3
x4
x5
x6
f
x2
x1
Evolutionary
Predicted solution
Computing
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Evolutionary Computation - Pros and Cons
Pro
can be applied to almost any kind of problem
can handle a large number of parameters
can adapt to a changing definition of the problem
easy to implement
Contra
slow performance compared to other search techniques
problem and parameter sensitive performance
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
DNA Sequence Analysis Multiple Sequence Alignment
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Multiple Sequence Alignment (MSA)
What is MSA?
A set of sequences arranged in a matrix where each row
consists of one sequence padded by gaps
Columns highlight similarity/conservation between positions
An alignment is optimal if it has maximum similarity for a
certain scoring function
ACTAA
TAATCAA
ATCA
TAATCAT
ATGATT
-A--CTAA-TAATC-AA--A-TC-A--TAATC-A--T
-A-TG-A-TT
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Multiple Sequence Alignment (MSA)
Why is MSA important?
Measure similarity between species
Reconstruction of evolutionary trees
Assist algorithms for protein structure prediction
The problem
To find an optimal alignment of n sequences is NP-hard
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Multiple Sequence Alignment (MSA)
The state-of-the-art
Clustal W - iterative search using pairwise alignments
very fast with quite reasonable results
Our approach (MSA EA)
(Thomsen et al., 2002)
very fast and better results using evolutionary computation
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Initialization with Seeding
Random
alignments
Clustal V
alignment
(Thomsen et al., 2002)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Single Parent Operators
(Thomsen et al., 2002)
Block Shuffle
Random Shuffle
Local Shuffle
Local Shuffle One
Grow Matched
Columns
Grow Matched Column
Parent
--------123456789
-AT-CA--A
T--AATCAA
AT--CA--T-AAT-CAT
A-TGAT-TA-TGAT-T-
Parent
--------123456789
-AT-CA-AA
T--AATCAA
AT--CA--T-AAT-CAT
A-TGAT-T-
Parent
--------123456789
-AT-CA-TC
TA-C-ATAA
AT--CA--T
T-C--A--T
A-TC-A-T-
Offspring
--------123456789
-AT-CA--A
T-AATCA-A
AT--CA--T-AAT-CAT
A--TGATTA-TGAT-T-
Offspring
--------123456789
-AT-CA-AA
T--AATCAA
AT--C--AT-AAT-CAT
A-TGAT-T-
Offspring
--------123456789
-AT-CAT-C
TA-C-ATAA
AT--CAT-T-C--AT-A-TC-AT--
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Two Parents Operators
(Thomsen et al., 2002)
Recombine
Recombine Matched
MatchedColumns
Columns
Parent 1
123456789
----*----ATCA--AT
T--AATCAA
----ATCAT-AAA-C-T
A-TGAT-T-
Parent 2
1234567890
------*---ATCAAT--T--A-ATCAA
-----ATCAT-AAACT--A-TG-AT-T-
123456789012
----*---*---ATCA--AT--T--AA---TCAA
----A---TCAT-AAA-C-T--A-TGA---T-T-
Offspring
”Clean Up Gap Columns” is applied after all other operators
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Fitness evaluation
(Thomsen et al., 2002)
Fitness = SymbolScore – GapPenaltyScore
n −1
n
w
SymbolScore =
i =1 j = i +1 k =1
PAM(l i [k ],l j [k ])
GapPenaltyScore = GOP + GAPS x GEP
GOP: gap opening penalty
GAPS: number of consecutive gaps
GEP: gap extension penalty
PAM = Point accepted mutation
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Data and Experiments
(Thomsen et al., 2002)
Seq. Length avg (min,max)
Data Set
N
Histone H4
71
101.1 (71, 107)
Globin
12
146.6 (136, 153)
Cytochrome C
6
108.0 (82, 135)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Results (scores)
Data Set
Histone H4
(Thomsen et al., 2002)
MSA EA
Clustal V
Score
Score
44207
44494.4
(44635)
Globin
-779.6
(-742)
-924
Cytochrome C
-1003.2
(-990)
-1078
Alignment scores (maximization); means of 30 runs
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Results (mean fitness – globin)
(Thomsen et al., 2002)
Classic EA
Clustal V
Fitness
Data set: Globin
#Evaluations
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
MSA-EA: Results (relative improvement)
(Thomsen et al., 2002)
Improvement (%)
Data set: Globin
Time (seconds, PC Pentium III, 800 MHz)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Mass extinction and
Self-organized Criticality
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Has mass extinction been important?
Punctuated Equilibrium
Time
Gradualist Model
Change
Charles Darwin, John Maynard Smith,
Richard Dawkins
Change
Stephen J. Gould, Stuart Kauffman,
Per Bak
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
log (frequency)
Percent extinction
Mass extinction is power law distributed
96%of all marine species
went extinct (Raup, 1986)
log (percent extinction)
Geological time (million years)
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
(Sepkoski, 1993)
Results - An example
Objective function (minimization)
10
Classic EA
SOC Extinction EA
8
6
4
2
0
0
100
200 300 400 500 600 700
Generations (number of iterations)
Benchmark: Rastrigin 20D
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
800
Life cycles
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Life cycles as an inspiration
Egg
Butterfly
Larva
Pupa
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
Life cycles as an inspiration
Inspiration
the life cycle of symbion pandora (Cycliophora)
Co-existing 'populations' of different life stages
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
The Life Cycle EA
Motivation for application
The performance of adaptive search heuristics is highly
problem dependent
Idea
Self-adaptive approach inspired by the life cycle of
symbion pandora, which combines search heuristics
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
The Life Cycle EA
Features of the algorithm
subpopulations for each search heuristic
individuals migrate if they make no progress for a while
heuristic1
heuristic2
heuristicn
candidate solution
…
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark
heuristic3
Results - An example
"Biologically-inspired computing", T. Krink, EVALife Group, Univ. of Aarhus, Denmark