Download An Introduction to Artificial Immune Systems

An Introduction to Artificial Immune
Systems
Jonathan Timmis
Computing Laboratory
University of Kent at Canterbury
CT2 7NF. UK.
[email protected]
http:/www.cs.kent.ac.uk/~jt6
AIS – October 2003 – 1
Novel paradigms are proposed and
accepted not necessarily for being faithful
to their sources of inspiration, but for
being useful and feasible
AIS – October 2003 – 2
What do I want to achieve?
O
O
Give you a taster of what AIS is all about
‹
Why do we find the immune system useful?
‹
Explain what AIS are
‹
Show you where they are being used
‹
Comments for the future
I won’t:
‹
‹
‹
Talk about all areas of AIS and applications
Talk too much about how AIS relate to other bioinspired
ideas (although I will mention it)
Go into too much detail: this is an introduction
AIS – October 2003 – 3
Outline
OWhat
are AIS?
OUseful
immunology
OThinking
about AIS
OApplication
OThe
Areas
Future
AIS – October 2003 – 4
Why the Immune System?
O
Recognition
‹
Anomaly detection
‹
Noise tolerance
O
Robustness
O
Changing nature of self
O
Diversity, Adaptive
O
Reinforcement learning
O
Memory; Dynamically changing coverage
O
Distributed
O
Multi-layered
AIS – October 2003 – 5
A Definition
AIS are adaptive systems inspired by theoretical
immunology and observed immune functions,
principles and models, which are applied to complex
problem domains
AIS – October 2003 – 6
Some History
O
Developed from the field of theoretical immunology in
the mid 1980’s.
‹ Suggested
O
we ‘might look’ at the IS
1990 – Bersini first use of immune algos to solve
problems
O
Forrest et al – Computer Security mid 1990’s
O
Hunt et al, mid 1990’s – Machine learning
AIS – October 2003 – 7
Scope of AIS:
O
O
O
O
O
O
Computer Security(Forrest’94’96’98, Kephart’94,
Lamont’98’01,02, Dasgupta’99’01, Bentley’00’01,02)
Anomaly Detection (Dasgupta’96’01’02)
Fault Diagnosis (Ishida’92’93, Ishiguro’94)
Data Mining & Retrieval (Hunt’95’96, Timmis’99’ 01,
’02)
Pattern Recognition (Forrest’93, Gibert’94, de Castro
’02)
Adaptive Control (Bersini’91)
AIS – October 2003 – 8
Scope of AIS (Cont……):
O
Job shop Scheduling (Hart’98, ’01, ’02)
Chemical Pattern Recognition (Dasgupta’99)
Robotics (Ishiguro’96’97,Singh’01)
Optimization (DeCastro’99,Endo’98, de Castro ’02)
Web Mining (Nasaroui’02)
Fault Tolerance (Tyrrell, ’01, ’02, Timmis ’02)
Autonomous Systems (Varela’92,Ishiguro’96)
Engineering Design Optimization (Hajela’96 ’98,
O
And so on …
O
O
O
O
O
O
O
Nunes’00)
AIS – October 2003 – 9
Outline
OWhat
are AIS?
OUseful
immunology
OThinking
about AIS
OApplication
OThe
Areas
Future
AIS – October 2003 – 10
How does it work: A simplistic view
M H C
p r o te in
A n tig e n
( I )
A P C
P e p tid e
( II )
T - c e ll
( III )
( IV
A c t iv a te d
)
)
L y m p h o k in e s
T - c e ll
A c t iv a te d
( p la s m a
( V
B - c e ll
( V I )
B - c e ll
c e ll)
( V II )
AIS – October 2003 – 11
Self/Non-Self Recognition
OImmune
system needs to be able to differentiate
between self and non-self cells
OAntigenic
therefore
encounters may result in cell death,
‹
Some kind of positive selection
‹
Some kind of negative selection
AIS – October 2003 – 12
Immune Pattern Recognition
BCR or Antibody
B-cell Receptors (Ab)
Epitopes
Antigen
B-cell
immune recognition is based on the complementarity between the
binding region of the receptor and a portion of the antigen called
epitope.
OThe
OAntibodies
present a single type of receptor, antigens might present
several epitopes.
‹
This means that each antibody can recognize a single antigen
AIS – October 2003 – 13
Immune Network Theory
OIdiotypic
OB
network (Jerne, 1974)
cells co-stimulate each other
‹
Treat each other a bit like antigens
OCreates
an immunological memory
Suppression
Negative response
Paratope
A g
1
2
Idiotope
3
Antibody
Activation
Positive response
AIS – October 2003 – 14
Learning
Antibody Concentration
Cross-Reactive
Response
Secondary Response
Primary Response
Lag
Lag
Response
to Ag1
Lag
Response
to Ag1
...
...
Antigen Ag1
Antigens
Ag1, Ag2
...
Response to
Ag1 + Ag3
Response
to Ag2
...
Antigen
Ag1 + Ag3
Time
AIS – October 2003 – 15
Danger Theory (1)
O
O
Proposed by Polly Matzinger, around 1995
Problem: Traditional self/non-self theory doesn’t
always match observations
‹ Immune system always responds to non-self
…apart from the nonself it doesn’t respond to (harmless
foreign)
‹ Immune
system always tolerates self
…apart from the self it doesn’t tolerate (dangerous
self)
O
T-cell activation by APCs
‹ Janeway’s infectious nonself model
AIS – October 2003 – 16
Danger Theory (2)
Danger theory relates innate and adaptive immune
systems:
‹ Tissues induce tolerance towards themselves
‹ Tissues protect themselves and select class of
response
1. Tissues induce tolerance by:
‹ Lymphocytes receive 2 signals
O
‹ Signal
1 – antigen/lymphocyte binding
‹ Signal 2 – antigen is properly presented by APC
‹ Signal
1 WITHOUT signal 2 = lymphocyte death
AIS – October 2003 – 17
Danger Theory (3)
2.
Tissues protect themselves
‹
Alarm Signals activate APCs
‹
‹
3.
Alarm signals come from
‹
Cells that die unnaturally
‹
Cells under stress
APCs activate lymphocytes
Tissues dictate response type
‹
Alarm signals may convey information
AIS – October 2003 – 18
Immune System: Summary
O
O
O
O
O
The host has to distinguish either between self/nonself or dangerous/non-dangerous
When an entity is recognised as foreign (or
dangerous)- activate several defense mechanisms
leading to its destruction (or neutralisation).
Subsequent exposure to similar entity results in rapid
immune response.
Overall behavior of the immune system is an
emergent property of many local interactions.
So it is useful?
AIS – October 2003 – 19
Outline
OWhat
are AIS?
OUseful
immunology
OThinking
about AIS
OApplication
OThe
Areas
Future
AIS – October 2003 – 20
This Section
OGeneral
Framework for describing and constructing
AIS models
OProvide
a few examples of applications
‹
Data Mining
‹
Optimisation
‹
Other areas from the labs research
AIS – October 2003 – 21
What do want from a
Framework?
O
In a computational world we work with
representations and processes. Therefore, we need:
‹ To
be able to describe immune system components
‹ Be
able to describe their interactions
‹ Quite
high level abstractions
‹ Capture
general purpose processes that can be
applied to various areas
AIS – October 2003 – 22
General Framework for AIS
Immune Algorithms
Affinity Measures
Representation
Application Domain
AIS – October 2003 – 23
The Framework ..
O
Think about the use of AIS in terms of:
‹ Representation;
O
O
‹ Affinity
Measures;
‹ Immune
Algorithms;
…and how we apply them and to what we apply them
Think about the bias of all of the above and what
affect that may have on the result (if any).
AIS – October 2003 – 24
Representation
OVectors
Ab = 〈Ab1, Ab2, ..., AbL〉
OReal-valued
OInteger
OBinary
Ag = 〈Ag1, Ag2, ..., AgL〉
shape-space
shape-space
shape-space
OSymbolic
shape-space
AIS – October 2003 – 25
Define their Interaction
O
Define the term Affinity
O
Affinity is related to distance
‹ Euclidian
D=
L
2
(
Ab
−
Ag
)
∑ i
i
i =1
• Other distance measures such as Hamming,
Manhattan etc. etc.
• Affinity Threshold
AIS – October 2003 – 26
Basic Immune Algorithms
ONegative
OClonal
Selection Algorithm
OImmune
OBone
Selection Algorithms
Network Algorithms
Marrow Algorithms
AIS – October 2003 – 27
Negative Selection (NS) Algorithms
O
O
O
Define Self as a normal pattern of activity or stable behavior of a
system/process
‹ A collection of logically split segments (equal-size) of pattern
sequence.
‹ Represent the collection as a multiset S of strings of length l
over a finite alphabet.
Generate a set R of detectors, each of which fails to match any
string in S.
Monitor new observations (of S) for changes by continually testing
the detectors matching against representatives of S. If any
detector ever matches, a change ( or deviation) must have occurred
in system behavior.
S e lf
s tr in g s ( S )
G e n e r a te
r a n d o m s t r in g s
(R 0)
M a tc h
Yes
R e je c t
D e te c to r S e t
(R )
No
D e te c to r
S e t (R )
P ro te c te d
S t r i n g s (S )
M a tc h
N o
Y e s
N o n - s e lf
D e te c te d
AIS – October 2003 – 28
Illustration of NS Algorithm:
r=2
Match
1011
1000
Don’t Match
1011
1101
Non_Self
Self
Self
AIS – October 2003 – 29
Clonal Selection Algorithm
1. Initialisation: Randomly initialise a population (P)
2. Antigenic Presentation: for each pattern in Ag, do:
2.1 Antigenic binding: determine affinity to each P’
2.2 Affinity maturation: select n highest affinity from P and
clone and mutate prop. to affinity with Ag, then add new
mutants to P
3. Metadynamics:
3.1 select highest affinity P to form part of M
3.2 replace n number of random new ones
4. Cycle: repeat 2 and 3 until stopping criteria
AIS – October 2003 – 30
Immune Network Algorithm
•
•
Initialisation: create an initial network from a sub-section of the antigens
Antigenic presentation: for each antigenic pattern, do:
2.1 Clonal selection and network interactions: for each network cell,
determine its stimulation level (based on antigenic and network interaction)
2.2 Metadynamics: eliminate network cells with a low stimulation
2.3 Clonal Expansion: select the most stimulated network cells and
reproduce them proportionally to their stimulation
2.4 Somatic hypermutation: mutate each clone
2.5 Network construction: select mutated clones and integrate
3. Cycle: Repeat step 2 until termination condition is met
AIS – October 2003 – 31
Outline
OWhat
are AIS?
OUseful
immunology
OThinking
about AIS
OApplication
OThe
Areas
Future
AIS – October 2003 – 32
Data mining
OMore
benchmark problem in this case
OAssume
a set of labelled vectors
OClassification
AIS – October 2003 – 33
AIRS: (Artificial Immune Recognition System)
OClonal
Selection
OBased
initially on immune networks, though found this
did not work
OResource
OSomatic
allocation
hypermutation
OAntibody/antigen
binding
AIS – October 2003 – 34
AIRS: Mapping from IS to AIS
Antibody
Feature Vector
Recognition
Combination of feature Ball
vector and vector class
Antigens
Training Data
Immune Memory
Memory cells—set of
mutated ARBs
AIS – October 2003 – 35
AIRS Algorithm
O
Data normalisation and initialization
O
Memory cell identification and ARB generation
O
O
Competition for resources in the development of a
candidate memory cell
Potential introduction of the candidate memory
cell into the set of established memory cells
AIS – October 2003 – 36
Classification Accuracy
OImportant
to maintain accuracy
AIRS1: Accuracy
AIRS2: Accuracy
Iris
96.7
96.0
Ionosphere
94.9
95.6
Diabetes
74.1
74.2
Sonar
84.0
84.9
AIS – October 2003 – 37
Features
ONo
need to know best architecture to get good results
ODefault
can get
settings within a few percent of the best it
OUser-adjustable
parameters optimize performance for
a given problem set
OGeneralization
and data reduction
AIS – October 2003 – 38
Unsupervised Learning
OAgain,
a benchmark problem in this case
OAssume
OWe
‹
‹
‹
a set of unlabelled vectors
can ask the questions:
Is there a large amount of redundancy?
Are there any groups or subgroups intrinsic to
the data?
What is the structural or spatial distribution?
AIS – October 2003 – 39
aiNET: Immune principles
employed
OB-cells
(antibodies)
OAntigens
OAntibody/antigen
OClonal
binding
selection process
OImmune
network theory
OCombined
with statistical analysis tools
AIS – October 2003 – 40
Data mining: Clustering
(aiNet)
O
O
O
O
Limited visualisation
Interpret via MST or
dendrogram
Compression rate of
81%
Successfully identifies
the clusters
Training Pattern
T r a in in g
1
1111111111111111111111 1111111111111
111111
111111111111 1
11111 11
11 1111
11111111
1111111111
5 5 5
111
1
22 22 2
11111111111111
1
1
1
555 55 555
222222
222222222222222222
111111111
111111111116 66 66666 5 555555555555555555555555555555555555555 5
222222222222222 2
2
1111111
2
2
2
2
2
2
2
2
2
111111
2222 222222222
555 5555555 5555 5555
22
111111
22222222
1
1
1
1
5 5555 5 55 5555
2
1111111
1 111
1 11
111111
11111111
1
1
1
1111111
1111 111 11
11111111111111111111111 11 11111111 1111
1
11
0 .8
0 .6
0 .4
0 .2
0
P a tte rn s
4 4
44
44 4 4
44 4 44 44
44 444444444444444 44 444
44 444444444444444444444444 44444
4444444444444 44
4
3 33 3333334 33
3
333333333333 33 3
3 33333333333333333333333333333333 33
3 3333 3333333333333 33 3
3 33
3
3 3333 3
0
7
77
777
77 7
7
7 77
88
77
77
88 8
888
777
7
88 8
8
7 77
77
8
8
7
8 8
7777
888
7
888
7 7 77
88
88888
77 77
88
777 7
888
7777 7
8
7
7
0 .2
0 .4
0 .6
0 .8
8 88
8 8
88 8
888
8
88 8
8
88
88 8
1
Result immune network
AIS – October 2003 – 41
Data mining: Adaptation to
multimodal optimisation
O
O
O
Supports the idea of
general purpose
algorithms
Initial population
Slight modification of
network algorithm
Combines local and
global search
Final population
AIS – October 2003 – 42
Results – Roots Function
aiNET
CLONALG
AIS – October 2003 – 43
Fun Immune Inspired Ideas
O
Immunised Fault Tolerance for Mechatronic devices
‹ Exploratory
‹ Increase
availability of machines
‹ Prediction
O
industrial sponsored work
of machine states
Software Mutation Testing
a vaccine for the software development
process
‹ Find
‹ Coevolution
of programme and test data to identify
common errors in the software development
lifecycle of a particular team
AIS – October 2003 – 44
More fun …
O
O
Danger theory and Web content mining
‹ Extracting information from the content of
web pages.
‹ Little AIS research in this area.
‹ A very large and very dynamic data set.
Domain characteristics include:
‹ Highly volatile data.
‹ High volume of data.
‹ Ever-changing content.
‹ Need for continuous adaptation.
AIS – October 2003 – 45
Outline
OWhat
are AIS?
OUseful
immunology
OThinking
about AIS
OApplication
OThe
Areas
Future
AIS – October 2003 – 46
The Future
O
Rapidly emerging field
O
Much work is very diverse
‹ Framework
‹ More
helps a little
formal approach required?
O
Wide possible application domains
O
What is it that makes the immune system unique?
O
More work with immunologists
‹ Theories
such as Danger theory, Self-Assertion
may have something to say to AIS
AIS – October 2003 – 47
Danger Theory and AIS
O
O
O
Advantages
‹ More scalable (activation in danger area only)
‹ More dynamic
Both do not have to be explicitly coded for
Characteristics
‹ Context dependent activation
‹ via
danger signal
‹ Notion
of danger area
‹ Localised response within danger area
AIS – October 2003 – 48
Integrating: Homeostasis
Ability of an organism to achieve a steady state of
internal body function in a varying environment
O
O
Lots of complex interactions
‹
Nervous system
‹
Endocrine system
‹
Immune System
Developed a simple neural network and endocrine
controller for a mobile robot
AIS – October 2003 – 49
The Future (2)
O
O
O
O
O
ARTIST: A Network for Artificial Immune Systems
(EPSRC funded network)
Work towards:
‹ A theoretical foundation for AIS as a new CI
‹ Extraction of accurate metaphors
‹ Immune System Modelling
‹ Application of AIS
Train PhD students
Fund workshops/meetings
Coordinate and Disseminate UK based AIS research
(links to Europe)
AIS – October 2003 – 50
AIS Resources: Books
O
O
O
Artificial Immune Systems and Their
Applications by Dipankar Dasgupta (Editor)
Springer Verlag, January 1999.
Artificial Immune Systems: A New
Computational Intelligence Approach
by Leandro N. de Castro, Jonathan Timmis,
Springer Verlag, November 2002.
Immunocomputing: Principles and Applications
by Alexander O. Tarakanov, Victor A.
Skormin, Svetlana P. Sokolova, Springer
Verlag, April 2003.
AIS – October 2003 – 51