Download A Binary PSO

Binary Particle Swarm
Optimization (PSO)
The Flowchart of Binary PSO
Generate and initialize particles with
random position (X) and velocity (V)
Particle m
…..
Particle 1
Evaluate position (Fitness)
If fitness(X) >fitness(Pbest)
Pbest=X
Update
Position
If fitness(X) >fitness(Gbest)
Gbest=X
Update
velocity
Termination criterion is met? (e.g., Gbest=sufficient
good fitness or maximum generations)
No
Return the best solution
Yes
A Binary PSO
A Particle
Position vector, X i = ( xi1 , xi2 ,..., xin )
Î {0,1} .
i = 1, 2,.., m (m is the total number of particles).
d = 1, 2,.., n (n is the dimension of data).
d
xi
Velocity vector, Vi = (v1i , vi2 ,..., vin )
d
vi is limited by
Vmax
A Binary PSO
• A particle = a solution or a gene subset.
• If bit is 1, gene is selected.
If bit is 0, gene is unselected.
Particle position
Gene expression data
A subset of selected
genes by a particle
An example of a particle position representation in PSO for gene selection.
A Binary PSO
Updating the velocity of a particle:
d
vi
=
d
w * vi
+
d
c1 r1 * ( pbesti
Inertia
W = inertial weight.
vid = velocity for particle
i at dimension d.
d
xi
-
d
) + c 2 r2 * ( gbest -
Personal influence
Global influence
c1
r1
c2
r2
gbest
= acceleration constant.
= random value.
xid = position for particle i at
dimension d.
d
xi
= acceleration constant.
= random value.
= the global best position
of all particles.
pbest = the best previous
position of the ith particle.
Updating the position of a particle:
if Sig (vid ) > r3 ,
Sig (vid )
=
1
1+ e
- vid
xid = 1;
else
xid = 0;
)
r3 = random value.
An Improved Binary PSO (IPSO)
Idea
A new and simple rule
Action
Position update
Based on the whole of
bits of a particle
(Not based on single bit)
Particle velocity should
be positive
Velocity update
An Improved Binary PSO (IPSO)
Analyzing the sigmoid function:
The properties of the sigmoid function
Sig (vid )
Sig (vid ) Î [0,1]
Sig(vid ) =
1
-vid
1+ e
d
lim
Sig
(
v
i ) =1
d
vi ®¥
d
lim
Sig
(
v
i )=0
d
vi ®-¥
vid Î [0,Vmax ]
vid
d
1) if vi = 0
=> Sig (0) = 0.5 or P( xid = 1) = 0.5
d
2) if vi < 0
=> Sig (vid < 0) < 0.5 or P( xid = 1) < 0.5
d
3) if vi > 0
=> Sig (vid > 0) > 0.5
d
or P( xi = 1) > 0.5
An Improved Binary PSO (IPSO)
1) Modify the rule of position update:
a) The diagnostic goal = to develop a medical procedure based on the least
number of possible genes for accurate disease detection.
b) Many previous works (biological and computational researches) have
proved that a smaller number of genes can possible to produce higher
classification accuracy.
A new and simple rule of position update:
if
Sig (Vi ) > r3 ,
d
xi = 0;
r3
= random value.
The whole of bits of a particle
else
d
xi
= 1;
An Improved Binary PSO (IPSO)
2) A simple modification of the formula of velocity update
Vi = w *Vi + c1r1 * ( Pbesti - X i ) + c2 r2 *(Gbest - X i )
Vi Î [0,Vmax ]
The whole of bits of particles
Calculation for the distance of two position.
Example:
Gbest = [1011101001]
X i = [0100110101]
Step 1) Calculate the difference of bits for
= [1 - 1110 - 11 - 100].
a=4
Gbest and X i
b=3
Step 2) Calculate the distance between
Gbest - X i =| a - b |=| 4 - 3 |= 1
Gbest and X i
An Improved Binary PSO (IPSO)
3) A Fitness Function:
fitness ( X i ) = w1 ´ A( X i ) + ( w2 ( M - R( X i )) / M )
A( X i ) Î [ 0,1] is leave-one-out-cross-validation (LOOCV) accuracy on
the training set using the only genes in
Xi.
R ( X i ) is the number of selected genes in X i .
M is the total number of genes for each sample
w1
and
w2
are two priority weights corresponding to the importance of
accuracy and the number of selected genes, respectively.
w1 Î [ 0.1, 0.9]
w2 = 1 - w1.