Download В работе [1] рассмотрена и проанализирована сеть

PARAMETRICAL NEURAL NETWORK
B.V. Kryzhanovsky, L.B. Litinskii
Institute of Optical Neural Technologies RAS, Moscow
A. Fonarev
Dep. of Engin. Science and Physics, The College of Staten Island, NY
Abstract
We develop a formalism allowing us to describe operating
of a network based on the parametrical four-wave mixing
process that is well-known in nonlinear optics. In the
network the signals propagate in the form of quasimonochromatic pulses at q different frequencies. The
retrieval properties of the network are investigated. It is
shown that the storage capacity of such a network is higher
compared with the Potts-glass neural network.
The work is supported by RBRF (grants 02-0100457 and 01-01-00090) and
the program “Intellectual Computer Systems” (the project 4.5).
References
[1] B.V. Kryzhanovsky, A.L. Mikaelian.
On recognition ability of neuronet based on neurons with parametrical
frequencies convertion. Dokladi RAS (2002), v. 383(3), pp. 1-4.
[2] A. Fonarev, B.V. Kryzhanovsky et al.
Parametric dynamic neural network recognition power.
Optical Memory & Neural Networks (2001), v.10(4), pp. 31-48.
[3] N. Bloembergen. Nonlinear optics. 1966.
[4] N. Chernov. Ann. Math. Statistics (1952), v. 23. pp. 493-507.
[5] I. Kanter. Potts-glass models of neural networks.
Physical Review A (1988), v. 37(7), pp. 2739-2742.
[6] D. Bolle, P. Dupont, J. Huyghebaert.
Thermodynamic properties of the Q-state Potts-glass neural network.
Physical Review A (1992), v. 45(6), pp. 4194-4197
INTRODUCTION
The goal of this work is to analyze the properties of a network that is capable
to hold and handle information encoded in the form of the phase-frequency
modulation. Schematically the work of this network can be described as
follows.
The network consists of N connected neurons. The signals propagate along
interconnections in the form of quasi-monochromatic pulses at q different
frequencies  k :

 xk ( t )  exp(i k t  i k )   exp(i k t ),
 where   { ,  ,..., },   {0, }.

k
1
2
q
k

When propagating along the interconnections the signals transform due to the
parametrical four-wave mixing processes of the form
 i -  j +  k  { r }1q only when j = k,
transmitting to the next neuron as a packet.
The information about p patterns
X  (exp(i 1 t  i 1 ), ..., exp(i N t  i N ) ),   1,2,.., p,
is stored in ( q  q ) - interconnection matrices Tij ( i, j =1,…,N). The
components of the patterns are preassigned quasi-monochromatic pulses.
For example, the pattern is a colored picture at the screen, and the state of
neurons encodes each pixel of the screen.
A neuron has a complex structure. It is composed of:
1) a summator of input signals;
q
2) a set of q ideal frequency filters { k }1 ;
3) a block comparing the amplitudes of the signals, and
q
4) q generators of quasi-monochromatic signals { k }1 .
The signals, which get the given neuron after transformation in
interconnections,
1) are summed up;
2) the summarized signal passes through q parallel filters;
3) the output signals are compared with respect to their amplitudes;
4) the neuron activates the signal with the frequency corresponding to
the maximal amplitude (and the relevant phase).
THE VECTOR FORMALISM
We consider a network consisting of N neurons. The states of the neurons

q
are given by vectors x j from a space R :


x j  1, j  1,.., N ;
 x j  x j  ek ,

0

 

 ... 
(1)

q



e

1

R
,
k

1,..,
q
.

k
 ... 

 

0


With the aid of the q-dimensional vectors x j the problem can be formulated
in terms of the vector spaces:


x j  x j  ek ~ exp( i k t  i k )  exp( i k )  exp( i k t )   exp( i k t ).
The p N-dimensional patterns are
Xμ  (xμ1 , xμ2 ,..., xμN ), μ = 1,2,...,p.

Their components x i are the vectors of the form (1).
According to the generalized Hebb rule, the patterns are stored in the
( q  q )  matrices Tij :

Tij 

T 
 ii
p
(
 1
 
, x j ) x i , i  j;
i, j = 1, 2,.., N.
0.

q
Then, if x  R , we have
 p   
Tij  x   ( x , x j ) x i ,
 1
and the matrix elements are
p
 
 
( kl )
 
Tij  ( Tijel , ek )   ( el , x j )( x i , ek ), 1  k, l  q.
 1
(2)
The dynamics of the network
Let at the time point t the network is in the state
 

X  ( x1 , x 2 ,..., x N ).
The local field acting on the i-th neuron is
q
N


(i) 
h i   Tij  x j    l  el .
j1
(3)
l 1
(i)
If in the expansion (3)  k is the amplitude that is maximal in modulus,
(i)
(i)
 k  max  l ,
1l q
then in the next time point, t+1, the i-th neuron has the value


(i)
x i ( t  1)  sgn( k )  ek .
(4)
"Spin" is oriented as close to the external field (3) as possible.
The fixed points of the network are the local minima of the energy functional
N
E ( X )  
p


 
 ( xi , x i )  ( x j , xj ).
i  j  1
In the case q=1, we have the ordinary Hopfield model:


x j  x j  ek

x j  1.
When q>1, the model is similar to the Potts-glass neural network
(I. Kanter, 1988; D. Bolle, 1992).
RETRIEVAL PROPERTIES OF THE NETWORK
Let p randomized patterns are stored in the network’s memory:
 

X   ( x 1 , x  2 ,..., x N ),   1,2,..., p.
Let the network starts from a distorted m-th pattern:



~
X m  ( a1b1 x m1 , a 2 b2 x m 2 ,..., a N bN x mN ),
where
with a probability a
1,
ai = 
; a is the probability of phase failure.
 1, with the probability 1 - a
with a probability b
x  xmi ,
bi xmi   μi
; b is the probability of frequency failure.
x
,
with
the
probability
1b

mi
Using the Chebyshev-Chernov method it can be shown that the probability of
the error in recognition of the pattern X m is equal to
 Nq 2

2
2
P  N  exp(1  2a ) (1  b ) .
 2 p

If N   , always the error probability vanishes, when the number of the
patterns p is less than
2
N (1  2a )
2
2
p
 q (1  b ) .
2 ln N
p is an asymptotically possible value of the storage capacity of the
parametrical neural network (PNN).
2
2
It differs by the factor q (1  b ) from the analogous characteristic of the
Hopfield model:
2
2
pPNN  q (1  b )  pHopfield .
Example 1
If q=10, and the probability of the frequency failure is b 
1
( the noise is
3
equal to 33%), then
2
2
q (1  b ) 
100  4
 44  pPNN is greater than pHopfield in 44 times.
9
Example 2
If q= 10, the number of neurons N= 100, the number of the patterns p= 200,
and the frequencies of the pattern are 40% noisy, then
the probability of restoring of the pattern is equal to 0.98.
FIXED POINTS OF PNN

1.
For any q  1 , the only fixed points of a network constructed with the
aid of two patterns X 1 and X 2 are these patterns themselves
(and their negatives  X 1 ,  X 2 ).

2.

Let us examine a case, when q = 2 and the vectors x j are without the
signs  1 (the signals are without the phase failure):
 2

x j  { ek }k 1 , j  1, 2,..., N .


As the neurons possess only one of the two values e1 or e 2 , formally
this case is alike the Hopfield model. Note, here a “contrast”
~
configuration X corresponds to every configuration X , but not its
~
negative - X. The components ~x j of the contrast configuration X take

the values that are alternative to x j .
This system has some nontrivial properties distinguishing it from the
Hopfield model .
For example, if a configuration X is a fixed point, then its contrast
~
configuration X is not a fixed point.
The other difference: a network constructed with the aid of three
patterns always has them as fixed points.
Restoration of the pattern with the frequency noise.
N=100, p=200, q=22. The pattern is a picture of a dog.
The frequency noise is 80% (a=0, b=0.8).
The gray squares are noisy pixels.
The states of the network after 50 and 100 steps are shown.
t=0
t=50
t=100
Restoration of the pattern with the phase noise.
N=100, p=200, q=15. The pattern is a picture of a dog.
The phase noise is 20% (a=0.2, b=0).
The states of the network after 50 and 100 steps are shown.
t=0
t = 50
t = 100
POTTS-GLASS NEURAL NETWORK
The properties of our model are close to the properties of PGNN model, which
was suggested in the end of 80-th and examined in 90-th.
PGNN is related to the Potts model of a magnetic solid , as well as the
Hopfield model is related to the Ising model.
In PGNN neurons take
 q different values, which are represented with the aid
of q special vectors  k :
 1 


... 




q
q 1
 j  1,.., N x j  {  k }k 1 , where  k ~  q  1   R , k  1,.., q .
 ... 



1



The vectors  k are nonorthogonal and they are linearly dependent:
q 
  k  0.
k 1
For the rest, the PGNN-model is similar to the aforesaid vector formalism.
In our PNN-model the signals are transformed due to action of elementary
matrices
 
A kl  ( , el )  ek , k, l  1,..,q .
In this case the filtration of the signals occurs: if the frequency of the signal
 m is not equal to the frequency  l , the signal does not transmit through


interconnection: A kl em   lm  ek .
In the PGNN-model the matrices  
B kl  ( ,  l )   k , k, l  1,..,q ,
play
the same role, however because of the nonorthogonality of the vectors

 k , the filtration does not occur:

q   lm  1 
B kl m 
 k .
q
In the PGNN-model the asymptotically possible value of the storage capacity
is twice less than in our PNN-model:
p
pPGNN  PNN .
2