* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download В работе [1] рассмотрена и проанализирована сеть
Survey
Document related concepts
Transcript
PARAMETRICAL NEURAL NETWORK B.V. Kryzhanovsky, L.B. Litinskii Institute of Optical Neural Technologies RAS, Moscow A. Fonarev Dep. of Engin. Science and Physics, The College of Staten Island, NY Abstract We develop a formalism allowing us to describe operating of a network based on the parametrical four-wave mixing process that is well-known in nonlinear optics. In the network the signals propagate in the form of quasimonochromatic pulses at q different frequencies. The retrieval properties of the network are investigated. It is shown that the storage capacity of such a network is higher compared with the Potts-glass neural network. The work is supported by RBRF (grants 02-0100457 and 01-01-00090) and the program “Intellectual Computer Systems” (the project 4.5). References [1] B.V. Kryzhanovsky, A.L. Mikaelian. On recognition ability of neuronet based on neurons with parametrical frequencies convertion. Dokladi RAS (2002), v. 383(3), pp. 1-4. [2] A. Fonarev, B.V. Kryzhanovsky et al. Parametric dynamic neural network recognition power. Optical Memory & Neural Networks (2001), v.10(4), pp. 31-48. [3] N. Bloembergen. Nonlinear optics. 1966. [4] N. Chernov. Ann. Math. Statistics (1952), v. 23. pp. 493-507. [5] I. Kanter. Potts-glass models of neural networks. Physical Review A (1988), v. 37(7), pp. 2739-2742. [6] D. Bolle, P. Dupont, J. Huyghebaert. Thermodynamic properties of the Q-state Potts-glass neural network. Physical Review A (1992), v. 45(6), pp. 4194-4197 INTRODUCTION The goal of this work is to analyze the properties of a network that is capable to hold and handle information encoded in the form of the phase-frequency modulation. Schematically the work of this network can be described as follows. The network consists of N connected neurons. The signals propagate along interconnections in the form of quasi-monochromatic pulses at q different frequencies k : xk ( t ) exp(i k t i k ) exp(i k t ), where { , ,..., }, {0, }. k 1 2 q k When propagating along the interconnections the signals transform due to the parametrical four-wave mixing processes of the form i - j + k { r }1q only when j = k, transmitting to the next neuron as a packet. The information about p patterns X (exp(i 1 t i 1 ), ..., exp(i N t i N ) ), 1,2,.., p, is stored in ( q q ) - interconnection matrices Tij ( i, j =1,…,N). The components of the patterns are preassigned quasi-monochromatic pulses. For example, the pattern is a colored picture at the screen, and the state of neurons encodes each pixel of the screen. A neuron has a complex structure. It is composed of: 1) a summator of input signals; q 2) a set of q ideal frequency filters { k }1 ; 3) a block comparing the amplitudes of the signals, and q 4) q generators of quasi-monochromatic signals { k }1 . The signals, which get the given neuron after transformation in interconnections, 1) are summed up; 2) the summarized signal passes through q parallel filters; 3) the output signals are compared with respect to their amplitudes; 4) the neuron activates the signal with the frequency corresponding to the maximal amplitude (and the relevant phase). THE VECTOR FORMALISM We consider a network consisting of N neurons. The states of the neurons q are given by vectors x j from a space R : x j 1, j 1,.., N ; x j x j ek , 0 ... (1) q e 1 R , k 1,.., q . k ... 0 With the aid of the q-dimensional vectors x j the problem can be formulated in terms of the vector spaces: x j x j ek ~ exp( i k t i k ) exp( i k ) exp( i k t ) exp( i k t ). The p N-dimensional patterns are Xμ (xμ1 , xμ2 ,..., xμN ), μ = 1,2,...,p. Their components x i are the vectors of the form (1). According to the generalized Hebb rule, the patterns are stored in the ( q q ) matrices Tij : Tij T ii p ( 1 , x j ) x i , i j; i, j = 1, 2,.., N. 0. q Then, if x R , we have p Tij x ( x , x j ) x i , 1 and the matrix elements are p ( kl ) Tij ( Tijel , ek ) ( el , x j )( x i , ek ), 1 k, l q. 1 (2) The dynamics of the network Let at the time point t the network is in the state X ( x1 , x 2 ,..., x N ). The local field acting on the i-th neuron is q N (i) h i Tij x j l el . j1 (3) l 1 (i) If in the expansion (3) k is the amplitude that is maximal in modulus, (i) (i) k max l , 1l q then in the next time point, t+1, the i-th neuron has the value (i) x i ( t 1) sgn( k ) ek . (4) "Spin" is oriented as close to the external field (3) as possible. The fixed points of the network are the local minima of the energy functional N E ( X ) p ( xi , x i ) ( x j , xj ). i j 1 In the case q=1, we have the ordinary Hopfield model: x j x j ek x j 1. When q>1, the model is similar to the Potts-glass neural network (I. Kanter, 1988; D. Bolle, 1992). RETRIEVAL PROPERTIES OF THE NETWORK Let p randomized patterns are stored in the network’s memory: X ( x 1 , x 2 ,..., x N ), 1,2,..., p. Let the network starts from a distorted m-th pattern: ~ X m ( a1b1 x m1 , a 2 b2 x m 2 ,..., a N bN x mN ), where with a probability a 1, ai = ; a is the probability of phase failure. 1, with the probability 1 - a with a probability b x xmi , bi xmi μi ; b is the probability of frequency failure. x , with the probability 1b mi Using the Chebyshev-Chernov method it can be shown that the probability of the error in recognition of the pattern X m is equal to Nq 2 2 2 P N exp(1 2a ) (1 b ) . 2 p If N , always the error probability vanishes, when the number of the patterns p is less than 2 N (1 2a ) 2 2 p q (1 b ) . 2 ln N p is an asymptotically possible value of the storage capacity of the parametrical neural network (PNN). 2 2 It differs by the factor q (1 b ) from the analogous characteristic of the Hopfield model: 2 2 pPNN q (1 b ) pHopfield . Example 1 If q=10, and the probability of the frequency failure is b 1 ( the noise is 3 equal to 33%), then 2 2 q (1 b ) 100 4 44 pPNN is greater than pHopfield in 44 times. 9 Example 2 If q= 10, the number of neurons N= 100, the number of the patterns p= 200, and the frequencies of the pattern are 40% noisy, then the probability of restoring of the pattern is equal to 0.98. FIXED POINTS OF PNN 1. For any q 1 , the only fixed points of a network constructed with the aid of two patterns X 1 and X 2 are these patterns themselves (and their negatives X 1 , X 2 ). 2. Let us examine a case, when q = 2 and the vectors x j are without the signs 1 (the signals are without the phase failure): 2 x j { ek }k 1 , j 1, 2,..., N . As the neurons possess only one of the two values e1 or e 2 , formally this case is alike the Hopfield model. Note, here a “contrast” ~ configuration X corresponds to every configuration X , but not its ~ negative - X. The components ~x j of the contrast configuration X take the values that are alternative to x j . This system has some nontrivial properties distinguishing it from the Hopfield model . For example, if a configuration X is a fixed point, then its contrast ~ configuration X is not a fixed point. The other difference: a network constructed with the aid of three patterns always has them as fixed points. Restoration of the pattern with the frequency noise. N=100, p=200, q=22. The pattern is a picture of a dog. The frequency noise is 80% (a=0, b=0.8). The gray squares are noisy pixels. The states of the network after 50 and 100 steps are shown. t=0 t=50 t=100 Restoration of the pattern with the phase noise. N=100, p=200, q=15. The pattern is a picture of a dog. The phase noise is 20% (a=0.2, b=0). The states of the network after 50 and 100 steps are shown. t=0 t = 50 t = 100 POTTS-GLASS NEURAL NETWORK The properties of our model are close to the properties of PGNN model, which was suggested in the end of 80-th and examined in 90-th. PGNN is related to the Potts model of a magnetic solid , as well as the Hopfield model is related to the Ising model. In PGNN neurons take q different values, which are represented with the aid of q special vectors k : 1 ... q q 1 j 1,.., N x j { k }k 1 , where k ~ q 1 R , k 1,.., q . ... 1 The vectors k are nonorthogonal and they are linearly dependent: q k 0. k 1 For the rest, the PGNN-model is similar to the aforesaid vector formalism. In our PNN-model the signals are transformed due to action of elementary matrices A kl ( , el ) ek , k, l 1,..,q . In this case the filtration of the signals occurs: if the frequency of the signal m is not equal to the frequency l , the signal does not transmit through interconnection: A kl em lm ek . In the PGNN-model the matrices B kl ( , l ) k , k, l 1,..,q , play the same role, however because of the nonorthogonality of the vectors k , the filtration does not occur: q lm 1 B kl m k . q In the PGNN-model the asymptotically possible value of the storage capacity is twice less than in our PNN-model: p pPGNN PNN . 2