Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
FUZZY LOGIC AND NEURAL NETWORKS Madan M. Gupta Intelligent systems Research Laboratory,University of Saskatchewan, Saskatoon, Canada, S7N OW0 Keywords: Fuzzy systems, neural systems, synaptic and somatic operations,fuzzy logic. ABSTRACT In this paper we give some basic principles of fuzzy neural computing using synaptic and somatic operations. We first briefly review the neural systems based upon conventional algebraic synaptic (confluence) and somatic (aggregation) operations. Then we provide a detailed neuronal morphology based upon fuzzy logic and its generalization in the form of Toperators. For such fuzzy logic based neurons. we then develop the learning and adaptation algorithm. 1. Introduction In this paper we elucidate the basic principle of neural computing - the neuronal morphology of the brain, and an artificial neural computing systems (1-31. In the conventional neural computing systems, the main mathematical tools employed are based upon algebraic and differential calculus. In the development of the cognitive machine we employ the learning and adaptation strengths of neural networks and the cognitive attributes found in fuzzy logic and its generalization in the form of T-operaton. Then. using the generalized form of fuzzy logic, we develop the fuzzy neural computing paradigms and extend the neural operations to the synaptic (confluence) and somatic (aggregation) operations. The theory presented in this brief exposition follows a pedagogical style. and starts from the very basic notion of neural computing and fuzzy logic learning to the advanced theory of cognitive neural computing. 2. The Neuron: Synaptic and Somatic Neural Operations Nature has developed a v u y complex neural smcture in most biological species [2-31. The biological neurons. over one hundred billion, in the central nervous systems (CNS)of humans play a very important role in various complex sensory. control and cognitive aspects of infomation processing and decision making [3-51. In neuronal information ptocessing then are a variety of complex mathematical operations and mapping functions involved which synergically act in a parallel-cascade structure forming a complex pattern of neuronal layers evolving into a sort of pyramidical pattern. The information flows from one neuronal layer to another in the forward direction with continuous feedback evolving into a dynamic p p i d i c a l structw. The pyramidical structure is in the sense of the extraction and convergence of information at each point in the forward direction. Biological neuronal morphology (suucture), indeed. provides a clue to as well as a challenge in the design of a realistic cognitive computing machine. In this paper we will provide only some basic insights into neuronal mathematical operations decimated into (i) synaptic (confluence) operations and (ii) somaric (aggregation and nonlinear mapping) operations. We will consider only a single neuron and study some of its intrinsic ProperneS. 636 92CH3179-9/92/$3.00 0 1992 IEEE From the neuro-biological, as well as the neuromathematical point of view. wc identify the two key neuronal elements in a biological neuron which are responsible for providing neuronal attributes such as learning, adaptation, knowledge (storage or memory to past experience), aggregation and nonlinear mapping operations on neuronal information [1-41. (i) Synapse: Synapse is a storage element of the past experience (knowledge); it learns from the neuronal environment and continuously adapts its strength. The past experience appears in the form of synaptic strength, what we call synaptic weight. There are over one hundred billion neurons in our central nervous systems and, on the average, there are over IO00 synapses per neuron. Mathematically, a synapse provides a confluence operation between the new neuronal inputs and the past experience. and sends signals to the dendrite - the input to the main body (soma) of the neuron. We will refer to the mathematical operation in synapse as 'synapticoperorion' or 'synupricconflrenceoperation'. (ii) Soma: Soma (neural cell) refers to h e main body of the neuron. It receives signals from a very few to over IO00 synapses through its dendrites and provides an aggregation operation. If the aggregated value of the dendritic inputs exceeds a certain threshold. it fires, providing an axonal (output) signal which is a sort of nonlinear function of the aggregated value. We will refer to the mathematical operations in soma as 'somurk operation'. In the somatic operation. one can identify three distinct mathematical operations: (i) aggregation, (ii) thresholding. and (iii) nonlinear transformation (mapping). Keeping in view the synaptic and somatic operations briefly described above. we depict a neuron as a mathematical processor which nccives an n dimensiond input signal vector x(t) E Rn and yields a scalar axonic output y(t) E R1. Mathematically, the neural process Ne can be depicted as a mapping operation from the neural input vector x(t) E Rn to be scalar n e m i output y(t) E R as Ne:x(t) E Rn -B y(t) e R1 (1) where. x(t) = [xl(t). x2(t) ... %(t) ...x&t)lT E Rn. Alternatively, we write y(t) =Ne [x(t) E Rn] E R' (2) Figure 2 shows a typical neuron depicting the mathematical operations of Equations (1) and (2). A biological neuron, as we have described above, provides two distinct mathematical operations distributed over the synupse (the junction point between an axon and the dendrite) and the somu. the main body of the neuron. We w ill call these two neuronal mathematical operations; (i) synapric operarions. and (ii) somaric operations. These two neuronal operations. which play two distinct mathematical functions in a biological neuron. are shown in Figure 3. From the biological - point of view, these two operations are physically separate. However, we will show shonly that from a mathematical point of view, certain attributes such as thresholding in soma can be transferred to the synaptic operation. Also, the mathematics of the synaptic and somatic operations can be customized. subject to certain rules of course, according to Ihe task in hand. Now. in the following paragraphs. we will provide a general mathematical description of synaptic and somatic operations. (i) Synaptic Operation: 'Ihe synapric weighting vector w(t) e Rn at the junction point between the neural input and the dendrite provides a storage (memory) to the past experience (knowledge-base). Thus. the synapric weight, wi(t). may be viewed as a representation of the past experience but which has the ability to adapt to the new experience (learning arrribure). The synaptic operation provides a confluence operation between the past experience w(t) E Rn and the neural inputs x(t) E Rn: Thus, the synaptic confluence operation. or just the synapric operation, assigns a relative weight (significance) to each incoming neural signal component xi(t) according to the past experience (knowledge) stored in wi(t). The weighted synaptic output (dcndrite signal) can be written as z$t) = wi(t) 0 xi(t), i = 1.2 ... n (3) where 0 is the synaptic confluence, opcration. The confluence operator, Q, as will be explained in the following sections, can be modeled by mathematical operations such as product and logical generalized AND operations. We define the synaptic operations for non-fuzzy and fuzzy signals as follows. (a) For non-fuzzy signals, xi(t). wi(t) E (--, -), and we define the confluence operation Q by the product operation. Thus. (4) zi(t) = wi(t) xi(t). i = 1,2 ... n. (b) For signals (fuzzy signals) bounded by the graded membership over the unit interval [0, 11, we define the confluence operation by the generalized AND operation. The generalized AND operation can be expressed using the notion of triangular noms (T-norms). Thus, we define the logical synaptic confluence operation as q t ) = wi(t) AND xi('), i = 1.2 ... n, E [O, 11. (5) For such signals (binary or fuzzy)* , the signal vector x(t) E Rn as well as the weighting vector w(t) E Rn each is defined over the unit hypercube [O, 11". n u(t) =+ i= 1 %(t) .or (6a) n u(t) = b Wi(t) Q Xi([) (6b) i= 1 The aggregation operation $[.I can be modeled by mathematical operations such as summation and logical generalized OR operations. We define this operation for nonfuzzy and fuzzy signals as follows: (i) For non-fuzzy signals xi(t), wi(t) E (--, OD). we define aggregation operation as u(t) n = .c U([) = 1= 1 qt)E R~ ,or .f1wi(t). xi([) E R1. (7a) (7P) I= One can view this aggregation operation as a linear mapping from n-dimensional dendritic inputs (zi(t)) to one dimensional space, Eqn. (7a). This somatic linear mapping operation can be combined with the synaptic confluence operation yielding a linear weighted mapping from ndimensional neural input x(t) e Rn to one dimensional space u(t) E R1, (7b). Alternatively, expressing this neuronal operation as a scalar product of two vectors, we write, u(t) = wT(t) . x(t) E R1 , where (74 ~ ( t=) [wl(t). w2(t) ... wn(t)lT E Rn = vector of synaptic weights, and ~ ( t= ) [xl(t). xz(t) ... x n ( t ) l T ~Rn = vector of neural inputs. Thus, the combined synaptic weighting and somatic aggregation operations provide a linear mapping of neural inputs x(t) E R" to u(t) E R ~ . (ii) For fuzzy signals bounded by the graded membership over the unit interval IO. 11. we define the aggregation operation by the generalized OR operation. The generalized OR operation can be expressed using the notion of triangular conorm (T-conorm). Thus, we define the logical somatic aggregation operation as, n u(t) = O R [~i(t)]E [0, 11 (84 i=l n = O R [wi(t) AND xi([)] E (0. I]. (8b) I= 1 Again, one can view this aggregation operation as a logical mapping from an n-dimensional neural input x(t) E [O. l]", to one dimensional space u(t) E [O. I] as shown in (8b). Expressing this neural operation as a scalar logical AND of two vectors, the neural input vector and the synaptic weighting vector, we write u(t) = wT(t) A N D x(t) (W where w(t). x(t) E [O. 11". and the vector ANDed operation is combined with individual ORed operation for i = 1.2, ... n. (ii) Soma& Operution: Figure 2. The somatic operation is carried out in the neural cell, the main body of the neuron, on the weighted neural input signals zi(t). i = 1. 2 ... n. and is a two step process as explained below. (a) Soma& Aggregation operation: Figure 2. The first somatic operation is the aggregation operation on the dendritic input signals zi(t). i = 1. 2 ... n, which essentially maps an nth order vector z(t) E R" into a scalar signal u(t) E Rn. For this somatic operation we introduce the generalized aggregation operation 0 . Thus, we have (ii) Nonlinear Somatic Openation with Thresholding: The biological neuronal processes generate some interesting mathematical mapping properties because of their nonlinear operations combined with a thresholding in the soma. Binary logic i s a subset of fuzzy logic. 637 As a matter of fact. if thc neuron would carry out only linear operations, the mathematical attractiveness as well as the robustness in many applications will disappear. The purpose of this section is to briefly explore these properties for their applications to neural computing systems. We will consider this important somatic operation again for two different situations, non-fuzzy signals and fuzzy signals. (i) For non-fuzzy signals, u(t) E ( --, -), we define the somatic nonlinear mapping operation on u(t) yielding a neural output y(t) as y(t) = f[u(t). wJ E R (9) where the nonlinear function fl.1 with thresholding wo i s shown in Figure 3. It should be noted, Figure 3. that the neural output y(t) is zero if the weighted aggregate U([) of the neural input signal x(t) E Rn is less than the threshold value wg. That is, the neuron will fire (will provide an output) only if the weighted aggregate of x(t) E Rn exceeds the threshold wo. If u(t) exceeds wo, the neural output y(t) increases monotonically with increasing U([) to a saturation value, say 1. Depending upon the mapping properties (shape) of the nonlinear function Q.1, the value of y(t) is distributed over the interval [0,1]. (E) For fuzzy signals with bounded graded membership u(t) over the unit interval [O, 11. we define the nonlinear mapping with thresholding wo E [O, 11 as follows: v(t) = u(t) OR wo, ~ ( t E) [O, 11 exclusively to fuzzy signals. Recalling Equation (9) and Figure 3 we define a new variable v(t) as v(t) = u(t) - WO (11) where u(t) is the weighted aggregate value of neural inputs defined in (71, and wo is the threshold (bias). Thus, if the weighted output U([) is less than wo, the neural output y(t) is zero. This implies that the neuron will fire only when the weighted aggregate value exceeds the threshold wg' Thus. we redefine y(t) as (12) YO) = flu(t), WO] = 4 w ) l and using Equation (7) and (1 1) we write the neural output y(t) as, n v(t) = # 0 Xi(t) (13a) i=O (13b) YO) = $[v(t)I where $[v(t)] is shown in Figure 4 with the threshold shifted to the origin, and 0 and 0 represent the generalized confluence and aggregation operations defined in (6). As a special case. representing the confluence operation by product, and aggregation operation by summation, we rewrite (12) and (13) as n v(t) = Wi(t) Xi(t) i=O (loa) T (13~) (13d) Y(t) =MW E R1 where $[.I is a sigmoidal nonlinear mapping function, and xa(t) is the augmented vectors defined as w,(t) = [WO, wl. w2 ... w2IT E Rn+' = augmented vector of synaptic weights including the threshold wo x,(t) = [x,. XI.xz ... x2IT E Rn+l, x0 = 1 = augmented vector of neural inputs, where xo = 1 accounts for the threshold (bias) term. A generalized neural model with threshold shifted to synapse is shown in Figure 4. 7. Conclusions In this paper we have developed a generalized neural model using the biological plausibility of synaptic and somatic operatlons. This generalized model can be specialized for nonfuzzy signals defined over (--, -) using the conventional mathematics as widely reported in the literature. However, the main interest of this paper was to develop a basic neuronal morphology using fuzzy logic operations rather than conventional mathematical operations. Reference 1. Amari. S.I.. (1977). "A Mathematical Approach to Neural Systems". in J. Metzler (Ed.), Systems Neuroscience, Academic Press,New York, pp. 67-1 17. 3. Generalized Mathematical Model of a Neuron In Section 2. we developed a model providing synaptic and somatic operations using a biological plausibility. In this section, we will generalize this mathematical model for realvalued non-fuzzy neural inputs and will devote the next section 2. 638 - - = wa (t) xa(t) E R1 ,and (lob) and. y(t) = vu([) where a is a positive constant. For 0 < a 5 1, the operation (lob) provides a contrast dilation to the membership v(t). whereas for a > 1 it provides a concentration operation. The contrast dilation, has the property of increasing the membership value, whereas the concentration, decreases it. The contrast intensification decreases the membership value for 0 e v 5 0.5. and increases it for 0.5 c v I 1. The blurring operation increases the membership value for 0 < v 10.5 and decreases it for 0.5 < v 5 1. Here. we have provided a few useful nonlinear operations on the graded membership v, the nonlinear operations can be expanded to some other nonlinear and or logical operations. Following the analogy of a biological neuron, we have divided the neural mathematical operations into two parts: (i) rhe synaptic confluence operarion, and (ii) the somaric aggregation and nonlinear mapping with rhreshulding (bias). We have defined these operations for signals over the interval (4.4.and for fuzzy signals with graded membership over the unit interval [O. 11. It is to be noted again that the neural processor modeled in this section may differ in many ways with its biological counterpart, however, they still retain many attributes for the confluence operation in the synapse and aggregation, thresholding and nonlinear operations of the somas. We will combine some of these operations, from the mathematical point of view. in order to produce a generalized mathematical framework for a neural processor [3]. Gupta. M.M.. [ 19921. "Introduction to Neural Computing Systems with Applications to Control and Vision". Class Notes, Intelligent Systems Research Laboratory, University of Saskatchewan. Canada. 3. Gupta, M.M..[ 19911, "Unctrrainty and Information The Emerging Paradigms", InternationalJ o u m l of New0 and Mass-Parallcl Computing and Information Systems, Vol. 2. pp. 65-70. 4. Gupta. M.M.and Qi. J. [lWl], "Theory of T-norms and Fuzzy Inference Methods". Fuzzy Sea and Systems North-Holland. V0140, pp. 43 1-450. 5. Zadeh, LA., (1965). "Fuzzy &U, Information and Control, Vol. 8. pp. 338-353. JwKtion points between pxw and dendrile~ \ Nr. Mapping Ncunl inputs x(1) E 'R Neural input fuld I II .-, U Soma N~urrlOUIPUI field / hiURON 1['Iz) Figure 2. Neuronal Synaptic and Somric mathematical operations. Figure 1. Neuron as a mathematical processor with the mapping function Ne: Ne: x(t) E Rn + y(t) E RI. u(i)e ill oulput Synapse .woj e R1 Figure 3. Somatic nonlinear mapping flu, w d with thresholding wg' f I Figure 4. Generalized neural model with augmented vectors n W & t h xa(t): v(t) =Jj Wi(t) Q Xi(t). y(t) = M W l . i d 639