Download Fuzzy logic and neural networks

FUZZY LOGIC AND NEURAL NETWORKS
Madan M. Gupta
Intelligent systems Research Laboratory,University of Saskatchewan, Saskatoon,
Canada, S7N OW0
Keywords: Fuzzy systems, neural systems, synaptic and somatic operations,fuzzy logic.
ABSTRACT
In this paper we give some basic principles of fuzzy
neural computing using synaptic and somatic operations. We
first briefly review the neural systems based upon conventional
algebraic synaptic (confluence) and somatic (aggregation)
operations. Then we provide a detailed neuronal morphology
based upon fuzzy logic and its generalization in the form of Toperators. For such fuzzy logic based neurons. we then
develop the learning and adaptation algorithm.
1. Introduction
In this paper we elucidate the basic principle of neural
computing - the neuronal morphology of the brain, and an
artificial neural computing systems (1-31. In the conventional
neural computing systems, the main mathematical tools
employed are based upon algebraic and differential calculus.
In the development of the cognitive machine we employ the
learning and adaptation strengths of neural networks and the
cognitive attributes found in fuzzy logic and its generalization
in the form of T-operaton. Then. using the generalized form of
fuzzy logic, we develop the fuzzy neural computing paradigms
and extend the neural operations to the synaptic (confluence)
and somatic (aggregation) operations.
The theory presented in this brief exposition follows a
pedagogical style. and starts from the very basic notion of
neural computing and fuzzy logic learning to the advanced
theory of cognitive neural computing.
2. The Neuron: Synaptic and Somatic Neural Operations
Nature has developed a v u y complex neural smcture
in most biological species [2-31. The biological neurons. over
one hundred billion, in the central nervous systems (CNS)of
humans play a very important role in various complex sensory.
control and cognitive aspects of infomation processing and
decision making [3-51. In neuronal information ptocessing
then are a variety of complex mathematical operations and
mapping functions involved which synergically act in a
parallel-cascade structure forming a complex pattern of
neuronal layers evolving into a sort of pyramidical pattern.
The information flows from one neuronal layer to another in
the forward direction with continuous feedback evolving into a
dynamic p p i d i c a l structw. The pyramidical structure is in
the sense of the extraction and convergence of information at
each point in the forward direction. Biological neuronal
morphology (suucture), indeed. provides a clue to as well as a
challenge in the design of a realistic cognitive computing
machine. In this paper we will provide only some basic
insights into neuronal mathematical operations decimated into
(i) synaptic (confluence) operations and (ii) somaric
(aggregation and nonlinear mapping) operations. We will
consider only a single neuron and study some of its intrinsic
ProperneS.
636
92CH3179-9/92/$3.00 0 1992 IEEE
From the neuro-biological, as well as the neuromathematical point of view. wc identify the two key neuronal
elements in a biological neuron which are responsible for
providing neuronal attributes such as learning, adaptation,
knowledge (storage or memory to past experience),
aggregation and nonlinear mapping operations on neuronal
information [1-41.
(i) Synapse: Synapse is a storage element of the past
experience (knowledge); it learns from the neuronal
environment and continuously adapts its strength. The past
experience appears in the form of synaptic strength, what we
call synaptic weight. There are over one hundred billion
neurons in our central nervous systems and, on the average,
there are over IO00 synapses per neuron. Mathematically, a
synapse provides a confluence operation between the new
neuronal inputs and the past experience. and sends signals to
the dendrite - the input to the main body (soma) of the neuron.
We will refer to the mathematical operation in synapse as
'synapticoperorion' or 'synupricconflrenceoperation'.
(ii) Soma: Soma (neural cell) refers to h e main body of
the neuron. It receives signals from a very few to over IO00
synapses through its dendrites and provides an aggregation
operation. If the aggregated value of the dendritic inputs
exceeds a certain threshold. it fires, providing an axonal
(output) signal which is a sort of nonlinear function of the
aggregated value. We will refer to the mathematical operations
in soma as 'somurk operation'. In the somatic operation. one
can identify three distinct mathematical operations: (i)
aggregation, (ii) thresholding. and (iii) nonlinear
transformation (mapping).
Keeping in view the synaptic and somatic operations
briefly described above. we depict a neuron as a mathematical
processor which nccives an n dimensiond input signal vector
x(t) E Rn and yields a scalar axonic output y(t) E R1.
Mathematically, the neural process Ne can be depicted as a
mapping operation from the neural input vector x(t) E Rn to
be scalar n e m i output y(t) E R as
Ne:x(t) E Rn -B y(t) e R1
(1)
where. x(t) = [xl(t). x2(t) ... %(t) ...x&t)lT E Rn.
Alternatively, we write
y(t) =Ne [x(t) E Rn] E R'
(2)
Figure 2 shows a typical neuron depicting the
mathematical operations of Equations (1) and (2).
A biological neuron, as we have described above,
provides two distinct mathematical operations distributed over
the synupse (the junction point between an axon and the
dendrite) and the somu. the main body of the neuron. We w
ill
call these two neuronal mathematical operations; (i) synapric
operarions. and (ii) somaric operations. These two neuronal
operations. which play two distinct mathematical functions in a
biological neuron. are shown in Figure 3. From the biological
-
point of view, these two operations are physically separate.
However, we will show shonly that from a mathematical point
of view, certain attributes such as thresholding in soma can be
transferred to the synaptic operation. Also, the mathematics of
the synaptic and somatic operations can be customized. subject
to certain rules of course, according to Ihe task in hand. Now.
in the following paragraphs. we will provide a general
mathematical description of synaptic and somatic operations.
(i) Synaptic Operation: 'Ihe synapric weighting vector
w(t) e Rn at the junction point between the neural input and
the dendrite provides a storage (memory) to the past
experience (knowledge-base). Thus. the synapric weight, wi(t).
may be viewed as a representation of the past experience but
which has the ability to adapt to the new experience (learning
arrribure). The synaptic operation provides a confluence
operation between the past experience w(t) E Rn and the neural
inputs x(t) E Rn: Thus, the synaptic confluence operation. or
just the synapric operation, assigns a relative weight
(significance) to each incoming neural signal component xi(t)
according to the past experience (knowledge) stored in wi(t).
The weighted synaptic output (dcndrite signal) can be written
as
z$t) = wi(t) 0 xi(t), i = 1.2 ... n
(3)
where 0 is the synaptic confluence, opcration.
The confluence operator, Q, as will be explained in the
following sections, can be modeled by mathematical operations
such as product and logical generalized AND operations. We
define the synaptic operations for non-fuzzy and fuzzy signals
as follows.
(a) For non-fuzzy signals, xi(t). wi(t) E (--, -), and we
define the confluence operation Q by the product operation.
Thus.
(4)
zi(t) = wi(t) xi(t). i = 1,2 ... n.
(b) For signals (fuzzy signals) bounded by the graded
membership over the unit interval [0, 11, we define the
confluence operation by the generalized AND operation. The
generalized AND operation can be expressed using the notion
of triangular noms (T-norms). Thus, we define the logical
synaptic confluence operation as
q t ) = wi(t) AND xi('), i = 1.2 ... n, E [O, 11. (5)
For such signals (binary or fuzzy)* , the signal vector
x(t) E Rn as well as the weighting vector w(t) E Rn each is
defined over the unit hypercube [O, 11".
n
u(t) =+
i= 1
%(t)
.or
(6a)
n
u(t) =
b
Wi(t) Q Xi([)
(6b)
i= 1
The aggregation operation $[.I can be modeled by
mathematical operations such as summation and logical
generalized OR operations. We define this operation for nonfuzzy and fuzzy signals as follows:
(i) For non-fuzzy signals
xi(t), wi(t) E (--, OD). we define aggregation operation as
u(t)
n
= .c
U([)
=
1= 1
qt)E
R~ ,or
.f1wi(t). xi([)
E
R1.
(7a)
(7P)
I=
One can view this aggregation operation as a linear
mapping from n-dimensional dendritic inputs (zi(t)) to one
dimensional space, Eqn. (7a). This somatic linear mapping
operation can be combined with the synaptic confluence
operation yielding a linear weighted mapping from ndimensional neural input x(t) e Rn to one dimensional space
u(t) E R1, (7b). Alternatively, expressing this neuronal
operation as a scalar product of two vectors, we write,
u(t) = wT(t) . x(t) E R1
, where
(74
~ ( t=) [wl(t). w2(t) ... wn(t)lT E Rn
= vector of synaptic weights, and
~ ( t=
) [xl(t). xz(t) ... x n ( t ) l T ~Rn
= vector of neural inputs.
Thus, the combined synaptic weighting and somatic
aggregation operations provide a linear mapping of neural
inputs x(t) E R" to u(t) E R ~ .
(ii) For fuzzy signals bounded by the graded
membership over the unit interval IO. 11. we define the
aggregation operation by the generalized OR operation. The
generalized OR operation can be expressed using the notion of
triangular conorm (T-conorm). Thus, we define the logical
somatic aggregation operation as,
n
u(t) = O R [~i(t)]E [0, 11
(84
i=l
n
= O R [wi(t) AND xi([)] E (0. I].
(8b)
I= 1
Again, one can view this aggregation operation as a
logical mapping from an n-dimensional neural input x(t) E [O.
l]", to one dimensional space u(t) E [O. I] as shown in (8b).
Expressing this neural operation as a scalar logical
AND of two vectors, the neural input vector and the synaptic
weighting vector, we write
u(t) = wT(t) A N D x(t)
(W
where w(t). x(t) E [O. 11". and the vector ANDed operation is
combined with individual ORed operation for i = 1.2, ... n.
(ii) Soma& Operution: Figure 2.
The somatic operation is carried out in the neural cell,
the main body of the neuron, on the weighted neural input
signals zi(t). i = 1. 2 ... n. and is a two step process as
explained below.
(a) Soma& Aggregation operation: Figure 2.
The first somatic operation is the aggregation operation
on the dendritic input signals zi(t). i = 1. 2 ... n, which
essentially maps an nth order vector z(t) E R" into a scalar
signal u(t) E Rn.
For this somatic operation we introduce the generalized
aggregation operation 0 . Thus, we have
(ii) Nonlinear Somatic Openation with Thresholding:
The biological neuronal processes generate some
interesting mathematical mapping properties because of their
nonlinear operations combined with a thresholding in the soma.
Binary logic i s a subset of fuzzy logic.
637
As a matter of fact. if thc neuron would carry out only linear
operations, the mathematical attractiveness as well as the
robustness in many applications will disappear. The purpose of
this section is to briefly explore these properties for their
applications to neural computing systems.
We will consider this important somatic operation again
for two different situations, non-fuzzy signals and fuzzy
signals.
(i) For non-fuzzy signals, u(t) E ( --, -), we define the
somatic nonlinear mapping operation on u(t) yielding a neural
output y(t) as
y(t) = f[u(t). wJ E R
(9)
where the nonlinear function fl.1 with thresholding wo i s
shown in Figure 3. It should be noted, Figure 3. that the neural
output y(t) is zero if the weighted aggregate U([) of the neural
input signal x(t) E Rn is less than the threshold value wg. That
is, the neuron will fire (will provide an output) only if the
weighted aggregate of x(t) E Rn exceeds the threshold wo. If
u(t) exceeds wo, the neural output y(t) increases monotonically
with increasing U([) to a saturation value, say 1. Depending
upon the mapping properties (shape) of the nonlinear function
Q.1, the value of y(t) is distributed over the interval [0,1].
(E) For fuzzy signals with bounded graded membership
u(t) over the unit interval [O, 11. we define the nonlinear
mapping with thresholding wo E [O, 11 as follows:
v(t) = u(t) OR wo, ~ ( t E) [O, 11
exclusively to fuzzy signals.
Recalling Equation (9) and Figure 3 we define a new
variable v(t) as
v(t) = u(t) - WO
(11)
where u(t) is the weighted aggregate value of neural inputs
defined in (71, and wo is the threshold (bias). Thus, if the
weighted output U([) is less than wo, the neural output y(t) is
zero. This implies that the neuron will fire only when the
weighted aggregate value exceeds the threshold wg' Thus. we
redefine y(t) as
(12)
YO) = flu(t), WO] = 4 w ) l
and using Equation (7) and (1 1) we write the neural output y(t)
as,
n
v(t) = #
0 Xi(t)
(13a)
i=O
(13b)
YO) = $[v(t)I
where $[v(t)] is shown in Figure 4 with the threshold shifted to
the origin, and 0 and 0 represent the generalized confluence
and aggregation operations defined in (6).
As a special case. representing the confluence operation
by product, and aggregation operation by summation, we
rewrite (12) and (13) as
n
v(t) =
Wi(t) Xi(t)
i=O
(loa)
T
(13~)
(13d)
Y(t) =MW E R1
where $[.I is a sigmoidal nonlinear mapping function, and xa(t)
is the augmented vectors defined as
w,(t) = [WO, wl. w2 ... w2IT E Rn+'
= augmented vector of synaptic weights including
the threshold wo
x,(t) = [x,. XI.xz ... x2IT E Rn+l, x0 = 1
= augmented vector of neural inputs, where xo = 1
accounts for the threshold (bias) term.
A generalized neural model with threshold shifted to synapse is
shown in Figure 4.
7. Conclusions
In this paper we have developed a generalized neural
model using the biological plausibility of synaptic and somatic
operatlons. This generalized model can be specialized for
nonfuzzy signals defined over (--, -) using the conventional
mathematics as widely reported in the literature. However, the
main interest of this paper was to develop a basic neuronal
morphology using fuzzy logic operations rather than
conventional mathematical operations.
Reference
1.
Amari. S.I.. (1977). "A Mathematical Approach to
Neural Systems". in J. Metzler (Ed.), Systems
Neuroscience, Academic Press,New York, pp. 67-1 17.
3. Generalized Mathematical Model of a Neuron
In Section 2. we developed a model providing synaptic
and somatic operations using a biological plausibility. In this
section, we will generalize this mathematical model for realvalued non-fuzzy neural inputs and will devote the next section
2.
638
-
-
= wa (t) xa(t) E R1 ,and
(lob)
and. y(t) = vu([)
where a is a positive constant. For 0 < a 5 1, the operation
(lob) provides a contrast dilation to the membership v(t).
whereas for a > 1 it provides a concentration operation.
The contrast dilation, has the property of increasing the
membership value, whereas the concentration, decreases it.
The contrast intensification decreases the membership value for
0 e v 5 0.5. and increases it for 0.5 c v I 1. The blurring
operation increases the membership value for 0 < v 10.5 and
decreases it for 0.5 < v 5 1.
Here. we have provided a few useful nonlinear
operations on the graded membership v, the nonlinear
operations can be expanded to some other nonlinear and or
logical operations.
Following the analogy of a biological neuron, we have
divided the neural mathematical operations into two parts: (i)
rhe synaptic confluence operarion, and (ii) the somaric
aggregation and nonlinear mapping with rhreshulding (bias).
We have defined these operations for signals over the interval
(4.4.and for fuzzy signals with graded membership over the
unit interval [O. 11. It is to be noted again that the neural
processor modeled in this section may differ in many ways
with its biological counterpart, however, they still retain many
attributes for the confluence operation in the synapse and
aggregation, thresholding and nonlinear operations of the
somas. We will combine some of these operations, from the
mathematical point of view. in order to produce a generalized
mathematical framework for a neural processor [3].
Gupta. M.M.. [ 19921. "Introduction to Neural
Computing Systems with Applications to Control and
Vision". Class Notes, Intelligent Systems Research
Laboratory, University of Saskatchewan. Canada.
3.
Gupta, M.M..[ 19911, "Unctrrainty and Information The
Emerging Paradigms", InternationalJ o u m l of New0
and Mass-Parallcl Computing and Information
Systems, Vol. 2. pp. 65-70.
4.
Gupta. M.M.and Qi.
J. [lWl], "Theory of T-norms and
Fuzzy Inference Methods". Fuzzy Sea and Systems
North-Holland. V0140, pp. 43 1-450.
5.
Zadeh, LA., (1965). "Fuzzy &U, Information and
Control, Vol. 8. pp. 338-353.
JwKtion points between pxw and dendrile~
\
Nr. Mapping
Ncunl inputs
x(1) E
'R
Neural input fuld
I
II
.-,
U
Soma
N~urrlOUIPUI field
/
hiURON
1['Iz)
Figure 2. Neuronal Synaptic and Somric mathematical
operations.
Figure 1. Neuron as a mathematical processor with the
mapping function Ne: Ne: x(t) E Rn + y(t) E RI.
u(i)e
ill oulput
Synapse
.woj e
R1
Figure 3. Somatic nonlinear mapping flu, w d with
thresholding wg'
f
I
Figure 4. Generalized neural model with augmented vectors
n
W & t h xa(t): v(t) =Jj Wi(t) Q Xi(t).
y(t) = M W l .
i d
639