Download GRANULAR COMPUTING AND ITS APPLICATION IN RBF NEURAL

Journal of Information, Control and Management Systems, Vol. X, (200X), No.X
7
GRANULAR COMPUTING AND ITS APPLICATION IN RBF
NEURAL NETWORK WITH CLOUD ACTIVATION
FUNCTION
Zuzana MEČIAROVÁ
University of Žilina, Faculty of Management Science and Informatics,
Slovak Republic
e-mail: [email protected]
Abstract
This paper discusses using of granular computing components for representing
neurons in hidden layer of RBF neural network with cloud activation function.
After providing a brief introduction to granular computing, the paper describes
basic components of granular computing. Then the process of granulation is
analyzed. Finally, the application of granular computing in RBF neural network
is introduced.
Keywords: cloud activation function, granulation, granule, granular computing,
granulated level, granular structure, hierarchy, RBF neural network,
1
INTRODUCTION TO GRANULAR COMPUTING
Although granular computing is a separate research field, the basic idea of
granular computing, i.e. problem solving with different granularities, have been
explored in many fields such as artificial intelligence, interval analysis, quantization,
Dempster-Shafer theory of belief functions, rough set theory, divide and conquer,
cluster analysis, machine learning, databases and many others [19]. In the past few
decades we have witnessed rapid development of granular computing. Three branches
of science, namely fuzzy theory, artificial intelligence and rough set theory contributed
significantly to the study of granular computing.
In 1979, Zadeh first introduced and discussed the notion of information
granulation and suggested that fuzzy set theory may find potential applications in this
respect, which pioneers the explicit study of granular computing [20]. A general
framework of granular computing was given in a paper [19] by Zadeh based on fuzzy
set theory. According to Zadeh, granules are constructed and defined based on the
concept of generalized constraints. Relationships between granules are represented in
terms of fuzzy graphs or fuzzy if-then rules. The associated computation method is
known as computing with words.
2 number of the even pages is at left side
Title of the paper, size 10
The popularity of granular computing is also due to the rough set theory. Rough
set models enable us to precisely define and analyze many notions of granular
computing. In 1982, Pawlak proposed the rough sets to approximate a crisp set [13]
and investigated the granularity of knowledge from the point of view of rough set
theory [12].
The ideas of granular computing have been investigated in artificial intelligence
through the notions of granularity and abstraction. Hobbs proposed a theory of
granularity as the base of knowledge representation, abstraction, heuristic search and
reasoning in 1985. In this theory, we look at the world under various grain seizes and
abstract from it only those things that serve our present interests. Our ability to
conceptualize the world at different granularities and switch among these granularities
is fundamental to our intelligence and flexibility. It enables us to map the complexities
of the world around us into simpler theories that are computationally tractable to
reason in [3]. In 1992, Giunchiglia and Walsh presented a theory of abstraction to
improve the conceptualization of granularities. According to Giunchiglia and Walsh
abstraction is the process which allows people to consider what is relevant and to
forget a lot of irrelevant details which would get in the way of what they are tying to do
[1].
The term “granular computing” was first suggested by T.Y. Lin [6] in 1997 to
label this growing research field. Yao investigated the trinity model of granular
computing from three perspectives: philosophy, methodology and computation [18]. In
the past decade, different granular computing models have been conducted in various
aspects and applied in various application domains including machine learning, data
mining, bioinformatics, e-business, network security, wireless mobile computing etc..
The essence of these models has been addressed by researches to build efficient
computational algorithms for handling huge amounts of data, information and
knowledge. The objectives of these computation models are computer-centered and
mainly concern the efficiency, effectiveness and robustness of using granules such as
classes, clusters, subsets, groups and intervals in problem solving [2].
Although it may be difficult to have a precise and uncontroversial definition,
according to the [4] granular computing can be described as a general computation
theory for effectively using granules such as classes, clusters, subsets, groups and
intervals to build an efficient computational model for complex applications with huge
amounts of data, information and knowledge.
In this paper, we review using of basic components of granular computing for
representing hidden-layer neurons in RBF neural network with cloud activation
function. The paper is organized as follow. The basic components of granular
computing are given firstly. Next the process of granulation is analyzed. The section 4
describes application of basic components of granular computing to represent neurons
in hidden layer of RBF neural network with cloud activation function. Then the
conclusion is introduced.
Journal of Information, Control and Management Systems, Vol. X, (200X), No.X
2
7
BASIC COMPONENTS OF GRANULAR COMPUTING
Basic components of granular computing are granules. Granule is defined as “a
small particle, especially one of numerous particles forming a larger unit” in
Merriam-Webster’s Dictionary [11]. The meaning of granule in granular computing is
very similar to the one in above definition. Any subset, class, object or cluster of a
universe is called a granule [15]. For example, in a cluster analysis, a granule may be
interpreted as a subset of a universal set. A granule plays two distinctive roles. It may
by an element of another granule and is considered to be a part forming the other
granule. It may also consist of family of granules and is considered to be a whole. Its
particular role is determined by our focal points at different levels of problem solving.
Basic property of a granule is its size. Intuitively, the size may be interpreted as
the degree of detail, abstraction or concreteness. For example, in cluster analysis, the
size of granule can be the number of elements in cluster. In set theory, the cardinality
of a set may be regarded as the size of granule. Connections and relationship between
granules can be represented by binary relations. In concrete models, they may be
interpreted as dependency, closeness or overlapping.
Granules of a similar type may be collected together and studied their collective
properties. This leads to the notation of granular level. A level consists of granules of
similar size or nature those properties characterize and describe the subject matters of
study. There are two types of information and knowledge encoded in a level. A granule
captures a particular aspect of problem solving and collectively, all granules in the
level provide a global view over problem solving called granulated view.
An important property of granules and granular level is their granularity. The
granularity of a level refers to the collective properties of granules in a level with
respect to their size. The granularity is reflected by the size of all granules involved in
level and enables us to construct a hierarchical structure called a hierarchy. The term
hierarchy is used to denote a family of partially ordered granulated levels, in which
each level consists of a family of interacting and interrelated granules [17]. Hierarchy
offers different granular views over universe of discourse.
The interaction between granules, levels and hierarchies may be represented by
three types of structures:
- the internal structure of a granule,
- the collective structure of all granules, i.e. the internal structure of a
granulated view or level,
- the overall structure of all levels.
The internal structure of a granule provides a proper description, interpretation
and characterization of granule. All granules in a granular level may collectively show
a certain structure called the internal structure of a granulated view. The overall
structure of all levels is represented by a hierarchy. In a hierarchy, both the internal
structure of granules and the internal structure of granulated views are reflected. The
three structures as a whole are referred to as granular structure.
4 number of the even pages is at left side
Title of the paper, size 10
3
GRANULATION
The basic function of granular computing is granulation. Granulation means the
process of construction the three basic components of granular computing, i.e.
granules, granulated levels and hierarchies [16]. There are two types of granulation:
top-down decomposition of large granules to smaller granules and the bottom-up
combination of smaller granules into larger granules. In the set theory, a granule may
be viewed as a subset of universe of discourse, which may be either fuzzy or crisp. A
family of granules containing every element from universe of discourse is called a
granular level or granulated view. A granulated view may consist of a family of either
disjoint or overlapping granules. There are many granulated views of the same
universe. Different views of the universe can be linked together and a hierarchy can be
established.
Granulation involves two important aspects, namely granulation criteria and
granulation methods. Granulation criteria address the question why two objects are put
into the same granule, that is to say granulation criteria determine whether a granule
should be granulated into smaller granules in top-down decomposition or whether
different granules should be put together to form a larger granule in bottom-up
construction. In many situations, objects are usually grouped together based on their
relationships, such as indistinguishability, similarity, proximity or functionality [19].
Granulation methods address the question how to put two objects into the same granule
based on granulation criteria. For example, cluster analysis may be regarded as one of
the granulation methods. Granule representation or description relate with the results of
granulation methods. It is necessary to describe, name and label granules using some
characteristics. For example, in the cluster analysis, the mean and standard deviation of
cluster may be used to describe granule, which is represented by this cluster.
4
APPLICATION OF GRANULAR COMPUTING IN RBF NEURAL
NETWORK WITH CLOUD ACTIVATION FUNCTION
In this section we describe construction of neurons in hidden layer of RBF neural
network from granular computing perspective. RBF neural network is a feed forward
network consists of three layers – an input layer, a hidden layer and an output layer
(Figure 1). The input layer is composed of variable number of input neurons. The
output layer consists of one output neuron with linear activation function given by
s
yˆ = ∑ o j v j ,
(1)
j =1
where oj are outputs from hidden neurons, vj are the weights between hidden and
output layer, s is the number of hidden neurons and ŷ is the output from the network.
The algorithm for computation the value of oj can be found in [7], [8], [9], [10]. The
number of input and output neurons is determined by the task, which neural network
solves.
7
Journal of Information, Control and Management Systems, Vol. X, (200X), No.X
s
yˆ = ∑ o j v j
j =1
Figure 1 Architecture of neural network with cloud activation function (CAF)
according to [7], [8], [9], [10]
The hidden layer consists of variable number of neurons. Granules as basic
components of granular computing are used to represent these neurons. Granules are
extracted from data in the form of clusters, i.e. entities embracing collections of
numerical data that exhibit some functional or descriptive commonalities. Clustering of
input data may be performed via competitive learning of a single-layer neural network
with the ability to find automatically the number of clusters [14], which represents the
granularity degree of data. We assume that N input data {xt }t =1,..., N are available. At
the beginning of learning process the minimal distance r is initialized and the number
of clusters equals to zero. Then, the input vector is presented and a cluster is found. If
the distance of input vector from a cluster is less then minimal distance r, the center of
cluster is updated using the form
c j ' = c *j ' + η t ( xt − c *j ' ) ,
(2)
where c *j ' is the center of cluster before update and c j ' after update, η t is learning rate,
xt are input patterns. If the distance of input vector from a cluster is greater then
minimal distance r, new cluster is constructed at the position of current input vector,
i.e. xt = c j ' +1 . As time progresses the learning rate is reduced and centers of clusters
do not change. The final number of clusters provides the number of granules
representing neurons in hidden layer. The centers of clusters are regarded as the means
of granules. A family of granules containing every value of input data is called
granulated view. Granulated view of input data consists of a family of overlapping
granules. Above-mentioned competitive learning algorithm is regarded as one of the
6 number of the even pages is at left side
Title of the paper, size 10
granular methods presenting bottom-up granulation, i.e. input data are combined into
larger granules. The standard deviations of granule are calculated by the form
σ=
1
M
M
∑c
j
2
− xm
,
(3)
i =1
where x m is the m-th input vector belonging to the cluster c j .
The granules extracted from the available data are then described by the three
digital characteristics of normal cloud model, namely expected value, entropy and
hyperentropy. The mean of a granule is regarded as the expected value of normal cloud
model and the standard deviation of a granule provides the entropy of normal cloud
model. Both characteristics are calculated in the process of competitive learning. The
hyperentropy of a granule is a measure of dispersion of the cloud drops and it can be
calculated using backward algorithm [5] or set up manually. Figure 2 illustrates the
cloud activation function of granules in hidden layer.
y
Granule 3
Granule 1
Granule 2
x
1
En1
En2
En3
He1
He3
He2
C1
C2
C3
Figure 2 Description of granules by normal cloud model
7
Journal of Information, Control and Management Systems, Vol. X, (200X), No.X
Then, the mathematical form of activation function based on normal cloud model
is given by form
 (xt − c j )2 
ψ 2 (u j ) = o = exp−
,
2 
 2(En') 
j
(4)
where oj are the outputs from the hidden neurons, xt(x1, x2, …, xk) is an input vector, cj
is the mean of cloud activation function and En’ is the mean of random numbers with
mean σ and standard deviation He. Figure 3 shows the derivation of granules from
input data and their description by cloud models.
Numerical data
Granules
Granulation
Competitive learning
y
Granule 1
Granule 3
Description of granules by
normal cloud model
Granule 2
x
Cloud
models
Figure 3 Derivation of granules from input data and their description by normal cloud
model
5
CONCLUSION
Granules and granulation are some of the key issues in the study of granular
computing. In this paper we have studied application of basic components of granular
computing for representing neurons in hidden layer of RBF neural network and
description of granules using the three digital characteristics of normal cloud model.
Acknowledgement: This work has been supported by the grant VEGA 1/0024/08.
8 number of the even pages is at left side
Title of the paper, size 10
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
GINCHIGLIA, F., WALSH, T.: A theory of abstraction. Artificial Intelligence,
Vol. 57, Issue 2-3, October 1992, pp. 323-389, 1992
HAN, J., DONG, J.: Perspectives of Granular Computing in Software
Engineering. Proceedings of IEEE International Conference on Granular
Computing 2007, 2nd 4th November 2007, pp. 66-71
HOBBS, J.R. Granularity. Los Angeles: Proceedings of the Ninth International
Joint Conference on Artificial Intelligence, Vol. 1, pp. 432-435, 1985
http://xandu.cs.sjsu.edu/~grc/grc2008/
LI, D., DU, Y.: Artificial intelligence with uncertainty. Chapman&Hall/CRC,
Taylor&Francis Group, Boca Raton, 2008
LIN, T.Y. Granular Computing. Announcement of the BISC Special Interest
Group on Granular Computing, 1997
MARČEK, M., MARČEK, D.: Approximation and Prediction of Wages Based
on Granular Neural Network. Chengdu: Proceedings of the 3rd International
Conference, RSKT 2008, Vol. 5009/2008, 17th – 19th May 2008, pp. 556-563
MARČEK, M., MARČEK, D.: Granular RBF Neural Network Implementation
of Fuzzy Systems: Application to Time Series Modeling. Philadelphia: Journal
of Mult.-Valued Logic&Soft Computing. Vol. 14, Number 3-5, 2008, 101-114
MARČEK, M., MARČEK, D.: RBF Neural Network Implementation of Fuzzy
Systems: Application to Time Series Modelling. Toronto: Proceedings of the
11th International Conference, RSFDGrC 2007, Vol. 4482/2007, 14th -16th May
2007, pp. 500-507
MARČEK, M., PANČÍKOVÁ, L., MARČEK, D.: Ekonometria a Soft
Computing. Žilina: EDIS – vydavateľstvo Žilinskej univerzity, 2008
Merriam-Webster’s Dictionary Online: http://www.merriam-webster.com
PAWLAK, Z.: Granularity of knowledge, indiscernibility and rough sets.
Proceedings of IEEE International Conference of Fuzzy Systems, pp. 106-110,
1998
PAWLAK, Z.: Rough Sets. International Journal of Computer and Information
Science, Vol. 11, pp. 341-356, 1982
ŠNOREK, M. Neurónové síte a neuropočítače. ČVUT Praha, 2002
YAO, J.T.: Information Granulation and Granular Relationships. Bejing:
Proceedings of IEEE International Conference on Granular Computing. 25th –
27th July 2005. Vol. 1. pp. 326-329 [
YAO, Y.Y. Granular Computing for Data Mining. Kissimmee: Congres of Data
Mining, Intrusion Detection, Information Assurance and Data Network Security
2006. Vol. 6241. 16th – 17th April 2006, pp. 624105.1-624105.12.
YAO, Y.Y.: Granular Computing: Past, Present and Future. Hangzhou:
Proceedings of the IEEE International Conference on Grnaular Computing 2008.
26th – 28th August 2008. pp 80-85
YAO, Y.Y.: Perspective of Granular Computing. Proceedings of IEEE
International Conference on Granular Computing 2005, pp. 85-90
Journal of Information, Control and Management Systems, Vol. X, (200X), No.X
7
[19] ZADEH, L.A.: Towards a theory of fuzzy information granulation and its
centrality in human reasoning and fuzzy logics. Fuzzy Sets and Systems, Vol.
19, pp. 111-127, 1997
[20] ZADEH, L.A.: Fuzzy sets and information granulation. Advances in Fuzzy Set
Theory and Applications. M. Gupta, R. Ragade and R. Yager (Eds.),
Amsterdam: North Holland Publishing Co., pp. 3-18, 1979