Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Journal of Information, Control and Management Systems, Vol. X, (200X), No.X 7 GRANULAR COMPUTING AND ITS APPLICATION IN RBF NEURAL NETWORK WITH CLOUD ACTIVATION FUNCTION Zuzana MEČIAROVÁ University of Žilina, Faculty of Management Science and Informatics, Slovak Republic e-mail: [email protected] Abstract This paper discusses using of granular computing components for representing neurons in hidden layer of RBF neural network with cloud activation function. After providing a brief introduction to granular computing, the paper describes basic components of granular computing. Then the process of granulation is analyzed. Finally, the application of granular computing in RBF neural network is introduced. Keywords: cloud activation function, granulation, granule, granular computing, granulated level, granular structure, hierarchy, RBF neural network, 1 INTRODUCTION TO GRANULAR COMPUTING Although granular computing is a separate research field, the basic idea of granular computing, i.e. problem solving with different granularities, have been explored in many fields such as artificial intelligence, interval analysis, quantization, Dempster-Shafer theory of belief functions, rough set theory, divide and conquer, cluster analysis, machine learning, databases and many others [19]. In the past few decades we have witnessed rapid development of granular computing. Three branches of science, namely fuzzy theory, artificial intelligence and rough set theory contributed significantly to the study of granular computing. In 1979, Zadeh first introduced and discussed the notion of information granulation and suggested that fuzzy set theory may find potential applications in this respect, which pioneers the explicit study of granular computing [20]. A general framework of granular computing was given in a paper [19] by Zadeh based on fuzzy set theory. According to Zadeh, granules are constructed and defined based on the concept of generalized constraints. Relationships between granules are represented in terms of fuzzy graphs or fuzzy if-then rules. The associated computation method is known as computing with words. 2 number of the even pages is at left side Title of the paper, size 10 The popularity of granular computing is also due to the rough set theory. Rough set models enable us to precisely define and analyze many notions of granular computing. In 1982, Pawlak proposed the rough sets to approximate a crisp set [13] and investigated the granularity of knowledge from the point of view of rough set theory [12]. The ideas of granular computing have been investigated in artificial intelligence through the notions of granularity and abstraction. Hobbs proposed a theory of granularity as the base of knowledge representation, abstraction, heuristic search and reasoning in 1985. In this theory, we look at the world under various grain seizes and abstract from it only those things that serve our present interests. Our ability to conceptualize the world at different granularities and switch among these granularities is fundamental to our intelligence and flexibility. It enables us to map the complexities of the world around us into simpler theories that are computationally tractable to reason in [3]. In 1992, Giunchiglia and Walsh presented a theory of abstraction to improve the conceptualization of granularities. According to Giunchiglia and Walsh abstraction is the process which allows people to consider what is relevant and to forget a lot of irrelevant details which would get in the way of what they are tying to do [1]. The term “granular computing” was first suggested by T.Y. Lin [6] in 1997 to label this growing research field. Yao investigated the trinity model of granular computing from three perspectives: philosophy, methodology and computation [18]. In the past decade, different granular computing models have been conducted in various aspects and applied in various application domains including machine learning, data mining, bioinformatics, e-business, network security, wireless mobile computing etc.. The essence of these models has been addressed by researches to build efficient computational algorithms for handling huge amounts of data, information and knowledge. The objectives of these computation models are computer-centered and mainly concern the efficiency, effectiveness and robustness of using granules such as classes, clusters, subsets, groups and intervals in problem solving [2]. Although it may be difficult to have a precise and uncontroversial definition, according to the [4] granular computing can be described as a general computation theory for effectively using granules such as classes, clusters, subsets, groups and intervals to build an efficient computational model for complex applications with huge amounts of data, information and knowledge. In this paper, we review using of basic components of granular computing for representing hidden-layer neurons in RBF neural network with cloud activation function. The paper is organized as follow. The basic components of granular computing are given firstly. Next the process of granulation is analyzed. The section 4 describes application of basic components of granular computing to represent neurons in hidden layer of RBF neural network with cloud activation function. Then the conclusion is introduced. Journal of Information, Control and Management Systems, Vol. X, (200X), No.X 2 7 BASIC COMPONENTS OF GRANULAR COMPUTING Basic components of granular computing are granules. Granule is defined as “a small particle, especially one of numerous particles forming a larger unit” in Merriam-Webster’s Dictionary [11]. The meaning of granule in granular computing is very similar to the one in above definition. Any subset, class, object or cluster of a universe is called a granule [15]. For example, in a cluster analysis, a granule may be interpreted as a subset of a universal set. A granule plays two distinctive roles. It may by an element of another granule and is considered to be a part forming the other granule. It may also consist of family of granules and is considered to be a whole. Its particular role is determined by our focal points at different levels of problem solving. Basic property of a granule is its size. Intuitively, the size may be interpreted as the degree of detail, abstraction or concreteness. For example, in cluster analysis, the size of granule can be the number of elements in cluster. In set theory, the cardinality of a set may be regarded as the size of granule. Connections and relationship between granules can be represented by binary relations. In concrete models, they may be interpreted as dependency, closeness or overlapping. Granules of a similar type may be collected together and studied their collective properties. This leads to the notation of granular level. A level consists of granules of similar size or nature those properties characterize and describe the subject matters of study. There are two types of information and knowledge encoded in a level. A granule captures a particular aspect of problem solving and collectively, all granules in the level provide a global view over problem solving called granulated view. An important property of granules and granular level is their granularity. The granularity of a level refers to the collective properties of granules in a level with respect to their size. The granularity is reflected by the size of all granules involved in level and enables us to construct a hierarchical structure called a hierarchy. The term hierarchy is used to denote a family of partially ordered granulated levels, in which each level consists of a family of interacting and interrelated granules [17]. Hierarchy offers different granular views over universe of discourse. The interaction between granules, levels and hierarchies may be represented by three types of structures: - the internal structure of a granule, - the collective structure of all granules, i.e. the internal structure of a granulated view or level, - the overall structure of all levels. The internal structure of a granule provides a proper description, interpretation and characterization of granule. All granules in a granular level may collectively show a certain structure called the internal structure of a granulated view. The overall structure of all levels is represented by a hierarchy. In a hierarchy, both the internal structure of granules and the internal structure of granulated views are reflected. The three structures as a whole are referred to as granular structure. 4 number of the even pages is at left side Title of the paper, size 10 3 GRANULATION The basic function of granular computing is granulation. Granulation means the process of construction the three basic components of granular computing, i.e. granules, granulated levels and hierarchies [16]. There are two types of granulation: top-down decomposition of large granules to smaller granules and the bottom-up combination of smaller granules into larger granules. In the set theory, a granule may be viewed as a subset of universe of discourse, which may be either fuzzy or crisp. A family of granules containing every element from universe of discourse is called a granular level or granulated view. A granulated view may consist of a family of either disjoint or overlapping granules. There are many granulated views of the same universe. Different views of the universe can be linked together and a hierarchy can be established. Granulation involves two important aspects, namely granulation criteria and granulation methods. Granulation criteria address the question why two objects are put into the same granule, that is to say granulation criteria determine whether a granule should be granulated into smaller granules in top-down decomposition or whether different granules should be put together to form a larger granule in bottom-up construction. In many situations, objects are usually grouped together based on their relationships, such as indistinguishability, similarity, proximity or functionality [19]. Granulation methods address the question how to put two objects into the same granule based on granulation criteria. For example, cluster analysis may be regarded as one of the granulation methods. Granule representation or description relate with the results of granulation methods. It is necessary to describe, name and label granules using some characteristics. For example, in the cluster analysis, the mean and standard deviation of cluster may be used to describe granule, which is represented by this cluster. 4 APPLICATION OF GRANULAR COMPUTING IN RBF NEURAL NETWORK WITH CLOUD ACTIVATION FUNCTION In this section we describe construction of neurons in hidden layer of RBF neural network from granular computing perspective. RBF neural network is a feed forward network consists of three layers – an input layer, a hidden layer and an output layer (Figure 1). The input layer is composed of variable number of input neurons. The output layer consists of one output neuron with linear activation function given by s yˆ = ∑ o j v j , (1) j =1 where oj are outputs from hidden neurons, vj are the weights between hidden and output layer, s is the number of hidden neurons and ŷ is the output from the network. The algorithm for computation the value of oj can be found in [7], [8], [9], [10]. The number of input and output neurons is determined by the task, which neural network solves. 7 Journal of Information, Control and Management Systems, Vol. X, (200X), No.X s yˆ = ∑ o j v j j =1 Figure 1 Architecture of neural network with cloud activation function (CAF) according to [7], [8], [9], [10] The hidden layer consists of variable number of neurons. Granules as basic components of granular computing are used to represent these neurons. Granules are extracted from data in the form of clusters, i.e. entities embracing collections of numerical data that exhibit some functional or descriptive commonalities. Clustering of input data may be performed via competitive learning of a single-layer neural network with the ability to find automatically the number of clusters [14], which represents the granularity degree of data. We assume that N input data {xt }t =1,..., N are available. At the beginning of learning process the minimal distance r is initialized and the number of clusters equals to zero. Then, the input vector is presented and a cluster is found. If the distance of input vector from a cluster is less then minimal distance r, the center of cluster is updated using the form c j ' = c *j ' + η t ( xt − c *j ' ) , (2) where c *j ' is the center of cluster before update and c j ' after update, η t is learning rate, xt are input patterns. If the distance of input vector from a cluster is greater then minimal distance r, new cluster is constructed at the position of current input vector, i.e. xt = c j ' +1 . As time progresses the learning rate is reduced and centers of clusters do not change. The final number of clusters provides the number of granules representing neurons in hidden layer. The centers of clusters are regarded as the means of granules. A family of granules containing every value of input data is called granulated view. Granulated view of input data consists of a family of overlapping granules. Above-mentioned competitive learning algorithm is regarded as one of the 6 number of the even pages is at left side Title of the paper, size 10 granular methods presenting bottom-up granulation, i.e. input data are combined into larger granules. The standard deviations of granule are calculated by the form σ= 1 M M ∑c j 2 − xm , (3) i =1 where x m is the m-th input vector belonging to the cluster c j . The granules extracted from the available data are then described by the three digital characteristics of normal cloud model, namely expected value, entropy and hyperentropy. The mean of a granule is regarded as the expected value of normal cloud model and the standard deviation of a granule provides the entropy of normal cloud model. Both characteristics are calculated in the process of competitive learning. The hyperentropy of a granule is a measure of dispersion of the cloud drops and it can be calculated using backward algorithm [5] or set up manually. Figure 2 illustrates the cloud activation function of granules in hidden layer. y Granule 3 Granule 1 Granule 2 x 1 En1 En2 En3 He1 He3 He2 C1 C2 C3 Figure 2 Description of granules by normal cloud model 7 Journal of Information, Control and Management Systems, Vol. X, (200X), No.X Then, the mathematical form of activation function based on normal cloud model is given by form (xt − c j )2 ψ 2 (u j ) = o = exp− , 2 2(En') j (4) where oj are the outputs from the hidden neurons, xt(x1, x2, …, xk) is an input vector, cj is the mean of cloud activation function and En’ is the mean of random numbers with mean σ and standard deviation He. Figure 3 shows the derivation of granules from input data and their description by cloud models. Numerical data Granules Granulation Competitive learning y Granule 1 Granule 3 Description of granules by normal cloud model Granule 2 x Cloud models Figure 3 Derivation of granules from input data and their description by normal cloud model 5 CONCLUSION Granules and granulation are some of the key issues in the study of granular computing. In this paper we have studied application of basic components of granular computing for representing neurons in hidden layer of RBF neural network and description of granules using the three digital characteristics of normal cloud model. Acknowledgement: This work has been supported by the grant VEGA 1/0024/08. 8 number of the even pages is at left side Title of the paper, size 10 REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] GINCHIGLIA, F., WALSH, T.: A theory of abstraction. Artificial Intelligence, Vol. 57, Issue 2-3, October 1992, pp. 323-389, 1992 HAN, J., DONG, J.: Perspectives of Granular Computing in Software Engineering. Proceedings of IEEE International Conference on Granular Computing 2007, 2nd 4th November 2007, pp. 66-71 HOBBS, J.R. Granularity. Los Angeles: Proceedings of the Ninth International Joint Conference on Artificial Intelligence, Vol. 1, pp. 432-435, 1985 http://xandu.cs.sjsu.edu/~grc/grc2008/ LI, D., DU, Y.: Artificial intelligence with uncertainty. Chapman&Hall/CRC, Taylor&Francis Group, Boca Raton, 2008 LIN, T.Y. Granular Computing. Announcement of the BISC Special Interest Group on Granular Computing, 1997 MARČEK, M., MARČEK, D.: Approximation and Prediction of Wages Based on Granular Neural Network. Chengdu: Proceedings of the 3rd International Conference, RSKT 2008, Vol. 5009/2008, 17th – 19th May 2008, pp. 556-563 MARČEK, M., MARČEK, D.: Granular RBF Neural Network Implementation of Fuzzy Systems: Application to Time Series Modeling. Philadelphia: Journal of Mult.-Valued Logic&Soft Computing. Vol. 14, Number 3-5, 2008, 101-114 MARČEK, M., MARČEK, D.: RBF Neural Network Implementation of Fuzzy Systems: Application to Time Series Modelling. Toronto: Proceedings of the 11th International Conference, RSFDGrC 2007, Vol. 4482/2007, 14th -16th May 2007, pp. 500-507 MARČEK, M., PANČÍKOVÁ, L., MARČEK, D.: Ekonometria a Soft Computing. Žilina: EDIS – vydavateľstvo Žilinskej univerzity, 2008 Merriam-Webster’s Dictionary Online: http://www.merriam-webster.com PAWLAK, Z.: Granularity of knowledge, indiscernibility and rough sets. Proceedings of IEEE International Conference of Fuzzy Systems, pp. 106-110, 1998 PAWLAK, Z.: Rough Sets. International Journal of Computer and Information Science, Vol. 11, pp. 341-356, 1982 ŠNOREK, M. Neurónové síte a neuropočítače. ČVUT Praha, 2002 YAO, J.T.: Information Granulation and Granular Relationships. Bejing: Proceedings of IEEE International Conference on Granular Computing. 25th – 27th July 2005. Vol. 1. pp. 326-329 [ YAO, Y.Y. Granular Computing for Data Mining. Kissimmee: Congres of Data Mining, Intrusion Detection, Information Assurance and Data Network Security 2006. Vol. 6241. 16th – 17th April 2006, pp. 624105.1-624105.12. YAO, Y.Y.: Granular Computing: Past, Present and Future. Hangzhou: Proceedings of the IEEE International Conference on Grnaular Computing 2008. 26th – 28th August 2008. pp 80-85 YAO, Y.Y.: Perspective of Granular Computing. Proceedings of IEEE International Conference on Granular Computing 2005, pp. 85-90 Journal of Information, Control and Management Systems, Vol. X, (200X), No.X 7 [19] ZADEH, L.A.: Towards a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logics. Fuzzy Sets and Systems, Vol. 19, pp. 111-127, 1997 [20] ZADEH, L.A.: Fuzzy sets and information granulation. Advances in Fuzzy Set Theory and Applications. M. Gupta, R. Ragade and R. Yager (Eds.), Amsterdam: North Holland Publishing Co., pp. 3-18, 1979