Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
INTERNATIONAL JOURNAL OF INTELLIGENT CONTROL AND SYSTEMS VOL.12, NO.2 JUNE 2007 142-149 Hierarchical Knowledge Representation: Symbolic Conceptual Trees and Universal Approximation Luis Fernando de Mingo, Fernando Arroyo, Miguel Angel Dı́az and Juan Castellanos Abstract— This paper presents a practical example of a system based on neural networks that permits to build a conceptual hierarchy. This neural system classifies an input pattern as an element of each different category or subcategory that the system has, until an exhaustive classification is obtained. The proposed neural system is not a hierarchy of neural networks, it establishes relationships among all the different neural networks in order to transmit the neural activation when an external stimulus is presented to the system. Each neural network is in charge of the input pattern recognition to any prototyped class or category, and also of transmitting the activation to other neural networks to be able to continue with the classification. Therefore, the communication of the neural activation in the system depends on the output of each one of the neural networks, so as the functional links established among the different networks to represent the underlying conceptual hierarchy. Index Terms— Neural Networks, Function Approximation, Hierarchical Architecture. 1. I NTRODUCTION ONCEPTS and categories are being objects to study in practical Psychology and in Artificial Intelligence (AI) years ago. It deals with the research of how the knowledge and meaning are represented in the memory [4]. Why are concepts important?. Concepts or conceptual categories are stable representations, stored in the memory, that permit to treat different samples as members of a same class; without them we will be slaves of particular [5]. From a psychologist point of view, when dealing with objects, situations and actions as members of conceptual categories, the perceived reality is mentally being divided into different groups. A structure is imposed to the world, dividing mentally the reality [3], [4]. Categories seem to be ruled by the cognitive economy principle, this is, to extract essential things from the known world and to represent them in the most economical way in order to minimize the cognitive effort. According to Collins and Quillian [2], [3], the importance of a concept is not the concept by itself, it is determined by the set of relationships that are established with other different concepts. These relationships can be of two types: Subset and Property. The former type expresses the specialization of some C L.F. de Mingo and M.A. Dı́az are with Dept. Organización y Estructura de la Información, Escuela Universitaria de Informática - Universidad Politécnica de Madrid, Crta. de Valencia km. 7 - 28031 Madrid, Spain. {lfmingo,mdiaz}@eui.upm.es F. Arroyo is with Dept. Lenguajes, Proyectos y Sistemas Informáticos, Escuela Universitaria de Informática - Universidad Politécnica de Madrid, Crta. de Valencia km. 7 - 28031 Madrid, Spain. [email protected] J. Castellanos is with Dept. Inteligencia Artificial, Facultad de Informática - Universidad Politécnica de Madrid, Boadilla del Monte - 28660 Madrid, Spain. [email protected] concepts into others, it produces the categorization, allowing to mentally structure the perceived reality. What could be added from a connectionist point of view to the concept representation in memory? [6]. Why could not it be thought that the concepts are supported by some small sets of neural assemblies? These ones form a mix of specialized neural networks concerning the samples recognition task and they determined if a sample belongs or is classified into a concept. Concepts are not isolated into the human cognitive system. They are immersed into a hierarchical structure that facilitates, among others, classification tasks to the human cognitive system. This conceptual hierarchy expresses a binary relationship of inclusion defined by the following criterions: • Inclusion criterion: Each hierarchy node determines a domain included into domain of its father node. Each hierarchy node determines a domain that includes every domain of its son nodes. • Generalisation-Specialisation criterion: Every node in the hierarchy, has differentiating properties that make it different from its father node, if it exists, and from the others son nodes of its father, if they exist. This Inclusion relation makes possible to keep the maximal economy criterion in the conceptual hierarchy. The defined inclusion relation makes also possible to represent a conceptual hierarchy as an acycled directed graph in which nodes represent concepts and edges the inclusion relation among them. Edges connecting nodes establishes that the father nodes are situated in the upper hierarchical level of its son nodes. Obviously, concepts are generated in a natural manner and it will not be constrained by these formal restrictions. However, many of the classifications learnt during our school years are built using these rules. In this paper, we will focus our work in this kind of hierarchies, which are learnt with a teacher or in a supervised manner. Next sections deal with the representation of any conceptual hierarchy using a set of neural networks, permitting the complete or exhaustive classification of an input. Exhaustive classification means that an input belongs, or not, to each one of the supported concepts by the system and to their specializations. The hierarchical architecture is controlled using neural network outputs, that is, depending on the output a different neural network is activated. 2. P ROBLEM D ESCRIPTION L ET D domain of any samples. Let H a conceptual hierarchy to the elements of the domain D. Let T the 143 Mingo et al.: Hierarchical Knowledge Representation: Symbolic Conceptual Trees and Universal Approximation INTERNATIONAL JOURNAL OF INTELLIGENT CONTROL AND SYSTEMS VOL.12, NO.2 JUNE 2007 144 145 Mingo et al.: Hierarchical Knowledge Representation: Symbolic Conceptual Trees and Universal Approximation INTERNATIONAL JOURNAL OF INTELLIGENT CONTROL AND SYSTEMS VOL.12, NO.2 JUNE 2007 146 147 Mingo et al.: Hierarchical Knowledge Representation: Symbolic Conceptual Trees and Universal Approximation INTERNATIONAL JOURNAL OF INTELLIGENT CONTROL AND SYSTEMS VOL.12, NO.2 JUNE 2007 148 149 Mingo et al.: Hierarchical Knowledge Representation: Symbolic Conceptual Trees and Universal Approxination Irvine, CA: University of California, Department of Information and Computer Science. 1998. [2] Collins, A. M. and Quillian, M. R.: Retrieval time from semantic memory. Journal of Verbal Learning and Behavior, 8,240-247. 1969. Fernando Arroyo Ph.D. on Artificial Intelligence with a position of full professor at the Universidad Politécnica de Madrid. He has a great number of international publications in leading journals concerning Membrane Computing and Neural Networks. [3] Quillian, M. R. Semantic Memory. E. Minsky (Ed.): Semantic Information Processing. Cambrige, Mass.: MIT Press. 1968 [4] Tulving, E. Episodic and Semantic Memory. E. Tulving and W. Donaldson (Eds.): Organization of Memory. New York: Academic Press. 1972 [5] Wallace, J.G.; Silverstein, R.B.; Bluff, K. and Pipingas, A.: Semantic Transparency, Brain Monitoring and Evaluation of Hybrid Cognitive Architectures. Connection Science, Vol. 6, N. 1, 1994. [6] F. Arroyo, J. Castellanos, C. Luengo, L.F. Mingo: Frames Neural Networks for Cognitive Process. Proceedings of SCI?97 ?World Multiconference on SYSTEMICS, CYPERNETICS AND INFORMATICS. Caracas July 1997, pp. 288-295. 1997. [7] F.Arroyo, J. Castellanos, C. Luengo, L.F. Mingo: Connectionist Systems for a Hierarchical Knowledge Representation. Proceedings of ICNNB98 - 1998 International Conference on Neural Networks and Brain. Beijing, pp. 471-474. 1998. [8] F. Arroyo, C. Luengo, F. Mingo, J. Castellanos: Evolving Connectionist and Symbolic Processes in a Hierarchical Memory Model. Proceedings of World Multiconference on Systemics, Cybernetics and Informatics. SCI’99 and ISAS’99, Orlando, Florida, 1999, vol 8, pp. 255-261. 1999. [9] Levon Aslanyan, Luis de Mingo, Juan Castellanos, Vladimir Ryazanov, F. Chelnokov, Alexander Dokukin: On Logical Correction of Neural Network Algorithms for Pattern Recognition. International Journal on INFORMATION THEORIES & APPLICATIONS. Vol. 14. No. 2. ISSN 1310-0513. Pp.: XX-XX. 2006. Miguel Angel Dı́az Full professor at the Universidad Politécnica de Madrid. His research interests lie in the area of membrane computing, especially in networks of evolutionary processors, but also in the area of neural networks.He has participated in several local projects. He has some papers published in visible journals and in proceedings from notable conferences (WSEAS, etc.) [10] Luis Mingo, Levon Aslanyan, Juan Castellanos, Miguel Diaz, and Vladimir Riazanov. Fourier Neural Networks: An Approach with Sinusoidal Activation Functions. International Journal on INFORMATION THEORIES & APPLICATIONS. Vol. 11. No. 1. ISSN 1310-0513. Pp.: 52-55. 2004. [11] Anderson, James A. and Edward Rosenfield (1988). Neurocomputing: Foundations of Research Cambridge, MA: MIT Press ISBN: 0-26201097-6. [12] Anderson, James A. (1995). An Introduction to Neural Networks Cambridge, MA: MIT Press ISBN: 0-262-01144-1. Luis F. de Mingo Contracted professor at the Universidad Politécnica de Madrid since 1998 and Ph.D. on Artificial Intelligence. His research interests lie in the area of learning models, in particular in the theoretical parts of Neural Networks and Pattern Recognition, but also in the area of Networks of Evolutionary Processors. He has participated in several INTAS projects and some local projects. Member of the editorial board of IJITA, and international reviewer of KDS, ITECH conferences. Active member of European Molecular Computation Consortium. Juan Castellanos Full professor at the Universidad Politécnica de Madrid with a Ph.D. on Artificial Intelligence. His research interests lie in the area of bio-computing, in particular in the theoretical parts of DNA computing and membrane systems, but also in the area of neural networks. He has more than sixty papers in these branches of bioinformatics, many of them published in visible journals and in proceedings from notable conferences (7th, 8th International Meeting on DNA Based Computers, etc.)