Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Annual report 2001 SNN P.O.Box 9101, interncode 231, 6500 HB Nijmegen, The Netherlands tel: +31 24 3614245, fax: +31 24 3541435, e-mail: [email protected] 1 Colofon: Editors: research directors Final editor: Stan Gielen Designer cover: ARGANTE-ARGANTE Lay-out: Annet Wanders Press: Trioprint 2 Contents Contacts 4 Addresses 5 Introduction 1 7 Research Groups 2A Intelligent Autonomous Systems Group University of Amsterdam 13 2B Laboratory for Biophysics University of Nijmegen 19 2C Algorithmics and Program Methodology Leiden University 33 2D Pattern Recognition Group Delft University of Technology 41 2E System Technology Cluster, Section Intelligent Modelling University of Groningen 47 2F Civil Engineering Informatics Group Delft University of Technology 53 2G Institute for Knowledge and Agent Technology (IKAT) University Maastricht 57 2H Evolutionary Systems and Applied Algorithmics CWI, Amsterdam 61 3A Industrial Partners 70 3B Commercial Spin-off activities of SNN 72 Commercial Applications Publications 4 75 3 Addresses SNN Prof.dr. C.C.A.M. Gielen (Director), Dr. H.J. Kappen (Vice Director), A. Wanders (Accounting). University of Nijmegen, PO Box 9101, intern code 231, 6500 HB Nijmegen, The Netherlands Tel: +31 24 3614245, fax: +31 24 3541435, e-mail: [email protected], http://www.snn.kun.nl BOARD of SNN Prof.drs. M. Boasson, Kuipers-Rietberglaan 17, 7271 EJ Borculo Ir. G. Hiemstra, Van der Meer & van Tilburg, Zeistoever 11, 3704 GB Zeist Dr.ir. G. van Oortmerssen, CWI, Kruislaan 413, PB 94079, 1090 GB Amsterdam Drs. P.J.M. Timmermans (treasurer), Faculteit Natuurwetenschappen,Wiskunde en Informatica, Toernooiveld 1, 6525 ED Nijmegen Dhr. J.H.M. Uylings, Twinning Center Amsterdam, Kruislaan 400, 1098 SM Amsterdam Drs. J.P. Veen (until June, 2001) Dr.ir. P. Zuidema (chairman), CMG, Prof. Meyerslaan 2, 1183 AV Amstelveen RESEARCH GROUPS Intelligent Autonomous Systems Group, University of Amsterdam Prof.dr.ir. F.C.A. Groen, Dr. B. Kröse Kruislaan 403, 1098 SJ Amsterdam, The Netherlands Tel:+31-20-5257463, fax: +31-20-5257490, e-mail: [email protected] http://www.science.uva.nl/research/ias/ Laboratory for Biophysics, University of Nijmegen Prof.dr. C.C.A.M. Gielen, Dr. H.J. Kappen Geert Grooteplein 21, Intern 231, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands. Tel: +31-24-3614245, fax: +31-24-3541435, e-mail:[email protected], http://www.snn.kun.nl Algorithmics and Program Methodology, Leiden University Prof.dr. J.N. Kok P.O. Box 9512, 2300 RA Leiden, The Netherlands Tel: +31-71-5277057, fax: +31-71-5276985, e-mail: [email protected] http://www.wi.leidenuniv.nl/CS/ALP Neural Network Research, Pattern Recognition Group, Delft University of Technology Dr.ir. R.P.W. Duin Department of Applied Physics, Lorentzweg 1, 2628 CJ Delft, The Netherlands Tel: +31-15-2786143, fax: +31-15-2786740, e-mail: [email protected] http://www.ph.tn.tudelft.nl/Research/neural/index.html 4 System Technology Cluster, University of Groningen, Section Intelligent Modelling Prof.dr.ir. L. Spaanenburg P.O. Box 800, 9700 AV Groningen, The Netherlands Tel: +31-50-3633925, fax: +31-50-3633800 e-mail: [email protected], http://www.cs.rug.nl/Research/St/ Civil Engineering Informatics Group, Delft University of Technology Prof.dr.ir. P. van der Veer P.O. Box 5048, 2600 GA Delft, The Netherlands Tel: +31-15-2781642, fax: +31-15-2787700, e-mail: [email protected], http://www.cti.ct.tudelft.nl/ Institute for Knowledge and Agent Technology (IKAT), Universiteit Maastricht Prof. H.J. van den Herik Universiteit Maastricht, Fac. Alg. Wetenschappen IKAT, Postbus 616, 6200 MD Maastricht, The Netherlands Tel: +31-43-3883485, fax: +31-43-3884897, email: [email protected], http://www.cs.unimaas.nl Evolutionary Systems and Applied Algorithmics; CWI, Amsterdam Prof.dr.ir. J.A. La Poutré P.O. Box 94079, 1090 GB Amsterdam, The Netherlands Tel: +31-20-5924082, fax: +31-20-5924199, email: [email protected], http://www.cwi.nl INDUSTRIAL PARTNERS Unit Energy Efficiency, Energy Research Foundation (ECN) Drs.Ing. J.J. Kok, Ir. J.J. Saurwalt P.O. Box 1, 1755 ZG Petten, The Netherlands Tel: +31-224-564116, fax: +31-224-561407, e-mail: [email protected], http://www.ecn.nl/ COMMERCIAL SPIN-OFF COMPANIES SMART Research BV Dr. T.M. Heskes P.O. Box 31070, 6503 CB Nijmegen, The Netherlands. Tel: +31-24-3615039, fax: +31-24-3541435, email: [email protected], http://www.smart-research.nl Nutech Solutions GmbH Dr. Thomas Bäck, Managing Director Leiden University, LIACS, Niels Bohrweg 1 NL-2333 CA Leiden, The Netherlands emaill: baeck@ liacs.nl 5 Contacts SNN had collaborations with the following companies and organisations in 2001 Aegon schadeverzekering NV Ahold ASTRON BrandmarC Buma Stemra Cap Gemeni Ernst & Young CLB Coda Music Technology CORUS Group Plc Dacolian De Telegraaf DSM ECN Emagic Soft- und Hardware GmbH Fokker Hogeschool voor de Kunsten IBM Watson Center New York Kennis Centrum Papier en Karton KiQ Korg Italy KPN Landustrie Sneek BV MatchCare Data Compare Midesa NATO C3 Agency Nederlandse Groeistichting NLR Noldus Information Technology BV OPG Groothandel BV Pfizer BV Philips Philips Research Labs RaInteray Rand Corporation RIVM Sappi SaurECN Schuitema NV Shell SIEP BV SKF SMART Research BV SoundPalette Space Hiscom BV Technische Unie Technofysica TNO-FEL Tomandandy Music Inc TVA Developments Tweehuysen Consultancy BV UMC Maastricht UMC Nijmegen UMC Utrecht Unilever Unilever Research Vlaardingen Witteveen+Bos Yamaha Corporation SNN Research is funded in part by: European community, Japanese Ministry of International Trade and Industry (MITI), Technology Foundation (NWO), Princes Beatrix Fonds. 6 1 Introduction 7 On the 9-th of July, Drs. Jean-Pierre Veen, member of the Board of SNN, suddenly passed away in a car accident. Jan Pierre has been a member of the SNN-board since 1994, and as such, he was one of the “senior” members. Jean-Pierre has played a very active and important role in SNN. He had a keen eye for good applied research and for applications within the market. His knowledge about the world of applied scientific research was unsurpassed. For every good research plan with a prospect for applications, he always managed to suggest successful ways to funding by national or international funding agencies. His advice always came with inspiring ideas about future applications. We will miss Jean Pierre’s enthusiastic contributions in our board meetings. We will remember him as a stimulating, intellectual and pleasant colleague. Dr. ir. Peter Zuidema, chairman 8 Introduction The human brain is one of the great mysteries of nature. It strongly appeals to human curiosity trying to understand itself. In this context it is remarkable that the human brain has often been compared with the most advanced computers. This is remarkable since computers differ from the brain both in the “hardware” and the “software” with obvious consequences for the different principles, which underlie information processing in biological and artificial systems. One of the aims of the Foundation for Neural Networks (SNN) is to bridge this gap between natural intelligence and artificial intelligence. SNN aims to investigate the computational principles of biological and artificial systems and to apply the results in novel applications. SNN plays a leading role on the international scene in fundamental research and advanced applications. This annual report presents an overview of the various research projects within SNN. Some of the high-lights of projects, which were finished in 2001, will be described in more detail. Major achievements in 2001 The Japanese Real World Computing Partnership (RWCP) has been one of the major funding agencies of SNN. In 2001 this project, which had a duration of 10 years, entered its last year. It was concluded by a large symposium in Tokyo, where all participating groups demonstrated the results of their research efforts. One of the demonstrations was presented by the IAS group of SNN at the University of Amsterdam, who demonstrated an autonomous ‘office robot’ which was built in collaboration with a number of Japanese research groups (AIST, Fujitsu, NTT) and a Swedish group (SICS). This robot is able to navigate in office-like environments, to have a conversation with users, and to dock to a loading station when it is running out of power. The IAS group developed the environment learning and localization module of the system. The robot learns a model of the environment from its `omni-directional’ vision system and uses a probabilistic model to estimate its location within the environment. Figure 1: Visitors and IAS-robot at the RWCP symposium Various industries have showed interest in the developed system and the underlying computer science methods. Based on the results obtained in this project, a European (ITEA) research proposal by a consortium of European industrial and academic partners was accepted for funding. A ‘personal’ robot, which operates in an intelligent home environment, will be developed in collaboration with Philips Research, Epictoid (a CWI spin-off), University of Leuven and the IAS group of the University of Amsterdam. Such a robot should be able to communicate with users using speech dialogue and emotional feedback and to navigate in a home environment. In another project, entitled “Hybrid modeling techniques for data-base mining” by Machiel Westerdijk (KUN), various methods for data mining were explored. The aim of this project was: • to find hidden information in large data bases. • to present the newly found information in terms of simple rules explaining the structure of the data and relations between data. 9 An explanation about which parameters are relevant and how parameters combine is equally important. In such a case a solution should meet 2 constraints: to find simple rules to obtain the best possible performance. The main scientific value of this project lies in the combination of various techniques (using so-called ‘hidden variable models’) which provide an optimal balance between optimizing performance and optimizing insight into the data and the rules explaining relations between the data to human experts. Our algorithm gave a much better performance than any other algorithm in the literature (such as the well-known C4.5 algorithm) on well-known bench-mark problems such as the diagnosis of heart diseases and diabetes, and in optimal credit assessment in financial applications. The results of this project are now used by one of the leading international companies on IT services and business consultancy for real-world applications. More information about the project can be optained at: http://www.snn.kun.nl/nijmegen/hybrid.html The group at the Technical University in Delft has completed a project on machine diagnostics by neural networks. The main objective of this project was to develop a diagnostic tool, based on neural network technology, for on-line analysis and evaluation of data in a complex process to monitor changes within the process. Typical analyses include the use in machine monitoring (on-line pump monitoring, off-line analysis of ship engine vibration, monitoring of rotating equipment in paper industry) and medical monitoring problems (depth of anesthesia during surgery, detection of eyeblink rate in Tourette’s syndrome patients, analysis of EEG/MEG signals for detection of epilepsy or Alzheimer’s disease). This project received the 2001 SKBS Award for best demo/application at the 13th conference on Artificial Intelligence in Belgium and The Netherlands. More information about the project can be obtained at: http://www.mbfys.kun.nl/~ypma/project/monisom/index.html. Some of the results have been implemented in a commercially available software system for machine health monitoring with self-organizing maps, called MONISOM. Further information about commercial use of the results can be obtained from: J. Valk, Landustrie Sneek BV (e-mail: [email protected]) Figure 2: MONISOM: a SOM-based system for machine health monitoring, applied to a dataset representing a progressively loose foundation in a submersible pump. 10 2 Research Groups 11 12 2A Intelligent Autonomous Systems Group, University of Amsterdam Prof.dr.ir. F.C.A. Groen, Dr.ir. B.J.A. Kröse, Drs. R. Bunschoten, Dr.ir. S. ten Hagen, Drs. J. Portegies Zwart, B. Terwijn, Drs. J. Verbeek, Dr. N. Vlassis, Drs. W. Zajdel Description of the group Objective The IAS group is engaged in the development of theoretical understanding and computing methods for autonomous systems, including methods for sensor data processing, reasoning, learning and distributed systems. Here we report only the work on probabilistic reasoning and learning by autonomous systems. More information about the other topics of the group can be found at: http://www.science.uva.nl/research/ias. Probabilistic methods for data modeling Drs. J. Verbeek, Dr. N. Vlassis Objective In many engineering problems, we are given a collection of data that correspond to observations of a physical system. Then we are asked to make a model of this system and to use it later to make predictions, take decisions, etc. Because the data are normally corrupted by noise, probabilistic methods for data modeling (probabilistic techniques) are playing an important role in the modeling process of these data. Approach We have recently developed a statistical technique for the problem of probabilistic clustering. In this problem we want to group the set of data into a number of clusters, while at the same time assigning to each point a degree of ‘certainty’ as to which cluster it belongs. We have proposed a novel algorithm for fitting the parameters of a Gaussian mixture in a greedy fashion: adding components to the mixture one after the other. Our algorithm is based on the Expectation-Maximization algorithm, a wellknown statistical technique for learning such models. Our experiments have shown that such a ‘greedy’ method for learning Gaussian mixtures can give results superior to other, state-of-the-art methods. We have demonstrated our algorithm in synthetic and realistic experiments with image data, and in all cases our algorithm gives better or at least as good solutions as other existing methods (Vlassis and Likas, 2002). Figure 3: A six-component mixture and the component allocation 2 2 1.5 1.5 1 1 0.5 0.5 0 0 −0.5 −0.5 −1 −1 −1.5 −1.5 −2 −1.5 −1 −0.5 0 0.5 1 −2 1.5 2 2 1.5 1.5 1 1 0.5 0.5 0 0 −0.5 −0.5 −1 −1 −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −1.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2 1.5 1.5 1 1 0.5 0.5 0 0 −0.5 −0.5 −1 −1 −1.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 13 Tools for non-linear data analysis Drs. J. Verbeek, Dr. N. Vlassis, Dr.ir. B.J.A. Kröse Objective Sensory, ‘real world data’ is generally high dimensional and noisy while the underlying process generating this data may have a lower dimension. The generated measurements can be considered as noisy points lying on a lowdimensional manifold embedded in a high-dimensional space (the space of measurements). In this project we study methods that map high dimensional data to a low dimensional space that preserves the structure of the data (i.e. it keeps nearby points nearby). Such methods are of interest for data visualization, where one needs to map data to two dimensional coordinates, and for regression, inter- and extrapolation tasks, where it is helpful to map the data to a low dimensional space. Linear projection techniques are the standard tool to perform these tasks. However, if there is a non-linear relationship between the intrinsic dimensions of the data and the observed dimensions, linear techniques may fail to produce sensible mappings. Approach The approach we consider in our research is a combination of several local linear mappings. Each local linear map is responsible for a region in the data space and produces a linear map of the data to the low dimensional space. A probabilistic framework allows us to find appropriate locations for the local linear maps and the regions for which they are responsible. Robust and fast methods have been developed (Verbeek et al., 2001; Verbeek et al., 2002). Figure 4: The one-dimensional structure (the curve) in a two-dimensional data set (the dots) discovered by the method developed in this project. The project is a collaboration with the Pattern Recognition Group of the TU Delft and is funded by STW. Classification of radar range profiles Drs. J. Portegies Zwart, Dr.ir. B.J.A. Kröse, Prof.dr.ir. F.C.A. Groen Objective The goal of this project is to automatically recognize aircrafts by examining how they reflect radar waves. Traditional methods of aircraft identification rely on the cooperation of the target aircraft. These methods are prone to both human and technical errors. Since misidentifications can have severe consequences, there is a demand for non-cooperative identification techniques. Approach In this project we investigate the use of High Resolution Radar (HRR) range 14 profiles as features for a non-cooperative classification algorithm for aircrafts. HRR range profiles are measurements of the reflectivity of aircrafts, measured along the line of sight between the radar and the target. In figure 5 an example of a HRR range profile is shown. As can be seen in the figure, the shape of the range profile depends on the shape of the aircraft. Therefore, it might be possible to determine the type of aircraft by measuring a range profile, and comparing it with a database of previously measured profiles of known aircrafts. HRR range profiles depend heavily on the pose of the aircraft at the time of measurement. Estimates of this pose are available through the use of a secondary tracking radar. Our research has resulted in a method of using these estimates to construct an estimate of the statistical distribution of HRR profiles. We have been able to obtain a more accurate estimate of the pose by combining information from the pose estimates and the behavior of certain peaks in the HRR range profiles. Other research included automatic alignment of HRR range profiles, and the use of linear feature extraction to improve classification accuracy. The project is funded by TNO-FEL. radar line of sight Figure 5: Example of a HRR range profile. The radar is located at the left, and looking to the right. Locations on the target aircraft, which reflect the radar signal highly, result in distinctive peaks in the measured signal. Therefore, part of the geometry of the target aircraft is encoded in the measured HRR range profile. Probabilistic models for distributed surveillance Drs. W. Zajdel, Dr.ir. B.J.A. Kröse Objective Surveillance systems often need to track a moving object through larger areas or buildings. In case the cameras (or other sensors) do not overlap, the sys- Figure 6: Automatic tracking of moving objects across different camera’s. 15 tem is faced with the problem whether an object observed with a camera at some time is the same object as observed by some other camera some time ago. Approach To deal with the uncertainty we use probabilistic networks. These methods are able to model the belief in target trajectories. Hidden Markov models can be seen as a special class. At the moment we adapt probabilistic methods developed for traffic surveillance to indoor tracking problems. An experimental set-up will be built. The project is funded by STW, in collaboration with TNO-FEL. Omnidirectional stereo Drs. R. Bunschoten, Dr.ir. B. Kröse Objective Range estimation from multiple images is sensitive to noise and lack of texture. We developed methods, which integrate information from multiple image pairs to obtain a robust depth estimate. Approach Our robot is equipped with an ‘omnidirectional’ vision system (see figure 7). While traversing the environment, images are captured. It is possible to make a spatial reconstruction of the environment from the images. We have developed a multi-baseline stereo vision algorithm for panoramic image data (Bunschoten and Kröse, 2001a; Bunschoten and Kröse, 2001b). For increased robustness, more than the two images required for reconstruction are used. Stereo data obtained from multiple stereo pairs are fused in a probabilistic fashion. Some results are displayed in figure 7. Bayesian methods for mobile robot navigation Drs. R. Bunschoten, Dr. N. Vlassis, B. Terwijn, Dr.ir. S. ten Hagen, Dr.ir. B. Kröse Objective Future ‘personal’ robots should be able to learn a representation of the world, in which they operate, autonomously. We develop probabilistic methods for environment learning, robot localization and navigation. Figure 7: The omnidirectional image from our vision system (left) and the depth estimates reconstructed from five images. 16 Approach For the environment representation we developed a method, which, instead of building a ‘geometric’ representation of the environment, builds an ‘appearance model’, consisting of all sensory representations. With this model the robot can localize itself after a new observation using a Markov procedure (Kröse et al., 2001; Vlassis et al., 2001). The implementation of the Markov localization procedure was made using a Monte Carlo (particle filtering) method. The robustness of this method to ‘kidnapping’ of the robot was shown at the RWCP symposium (see also figure 8). For the learning stage we developed a method which corrects odometric pose estimates (Hagen, ten and Kröse, 2001). The RWCP funding has ended, but the work is continued within a European (ITEA) project ‘Ambience’, in collaboration with Philips Research. 400 400 Monte Carlo Estimated Position 350 350 300 300 250 250 200 200 150 150 100 100 Monte Carlo Estimated Position 50 −150 50 −150 −100 −50 0 50 100 150 200 −100 −50 0 50 100 150 200 250 250 Figure 8: The influence of kidnapping the robot on the particle filter representation. In the middle we see the distribution of samples (representing the position of the robot) before kidnapping. The particles are condensed to the real location. In the right figure we see the distribution of particles indicating the robot’s position estimate after the kidnapping: the robot is uncertain about its position. It takes about five novel observations to reach again a condensed estimation. Reinforcement learning and neurocontrol Dr.ir. S. ten Hagen, Dr.ir. B. Kröse Objective In a joint project with the Control Group from the Delft University of Technology we investigate Reinforcement Learning (RL) for control applications, in particular for the continuous state domain and the applicability for real-world control problems. Approach We investigated how much exploration (excitation) we have to add to the control signal in order to obtain good learning performance. For non-linear systems we developed ‘Neural-Q’ learning which requires far fewer trials then other neural reinforcement learning approaches. The resulting feedback controller can be regarded as a linear feedback with a non-linear correction that compensates for the non-linearities in the system. The non-linear correction allows this feedback to operate in a larger region than conventional linear feedback, based on local linearities of the non-linear system. A Ph.D. thesis was defended in February 2001 (Hagen, ten, 2001). This project was funded by STW. 17 Kat holieke U n i v e r s i t e i t Ni j me g e n 18 2B Laboratory for Biophysics, University of Nijmegen Prof.dr. C.C.A.M. Gielen, Dr. H.J. Kappen, Dr. T.M. Heskes, Dr. W.A.J.J. Wiegerinck, Dr. A. Ypma, Drs. M.J.D. Westerdijk, Drs. M. Leisink, Drs. B. Bakker, Ir. T.A. Cemgil, Drs. L. Pantic, Drs. O. Zoeter, Ir. M.J. Nijman, Drs. J.J. Spanjers, Drs. N. Keyzer Description of the group The laboratory of Biophysics of the KUN is engaged in research on the theory of neural networks since 1988, with links to computational issues in neurobiology as well as applications in industry. Important research themes are stochastic learning processes; Learning of structure in large databases using probability models based on Bayes networks and Boltzmann Machines; Active decision processes with application in medical diagnostics; Confidence estimation for neural network prediction and relevance detection; Bayesian statistics; Recognition of musical rhythms; The functional role of dynamical synapses and pattern formation in neural dynamics. The group receives significant funding from the Japanese Real World Computing program and from various STW project subsidies. More information on the group can be found at: http:/www.snn.kun.nl/nijmegen Long term research projects Graphical models Wim Wiegerinck, Bert Kappen, Marcel Nijman, Martijn Leisink Funding: STW Pionier, RWC The aims of this project are to develop novel theory, methods and implementations for learning and reasoning in a complex dynamic multi-sensory environment. The approach to reasoning and learning is based on the axioms of probability theory. It encompasses directed graphical models and Boltzmann Machines. The research aims at the design of algorithms to enable learning and reasoning involving up to the order of 1000 variables. This allows for applications, which are order of magnitude larger than currently possible. The fundamental operation that must be performed during learning or reasoning tasks is to compute marginal probabilities for subsets of variables. In general, these computations are intractable since they require O(2 n) operations, with n the number of variables. Approximate schemes are therefore of great importance for real world computation. An introduction to this research can be found in (Kappen et al., 2001a; Kappen et al., 2001b; Kappen, 2001). In 2001, we have obtained the following results: • Standard mean field theory can be derived from Jensen’s inequality yielding a first-order lower-bound of the partition function. The quality of the bound is of crucial importance for the quality of the approximation. For directed graphical models, optimization of this lower bound provides an approximate EM-learning scheme with guaranteed convergence. We derived a systematic sequence of odd-order lower-bounds of the partition function that are guaranteed to improve upon the first-order lower-bound (Leisink and Kappen, 2001b; Leisink and 19 Kappen, 2001c). We applied this approach to compute bounds on the means and correlation’s in Boltzmann Machines (Leisink and Kappen, 2001a). • For (symmetric) Boltzmann Machines, the mean field equations can be derived as the first order approximation in a Taylor expansion of the marginals. Second order corrections in this expansion are known as the TAP correction. We generalized this expansion for asymmetric Boltzmann machines and graphical models of general form. First and second order approximations were derived and studied numerically. For sigmoid belief networks, the expansion method gives an extremely fast alternative to the standard approximation schemes (Kappen and Wiegerinck, 2001b; Kappen and Wiegerinck, 2001a). • A very recent development is the application of the Cluster Variation method (CVM) to probabilistic inference. CVM is a method that has been developed in the physics community to approximately compute the properties of the Ising model. The CVM approximates the probability distribution by a number of (overlapping) marginal distributions (clusters). The quality of the approximation is determined by the size and number of clusters. We derive novel iteration schemes for the Cluster Variation method (Kappen and Wiegerinck, 2002; Kappen, 2002). Figure 9: BayesBuilder graphical software environment, showing part of the Anemia network. The network consists of 91 variables and models some specific Anemia’s. The software implements some of the approximation techniques described above. Computational neuroscience Associative memory with dynamic synapses Lovorka Pantic, Joaquin J. Torres, Bert Kappen Funding: University of Nijmegen Recent biological findings show that the electrical properties of synapses depend strongly on recent presynaptic neuron activity. This phenomenon is known as synaptic depression and shows that the synapse has to recover after transmission of an action potential (Abbott at al., 1997, Tsodyks and Markram, 1997). In our previous studies (Pantic et al., 2000; Pantic et al., 2001a) we have shown that synaptic depression improves the ability of neurons to detect coincident spikes. We have also studied the effects of synaptic depression on associative memory. Our results reveal, in addition to the standard memory 20 states, a new type of behavior where the network resides in one memory state for some time and then rapidly switches to another memory state (see figure 10). Using linear stability analysis we obtain a phase portrait defining the parameter values for which this new phase occurs. We conclude that the functionality of synaptic depression affects the stability of memory attractors and enables flexible transitions among them (Pantic et al., 2001b). s 20 20 15 τrec neurons Paramagnetic phase 40 Oscillatory phase 10 60 5 Ferromagnetic phase 80 0 100 20 40 60 time 80 100 0 2 4 β 6 8 10 Figure 10: The rapid switching behavior of an associative memory network consisting of 100 neurons (vertical axis) with dynamic synapses and 10 stored patterns during 100 msec (horizontal axis). Blue indicates neuron firing. The network settles in a combination of one or more patterns for some time and then rapidly switches to another activation. This behavior may be relevant for sequential processing of perceptual information. Calcium signaling in non-excitable cells Dr. J.J. Torres, University of Nijmegen and University of Granada Dr. H.J. Kappen, Dr. P.H.G.M. Willems and Dr. W.J.H. Koopman, University of Nijmegen Funding: STW Pionier, MEC, MCyT and FEDER (Ramon y Cajal program) Ionized calcium (Ca2+) not only represents the most common signal transduction element relaying information within cells to control a wide array of activities, including secretion, contraction and cell proliferation, but also is invariably involved in cell death. To coordinate all of these functions, cytosolic Ca 2+ needs to be precisely regulated in space, time and amplitude. In this project we aim to model calcium signaling in different types of non-excitable cells and study calcium wave propagation in monolayers of those cells. As a first biological example we started with the pancreatic acinar cell. This cell has been used extensively as a biological model for studying the relationship between the spatio-temporal aspect of cystosolic calcium signaling and cellular activities such as fluid and protein secretion. Following this motivation, we have developed a mathematical model that includes realistic assumptions about different calcium fluxes described in living cells (see figure 11). The model reproduces qualitatively and quantitatively the dynamics of the experimentally observed calcium signals (Torres et al., 2001a; Torres et al., 2001b). Secondly and using this mathematical model, we have modeled an extended medium consisting of a monolayer of these non-excitable cells to explore the regions in the space of parameters where calcium waves can propagate in the medium (Torres et al., 2001a). The study has been useful to find regions in the space of parameters 21 where different oscillatory regimes appear. The ability of our model to generate traveling waves, allows to investigate the mechanism that underlies the highly complex spatio-temporal Ca2+ signals between and within non-excitable cells. B 2mM 100nM V leakER V leakPM V IP 3 V PMCA V CRAC [Ca2+] ER [Ca2+] cyt IP 3 DAG V SERCA PLC PIP 2 Receptor Hormone Figure 11: Schematic representation of the Ca 2+ fluxes present in the model for calcium oscillations in pancreatic acinar cells, and the diverse regulatory interactions of Ca 2+ : Grey background: compartments with high [Ca 2+]; white background: compartments with low [Ca 2+ ]. Modeling electrical activity in normal rat kidney fibroblast Dr. J.J. Torres, University of Granada and University of Nijmegen. Drs. N.L.N. Cornelisse, Dr. L. Theuvenet, Dr. E. Harks, Prof.dr. C.C.A.M. Gielen, University of Nijmegen. Prof.dr. D.L. Ypey, University of Leiden and University of Nijmegen Funding: NWO, MEC, MCyT and FEDER (Ramon y Cajal program) Membrane voltage (mV) 40 20 0 -20 -40 -60 -80 -//5 min Using standard Hodgkin-Huxley (HH) and Goldman-Hodgkin-Katz (GHK) formalisms we have developed a mathematical model to study excitability and action potential propagation in quiescent normal rat kidney fibroblasts (NRK) (Torres et al., 2002). One can induce artificially growing of NRK fibroblasts forming a two dimensional monolayer of such cells. In the monolayer the cells are connected electrically by gap-junctions. Using standard techniques it is possible to record the electrical activity of the cells when they are isolated, when they are growing normally and when the cell are growing anomalously (see figure 12). Theoretically, we can build two-dimensional networks of HH units to investigate what the conditions are 50 µM BAPTA-AM in which an artificial mono layer is able to reproduce the observed behavior in the recordings. Using standard techniques we were able to describe the main ionic currents responsible for the excitability of NRK fibroblasts. We have introduced this information in the model and we were able to reproduce the voltage-clamp and current-clamp experiments in NRK fibroblasts in both quiescent and densityarrested NRK fibroblasts (see, for instance figure 12). ICl(Ca)=0 pA 40 Membrane voltage (mV) 20 0 -20 -40 // -60 -80 22 4 min Figure 12: Electrical activity in density-arrested NRK fibroblasts. Top layer: Electrical signals recorded in a growing monolayer of NRK fibroblast before and after cytosolic Ca 2+ is buffered by BAPTA-AM. Erasing calcium from cytoplasm has the effect of reducing the calcium dependent chloride conductance. Bottom panel: Model simulations for the same conditions as the experiments in the top panel. Regulation of melanotrope cell activity in Xenopus laevis Drs. N. Cornelisse, Prof.dr. C. Gielen, Prof.dr. E. Roubos, Dr. W. Scheenen, Prof.dr. D. Ypey One of the mysteries in neuronal communication deals with the problem how extracellular information, which arrives from many other cells onto a cell, is coded into intracellular signals to initiate new cell activity (for example hormone secretion, DNA expression, protein synthesis). One aspect of this problem is that a large number of extracellular neurotransmitters is known to be involved. However, only very few intracellular messengers are available to transport the extracellular infor- mation from the cell membrane to the cell nucleus. In this project we investigate this problem in the melanotrope cell of Xenopus laevis, which is known to produce a hormone, which adapts the color of the skin to the background of the animal to camouflage it for predators. A minimal model is presented to explain changes in frequency, shape, and amplitude of intracellular Ca 2+ (second messenger) oscillations in the melanotrope cell (see Cornelisse et al., 2001 and figure 13). Local spatiotemporal changes in intracellular Ca2+ concentration are known to transport the extracellular signals near the membrane of the cell to the cell nucleus. The model describes the cell as a plasma-membrane oscillator (corresponding to the well-known Hodgkin-Huxley model) with influx of extracellular Ca 2+ via voltagegated channels in the plasma membrane. Figure 13 shows an example of changes in intracellular Ca 2+ concentration (top panel) as a function of the action potentials of the cell membrane (lower panel). Different extracellular neurotransmitters each have a different effect on the dynamics of various ion-channels in the cell membrane, which affect the membrane potential of a cell. Therefore, we have characterized the time constants, which determine opening and closure of the Ca 2+, Na+ , and K+ ionchannels. The relationship between the concentration of extracellular neurotransmitters and the time constants of ion-channel dynamics allow us to provide an accurate description of the communication between cells and of the intracellular Ca2+ oscillations. [Ca ]i (mM) 0.35 0.3 2+ 0.25 0.2 0.15 0.1 10 15 20 25 15 20 25 30 35 40 45 30 35 40 45 Figure 13: Simulation of calcium oscillations coupled to electrical bursting Voltage(mV) 20 0 -20 -40 -60 10 time(s) 23 Applied research projects Graphical models for data mining Alexander Ypma and Tom Heskes Funding: Technology Foundation STW, since May 1 2001 Goal In this project, we focus on the unsupervised modeling of dynamic processes with graphical models. We want to know how prior knowledge about a particular problem can be used to find meaningful models, while at the same time allowing for unsupervised ‘discovery’ of underlying patterns. In case studies we will explore which approximations are valid and necessary for obtaining a practically usable method. Example: modeling of heterogeneous time series Practical applications may be found, for example, in modeling of consumer behavior using sequences of phone calls or web navigation patterns. Here, we have prior knowledge about the origin of the data like e.g. the hierarchical structure of a website and possibly other information about a user. Since many different user types will visit a website, one should account for heterogeneity. One way to take heterogeneity into account is by adopting a probabilistic generative model for description of individuals. Learning of the individual model parameters and the labels of each data item (e.g. a sequence of web-pages or phone-numbers) can be done in an unsupervised manner using the EM algorithm. Two examples of this approach are shown in figure 14. The left figure illustrates the simultaneous learning of parameters of different regression models. The right figures show two transition matrices that describe the surfing behavior at a large commercial website, using a mixture of Markov chains. Each entry in a transition matrix yields the probability of clicking on a certain page-type given the currently visited page-type (e.g. ‘sports’ or ‘weather’ in a news-website). We assume that a certain surfer-type can be characterized by such a page-type transition matrix. Mixture models and (hidden) Markov models are instances of the broader class of graphical models. Outlook Other applications include the prediction of brand market shares as a function State transition matrix for user type 9 2 2 4 4 6 6 From state From state State transition matrix for user type 7 8 8 10 10 12 12 14 14 2 4 6 8 To state 10 12 14 2 4 6 8 10 12 14 To state Figure 14: Describing heterogeneity with mixture models. Left figure: simultaneous learning of parameters from three regression models using noisy realizations of these models; right two figures: modeling web surfing behavior with mixtures of Markov chains; frequently visited entries in page-to-page transition matrix are shown as bright (yellow) values; each surfer-type is modeled by a separate transition matrix. 24 of several macro- and microeconomic variables and prediction of option prices. Industrial partners in this project are currently: KPN Research and BrandmarC. Ultimately, we intend to develop a data mining tool based on graphical models that can be used in a practical setting. Multitask learning Bart Bakker, Bert Kappen, and Tom Heskes Funding: STW Pionier Approach Many real-world problems can be seen as a series of similar, yet self contained tasks. An example is newspaper sales, where we predict sales at different locations within the country. In neural networks this learning process can be implemented as follows. We build a feed-forward model (see figure 15) to represent the series of parallel tasks. In this model the inputs correspond to the inputs of the parallel tasks, e.g. weather figures and previous sales (both long term and short term) for the newspaper example. Each output corresponds to the output of one of the tasks (where for simplicity we have taken this output to be one-dimensional). All weights W connected to the inputs in this model are the same for all tasks, whereas the weights Ai connected to the outputs are different for each task. This architecture allows the model to detect ‘features’ in the inputs, and to use these features for regression. We use the same architecture for our multitask learning project, and add to it a Bayesian prior over the task specific parameters Ai . This prior introduces a ‘soft sharing’ of the parameters Ai between the tasks. In our approach, all shared parameters, including but not restricted to the hyperparameters specifying the prior, are inferred through a maximum likelihood procedure. The remaining parameters specific to each task are treated in a Bayesian manner. Results We apply the model to the prediction of newspaper sales for ‘De Telegraaf’. The data that we use to train the model consist of three years of Saturday newspaper sales figures at 343 outlets in The Netherlands, and corresponding inputs. Inputs include recent sales (-4 to -6 weeks), last year’s sales (-51 to 53 weeks), weather information (temperature, wind, sunshine, precipitation quantity and duration) and season (cosine and sine of scaled week number). We compared our Bayesian approach to ‘single task learning’, where we trained µ outputs yi A i µ bias hik W Figure 15: Neural network model. inputs xµ ij bias 25 one feedforward model for each task (so all parameters were task specific), and ‘non-Bayesian multitask learning’, where we used the model architecture shown in figure 16, and inferred all parameters through maximum likelihood optimization. We found that our approach yielded predictions that were more accurate than predictions made through non-Bayesian multitask learning, which in turn performed much better than single task learning. The task clustering aspect of our approach revealed hidden structure within the data. When we allowed two tasks clusters, we obtained one cluster corresponding to a model that was very sensitive to long term sales and season, whereas the other was most sensitive to short term sales. A closer study of the outlets assigned to each of the clusters (see figure 16) revealed a sensible division. Outlets, that were assigned to the long term/seasonal cluster, were located in the more touristic areas of Holland (where sales are strongly influenced by the holidays), whereas those in the short term cluster were located in the larger cities. This distinction between touristic and urban locations was detected by the model, even though it was not explicitly coded in the data. Figure 16: Clustering of Dutch outlets. Circles mark outlets assigned to either the ‘seasonal’ cluster (left panel) or the ‘short term’ cluster (right panel). Analysis and classification of dyskinesia in Parkinson Disease Drs. N. Keijser, Dr. M. Horstink, Prof.dr. C.C.A.M. Gielen Parkinson patients usually receive Levodopa medication. However, after using Levodopa for about 10 years, it becomes more difficult to adjust the proper dose of Levodopa. As a result patients either develop Parkinson symptoms, or (more frequently) develop involuntary movements (dykinesia) which cannot be suppressed by the patients. This is a very well known and hard problem in the treatment of Parkinson Disease. The aim of this project is to develop an automatic and objective procedure to detect dyskinesia and to find the proper dose and timing of Levodopa administration. Although many groups are working to solve this problem, the main difficulty so far has been to distinguish involuntary dyskinesia from voluntary movements. Movements of patients were measured using accelerometers on the wrist, upper arm, trunk and leg, which measure 3-D movements. Accelerometer signals are recorded on a small portable recorder and analyzed offline. After preprocessing of the accelerometer signals to obtain specific movement parameters, a neural 26 network was trained with these parameters as input and the rating of the movements of patients by experienced neurologists as output. After training the neural network was able to classify movements correctly into normal movements and dyskinesia (score 95%). The neural network also correctly rated the amount of dyskinesia on the usual m-AIMS score (scale between 0 and 4) which is used by neurologists to rate dyskinesia. The performance obtained in this study is by far the best in the literature reported so far. Moreover, this is the first study, which is able to distinguish between normal voluntary movements and dyskinesia. Quite interestingly, the neural networks also provide detailed information about the parameters, which are used to rate dyskinesia and to distinguish dyskinesia from voluntary movements. For example, it appears that movements of different body segments move in a much more coordinated way in voluntary movements than in dyskinesia. Information about the parameters, which distinguish between voluntary movements and dyskinesia, provide valuable information to assist neurologists in the diagnosis and treatment of dyskinesia. 2 physicians rating neural network rating AIMS score 1.5 1 0.5 eat sweep enter a door wash hands enter a door change clothes Sit & mental stand & mental walk clean the sheets serve & drink coffee wipe off the dishes knit wash the dishes Prepare & eat luncg set the table make coffee sit telephone pack−board stand take off shoes write a letter 0 Figure 17: Example of the dyskinesia rating given by the physicians (circles) and predicted by the neural network (dots with error bars) for the trunk for eighty-one one-minute intervals of various activities. one−minute intervals and the tasks performed in these one−minute intervals Neural networks for the paper industry Onno Zoeter, Stan Gielen, and Tom Heskes Funding: Kenniscentrum Papier en Karton Approach Papermaking is a difficult and only partially understood process. Traditionally, the production process is controlled and optimized by human experts that have gathered insights and rules of thumb through years of experience. The increase in the number of sensors and the amount of process data stored facilitates a more quantitative analysis. The goal is to obtain insight in the production process and to use this insight to help the machine operator improve the production. Our first attempt in this direction is the development of a model for the visualization of highly dimensional dynamic data, a typical example being the sensor readings of a running paper mill. The model that we use is a dynamic extension of a hierarchical mixture of principal component analyzers. The generative model 27 for each level of the hierarchy corresponds to a particular kind of switching dynamical system. Its graphical structure is given in figure 18. Results An example of the visualization obtained with our model is given in figure 19. The top level visualizes the complete data set, with cluster and subclusters of data points visualized at deeper levels. In a “parent” plot, the user can select interactively the number of “children” and their approximate center locations. In this way, the visualization focuses on particular aspects of the data. A problem with switching linear dynamical systems is that exact inference (computation of the probabilities of switches and visualizations given all observations) is intractable, i.e. it scales exponentially with the length of the time sequence. We have developed a new method for approximate inference, which scales linearly with the length of the sequence and can be generalized to inference in general dynamic Bayesian networks. Case study with Sappi Nijmegen In collaboration with Joost Dercksen and Tim Mulder (Sappi) and Dario Lo Cascio (TNO), we have been involved in a case study for the paper mill Sappi Nijmegen. Figure 18: Graphical structure corresponding to the third level of the hierarchical switching Kalman filter. Square nodes refer to switches (open: corresponding to subplots at the third level; shaded: subplots at the second level), elliptical nodes to continuous variables (open: latent variables corresponding to the location on each subplot; shaded: observations, e.g. sensor readings). The goal of this study was to search for relationships between pulp and process parameters and off-line paper quality parameters such as internal bond. Using self-organizing maps for data mining, we found interesting relationships, some of them novel to the process engineers. The quality of the data appears to be insufficient (many missing variables) to apply more quantitative supervised models. This will be a topic for further research. Decision support systems for medical diagnosis Wim Wiegerinck, Bert Kappen, Marcel Nijman, Ender Akay, SNN Nijmegen Jan Neijt, Eelco de Koning, UMC Utrecht Funding: STW Companies: Hiscom, Pfizer, Rand Corporation Computer-based diagnostic decision support systems (DSS) will play an increasingly important role in health care. Due to the inherent probabilistic nature of medical diagnosis, a DSS should be preferably based on a probabilistic 28 Paper Paper Paper Paper type type type type 1 2 3 4 Figure 19: Ten hours of production data from a paper mill projected using a hierarchy of switching linear dynamical systems. The labels encode different paper types. Bars below the subplots visualize the probabilities of the switches as a function of time. Note that the algorithm does not use the labels themselves. model. In particular Bayesian networks provide a powerful and conceptually transparent formalism for probabilistic modeling. A system modeled with a Bayesian network can typically generate a differential diagnosis based on the patient findings, a suggestion of additional tests that are expected to be most relevant to refine the diagnosis, and some insight in the disease mechanisms of the patient. In this joint project with the UMC Utrecht, we aim to build a large and detailed Bayesian network for diagnostic support in internal medicine. The practical application of Bayesian networks in large, complex domains such as internal medicine requires that several major obstructions are taken. One obstruction is that complex probabilistic models are intractable for exact computation (i.e. exact computation will cost too much CPU time). For this reason, we need fast and accurate approximate inference methods. Recently, cluster variation techniques gained much interest in this context. We developed methods to improve convergence of this approximation technique. Simulation results on large networks show that cluster variation can be considerably more accurate than other state-of-the-art methods such as the mean field approximation (Kappen and Wiegerinck, 2002). Another obstruction is that in internal medicine, patient-data is insufficient to train the model. In such a case it is common practice that model parameters are set by hand by human domain experts. Often, however, human domain experts are not able to express their knowledge in such a way that it can directly be used to set the model parameters. To overcome this problem we have developed methods that can use available expert knowledge to train the model (Wiegerinck and Heskes, 2001a; Wiegerinck and Heskes, 2001b). Furthermore, we develop software tools for modeling and inferencing large Bayesian networks. One of these tools is BayesBuilder, which is software for construction of Bayesian networks and for efficient computation. A public version of BayesBuilder can be downloaded from: http://www. mbfys.kun.nl/snn/Research/bayesbuilder/. 29 Hidden variable models for data mining Machiel Westerdijk, Wim Wiegerinck, Stan Gielen Funding: STW Hidden variable models are well suited to describe complicated data distributions in a transparent and compact manner. Therefore they provide an important tool for data mining. In data mining, however, one is often also interested in classification. However, usually hidden variable models are optimized by maximizing the joint (input-output) data likelihood. Therefore, they are suboptimal for classification tasks. On the other hand, models that are directly optimized for classification, e.g. support vector machines and multi-layer perceptrons, often do not provide insight in the underlying joint data distribution. In this project we aim to combine the advantages of hidden variable models and models that are optimized for classification. We developed methods to increase classification performance of hidden variable models, while keeping the likelihood as large as possible. In addition, we have developed methods to induce classification rules. These are if-then-else rules that explain to a user why classifications are reached in particular cases. The methods were tested on a generative vector quantizer and on a principal component analysis model on a number of real world data sets. Compared to maximum likelihood models, the results showed a significant increase in classification performances while the data likelihood remained close to that of their maximum likelihood counterpart. Furthermore, induced classification rules are found to be short and accurate (Westerdijk et al., 2000; Westerdijk and Wiegerinck, 2001; Westerdijk et al., 2001; Westerdijk, 2001). Genetic linkage analysis Martijn Leisink, Bert Kappen, Stan Gielen. Han Brunner, Antropogenetica KUN Funding: STW Pionier Localization of genes involved in a genetic disease is a promising real world application for several techniques discussed above. Currently we are developing a general scheme to find multilocus diseases in third party databases. In short, the procedure is as follows: The DNA of the individuals in several pedigrees is sampled at about 200 known locations. Furthermore the status (affected or unaffected) is known for each individual. Using a Hidden Markov Model we reconstruct the mode of inheritance based on the available data (see figure 20). Grandfather Grandmother Grandfather Grandmother A B C D B E B C 0 Selector bit 1 Phase bit A 1 Selector bit 0 Selector bit Father 1 0 Phase bit B 0 Selector bit Child A B B B B Phase bit A 0 Selector bit Mother 0 D A Selector bit 30 D B Figure 20: All parents-child relations are similarly modeled. This results in a graphical model for the whole available pedigree. Each child has two selector bits, which indicate whether the paternal or the maternal chromosome is inherited, and one phase bit, which holds the order of the alleles. Once we have this information, a Bayesian model gives us the likelihood that a specific location (or gene) is ‘linked’ to the disease. An extension to multiple genes is straightforward. The first experiments show that for single locus diseases our approach yields better results than the standard techniques in this field. Quantization of temporal patterns by neural networks Dr. H.J. Kappen, Ir. T.A. Cemgil, Biophysics, University of Nijmegen. Dr. P. Desain, Dr. H.J. Honing, Ing. P. Trilsbeek, NICI, University of Nijmegen. J. de Haan, D. Weiermans, SoundPalette, Utrecht. H. Timmermans, Hogeschool voor de Kunsten, Utrecht. Leigh Smith, Tomandandy, New York. R. Boulanger, Berklee College of Music, Boston. M. Dunn, Coda Music. G. Lengeling, Emagic Funding: STW Automatic music transcription refers to a computer generation of a high level description of musical performance, for example in the form of western music notation. To generate a meaningful transcription, one has to model timing deviations from a strict mechanical template (i.e. music notation) (Desain et al., 1999). We can identify deviations roughly in two time scales: in the short scale due to expressive timing deviations and in the long scale due to tempo fluctuations. A robust transcription system has to track tempo fluctuations and filter out the expressive timing deviations to produce an acceptable score. The former task is tempo tracking and the latter is rhythm quantization. We formulate tempo tracking as a hidden state estimation problem in a linear dynamical model. The hidden states of the model represent the beats of the tempo track. In absence of data, this model describes likely tempo fluctuations (Cemgil et al., 2000b; Cemgil et al., 2000c). We use a quantizer as a measurement model. Quantization plays an important role in identification of the correct tempo interpretation by defining it as the one that results in simpler quantization. We use the quantization model to infer the likelihood of a beat given an onset sequence (Cemgil et al., 1999; Cemgil et al., 2000a; Cemgil and Kappen, 2001c). We have also extended our model where model parameters are adapted online and in real-time using an EM algorithm (Cemgil and Kappen, 2001b). The model is implemented in real-time and demonstrated in (Cemgil and Kappen, 2001a). Tempo Tempo > Score Figure 21: When a musical score is performed, expressive timing and tempo changes cause deviations from exact timing implied by the score notation. The figure displays a graphical model of such deviations. When a performance is observed, the most likely score and tempo are estimated by application of the Bayes rule. > Score Perf Perf 31 32 2C Algorithms and Program Methodology, Leiden University Prof.dr. J.N. Kok, Prof.dr. G. Rozenberg, Prof.dr. H. Spaink, Prof.dr. A. Ijzerman, Dr. Th. Bäck, Dr. C. Schmidt, Dr. M.W. Beukers, Dr. W. Kosters, Drs. E.-W. Lameijer Description of the Group The research within the section Algorithms and Program methodology (ALP) is concentrated around the topics Bioinformatics, Coordination, Optimization and Data Mining. • Bioinformatics. The application of computer science technology for solving problems in biochemistry and genomics. The expertise in molecular biology, medicinal chemistry and computer science with particular aspects on data mining and adaptive search and optimization comes together in a way that allows for the exploitation of interdisciplinary synergies between these disciplines. • Coordination. Software systems are difficult to maintain due to their inherent complexity. Coordination of components in Software Systems studies how complex systems can be constructed from components using a clear distinction between individual components and their coordinated interaction. • Optimization. Many problems can be described as an optimization problem in which, within certain constraints, optimal values have to be found for the variables. The techniques that are used are stemming from Simulated Evolution, in particular from Evolutionary Computation. Research is being done on the development of new algorithms, on the applicability on benchmark problems and on concrete “real-life” cases. • Data Mining. Data Mining looks for hidden patterns in large amounts of data. Within ALP the focus is on association rules and on techniques from the area of Natural Computation. Below we report on all topics, except for the Coordination topic. More information on the group can be found at: http://www.wi.leidenuniv.nl/CS/ALP Long-term Research Projects Evolutionary Computation Today, evolutionary computation is one of the key technologies for implementing adaptive and self-adaptive algorithms useful for a variety of tasks in fields such as optimization, business intelligence, and machine learning. Evolutionary algorithms, encompassing methods such as genetic algorithms and genetic programming as well as evolution strategies, are gleaned from the model of organic evolution. They adapt to an unknown environment and search for optimal solutions by means of an evolutionary process, improving a set of candidate solutions by selection and by variation operators such as mutation and recombination. The state-of-the-art in evolutionary computation is now 33 characterized by a mature understanding of the basic working principles of these algorithms and a lack of an in-depth understanding of many of the more specific features of their functioning. Concerning practical applications, evolutionary algorithms have proven their capabilities and their economic relevance by many successful applications to industrial problems. Our research focuses on practical as well as theoretical aspects of evolutionary algorithms, including among others their behavior in dynamic environments, with a particular interest in exploiting the self-adaptive capabilities of evolution strategies and genetic programming for such kind of problems, their convergence velocity analysis in case of dynamic/bimodal test problems, their efficiency in comparison to classical optimization methods, the deeper understanding of variation operators (mutation, multi-parent recom bination), and the general principles of adaptable and self-adaptable software. Concerning their applications, current topics of research include the development of an evolutionary DNA-computing approach implementing an evolutionary algorithm in vitro, with the goal of exploiting massive population sizes, the exploitation of evolutionary algorithms and data mining techniques in bioinformatics applications, and the application to technical design problems occurring in industry. Although it is a common assumption that a problem and its quality measure never change during the run of an optimization algorithm, this restriction is too simplistic in many real-world situations. As an important aspect of research, the consequences on evolutionary optimization when the problem is allowed to change over time are studied, in contexts such as control problems as well as adaptive and self-adaptive properties of software. In some cases, every problem dealing with changing situations can be considered as a new optimization problem, which has to be solved independently of previous and future problems. Research can then be concentrated on the exploitation of previous solutions to speed-up finding new solutions. More complex situations arise when solutions and problems interact with each other. This is typically the case with dynamic scheduling problems, such as dynamic job shop scheduling problems. Here, new jobs can arrive which have not been taken into account when the optimization started. If some old jobs are already being processed, the set of new schedules is limited. A schedule should therefore not only be good, but also sufficiently robust to allow for efficient solutions of yet unknown situations, which are likely to arise later on. The target is to obtain a better overall solution than can be obtained by local optimization only. The application of robust optimization techniques is investigated both from a theoretical point of view and from a practical point of view. In cooperation with the Dutch Air Traffic Control, these techniques will be applied to problems Figure 22: Abstract sketch of an evolutionary algorithm operating in a dynamic environment. 34 occurring in air traffic control. This year we published on evolutionary art (Hemert and Jansen, 2001), dynamic environments (Hemert van et al., 2001), constraint satisfaction (Hemert van, 2001), regression problems (Eggermont and van Hemert, 2001), case-based memory (Eggermont et al., 2001), overview of Evolution Strategies (Bäck, 2001), theory of genetic algorithms (Bäck et al., 2001a), noisy Evolution Strategies (Markon et al., 2001), distributed direct optimization (Emmerich et al., 2001), engineering applications (Giotis et al., 2001), the shape of evolutionary search (Jelasity, 2001) and genetic programming (Banzhaf et al., 2001a, and 2001b. Self-Adaptive Software Self-adaptivity is one of the key discoveries in the field of evolutionary computation, originally invented in the framework of the so-called evolution strategies in Germany. The concept enables the algorithm to dynamically adapt to the problem characteristics and even to cope with changing environmental conditions - as they occur in unforeseeable ways in many real-world business applications. In evolution strategies, self-adaptivity is generated by means of an evolutionary search process that operates on the solutions generated by the method, as well as on the evolution strategy parameters and configuration, i.e. components of the algorithm itself. This concept therefore enables the algorithm to adapt itself to varying conditions of its environment, which is a desirable property for software in general. Therefore, a deeper understanding of the principles underlying self-adaptation is desirable and will have an influence to fields outside evolutionary computation as well. Genetic Programming complements the self-adaptive principles of evolution strategies from the automatic programming point of view. Genetic Programming is a label for that set of Evolutionary Algorithms, which, for the purpose of quality evaluation, represent a potential solution to a decision problem as a computer program. The long-term objective of our research is an autonomous Genetic Programming paradigm, which accepts an arbitrary problem representation and does not require manual interference. We currently pursue the following theoretical and applied approaches to this objective. The use of a Turing-complete target language implies that Genetic Programming is applicable to an arbitrary problem domain. In particular, Genetic Programming is increasingly being applied to real-world decision problems, and especially to data mining. A generic key task in data mining is the identification of problem-relevant decision variables and their relationships. Often, traditional approaches require manual analysis and processing prior to the employment of a standard solver. Thus, we focus on an autonomous identification of the variables and their relationships by coevolving genotype-phenotype mappings with the genotypes such that beneficial mappings encode and exploit highly relevant variables. Agent-based strategy design in dynamic environments with incomplete knowledge is a field in its own right with significant application potential to a wide range of domains such as economics, robotics, and sociology. We focus on the automatic production of autonomous decision makers which exhibit problemoriented behavior under the control of an individual Machine Learning paradigm, such as Genetic Programming. Autonomy of Genetic Programming requires 35 the effectiveness of solely implicit principles in order to turn manual interference obsolete. Natural phenomena imply the emergence of autonomous agents, such as biological organisms. We focus on the identification of algorithmic metaphors of such phenomena in order to design a Genetic Program-ming system that autonomously deals with generic tasks. Ultimately, we focus on a phenomenon we call autopoietic programming, i.e. the emergence of self-maintaining algorithms in a noisy working memory which exhibit problem-oriented behavior. In 2001 we published on self-adaptation (Bäck et al., 2001b), dynamic optimization (Nijssen, 2001), the evolution of genetic code (Keller and Banzhaf, 2001), and autopoietic solutions (Keller, 2001a). Data-Mining Data mining is concerned with the analysis of large amounts of data. Usually these data are gathered in some automated fashion, for instance in the case of market basket analysis. In Leiden we have a special interest in the application of techniques from natural computing, such as neural networks and evolutionary programming. In many cases the programs find there own route through the problem space, but for most applications it turns out to be important to have human guidance during the process. Human experts are always necessary to raise the questions - and to interpret the answers. The main goal of data mining is to extract (possibly unanticipated) information from the data. Interesting topics are classification, including clustering and prediction. We are in particular interested in association rules: these rules try to find statistically significant if-then rules in large datasets. The rules can be used to understand what is happening, and provide, e.g., the management with proper information about business processes such as purchase behavior. These association rules may connect different databases; in this case inductive logic programming is a good tool to fasten the queries. We have performed several projects for the industry, including insurance companies, wholesale departments, supermarkets and food producers. Briefly speaking, such a project starts from existing (but sometimes unused as yet) databases, where several of the above mentioned techniques are used to provide the information needed. In the case of association rules, the result may be an ordered list of if-then rules that are present, combined with information about statistical significance, and coupled with data about pricing, season and so on. On the one hand, known rules should be present (their occurrence is a proof of principle for the method), on the other hand new rules should appear otherwise the method would not be necessary. In 2001 we published on knowledge discovery and evolutionary computation (Eggermont, 2001), evolutionary algorithms in data-mining (Bäck and Schütz, 2001; Keller, 2001b), association rules for multiple relations (Nijssen and Kok, 2001), fuzzy association rules (Graaf de et al., 2001), natural data mining techniques (Kok and Kosters, 2001), intelligent data analysis (Kok, 2001), mining E-markets (Putten van der and den Uyl, 2001), multidimensional scaling (Wezel van et al., 2001) and data fusion (Smith et al., 2001). 36 Bioinformatics Deriving knowledge from large databases and identifying structure in highdimensional spaces are some of the key problems that play a strong role in bioinformatics, i.e. the application of computer science technology for solving problems in biochemistry and genomics. At Leiden University, the expertise in molecular biology, medicinal chemistry and computer science with particular aspects on data mining and adaptive search and optimization come together in a way that allows for the exploitation of interdisciplinary synergies between these disciplines. At present, research is conducted on projects related to using molecular biology to implement evolutionary algorithms (Evolutionary DNA Computing), using data mining techniques for drug design (intelligent data analysis for drug design), and using optimization methods to support the drug design process (optimization of drug structures). A brief overview of these activities is given in the next section. In 2001 we published on Environmental Epidemiology (Lamers, 2001) and DNA Computing (Schmidt et al., 2001). Applied Research Projects Evolutionary DNA Computing Prof.dr. J.N. Kok, Prof.dr. G. Rozenberg, Prof.dr. H. Spaink, Dr. Th. Bäck, Dr. C. Schmidt, Drs. C. Henkel, Leiden University Funding: NWO Project DNA computing studies the use of nucleic acids (the molecules that store and transmit information in living cells) for attacking various computational problems. DNA is especially suitable for dealing with NP complete optimization problems, which require huge search spaces but relatively simple operations. Since billions of DNA molecules can act as billions of parallel processors, the potential to reduce computing time is enormous. Current limitations in the implementation of DNA computing are scalability and the rather high error-rates of biochemical operations. In order to solve problems of realistic sizes, laboratory procedures must become both less error-prone and much faster. The latter is being achieved by increased automation and miniaturization in molecular biology, including advanced liquid handling robots, DNA arraying techniques and single molecule Figure 23: The evolutionary DNA computing loop illustrated, with emphasis on using amplification and diversification to model variation operator, and a selection operator such as (µ,)-selection which eliminates a large portion of unfit molecules. 37 detection. Errors are harder to avoid, since many biochemical procedures are intrinsically ‘noisy’. At Leiden University, we try to exploit this fuzziness by implementation of evolutionary algorithms in DNA. Pools of potential solutions can be generated by mutation and recombination, and screened for ‘fitter’ molecules. Iteration of this process will provide DNA computing with enormous search spaces, yielding a potentially powerful application. In addition, the similarity of evolutionary DNA computing to natural molecular evolution may inspire new abstractions to be used in evolutionary computation in general, as well as computationally inspired biotechnologies. Our current research focuses on both the adaptation of molecular biological and biophysical techniques and the development of algorithms. More information at www.lcnc.nl and wwwimp.leidenuniv.nl/~henkel. Intelligent Data Analysis for Drug Design Prof.dr. A. IJzerman, Prof.dr. J.N. Kok, Dr. Th. Bäck, Dr. M.W. Beukers, Dr. W. Kosters, Leiden University. Prof.dr. G. Vriend, Dr. E. Bettler, Nijmegen University Funding: NWO-BMI Project Data mining and evolutionary computation have a strong potential in analyzing molecular biology databases and supporting the drug design process in many ways. In cooperation with the Leiden / Amsterdam Center for Drug Research (LACDR) and the Center for Molecular Bioinformatics (CMBI) in Nijmegen, the ALP group is involved in a bioinformatics project focusing on so-called G proteincoupled receptors (GPCRs), as they are the major target of the majority of today’s medicines. These receptors are membrane-bound proteins that serve as anchor points for hormones and neurotransmitters, such as the classical biogenic amines (nor)adrenaline, dopamine and serotonin. It is estimated that the human genome consists of at least 1000 different receptors, far more than the 200-300 presently known. This poses formidable questions and challenges, e.g.: What are the endogenous ligands for these orphan receptors, what is their function, are these validated targets for novel drugs etc.? Within this project, data mining and evolutionary computation techniques, developed for intelligent data analysis and efficient search processes in vast Figure 24: Left: Charge distribution on VdW surface of CGS15943 (red zone: negative; blue zone: positive); middle: same as previous, but now as ‘fingerprint’; right: new derivative with good receptor affinity. The transition from middle to right should generate many structures in an “automated” way by means of an evolutionary optimization rather than the trial-and-error approach used so far. 38 search spaces - such as biological sequence spaces - are introduced to the fundamental problems in determining function and used to finally design novel chemical entities that may be the lead structures for new drugs. In particular, the work aims at using these methods for directly linking sequence with function, thereby avoiding the diffcult structural element mentioned above. Optimization of Drug Structures Prof.dr. A. Ijzerman, Prof.dr. J.N. Kok, Dr. Th. Bäck, Drs. E.-W. Lameijer, Leiden University In addition to the data mining elements, the aspect of search and optimization in chemical structure space is also relevant within the framework of drug design related to GPCRs as outlined in the previous project. Direct (i.e. with known protein structures) and indirect molecular modeling approaches (i.e. in the absence of a protein structure, and based solely on the structures of compounds binding to such proteins) have been developed over the years. Today the indirect approach is most feasible for GPCRs in view of the absence of experimentally determined atomic coordinates for any member of this protein family. However, it is anticipated that rather soon the first realistic structures will be available given the rumors that abound. Genetic and evolutionary algorithms are powerful search methods gleaned from the model of organic evolution that can be used for such tasks. The basic idea is to use such an evolutionary algorithm to optimize molecules composed of components taken from a structure database so as to fit a given target shape as precisely as possible, i.e. the algorithm creates a chemical structure that matches the target so as to yield a certain desired functionality. In the absence of an experimentally determined receptor protein structure, the target in this case can be a so called supermolecule or receptor fingerprint, composed of the characteristics of one (see figure 25) for more ligands known to bind to a given receptor, in fact constituting a perfect example of the indirect approach. The mapping (not the generation) of a chemical structure to the shape and electronic characteristics of such a target can be performed by means of existing software tools (CORINA, SPARTAN, Sybyl, INSIGHT/Discover, etc most of them available at Leiden University), while the generation (i.e. optimization) of the structure is done by the evolutionary algorithm. Figure 25: Examples of applications of evolution strategies: Traffic light control optimization (left), elevator control optimization (middle), metal stamping process optimization in automobile industry (right). 39 40 2D Pattern Recognition Group, Delft Technical University Dr.ir. R.P.W. Duin, Dr. A. Ypma, Dr.ir. D. de Ridder, Dr. D.M.J. Tax, Dr. M. Skurichina, Drs. E. Pekalska, Ing. P. Paclik, Ir. P. Juszczak Description of the group The topic of statistical pattern recognition has been studied for more than 30 years within the Delft Pattern Recognition Group. Since 1988 this research has been focused on an understanding of neural network classifiers and a comparison with traditional classification techniques. Recently, methods for novelty detection and combining classifiers have been studied. In addition the question of how real world objects can be optimally represented for recognition purposes receives more and more attention. As the research is focused on sensor based applications observing natural scenes it is called Sensory Pattern Analysis (Duin, 2001; Jain et al., 2001; Kuncheva et al., 2001; Loog et al., 2001). More information on the group can be found at: http://www.ph.tn.tudelft.nl/Reesearch/neural/index.html Research projects Machine diagnostics by neural networks Dr. A. Ypma, Dr.ir. R.P.W. Duin, Delft University of Technology. Partners: Landustrie (Sneek), TechnoFysica (Barendrecht) Funding: STW This project was finished in 2001. Some of the results have been implemented in a commercially available software system for machine health monitoring with selforganizing maps, called MONISOM. MONISOM consists of tools for reading and processing datasets, (re)training a SOM (a neural network realizing a Self Organizing Map) on a dataset, analyzing the trained map and evaluation of a new dataset onto a trained map. Typical analyses include: map cluster analysis, novelty analysis of a new dataset compared to a known dataset, novelty and cluster analysis per feature, setting of thresholds, inspecting regions of the map where a new dataset has many ‘hits’ and generation of a trajectory plot over the map along with monitoring of the novelty through time. The user group is investigating the use of MONISOM in practical machine monitoring (on-line pump monitoring, off-line analysis of ship engine vibration, monitoring of rotating equipment in paper industry). Moreover, medical monitoring problems (depth of anesthesia during surgery, detection of eyeblink rate in Tourette’s syndrome patients, analysis of EEG/MEG signals for detection of epilepsy or Alzheimers disease) may be approached with MONISOM.MONISOM has received the 2001 SKBS Award for Best demo/application at the 13th conference on AI in Belgium and The Netherlands. More information can be obtained at: http://www.mbfys.kun.nl/ ~ypma/project/monisom/index.html, http://www.mbfys.kun.nl/~ypma/papers/ list_of_papers.html, and in (Ypma, 2001b; Ypma et al., 2001a; Ypma et al., 2001b; Ypma et al., 2001e). Further information about commercial use of MONISOM can be obtained from J. Valk, Landustrie Sneek b.v. P.O.Box 199, 8600 AD Sneek, The Netherlands, Phone: +31 515 486888 E-mail: [email protected]. 41 Nonlinear-feature extraction from image data for classification tasks Dr.ir D. de Ridder, Dr.ir. R.P.W. Duin, Delft University of Technology Contacts: Prof.dr. J. Kittler, University of Surrey, UK Funding: NWO Feature extraction in pattern recognition is related to three goals: a better understanding of the objects to be classified, more accurate classifiers and cheaper classifiers. This topic has been extensively studied in the past for feature selection and for linear feature extraction. Nonlinear feature extraction, however, has hardly been studied until now due to the lack of a well defined restriction of the infinite universe of possible nonlinearities and due to insufficient tools. This project aims to investigate nonlinear feature extraction capabilities of adaptive methods, such as neural networks. The project was ended in 2001 with a thesis. It focused on three main questions: 1. Can adaptive methods (among others neural networks) be trained to solve image processing problems? 2. How can one use prior knowledge about the problem as well as possible in designing and training neural networks? 3. To what extent can one learn more about problems by studying neural networks trained to solve them? Three techniques were applied: supervised feed-forward networks for both object recognition (classification), image filtering (regression), and unsupervised clustering methods for segmentation and object recognition. The main conclusion is that neural networks are mostly applicable to complex problems which cannot easily be split into sub-problems, but for which a good performance criterion can be given. Inspection of neural networks after training is difficult, as the solution found is hard to express in terms of image processing primitives. If sufficient prior knowledge of the problem is present, it is pre-ferable to specify a simpler adaptive model, as besides good performance this will give solutions, which provide more insight (Ridder de, 2001; Ridder de et al., 2001; Musa et al., 2001a; Musa et al., 2001b). Dissimilarity-based Pattern Recognition Drs. E. Pekalska, Dr.ir. R.P.W. Duin, Delft University of Technology Funding: NWO For learning purposes, objects are usually represented by features. Defining welldiscriminating features for a given task is not trivial, and sometimes not even µ − 0.5 λ1 e 1 µ µ + 0.5 λ1 e 1 e1 P1 P2 P3 P4 P5 P6 P7 P8 P9 42 Figure 26: This illustrates a set of 9 subspaces found for the class of the handwritten digit 5. In each row the mean of the subspace is shown and the deviations in the direction of the first basis vector, shown on the right. possible. An alternative is a representation built by using the concept of dissimilarity (distance). An object is then characterized in a relative way, i.e. by its dissimilarities to a set of prototypes. The use of dissimilarities is especially appealing when features can hardly be defined in a natural manner, e.g. when some particular characteristics of objects or measurements, like curves or shapes, are considered. This approach not only offers a good alternative for the traditional feature base approach. It may asymptotically have a very good performance: under rather general conditions the classification error for dissimilarity-based classifiers goes to zero while in feature based approaches an intrinsic class overlap exists that cannot be removed by more training examples, see figure 27 (Pekalska and Duin, 2001a; Pekalska et al., 2001; Duin and Pekalska, 2001; Pekalska and Duin, 2001c; Pekalska and Duin, 2001b). Combining classifiers Drs. M. Skurichina, Dr.ir. R.P.W.Duin When a pattern recognition problem is complex and the amount of training data to learn a classifier is limited, it may be difficult to construct a good single classification rule to solve the problem. A well-known approach that allows us to improve the performance of a weak (bad performing) classifier is to use the combined decision of an ensemble of weak classifiers instead of a single decision of one weak classifier. It is known that the ensembles often outperform single classifiers. In classifier combination, it is believed that diverse ensembles have a better potential for improvement on the accuracy than non-diverse ensembles. We put this hypothesis to test for two methods for building the ensembles: Bagging and Boosting, renowned for their success, as reported in numerous experimental studies. Our study showed a vague support for the hypothesis of the beneficial diversity for Boosting and virtually no support for Bagging. However, the performance of ensembles in bagging and boosting is affected by the choice of the classification rule, by the training sample size used to train classifiers and by the data distribution. Therefore, these factors should also be involved in future studies in order to test the hypothesis properly (Skurichina and Duin, 2001; Skurichina, 2001). One-class classification Dr. D.M.J. Tax, Ir. P. Juszczak, Dr.ir. R.P.W. Duin Funding: NWO This new project is inspired on the project ‘Confidence levels in neural networks’, where the problem of one-class classification was posed. In that research project, A B Dissimilarity matrix X? D Training set Is there a feature space X x2 for which Dis Rk Euclidean distances D x1 Figure 27: A possible approach to dissimilarity based pattern recognition: embedding the dissimilarities in a hypothetical feature space. 43 three types of one-class classifiers were distinguished, (1) the density estimators, (2) the boundary estimators and (3) the reconstruction methods. A very important issue that should be considered, is the representativity of the training set. When the data is represented well, in the sense that the distribution of the training set is very comparable to the distribution which will occur in real applications, the density methods will perform well (also assuming reasonable sample sizes). In the case of an unknown outlier distribution, the density methods completely fail and the boundary methods are to be preferred. Finally, when the generating model behind the data is known (or can be approximated with reasonable precision), it is usually better to use this model and apply a reconstruction model to detect outliers. These one-class classifiers were applied to several artificial and real world problems, like machine diagnostics, handwritten digit recognition and image database retrieval. In all cases satisfactory performances were achieved, but it appears that the results depended on the chosen representation of the objects to be classified. The oneclass classifiers suffer significantly more from a poor representation, which ignores important characteristics of the data, than a normal classifier. Poor representation will result immediately in a poor distinction between target and outlier objects, while in normal classification it might be still improved by feature selection or feature extraction. One of the major tasks in this research is therefore to find automatic procedures to perform feature selection in the one-class classification problem. When only example objects of the target class are present, it is not clear which features are useful and which are useless. When some example objects from the outlier class are present, it may be possible to apply the classical feature selection tools. It should be investigated if these well-known selection tools suffice, and what the best methods are for obtaining outlier objects (Tax, 2001; Tax and Duin, 2001b; Tax and Duin, 2001a; Tax and Duin, 2001c). Tools for nonlinear data analysis Drs. J.J. Verbeek, Dr.ir. B.J.A. Kröse, University of Amsterdam Dr.ir. D. de Ridder, Dr.ir. R.P.W. Duin, Delft University of Technology Partners: Noldus BV, Shell EPT/RF, NATO, TNO-FEL, KIQ, Unilever Research Funding: STW Current computerized measurement systems and data acquisition systems deliver a huge amount of data. Because several sensors are often measuring the same physical phenomenon, the intrinsic dimensionality of the data will in many cases be lower than the dimensionality of the data itself and only depend on the degrees of freedom of the observed phenomenon. If the dimensionality of the measurement space is not reduced correspondingly by some mapping, the outcome of any analysis of the measurements may suffer from an increased noise resulting from more sensor signals, instead of taking advantage of the increased information or resolution. Feature extraction and feature reduction thereby become more and more important in relation with increasing sensor capabilities. However, standard analysis packages are often limited to linear projections, while the data not necessarily reside on a linear manifold. Recently, a number of novels promising techniques for nonlinear projections have been developed. These techniques show to be useful if the number of datapoints is larger than the dimensionality of the data, but may perform sub-optimally if the 44 number of datapoints is small. In this project we focus on situations in which the number of datapoints is in the order of the dimensionality of the data. Depending on the application (visualization, compression or classification) we will define criteria to assess the performance. Novel methods will be tested on these criteria and on speed. Beside several theoretical studies, we worked on a real-life data set supplied by Noldus BV. This set contains movies of observed animal behavior and sets of derived features. In the first instance, prior work on this set has been successfully repeated. Application of mixture-of-subspace models was not yet successful. Figure 28: Some profiles used for the study on automatic analysis of animal behavior. Multi-spectral image segmentation Ing. P. Paclik, Dr.ir. R.P.W. Duin Partner: Dr.ir. G.M.P. van Kempen, Dr. R. Kohlus, URV, Vlaardingen Backscatter images (BSE) acquired by scanning electron microscopy (SEM) play an important role in structural analysis of laundry detergents. Segmentation of BSE images is an essential step in the analysis of powder properties. We have developed a supervised algorithm for the segmentation and we proposed a novel way to shorten feature computation time by feature selection. Currently, the product analyst segments these images interactively, which is both time consuming and an inaccurate operation. Moreover, the results delivered by different experts are considerably varying. One of the reasons is that the analyst bases her decisions on the single-band BSE that lacks information about underlying chemical composition. The segmentation algorithm may be split into two parts. In the first one, a classifier is trained using multi-band images. In the second, new backscatter images are labeled by the trained classifier (Paclik et al., 2001a; Paclik et al., 2001b). The segmentation process is shown in figure 29. training backscatter classifier: backscatter image aligning images labeling backscatter image computing features dataset with all features multi−spectral image aligned images feature selection training classifier dataset with selected features trained classifier labeled backscatter labels segmenting new image: backscatter image computing features labeling backscatter image dataset with selected features labeled backscatter Figure 29: The procedure for automatic segmentation of backscatter images trained by multi-spectral electronmircroscope images 45 46 2E System Technology IWI-Cluster, University of Groningen Prof.dr.ir. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, Ir. S. Achterop, Dr.ir. R. Moddemeijer, Drs. M. van Veelen. Associated Researcher: Ir. A.J.W.M. ten Berg (Philips), Drs. M.H. ter Brugge, Ir. W.J. Jansen (Dacolian), Drs. M. Diepenhorst (vanVliet), Ir. B.J. van der Zwaag (University Twente), Dr. R.S. Venema (Boise State University), Ir. F.W. Greuter (Rohill) Description of the group The System Technology Cluster of the RuG Institute for Mathematics and Computing Science (IWI) is engaged in research on the systematic design of computational intelligent units and their integration in large, distributed systems. The Cluster is part of the KNAW Institute for Programming and Algorithmic Research (IPA) and co-operates with the Chair in Signals & Systems of the University of Twente. More information on the group can be found at: http//www.cs.rug.nl/Research/St/ Aims and scope Computational Intelligence (CI) has rapidly become an accepted technology within a broadening area of applications. From the many different CI disciplines we will primarily target on a neural executionable model to fuse a more heterogeneous set of knowledge. Such a model will be a multi-net, abundantly present in nature and so far little used in artificial neural systems. The popular monolithic neural networks have been studied in the past decennium to find the principles of operations and most importantly the reasons for not operating. It appeared that an elaborate feature identification was required to facilitate the neural capability by principle, while further developments in network topology and learning algorithms were demanded to realize the promised quality. The more flexible alternative is the multi-net: a deliberate composition of neural nets and other algorithms, that have the promise to be easier to design, mold and understand. They have a special relevance in distributed environments, where the neural parts may easily be implemented as agents in a telematic meaning. Research “Module”, “object” and “agent” are different names for the single purpose of describing complex problems using concepts of local autonomy. The words reflect the differences in development of the respective fields of technology. Neural networks are at the start of such a development, where we only are starting to devise more complex systems with neural behavior. We are pursuing such developments in three related directions. In the past, neural nets have largely been cast into software, and hardware realizations have been few and only moderately successful. This is expected to change with the coming of Polymorph Computing. Character recognition (or the super class of text mining) is based on the intensive and large-scale application of clustering and classification techniques. Especially in mobile environments, neural networks will play a major role. Lastly we devote our attention to process identification and the early detection of abnormalities. 47 Neural Networks in Polymorph Computing Prof.dr.ir. L. Spaanenburg, Ir. S. Achterop The history of micro-electronic hardware is governed by the quest to find products that create a new and larger market for the new technology. From the gate came the adder, followed by the arithmetic unit, the micro-processor, the computer and now the system-on-a-chip. Steadily the devices became larger and more programmable. Meanwhile more ways to program are invented, bridging the gap between hardwired and soft programmable functions. The recently coined name for creating functions in an arbitrarily programmable realization is Polymorph Computing, indication that a single function may have many SW/HW realizations. Typical polymorph architectures are based on the assembly of computer cores, memory fields and field-programmable parts. Functions can be executed in software on the cores, soft wired in the Field-Programmable Gate Arrays (FPGAs) or stored in the memory. This gives the flexibility of a general-purpose device and the speed of a dedicated one. Personalization is facilitated during fabrication, during product definition and during application. In other words, a neural network can be soft wired when needed and still adapted later in the product life cycle. Polymorph devices are not yet widely available. But existing parts mimic such capability already on board-level or within microcontrollers. This provides for a testing ground, preparing for the future. Notably we aim for the use of modular structures, also in neural networks. Structured design of neural networks uses the concepts of modularity and hierarchy. Modularity involves the partitioning of the net into independent cooperating parts; hierarchy shows how an abstract building block can be created as a net of neurons each with a lower-level function. It has been shown that hierarchy supports the piecewise linear composition, thereby removing the need for non-linear basic elements, and therefore eases the realization. Together with earlier results on minimum word width requirements of digital realizations this allows to create neural nets on a small footprint. Sizable neural nets can already be created on 8-bit, 2k micro-controllers in a networked environment, as forecasted for Home Automation (the Virtual Network). Special attention has been given to the extension of learning rules for handling modular structures. (Spaanenburg, 2001e; Spaanenburg et al., 2001b; Spaanenburg, 2001b; Spaanenburg et al., 2001a; Berg and Spaanenburg, 2001; Achterop et al., 2001; Spaanenburg and ten Berg, 2001; Veelen van et al., 2001b; Spaanenburg and Stevens, 2001; Barakova and Spaanenburg, 2001; Bijma et al., 2001; Spaanenburg, 2001a). Industrial contacts Polymorph computing is part of a co-operation with ASTRON (Dwingelo) as part of the development of the LOFAR radio-telescope. LOFAR is a computing network, that operates in the GHz range of signal transfer over a 400 km square area. It requires highly programmable, high-speed nodes while handling terabytes of data. Such nodes will not only be polymorph, but also need to be highly autonomous and are therefore susceptible to the type of network catastrophes that make self-healing by neural intelligence necessary. 48 Understanding printed text Prof.dr. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, Drs. B.J. van der Zwaag Most of the non-pictorial information comes to us as printed text. Getting the information (or even knowledge) out of text is by no means a trivial task. This is the challenge of text mining, where a range of clustering and classification methods are applied to create global information on the content of the communication. A typical example is in the automated maintenance of search structures on the Net. It has been shown how text mining over the messages exchanged between users of the Clubs on the KPN Net may reveal the content need and therefore indicates the proper arrangement of Clubs to optimally facilitate searches. Optical Character Reading has developed into a mature technology. Various high-quality, low price software tools are available and support the scanning of printed text into a computer file with an acceptable small amount of errors. A notable area, where the standard OCR is not sufficient, is in license-plate reading. The amount of pixels in the image is too small to easily separate the characters. Nevertheless reading such license plates has been achieved for an impressive 98%. This architecture as developed at the Rijks University Groningen uses a judicious mix of neural networks, template matchers and voters to reach this figure at a lower than 0.02% false readings. Figure 30: Site-test for license plate recognition systems at Woerden It has been noted that even a small amount of occlusion will drastically reduce the performance. Occlusion of characters on a license plate can easily occur when part of the characters is obscured by utilities such as a bicycle rack. Most of the practical problems are caused by top occlusion, i.e. where the top of the characters is blocked from vision. Different characters have different sensitivities for top occlusion; threshold figures range from 8 to 80%. The essence of the reading problem is that a neural network trained partly with occluded characters only deteriorates the performance. This is in contrast to the popular belief that a net learns from its problems. The solution has been to create a dedicated recognizer for characters with 10 to 35% top occlusion and 49 Figure 31: The intelligent reading and writing pen of C-Technology merge that in the existing architecture. Such multiple classifiers have been widely studied in literature. A basic requirement is it’s “Plug-and-Play” characteristic. So occlusion is tested when no ideal character has been read with sufficiently high confidence. If, then again, no occluded character has been read with sufficient confidence we do not take a vote but strictly decide on an error. As a result, the overall classification system takes care of both occluded and non-occluded characters. Other OCR-like problems occur with the message entry devices that are being popularized in the mobile environment. With the PDA comes the need for reading handwriting. Both research results and professional offerings show a very mature technology. The demands become even more stretched with devices like the intelligent pen. The use on plain paper seems already advanced; reading from different material such as blotting paper, curved screens and even wrapping material seems possible when deformations like occlusion can be taken care of (Oudshoff et al., 2001; Spaanenburg and Stevens, 2001; Bijma et al., 2001; Wichers et al., 2001; Zwaag van der, 2001). Industrial contacts This research has been performed in combination with Dacolian, KPN Research, and MatchCare. On novelties and abnormalities Prof.dr. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, and Dr. M. van Veelen In many real-world situations processes feature a time-dependent behavior with unknown variations in time. Where such processes are part of a larger network, small local failures may easily lead to large errors. The past decade has shown a number of such dramatic situations, on the power grid, on the telecommunications grid and on the transport grid. For instance, in 1996 a small power dip near the Keeler-Alliston dam put 11 American states and 2 Canadian provinces in the dark within a couple of seconds. The central monitor of the network came too late to block the spreading of the disaster; later one concluded that a small local correction would have given ample time to control the abnormal process behavior. The classical Fault Diagnosis and Isolation (FDI) procedures do not allow for the early detection of unknown abnormalities. Firstly, it requires all abnormalities 50 to be modeled in their effect by a dedicated fault model. Where faults are unknown, such models do not exist. Secondly, faults must have a clear impact to become distinguishable from other behavior. This makes it hard to label small aberrations as leading to larger faults in the near future. Early detection needs a look inside the process. Neural networks, and markedly the modular ones, help to get a glimpse of the process internals. Training a neural knowledge model helps to demarcate the input/output space and allows to detect disturbances without the requirement of a pre-diagnosed fault model. Performance of this method has outperformed existing methods and leads in a number of benchmark tests to a high confidence level with no markedly false warnings. The ability to find a physically plausible structure within the measurement set that can be explained through (and leads to) a better understanding of the process is core to the identification of the process and therefore to the labeling of distinct measurements as novelties or abnormalities. It is necessary to document the knowledge within the neural network to validate abnormality detection. Such may involve the automated extraction of rules in a manner and order that allows for operator inspection. As rule extraction is a combinatorial problem, the use of modular nets is called for (Spaanenburg, 2001f; Venema and Spaanenburg, 2001; Spaanenburg, 2001d; Steen van der et al., 2001; Veelen van et al., 2001a; Spaanenburg, 2001c). Industrial contacts The research on abnormality detection has been spurred by a contract with ECN. Much of the experimental evidence has come forward on basis of data from Hoogovens. Further research will be done in collaboration with Twente University. 51 52 2F Civil Engineering Informatics Group, Delft University of Technology Prof.dr.ir. P.van der Veer, Prof.ir. F.Tolman, Dr.Ir. R. Beheshti, Ir. H. Boere, Dr.Ir. J.Cser, Ir. D. Roozemond, Ir. L. Aarts, Ir. E.Dado, Ir. H. Havinga, Ir. M. Hooimeijer, Drs. L. Oosterlaan, Ir. S. Öszariyildiz, Ir. S. Özmutlu, Ir. T. Rientjes, Ir. F. Schulze, Ir. T. Botterhuis, Ir. H. van Lint, Ir. R. Wu, Ir. H. Schevers, Ir. M.Werner, Dipl.Ing. A. Zijderveld Description of the group The Civil Engineering Informatics group is engaged in the field of computational intelligence for civil engineering systems. The research focuses on new modeling methods, integrated method modeling and contributions to knowledge of physical phenomena by data analysis. The main themes of research are Hydro informatics and Building & Construction IT. The neural network research is concentrated in the Hydro informatics theme and involves an integration of computer science, hydro sciences and mathematics. It deals with bringing physical knowledge into data oriented methods, and with integrating data oriented methods with classical mathematical models. More information on the group can be found at: http//www.cti.ct.tudelft.nl/ Research projects Neural network method for solving partial differential equations Ir. L.P. Aarts, Civil Engineering Informatics, Delft University of Technology Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology Users: Rijkswaterstaat, TNO-NITG Funding: TU Delft This project concerns the problem of the ‘black box’ nature of some neural network models. In many situations this is a disadvantage besides the great advantages like the ability to cope with strongly complex non-linear behavior of systems, the adaptive properties, the performance as fast simulators and the integrateability with various model systems. This project aims to introduce physical knowledge into neural network modeling. The aim is to find neural network architectures that represent directly partial differential equations. This may lead to modeling methods based upon new types of neural networks that can easily be integrated with measurement-data oriented neural networks. The results about the basic concept are very promising, even in sets of nonlinear partial differential equations. This research may lead to results with generally applicable methods for all fields where physical phenomena are described by partial differential equations. The very promising results led to funding for a PhD-position. Neural network method for solving potential flow Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology Users: Rijkswaterstaat, TNO-NITG Funding: TU Delft This project concerns a neural network method for solving potential flow problems, described by second order differential equations, known as Laplace 53 Equations. The method uses complex function theory, especially analytical functions to describe basis components of the solutions. Problems of cyclic behavior and many-valuedness of the basic components have to be coped with. This method can be seen as a specific case of the generic method as mentioned in the previously described project. The main advantage of this approach is that the neural network needs only to be trained on boundary conditions, not on a grid. In this sense the method has similarities with the family of boundary element methods and integral equation methods. Neural network methods for the simulation a complex water system. Ir. A. Zijderveld, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology. Ir. P.Heinen, Rijkswaterstaat, Ministery of Public Works Users: Rijkswaterstaat, TNO-NITG Funding: Rijkswaterstaat Research on neural network models for the prediction of water levels along the coast and in the main Dutch rivers resulted in an integrated approach with classical numerical methods and expert system technology. This combines data analysis of measurement data, analysis and classification of storm situations, physical knowledge representation in numerical models and human knowledge and experience representation. Results are scientifically interesting, as this integrated method modeling is able to cope with extreme values in water level predictions much better than classical methods. For practical applications, the prediction accuracy of extreme water level values is important as it is directly related to human safety. Rijkswaterstaat (Ministry of Public Works) has implemented the results of the project. A dissertation on this subject will be finalized in 2002. The use of neural networks in simulation of complex morphological systems Ir. L.Oosterlaan, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil Engineering Informatics. Dr. J. van de Graaff, Hydraulic Engineering. Prof.dr.ir. A.W. Heemink, Information Technology and Systems, Delft University of Technology Users: Rijkswaterstaat Funding: LWI The morphodynamics of a coastal system consists of complex, not yet fully understood, relationships between hydrodynamics (waves and currents), sediment transport processes, morphology and environmental conditions. Hydrodynamics induces sediment transport, which is reflected in a change of morphology. These interactions however, are non-linear and often display a time lag between the initial hydrodynamic conditions and morphological response. In addition, in many cases a hydrodynamic threshold value has to be exceeded before sediment movement starts. Here we deal with a situation where all research had to be done on the basis of measurement data. The research is focussed on integration of neural networks with other methods. An integration of an empirical orthogonal function approach with neural networks appeared to provide promising results. 54 Integrated Method Modeling in water management systems Ir. H.N.J.Havinga, Ir. M.G.F.Werner, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology Users: Rijkswaterstaat, WL Delft Funding: LWI and TU Delft In many fields of civil engineering practical situations are too complex to be modeled analytically. Often there is a large gap between measurement data and the output of a mathematical model. In this research activities method integration is investigated to achieve models with greater predictive properties. Integration of neural network technology with fuzzy logic, genetic algorithms is studied. Combinations with Geo Informatic Systems (GIS) are investigated, in order to obtain infrastructural decision support systems that can better cope with uncertain or incomplete data (Aarts and van der Veer, 2001b; Aarts and van der Veer, 2001a; Aarts and van der Veer, 2001c; Cser and e.a, 2001; Hooimeijer, 2001; Khu et al., 2000). 55 56 2G Institute for Knowledge and Agent Technology (IKAT), University Maastricht Prof. H.J. van den Herik, Prof. A.J. van Zanten, Prof. L.A.A.M. Coolen, Prof. H. Visser, Prof. J.M.J. Murre, Dr. E.O. Postma, Dr. J.W.H.M. Uiterwijk, Dr. S. Etalle, Dr. N. Roos, Dr. I. Sprinkhuizen-Kuyper, Dr. F.J. Wiesman, Dr. P.A. Vogt, Dr. E.N. Smirnov, M.Sc, G.W. Boers, M.Sc., N.I. Bourdonskaia, M.Sc, H.H.L.M. Donkers, M.Sc., M.F. van Dartel, M.Sc., C.A. van Dorp, M.Sc., L. Kocsis, M.Sc., L.J. Kortmann, M.Sc., P.A.M. van der Krogt, M.Sc., E.H.N. Mathijsen, M.Sc., Y.P. Ran, M.Sc., A. Sprinkhuizen, M.Sc., J.P.G.M. Verbeek LLM, M.H.M. Winands, M.Sc. Description of the group IKAT’s research is organized in five main research groups: agent technology, neural networks and adaptive behavior, search and games, public services and knowledge management. Neuro-computational and machine learning methods are mainly employed in the neural networks and adaptive behavior group, and to a lesser extent in the agent technology and search and games groups. More information on the group can be found at: http//www.cs.unimaas.nl Learning to recognize (painted) images Recent developments in image classification have focused on efficient preprocessing of visual data to improve the performance of neural networks and other machine learning algorithms when dealing with content-based classification tasks. Pre-processing techniques are used to bring out the visual characteristics relevant for the classification. This project studies modern neural pre-processing techniques to enhance the performance on image classification tasks. The performance of the techniques is assessed on the difficult classification task of recognizing a painter from the contents of a painting. The basic assumption is that the maker of a painting can be recognized from his/her “hidden signature”, i.e., brushstroke, spatial and color composition, and texture. Such painterspecific features are to be retained in the pre-processed representation to ensure successful classification. Domain-specific knowledge provides a rough albeit indispensable guideline for determining the appropriate type of pre-processing. Figure 32: Illustration of the application of neuro-computational techniques to the analysis of paintings. Left: digital reproduction of the painting Supper at Emmaus by Rembrandt (1648). Right: brush-stroke representation of the same painting obtained by applying neuralnetwork techniques. The shades of white indicate the degree to which the corresponding parts contain brush stroke patterns that are typical for Rembrandt. 57 For this reason we collaborate with the Rembrandt Research Project and the Amsterdam Rijksmuseum (as part of the NWO funded ToKeN 2000 project). In 2001, special learning algorithms have been developed to mimic the eye movements of observers. These active-vision algorithms have been successfully applied to the recognition of faces (Postma et al., 2001a) and will be applied to select the relevant parts of paintings for further analysis. Neural robots The neural robots research deals with the study of autonomous robots that are controlled by neural networks. More specifically, the research focuses on the automatic learning of elementary behavioral tasks such as target following (András et al., 2001; Kortmann et al., 2001) and object pushing (Spronck et al., 2001a; Spronck et al., 2001b; Spronck et al., 2001c). By studying how robot models learn to perform these tasks, we hope to gain a deeper understanding of the way more complex behaviors can be learned in an automatic way. The task of pushing an object (i.e. a circular box) between two walls is relevant for robot soccer and underlies many more complex behaviors such as target following, navigation, and object manipulation. Despite the apparent simplicity of the task, training a robot to perform the object-pushing task turns out to be rather difficult. In 2001 our experiments focused on the evolutionary optimization of the neural networks that control robot behavior. In particular, the effects of various evolutionary strategies have been explored (Spronck et al., 2001a). Figure 33: Top view of pushing behavior for nine different starting positions by an artificially evolved robot. The figures illustrate the nine initial positions of the robot (bottom circle) and circular object (circle just above the robot) and the final positions (top two circles). Symbol grounding in multi-agent systems The symbol grounding problem concerns the question of how semantic interpretations of symbolic representations can be made intrinsic to the system, rather than being described in a formal way (e.g. as a set of rules). The project focuses on the study of systems composed of two or more robots that solve the symbol grounding problem through interaction with an environment of other robots and objects. The robots solve the symbol grounding problem by engaging in a series of so-called language games. In a language game the robots try to 58 name an object by categorizing the sensed object and subsequently assigning a name to the categorized object. When robots fail in categorizing or naming an object, they adapt their memory to improve their performance on future occasions. At the start of each experiment, the robots have no categories or names at all. During the experiment categories and names emerge from the linguistic and physical interactions with the environment. The experiments performed thus far reveal that the robots solve the symbol grounding problem through the co-evolution of meaning and lexicon. In addition, the automatic learning of ontology mapping through language games has been studied (Wiesman et al., 2001) ToKeN 2000 The ToKeN 2000 project is concerned with two fundamental problems: (1) How can the accessibility of information and knowledge be improved for the user in such a way that knowledge acquisition for the user is optimal? So as to make the information and knowledge available and accessible for large groups of users, the problem is: (2) How can the retrieved information and knowledge be upgraded and enriched? These problems give rise to the following research themes: (1) Control, (2) Navigation, Adaptation, and Learning, (3) Language Technology, (4) Delivery Techniques, and (5) Knowledge Enrichment. The research is applied and tested in three domains: Education & Culture, Police & Law, and Health Care. In the first year of the project an experimental platform will be built and tested. The Rijksmuseum’s collection database serves as a testbed. In the second phase of the project the experiment platform serves as a basis for further research. ToKeN 2000 is a joint research program of IKAT, IPO (TUE Eindhoven), NICI (KU Nijmegen), CWI, TU Delft, Universiteit Leiden Department of Psychology. IKAT provides three parts: (1) The Metabrowser (2) Automatic Image Recognition, and (3) Intelligent Information Filtering. Figure 34: Two robots in their environment of three objects (black cylinders). 59 60 2H Evolutionary Systems and Applied Algorithmics; CWI, Amsterdam Prof.dr.ir. J.A. La Poutré, CWI and Eindhoven University of Technology. Drs. F. Alkemade, Drs. S.M. Bohté, Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Dr.ir. P.J. ‘t Hoen, Drs. M.B. de Jong, Dipl. Math. E. Kutschinski, Dr. D.J.A. Somefun, CWI, Amsterdam Description of the group The research group focuses on the combination of two components, consisting of computer science techniques and application fields: • The techniques concern intelligent computation: evolutionary and multiagent systems, adaptive algorithms, and neural networks; • The application fields concern economics, management, and e-societies (like for instance e-commerce); important topics are markets and market mechanisms (economics), negotiations, auctions, and social aspects (game theory), and optimization and classification (management). Agent systems: e-business and economics. The concept of software agents in computer science, as well as the concept of societies of (human) agents in economics and social sciences, yield important areas of research. In both cases, adaptive behavior of agents is essential. An important aspect of adaptivity for an agent is the skill of learning. This is a growing field of research, important for both agent technology (how to build a really-learning agent) and economics (how to simulate learning agents). In order to allow learning in agent systems, machine learning techniques are necessary. In our research group, we focus on the investigation of evolutionary systems and neural networks in order to build the internals of learning agents for e-commerce applications as well as to simulate markets and market mechanisms in economics and in e-commerce agent systems. Focus areas are the following: • Adaptive strategies for trading, like negotiations, auctions, and dynamic pricing; this concerns learning behavior, especially for micro-transactions (a micro-transaction has a small value compared to the total value for one agent; this allows for learning and experimenting agents) (Bragt van et al., 2001; Bragt van and La Poutré, 2001a and 2001b). • The design and simulation of market mechanisms for e-commerce applications (Bohté et al., 2001a). • Agent-based computational economics (ACE), in particular the simulation of economic markets; this addresses emergent properties of markets, seen as complex adaptive systems (CAS) (Alkemade and Poutré, 2001). Optimization and Classification Neural networks as well as evolutionary algorithms are particularly investigated with respect to classification problems and active learning. Special attention is given to spiking neural networks, which are novel types of neural networks 61 (Bohté et al., 2001b). Application areas are e.g. remote sensing, data mining, and agent implementations. Adaptive discrete algorithms (Stee van and Poutré, 2001b; Stee van and Poutré, 2001a) as well as evolutionary algorithms are designed for e.g. decision making in dynamic environments, like on-line process management and quality of service in information technology. Other activities Based on the above research activities, the research group also participates (as initiator and coordinator) in the ICES/KIS-III expression of interest “Adaptive Intelligent Systems for Health Care (I-CARE)”, for designing agent systems and intelligent systems for e-health applications. More information on the group can be found at: http://www. cwi.nl Trade agents Prof.dr.ir. J.A. La Poutré, Drs. S.M. Bohté, Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Dr. D.J.A. Somefun, Dr.ir. P.J. ‘t Hoen The Trade Agents project (“Autonomous Systems of Trade Agents in E-Commerce”) concerns research on systems of trading software agents in electronic commerce (see http://www.cwi.nl/projects/ASTA). Partners in the project are CWI, TNO, ING, and KPN. The project is funded by the Telematics Institute. The major line of the CWI research addresses the development of models and algorithmic software solutions for new business applications that are based on interacting software components (agents), that are owned and controlled by different, autonomous market parties. Specific examples include market mechanisms, dynamic pricing, negotiation, bidding and buying, and (fast) profiling in e-business. Our focus is on the evolutionary simulation and development of trade agent systems; the development of adaptive, learning trade agents; and the creation of new e-business concepts. The amount of attention space available for recommending suppliers to consumers on e-commerce sites is typically limited. Within the Trade Agents project, we developed a competitive market-based recommendation mechanism (based on adaptive software agents) for efficiently allocating the “consumer attention space”, or banners (Bohté et al., 2001a). A patent application was filed (together with KPN Research) for this system. In our approach, agents get information about a consumer’s interest. Subsequently, each agent bids in an auction for Figure 35: The winning agents display their advertisement to the customer, where the attention space is sufficiently large to display three banners. Two lists of stores are shown for 2 types of consumers (both searching for jeans). 62 the momentary attention of that consumer. Winning agents may display their advertisement or banner in the available attention space to the consumer. Successive auctions allow agents to rapidly adapt their bidding strategy to focus on consumers interested in their offerings. The feasibility of this system, for variety of customer behavior models, was demonstrated by evolutionary simulation (as in agent-based computational economics). A scalable and extensible agent architecture was developed for the abovedescribed system. This architecture supports agents in a distributed bidding application, where the agents run on dedicated machines for maximum computational resources. Furthermore, as an extension, the agents can operate in multiple independent markets concurrently. bid 1.5 1 0.5 0 x y Figure 36: An example of an evolved bidding strategy for a two-dimensional consumer profile. We further studied the problem of applying price discrimination with an online adaptive algorithm. In particular, we considered the case where the seller of a product or service applies price discrimination by distinguishing between different delivery times. For this situation, we developed an efficient and robust algorithm that dynamically adjusts both the price and the size of the discount for delayed delivery of the product. The algorithm consists of a (multi-variable) derivative follower algorithm with an adaptive step-size. We showed that this algorithm has attractive (convergence) properties when operating in static and dynamic pro fit landscapes (unlike previously proposed algorithms). Compu-tational experiments show that the algorithm is able to generate high profit levels in a dynamic pricing setup with price discrimination. Consumer Consumer Mall Manager Agent Consumer W eb Site Supplier Agent Supplier Supplier Agent Supplier Supplier Agent Supplier Figure 37: The architecture of the competitive market-based distribution system for consumer attention space. 63 To explore the potential of active profiling by example in information filtering, selective sampling for text classification was further investigated. This work focused on so-called “committee-based” sampling methods. In particular, a technique to create more robust homogeneous committees was developed, and a novel approach of using several different classifier types within one committee for selective sampling was examined. These mixed committees perform significantly better in terms of learning speed and Classification performance than any of the individual classifiers. Generating successful negotiation strategies by evolutionary computation Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Prof.dr.ir. J.A. La Poutré The rapid growth of a global electronic market place, together with the establishment of standard negotiation protocols, currently leads to the development of multi-agent architectures in which artificial agents can negotiate on behalf of their users. Our long-term goal in this project is to develop artificial bargaining agents, which are able to deal successfully with a variety of opponents in such an electronic market place. To reach this goal, we combine and further refine techniques from the fields of multi-agent systems, learning by co-evolution, and evolutionary computing. In recent work (Bragt van and La Poutré, 2001b), we observed that most of today’s (prototype) systems for automated negotiations, like Kasbah or Têteá-Tête, use simple and static negotiation rules. We were able to show that such “fixed” bidding agents can be exploited by more sophisticated “adaptive” software agents (based upon evolving automata). These adaptive agents are able to learn strategies, which perform (almost) optimally against a variety of fixed opponents. Furthermore, they are able to adapt their strategies online to deal with changing opponents and open environments. We also developed an abstract model of multiple adaptive agents who are updating their strategies over time (Bragt van and La Poutré, 2001a). The bargaining strategies are represented in this model by special kind of finite automata, which require only two transitions per state. We show that such automata (with a limited complexity) are a suitable choice in a computational setting. We furthermore describe an evolutionary algorithm (EA) which generates s0 s1 τacc = 0.97 o0 = 1.0 o1 = 1.0 s*init τacc = 0.84 o0 = 0.81 o1 = 0.97 else o0 >= 0.42 o1 >= 0.90 o0 >= 0.74 o1 >= 0.35 else τacc = 0.48 o0 = 0.65 o1 = 0.0 s2 64 o0 >= 0.55 o1 >= 0.48 else τacc = 0.10 o0 = 1.0 o1 = 0.03 o0 >= 0.26 o1 >= 0.32 sinit s3 Figure 38: A finite automaton (with four states) which is capable of negotiating about two issues. An evolutionary algorithm has optimized the structure of this automaton has been optimized by an evolutionary algorithm. highly efficient bargaining automata in the course of time. A series of computational experiments shows that co-evolving automata are able to discriminate successfully between different opponents, although they receive no explicit information about the identity or preferences of their opponents. These results are important for the further development of evolving automata for real-life (agent system) applications. As part of the project on efficient bargaining strategies we studied a negotiation game with “out-side opportunities” for the players. In this model, players can negotiate with other opponents if negotiations with the current opponent are not successful. The emergent behavior in this model was studied by evolutionary simulation. These experiments show that “fair” agreements can evolve if the players have the opportunity to negotiate at a low cost with multiple opponents. Evolutionary exploration systems for electronic markets Drs. F. Alkemade, Prof.dr.ir. J.A. La Poutré, Prof.dr. H.M. Amman, Eindhoven University of Technology Funding: NWO-EW project The rapid growth of the Internet affects markets and commerce in a substantial way. For example, the rise of electronic markets occurs together with an important increase of scale due to globalization. The opportunities for interaction and information gathering also increase substantially for the participants (agents) in an electronic market. Evolutionary techniques appear to be very appropriate to model and study such electronic markets. We therefore develop (computational) evolutionary systems, which model electronic markets. This project falls within the novel field of “agentbased computational economics” (ACE) and combines research topics from computer science, cognitive science and evolutionary economics. We developed alternative evolutionary techniques, which yield more robust results in comparison with previous studies. We also started to develop a model of network economics, in which adaptive agents buy and sell goods. Transactions can only take place in this model if there is a link (network connection) between the two parties. Our goal is to investigate the role of intermediaries in such a setting. Because the complexity of our model is significant we use genetic algorithms to determine highly efficient outcomes. We use these outcomes as a benchmark to evaluate the performance of agentbased models. Evolutionary Economics Agent-Based Computational Economics Cognitive Science Computer Science 65 Computation in networks of spiking neurons Drs. S.M. Bohté, Prof.dr.ir. J.A. La Poutré. Prof.dr. J.N. Kok, Leiden University Artificial neural networks (ANNs) have received much attention in the last decades as powerful methods for adaptive classification and function approximation. Implicitly, such neural networks also claim to model the type of information processing and learning as takes place in systems of real neurons (e.g. the human brain). A particularly important assumption is that the artificial neurons as used in traditional neural networks sufficiently capture the information processing capabilities of real neurons. This assumption is increasingly being questioned, as more information on the actual behavior of cortical neurons and their connectivity becomes available. In addition, there are important computational questions like the manipulation of structured information by neural networks (e.g. the representation of visual objects and the construction and expression of language). The exploration of the computational properties of a more biologically realistic neuron model, spiking neurons, is the subject of this project (Bohté et al., 2001b). In particular, the (additional) computational abilities offered by the careful timing of individual spikes emitted by such neurons is being researched. Results so far include the development of unsupervised and supervised learning rules. Current work is focused on the structured representation of information with spiking neurons, where we have been exploring the use of distributed representations in individual neural nodes. This has resulted in architectures that are able to perform dynamic variable binding, and, in a vision application, the position invariant detection of feature-conjunctions (e.g. finding {yellow-and-rectangle} anywhere in a picture). Dynamic algorithms for on-line optimization Drs. R. van Stee, Prof.dr.ir. J.A. La Poutré. L. Epstein, The Interdisciplinary Center, Herzliya, Israel. S. Seiden, Louisiana State University, USA. Funding: NWO-EW project The project “Dynamic Algorithms for On-Line Optimization” addresses the design and analysis of efficient algorithms for on-line optimization problems that are fundamental to various management and design problems in computer systems and networks. Within this project, research on online bin packing was performed (Seiden and van Stee, 2001). New upper and lower bounds were presented for a multidimensional generalization of bin packing, called box packing. Several variants of this problem, including bounded space box packing, square packing, variable sized box packing and resource augmented box packing are also studied. Furthermore, work was done on on-line scheduling (Epstein and van Stee, 2001a; Epstein and van Stee, 2001c). In particular, the scheduling problem of minimizing the maximum starting time on parallel identical machines on-line was studied (Epstein and van Stee, 2001b). The goal is to minimize the last time that a job starts. We designed an algorithm with constant competitive algorithm for this problem. We also showed that a “greedy” algorithm is optimal for resource augmentation. 66 We also developed an algorithm to minimize the total completion time on-line on a single machine, using restarts, with a low, constant competitive ratio of only 3/2. This is the first restarting algorithm to minimize the total completion time that has a better competitive ratio than an algorithm that does not restart. Figure 39: (a) A complete remote-sensing image. Inset: image cutout that is actually clustered with various methods. (b) Classification of the cutout as obtained by clustering the entire image with the UNSUP algorithm. (c) Classification by Kohonen’s SOM algorithm. (d) Classification by a spiking neural network. Quality of service for multimedia systems Dr.ir. J. Korst, Ir. J. Aerts, Ir. W. Michiels, Philips Research Funding: NWO-EW project “Quality of Service” (QoS) is a typical feature of multimedia servicing. It yields important perspectives of providing services to clients in a flexible and meaningful way. The primary problem with QoS is resource management: i.e. the assignment of activities to scarce resources. In this NWO-EW project, we investigate how to use, handle, and optimize QoS for servicing requests in on-line multimedia systems, in order to obtain a good system performance. In the project, research was performed on reinforcement learning with multiple objectives. In co-operation with researchers from Philips research, work was also performed on scheduling strategies for near video-on-demand. 67 68 3 Commercial Applications 69 3A Participating Industrial Partner Energy research Centre of the Netherlands (ECN) Unit Technological Services and Consultancy Description of the Group Ir. J.J. Saurwalt, Drs.ing. J.K. Kok, Ir. G.P. Leendertse, Ir. J.J. Kaandorp, Ing. F.J. Kuijper The Netherlands Energy Research Foundation ECN is the leading institute for energy research in the Netherlands. Research at ECN is carried out under contract from the government and from national and foreign organizations and industries. Intelligent systems, incorporating neural networks and intelligent agent technology are developed as part of ECN’s activities in the following areas: solar energy, wind energy, clean fossil fuels, energy efficiency in industry, and renewable energy in the built environment. The department Technological Services and Consultancy (TSC) is engaged in the development, engineering and implementation of these intelligent systems, as well as those of industrial parties. Moreover, ECN - TSC applies neural networks as an advanced tool for data mining studies. More information on the group can be found at: http//www.ecn.nl Co-operation with SNN Projects ECN started a strategic-alliance with the SNN in 1999. The focus of the alliance has been on knowledge transfer in two directions. ECN contributes to technological innovation by developing and transferring specific knowledge and technologies for its target groups and clients. A further dissemination of artificial intelligence techniques towards energy-intensive sectors will enhance energy efficiency and will strengthen the position of sustainable energy sources. By co-operation with the SNN, ECN is able to utilize new technologies directly after they have reached maturity. Short-Term Forecasting of Sustainable Energy Production Sponsored by: Foundation for Sustainable Energy (SDE), System Integration ·ENGINE (ENergy Generation In the Natural Environment) Partners: KNMI, Meteo Consult, WEOM The contribution of wind and solar energy systems to the overall electrical power supply has a varying character. Due to liberalization of the energy market, short- 70 term supply and demand matching will gain importance. A reliable power output prediction system will contribute to the cost-effectiveness of sustainable energy sources. ECN units co-operate in a research project in which the technical and economical feasibility of a Power Output Prediction System is investigated. Further, a proof-of-principle prototype is designed and implemented. Neural networks operate next to physical models in the heart of the prediction system. Intelligent Data Analysis for High Speed Screening of Catalysts Sponsored by: European Union Partners: DSM Research, AMTEC, and others High Speed Screening (HSS) for discovery of catalysts is a relatively new field of research. By running tens or hundreds of experiments in parallel, the average time for an experiment is drastically reduced. In catalysis research the number of possible combinations is virtually infinite. It is not possible to try all of them. So, in addition to performing experiments, the need remains for being very smart in the selection of experiments. ECN - TSC is developing intelligent techniques for the optimal selection of these experiments as well as for knowledge extracfrom the database of experimental results. Figure 40: ECN investigates the application of neural networks for power output prediction of wind turbines. (Photo: A.A. Homan, ECN). 71 3B Commercial Spin-off activities of SNN SMART Research BV SMART Research BV is the commercial outlet of SNN Nijmegen. Within SMART, we conduct pilot and feasibility studies, but we also develop actual working applications. The techniques that we apply include advanced neural networks and graphical models like Bayesian networks. We have worked, among others, for companies and organizations in the consumer branch (Schuitema, Riedel, Vendex, Centraal Bureau Levensmiddelen), publishers (De Telegraaf, BauerVerlag, Edipresse), production industry (Shell, SKF, Kenniscentrum Papier en Karton), and finance (Simplex Credit Advisory). We will highlight some of the projects we have worked on in 2001. SKF, market leader in the production of all kinds of bearings, has chosen BayesBuilder as their development tool to build a probabilistic rule-based system for the analysis of bearing failures. Better than its competitors, BayesBuilder facilitates the implementation of huge Bayesian networks within a user-friendly interface and environment. Within this project SMART provides support and maintenance. In 1996, De Telegraaf started using JED, for “Just Enough Delivery”, to optimize the distribution of single-copy newspapers. JED is a software package based on neural-network technology: it learns from the past sales, to predict the sales in the future. For De Telegraaf, we have developed and implemented a completely new version. As shown in comparative studies for Bauer-Verlag (Germany) and Edipresse (Switzerland), this new version makes even better predictions, clearly outperforming alternative approaches. Furthermore, it contains many new features: tools to select and target all outlets in specific regions and/or branches,algorithms to calculate and visualize different scenarios, graphics displaying the characteristics and explaining the predictions for individual outlets, and so on. The new version is fully automatic that is predictions are continuously updated based on the latest available information. We expect JED 2.0 to be up and running at De Telegraaf early 2002. After having built a successful prototype, we will in 2002 also start implementing JED for Midesa, the distributor of the Portuguese newspaper Público. 72 50 40 30 20 sales prediction delivery (actual) delivery (suggested) 10 0 2000−14 2000−27 2000−40 2001−1 2001−14 2001−27 2001−40 2002−1 Figure 41: Just Enough Delivery: optimizes the trade-off between returns and sellouts More information can be obtained at: SMART Research BV Dr. T.M. Heskes P.O. Box 31070 6503 CB Nijmegen The Netherlands. Tel: +31-24-3615039 Fax:+31-24-3541435 email: [email protected] http://www.smart-research.nl 73 NuTech Solutions GmbH NuTech Solutions’ core business is in the field of adaptive business intelligence based on modern computer science methods. These methods are gleaned from the model of information processing in natural systems and enable industry to enter a new economic dimension of effectiveness and adaptability in applications in the field of business process optimization and data mining. Data Mining indicates the process of deriving hidden knowledge from large databases, like they occur in many industrial branches such as e.g. banks, health industry, automobile industry, chemical industry etc. Methods such as classical statistics, genetic programming, neural networks, rough sets and fuzzy logic are applied to such databases to derive compact, readable and interpretable pieces of knowledge from massive amounts of data. This knowledge is then used to derive optimized management recommendations for high-level decision support within companies, leading to huge savings, profitability increases or quality improvements. Business process optimization refers to the problem of optimizing complex industrial systems according to given criteria (e.g. quality, profitability, value). Applications are given e.g. in the domain of scheduling problems (e.g. trucks, production processes on machines, crews for airlines), transport optimization (e.g. lengths and duration’s of routes, cost of transport), technical design optimization (many different domains), management process optimization (modeling, simulation and optimization of decision making processes) and experimental optimization (e.g. in biotechnology, food and cosmetics industry). Many staff members of NuTech Solutions are among the leading experts worldwide in the above mentioned fields. In total, NuTech employs about 20 staff members in the field of computational intelligence. Software products in adaptive business intelligence as well as client-specific consulting activities are the main focus of the business of NuTech Solutions. Evolution strategies as a key technology give NuTech access to technologies which no other company world wide can offer to its clients. Contact and further information: Dr. Thomas Bäck Managing Director Leiden University, LIACS Niels Bohrweg 1 NL-2333 CA Leiden emaill: baeck@ liacs.nl http://www.liacs.nl 74 4 Publications 75 A B CD Aarts, L. and van der Veer, P. (2001a). Neural network method for solving parital differential equations. Neural Processing Letters,volume 17(3): pages 261-271. Aarts, L. and van der Veer, P. (2001b). Solving nonlinear differential equations by a neural network method. In Computational Science-ICCS 2001, volume Part II, pages 181-188. Aarts, L. and van der Veer, P. (2001c). Solving nonlinear differential equations by a neural network method. Lecture notes in Computer Science, volume 2074(2): pages 181-189. Achterop, S., DeVos, M., v.d.Schaaf, K., and Spaanenburg, L. (2001). Architectural requirements for a lofar generic node. In Proceedings ProRISC01, pages 234-239. Alkemade, F. and Poutré, J. L. (2001). Heterogeneous, boundedly rational agents in the Cournot duopoly. In Proceedings of the 13th Belgian-Dutch Conference on Artificial Intelligence (BNAIC ‘2001), pages 21-22, Amsterdam. András, P., Postma, E., and van den Herik, H. (2001). Natural dynamics and neural networks: Searching for efficient preying dynamics in a virtual world. Journal of Intelligent Systems, 11(3):173202. ISSN 0334-1860. Bäck, T. (2001). Evolution strategies: Overview and a CFD application. In Periaux, J., Joly, P., Pironneau, O., and Onate, E., editors, Innovative Tools for Scientific Computation in Aeronautical Engineering. CIMNE, Barcelona. Bäck, T., de Graaf, J. M., Kok, J. N., and Kosters, W. A. (2001a). Theory of genetic algorithms. In Paun, G., Rozenberg, G., and Salomaa, A., editors, Current Trends in Theoretical Computer Science: Entering the 21st Century, pages 546-578. World Scientific, Singapore. Bäck, T., Emmerich, M., and Schallmo, M. (2001b). Industrial applications of evolutionary algorithms: A comparison to traditional methods. In Parmee, I., editor, Proceedings of the International Conference on Optimisation in Industry III. Springer, Berlin. Bäck, T. and Schütz, M. (2001). Evolutionäre algorithmen im data-mining. In Hippner, H., Küsters, U., Meyer, M., and Wilde, K., editors, Handbuch Data-Mining im Marketing, chapter 10, pages 221244. Vieweg Verlag, Wiesbaden. Bakker, B. and Heskes, T. (2001b). Task clustering for learning to learn. In Kröse, B., Rijke, M. d., Schreiber, G., and Someren, M. v., editors, BNAIC 2001, pages 33-40. Bakker, B., Heskes, T., Neijt, J., and Kappen, B. (2001). Improving cox survival analysis with a neural-Bayesian approach. Statistics in Medicine (in press). Banzhaf, W., Nordin, P., Keller, R., and Francone, F. (2001a). Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications (3rd edition). Morgan Kaufmann, dpunkt.verlag. Banzhaf, W., Nordin, P., Keller, R., and Francone, F. (2001b). Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications (Japanese version). SciTech Press. Barakova, E. and Spaanenburg, L. (2001). Learning and reproducing. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Barber, D. and Heskes, T. (2002). An introduction to neural networks. In Encyclopedia of Life Support Systems. In press. Berg, A. B. t. and Spaanenburg, L. (2001). On the compositionality of neural networks. In Proceedings ECCTD, volume 3, pages 405-408. Bijma, M., Haseborg, H. T., Diepenhorst, M., and Nijhuis, J. (2001). Neural hardware implementations: an overview. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Boers, E. and I.G. Sprinkhuizen-Kuyper, I. (2001). Combined biological metaphors. In Patel, M., Honavar, V., and Balakrishnan, K., editors, Advances in the Evolutionary Synthesis of Neural Systems, pages 153-183. MIT Press, Cambridge, MA. SBN 0-262-16201-6. Bohté, S., Gerding, E., and Poutré, J. L. (2001a). Competitive market-based allocation of consumer attention space. In Proceedings of the 3rd ACM Conference on Electronic Commerce (EC 76 ’01), pages 202-205, USA. Bohté, S., Poutré, J. L., and Kok, J. (2001b). Learning in spike-time encoded neural networks. In Proceedings of the 5th International Conference on Cognitive and Neural Systems, Boston, USA. Bossi, A., Etalle, S., and Rossi, S. (2001a). Semantics of well-moded input-consuming programs. Computer Languages, volume 26(1): pages 1-25. ISSN 0096-0551. Bossi, A., Etalle, S., Rossi, S., and Smaus, J.-G. (2001b). ). semantics and termination of simply-moded logic programs with dynamic scheduling. In Sands, D., editor, Proceedings of the European Symposium on Programming, pages 402-416. LNCS 2028, Springer-Verlag, Berlin, Germany. ISBN 3 540 418628. Bragt van, D., van Kemenade, C., and Poutré, J. L. (2001). The influence of evolutionary selection schemes on the iterated prisoner’s dilemma. Computational Economics, volume 17(2/3): pages 253-263. Bragt van, D. and La Poutré, J. A. (2001a). Co-evolving automata negotiate with a variety of opponents. In Proceedings of the Eleventh Dutch-Belgian Conference on Machine Learning (BENELEARN-2001), pages 77-84. Bragt van, D. and La Poutré, J. A. (2001b). Generating efficient automata for negotiations - An exploration with evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), page 1093, San Francisco, California, USA. Morgan Kaufmann. Breuker, D., Uiterwijk, J., and Herik, H. v. d. (2001a). The pn2 -search algorithm. In van den Herik, H. and Monien, B., editors, Advances in Computer Games, pages 115-132. IKAT, Universiteit Maastricht, Maastricht. ISBN 90 6216 5761. Breuker, D., van den Herik, H., Uiterwijk, J., and Allis, L. (2001b). A solution to the ghi problem for best-first search. Theoretical Computer Science, volume 252(1-2): pages 121-149. ISSN 0304-3975. Bunschoten, R. and Kröse, B. (2001a). 3-d scene reconstruction from cylindrical panoramic images. In Proceedings of the 9th International Symposium on Intelligent Robotic Systems (SIRS’2001), pages 199-205, LAAS-CNRS, Toulouse, France. Bunschoten, R. and Kröse, B. (2001b). 3-d scene reconstruction from multiple panoramic images. In Proceedings of 7th annual conference of the Advanced School for Computing and Imaging (ASCI 2001), pages 49-54, Heijen, The Netherlands. ASCI. Cemgil, A. and Kappen, B. (2001a). A dynamic belief network implementation for realtime music transcription. In Proceedings of the Belgian-Dutch Conference on Artificial Intelligence 2001, Amsterdam. Cemgil, A., Kappen, B., Desain, P., and Honing, H. (2001c). On tempo tracking: Tempogram representation and kalman filtering. Journal of New Music Research. In press. Cemgil, A. and Kappen, H. (2001b). Bayesian real-time adaptation for interactive performance sytems. In Proceedings ICMC, Habana/Cuba. In press. Cemgil, A. and Kappen, H. (2001c). Tempo tracking and rhythm quantization by sequential monte carlo.In Proceedings NIPS. In press. Cliteur, P., Herik, H. v. d., Huls, N., and Schmidt, A. (2001). It ain’t necessarily so. E.M. Meijers Instituut, Leiden & Kluwer, Deventer, The Netherlands. ISBN 90 268 3854 9. Cornelisse, L., Scheenen, W. J., Koopmani, W. J., Roubos, E. W., and Gielen, S. C. (2001). Minimal model for intracellular calcium oscillations and electrical bursting in melanotrope cells of xenopus laevis. Neural Computation, volume 13: pages 113-137. Cser, J. and e.a, L. O. (2001). A neural network approach to predict nearshore morphology along the dutch coast. Journal of Coastal Research, 17:1-11. Donkers, H., Uiterwijk, J., and Herik, H. v. d. (2001a). Admissibility in opponent-model search. In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 373-380. 77 DEFG H I J K Universiteit van Amsterdam, Amsterdam. Donkers, H., Uiterwijk, J., and Herik, H. v. d. (2001b). Probabilistic opponent-model search. Information Scien-ces, volume 135(3-4): pages 123-149. ISSN 0020-0255. Dorp, C. v., Beulens, A., and Beers, G. (2001). Erp in semi-process: Requirements capture for tracking and tracing. In Strong, D., Straub, D., and DeGross, J., editors, Proceedings of the 7th Americas Conference on Information Systems (AMCIS 2001). Association for Information Systems, Boston, Massachusetts, USA. Duin, R. (2001). Four scientific approaches to pattern recognition. In Vossepoel, A. and Vos, F., editors, Fourth Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image Processing, pages 331-337. NVPHBV, Delft. Duin, R. and Pekalska, E. (2001). Complexity of dissimilarity based pattern classes. In Austvoll, I., editor, Proceedings of the 12th Scandinavian Conference on Image Analysis, SCIA 2001, pages 663-670. NOBIM, Norway. Eggermont, J. (2001). Knowledge discovery and evolutionary computation - an partial abstract of my ph.d. In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van Remortel, P., editors, Proceedings of the Brussels Evolutionary Algorithms Day ( bead-2001). Vrije Universiteit Brussel (vub). Eggermont, J., Lenaerts, T., Poyhonen, S., and Termier, A. (2001). Raising the dead; extending evolutionary algorithms with a case-based memory. In Miller, J., Tomassini, M., Lanzi, P., Ryan, C., Tetamanzi, A., and Langdon, W., editors, Proceedings on the Fourth European Conference on Genetic Programming (EuroGP’01), volume 2038 of LNCS, pages 280-290, Lake Como, Milan, Italy. Springer-Verlag. Eggermont, J. and van Hemert, J. I. (2001). Adaptive genetic programming applied to new and existing simple regression problems. In Miller, J., Tomassini, M., Lanzi, P., Ryan, C., Tetamanzi, A., and Langdon, W., editors, Proceedings on the Fourth European Conference on Genetic Programming (EuroGP’01), volume 2038 of LNCS, pages 23-35, Lake Como, Milan, Italy. Springer-Verlag. Emmerich, M., Willmes, L., and Bäck, T. (2001). Asynchronous evolution strategies for distributed direct optimisation. In Giannakoglu, K., Tsahalis, D., Periaux, J., Papailiou, K., and Fogarty, T., editors, Proceedings of the International Conference on Industrial Applications of Evolutionary Algorithms, EUROGEN. CIMNE, Barcelona. Epstein, L. and van Stee, R. (2001a). Lower bounds for on-line single-machine scheduling. In Proceedings of the 26th International Symposium on Mathematical Foundations of Computer Science (MFCS ‘2001), Lecture Notes in Computer Science, LNCS 2136, pages 338-350, Mariánské Lázne, Czech Republic. Springer Verlag. Epstein, L. and van Stee, R. (2001b). Minimizing the maximum starting time on-line. Technical Report SEN-R00133, CWI, Amsterdam. Epstein, L. and van Stee, R. (2001c). Optimal on-line flow time with resource augmentation. In Proceedings of the 13th International Symposium on the Fundamentals of Computation Theory (FCT ‘2001), Lecture Notes in Computer Science, LNCS 2138, pages 472-482, Riga, Latvia. Springer Verlag. Gao, X., Iida, H., Uiterwijk, J., and Herik, H. v. d. (2001). Strategies anticipating a difference in search depth using opponent-model search. Theoretical Computer Science, volume 252(1-2): pages 83-104. ISSN 0304-3975. Giotis, A. P., Emmerich, M., Naujoks, B., Bäck, T., and Giannakoglu, K. (2001). Low-cost stochastic optimisation for engineering applications. In Giannakoglu, K., Tsahalis, D., Periaux, J., Papailiou, K., and Fogarty, T., editors, Proceedings of the International Conference on Industrial Applications of Evolutionary Algorithms, EUROGEN. CIMNE, Barcelona. Graaf de, J., Kosters, W., and Witteman, J. (2001). Interesting fuzzy association rules in quantitative databases. In de Raedt, L. and Siebes, A., editors, 5th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD’01), volume 2168 of LNAI, 78 pages 140-151, Freiburg, Germany. Springer Verlag. Hagen ten, S. (2001). Continuous State Space Q-Learning for Control of Nonlinear Systems. PhD thesis, Computer Science Institute, University of Amsterdam, The Netherlands. Hagen ten, S. and Kröse, B. (2001). Autonomous map building by mobile robots. In Kröse, B., Rijke, M. d., Schreiber, G., and Someren, M. v., editors, Proceedings of the 13th Dutch-Belgian Artificial Intelligence Conference, BNAIC’01, pages 117-124. Hemert, J.I., v. and Jansen, M. (2001). An engineering approach to evolutionary art. In Spector, L., Goodman, E. D., Wu, A., Langdon, W., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation Conference (gecco-2001), page 177. Morgan Kaufmann Publishers. Hemert van, J. (2001). Evolutionary computation in constraint satisfaction and machine learning – an abstract of my ph.d. In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van Remortel, P., editors, Proceedings of the Brussels Evolutionary Algorithms Day ( bead-2001). Vrije Universiteit Brussel (vub). Hemert van, J., Hoyweghen, C. V., Lukschandl, E., and Verbeeck, K. (2001). A “futurist” approach to dynamic environments. In Branke, F. D. O. P. J. and Bäck, T., editors, Proceedings of the Work-shops at the Genetic and Evolutionary Computation Conference (gecco-2001), pages 35-38. Morgan Kaufmann. Herik, H. v. d. (2001). . it ain’t necessarily so. In Cliteur, P., van den Herik, H., Huls, N., and Schmidt, A., editors, Grensoverschrijdende Communicatie, pages 331-352. E.M. Meijers Institituut, Leiden & Kluwer, Deventer, The Netherlands. ISBN 90 268 3854 9. Herik, H. v. d. and Iida, H. (2001). Computer and games. Theoretical Computer Science, volume 252(1-2):ISSN 0304-3975. Heskes, T. (2001). Self-organizing maps, vector quantization, and mixture modeling. IEEE Transactions on Neural Networks, volume 12: pages1299-1305. Heskes, T. (2002). Approximate algorithms for neural-Bayesian approaches. Theoretical Computer Science. In press. Hooimeijer, M. (2001). Reduction of complex computational models. In Phd-dissertatie. Technical University Delft. Jain, A., Duin, R., and Mao, J. (2001). Statistical pattern recognition: A review. In Vossepoel, A. and Vos, F., editors, Fourth Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image Processing, pages 1-34. NVPHBV, Delft. Jelasity, M. (2001). The Shape of Evolutionary Search, discovering and representing search space structure. PhD thesis, Universiteit Leiden. Kappen, H. (2001). An introduction to stochastic neural networks. In Moss, F. and Gielen, S., editors, Neuro-Informatics and Neural Modelling, pages 517-552. North-Holland. Kappen, H. (2002). The cluster variation method for approximate reasoning in medical diagnosis. In Nardulli, G. and Stramaglia, S., editors, Modeling Bio-medical signals. World-Scientific. In press. Kappen, H., Gielen, C., Wiegerinck, W., Cemgil, A., Heskes, T., Nijman, M., and Leisink, M. (2001a). Approximate reasoning: Real world applications of graphical models. In Uesaka, Y., Kanerva, P., and Asoh, H., editors, Foundations of Real-World Intelligence, pages 73-121. CSLI, California, USA. Submitted to RWC in October 2001. Kappen, H. and Wiegerinck, W. (2001a). Mean field theory for graphical models. In Saad, D. and Opper, M., editors, Advanced mean field theory, pages 37-49. MIT Press. Kappen, H. and Wiegerinck, W. (2001b). Second order approximations for probability models. In Leen, T., Dietterich, T., Caruana, R., and Sa, V. d., editors, Advances in Neural Information Processing Systems 13, pages 238-244. MIT Press. Kappen, H. and Wiegerinck, W. (2002). A novel iteration scheme for the cluster variation method. In Dieterich, T., Becker, S., and Ghahramani, Z., editors, Advances in Neural Information Processing Systems, volume 14. In press. 79 KLMN OPQR Kappen, H., Wiegerinck, W., and Braak, E. t. (2001b). Decision support for medical diagnosis. In STT, editor, The future of data mining. In press. Keller, R. E. (2001a). Autopoietic solutions for real-world problems. In EvoDebates: The future of Evolutionary Computing. EvoNet-The European Network of Excellence in Evolutionary Computing. In press. Keller, R. E. and Banzhaf, W. (2001). The evolution of genetic code on a hard problem. In Spector, L., Goodman, E. D., Wu, A., Langdon, W., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation Conference (gecco-2001). In press. Khu, S., van der Veer, P., and Liong, S. (2001). Improving the performance of artificial neural network and its application to rainfall-runoff simulation. Canadian Journal of Civil Engineering, pages 1-21. Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001a). Learning time allocation using neural networks. In Marsland, T. and Frank, I., editors, Computers and Games, Proceedings of the Second International Conference, CG 2000: Lecture Notes in Computer Science 2063, pages 170-185. Springer Verlag, Berlin. ISBN 3-540-43080-6. Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001b). Move ordering using neural networks. In Monostori, L., Váncza, J., and Ali, M., editors, Engineering of Intelligent Systems: Lecture Notes in Artificial Intelligence, pages 45-50. Springer Verlag, Berlin, Germany. ISBN 3-540-42219-6. Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001c). Move ordering using neural networks. In Monostori, L., Váncza, J., and Ali, M., editors, Engineering of Intelligent Systems, Lecture Notes in Artificial Intelligence, pages 45-50. Springer Verlag, Berlin, Germany. ISBN 3-540-42219-6. Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001d). Search-independent forward pruning. In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 159-166. Universiteit van Amsterdam, Amsterdam, The Netherlands. Kok, J. (2001). Intelligent data analysis. In Kent, A., editor, Encyclopedia of Library and Information Science, volume 70, pages 218-224. Marcel Dekker Inc., New York. Kok, J. and Kosters, W. (2001). Natural data mining techniques. In Paun, G., Rozenberg, G., and Salomaa, A., editors, Current Trends in Theoretical Computer Science: Entering the 21st Century, pages 603-613. World Scientific, Singapore. Kortmann, R., Postma, E., and Herik, H. v. d. (2001). Evolution of visual resolution constrained by a trade-off. Artificial Life, volume 7(2): pages 125-145. ISSN 1064-5462. Kröse, B., Vlassis, N., Bunschoten, R., and Motomura, Y. (2001). A probabilistic model for appearance-based robot localization. Image and Vision Computing, volume 19(6): pages 381391. Kuncheva, L., Bezdek, J., and Duin, R. (2001). Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognition, volume 34(2): pages 229-314. Lamers, M. (2001). Neural Networks for Analysis of Data in Environmental Epidemiology - A Casestudy into Acute Effects of Air Pollution Episodes. PhD thesis, Universiteit Leiden. Leisink, M. and Kappen, H. (2001a). Means, correlations and bounds. In Proceedings NIPS. In press. Leisink, M. A. R. and Kappen, H. J. (2001b). A tighter bound for graphical models. In Leen, T., Dietterich, T., Caruana, R., and Sa, V. d., editors, Advances in Neural Information Processing Systems 13. MIT Press. In press. Leisink, M. A. R. and Kappen, H. J. (2001c). A tighter bound for graphical models. Neural computation, volume 13. In press. Loog, M., Duin, R., and Haeb-Umbach, R. (2001). Multiclass linear dimension reduction by weighted pairwise fisher criteria. IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 23(7): pages 762-766. 80 Mathijsen, E., Lipsch, M., Dorp, C. v., and Herik, H. v. d. (2001). A model for exchanging knowledge among virtual teams. In Proceedings of the 2nd European Conference on Knowledge Management. The IEDC-Bled School of Management, Bled, Slovenia. Musa, M., Duin, R., and de Ridder, D. (2001a). An enhanced em algorithm for principal component analyzers. In Leonardis, A. and Bischof, H., editors, Proceedings ICANN 2001, Workshop on Kernel and Subspace Methods for Computer Vision, pages 25-35. Musa, M., Duin, R., and de Ridder, D. (2001b). Modelling handwritten digit data using probabilistic principal component analysis. In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft. Nijssen, S. (2001). Dynamic optimisation, ec theory and data mining - a partial abstract of my ph.d. In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van Remortel, P., editors, Proceedings of the Brussels Evolutionary Algorithms Day ( bead-2001). Vrije Universiteit Brussel (vub). Nijssen, S. and Kok, J. (2001). Faster association rules for multiple relations. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI’01), pages 891-896. Oudshoff, A., I.E. Bosloper, Klos, T., and Spaanenburg, L. (2001). Categorizing virtual communities by text-mining. In Proceedings BNAIC’01, pages 195-202. Paclik, P., Duin, R., and van Kempen, G. (2001a). Multi-spectral image segmentation algorithm combining spatial and spectral information. In Austvoll, I., editor, Proceedings of the 12th Scandinavian Conference on Image Analysis, SCIA 2001, pages 230-235. NOBIM, Norway. Paclik, P., Duin, R., and van Kempen, G. (2001b). Segmentation of multi-spectral images using a combined classifier approach. In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft. Pantic, L., Torres, J., and Kappen, H. (2001). On the role of dynamical synapses in coincidence detection. In Proceedings CNS. In Press. Pekalska, E. and Duin, R. (2001a). Automatic pattern recognition by similarity representations. Electronics Letters, volume 37(3): pages 159-160. Pekalska, E. and Duin, R. (2001b). Is combining useful for dissimilarity representations? In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft. Pekalska, E. and Duin, R. (2001c). On combining dissimilarity representations. In Kittler, J. and Roli, F., editors, Multiple Classifier Systems, Proceedings Second International Workshop MCS 2001, Lecture Notes in Computer Science, volume 2096, pages 359-368. Springer Verlag, Berlin. Pekalska, E., Paclik, P., and Duin, R. (2001). A generalized kernel approach to dissimilarity based classification. Journal of Machine Learning Research, volume 2(2): pages 175-211. Postma, E., Dartel, M. v., and Kortmann, R. (2001a). Recognition by fixation. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 425-432. Universiteit van Amsterdam, Amsterdam, The Netherlands. Postma, E., Roebroeck, A., and Lacroix, J. (2001b). The dynamics of semantic network development. In French, R. and Sougné, J., editors, Connectionist Models of Learning, Development and Evolution, pages 293-302. Springer-Verlag, London, U.K. Putten van der, P. and den Uyl, M. (2001). Mining e-markets. In IT Monitor, volume 3. In press. Ridder de, D. (2001). Adaptive methods of image processing. PhD thesis, Delft University of Technology. ASCI Dissertation Series, 70. Ridder de, D., Lemmers, O., Duin, R., and Kittler, J. (2001). The adaptive subspace map for image description and image database retrieval. In Vossepoel, A. and Vos, F., editors, Fourth Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image Processing, pages 291-299. NVPHBV, Delft. Roos, N., Teije, A. t., Bos, A., and C., W. (2001). Multi-agent diagnosis: an analysis. In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium- 81 STUV Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 221-228. Universiteit van Amsterdam, Amsterdam, The Netherlands. Sasaki, N., Hirasawa, M., Hashimoto, T., Iida, H., Uiterwijk, J., and Herik, H. v. d. (2001). Hira, a four-handed shogi program. In van den Herik, H. and Monien, B., editors, Advances in Computer Games 9, pages 247-259. IKAT, Universiteit Maastricht, Maastricht, The Netherlands. ISBN 90 6216 5761. Schmidt, K., Henkel, C., Rozenberg, G., and Spaink, H. (2001). Experimental aspects of dna computing by blocking: Use of fluorescence techniques for detection. In Kraayenhof, R., Visser, A., and Gerritsen, H., editors, Fluorescence Spectroscopy, Imaging and Probes. Springer-Verlag. Seiden, S. and van Stee, R. (2001). New bounds for multi-dimensional packing. Technical Report SEN-R00132, CWI, Amsterdam. Skurichina, M. (2001). Stabilizing weak classifiers. PhD thesis, Delft University of Technology. Skurichina, M. and Duin, R. (2001). Bagging and the random subspace method for redundant feature spaces. In Kittler, J. and Roli, F., editors, Proceedings Second International Workshop MCS 2001, Lecture Notes in Computer Science, volume 2096, pages 1-10. Springer Verlag, Berlin. Smirnov, E. (2001). Conjunctive and Disjunctive Version Spaces with Instance-Based Boundary Sets. PhD thesis, IKAT, Universiteit Maastricht, Maastricht, The Netherlands. Smirnov, E., Herik, H. v. d., and Sprinkhuizen-Kuyper, I. (2001). An adaptable version-space representation. In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 233-240. Smith, K., Chuan, S., and van der Putten, P. (2001). Determining the validity of clustering for data fusion. In International Workshop on Hybrid Intelligent Systems 2001. In press. Spaanenburg, L. (2001a). Embedded intelligence glues the home together. In PT-ES Research Dossier 2002, pages 24-27. Ten Hagen & Stam. Spaanenburg, L. (2001b). Embedding virtual products by generic components. In Proceedings 2nd Embedded Systems Workshop, pages 237-242. Spaanenburg, L. (2001c). Intelligente systemen: wantrouwen is op zijn plaats. In Automatisering Gids, page 19. Ten Hagen & Stam. Spaanenburg, L. (2001d). Intelligentie robots stagneert door anti-techniek stemming. In Intelligentie robots stagneert door anti-techniek stemming, page 17. Ten Hagen & Stam Uitg. Spaanenburg, L. (2001e). Neural networks in fixed-point hardware. In On-line Symposium for Electronics Engineers. In Press. Spaanenburg, L. (2001f). Unlearning in feed-forward multi-nets. In Proceedings ICANNGA’01, pages 106-109. Spaanenburg, L., Diepenhorst, M., Haseborg, H. T., and Wang, P. (2001a). Training neural nets for small word width. In Proceedings AmiRA’01, pages 171-180. Spaanenburg, L., Haseborg, H. T., and Peng, W. (2001b). Bringing neural intelligence to 8-bit virtual peripherals. Int. Journal of Knowledge-based Intelligent Engineering Systems, volume 5(3): pages 171-178. Spaanenburg, L. and Stevens, H. (2001). Embedded intelligence to understand images. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Spaanenburg, L. and ten Berg, A. (2001). Neural networks with a small foot-print: a lesson in trading programs for data. In Proceedings AmiRA’01, pages 291-300. Spronck, P. (2001). Palm game design. In Mehdi, Q., Gough, N., and Al-Dabass, D., editors, GAME-ON 2001 2nd International Conference on Intelligent Games and Simulation, pages 9599. SCS Europe, Ghent, Belgium. Spronck, P. and Schilstra, K. (2001). Boks: A knowledge based system in support of the Dutch building materials regulations. In Kowalczyk, R., Loke, S.-W., Reed, N., and Williams, G., editors, Advances in Artificial Intelligence. PRICAI 2000 Workshop Reader: Lecture Notes in Computer 82 Science, volume 2112, pages 93-102. Springer-Verlag, Berlin, Germany. ISBN 3-540-42597-7. Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001a). Evolutionary learning of a neural robot controller. In Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation-CIMCA 2001, pages 510-518. ISBN: 0-858-89847-0. Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001b). Infused evolutionary learning. In Hoste, . V. and de Pauw, G., editors, Proceedings of the Eleventh Belgian-Dutch Conference on Machine Learning, pages 61-68. University of Antwerp, Antwerp, Belgium. ISBN: 0-858-89847-0. Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001c). Island-based evolutionary learning. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 441-448. Stee van, R. and Poutré, J. L. (2001a). Partial servicing of on-line jobs. Journal of Scheduling, volume 4(6): pages 379-396. Stee van, R. and Poutré, J. L. (2001b). Running a job on a collection of dynamic machines, with on-line restarts. Acta Informatica, volume 37(10): pages 727-742. Steen van der, E., Nijhuis, J., Spaanenburg, L., and van Veelen, M. (2001). Sampling casts plasticity in adaptive batch control. In Proceedings ProRISC01, pages 646-651. Tax, D. (2001). One-class classification; Concept-learning in the absence of counter-examples. PhD thesis, Delft University of Technology. ASCI Dissertation Series, 65. Tax, D. and Duin, R. (2001a). Combining one-class classifiers. In Kittler, J. and Roli, F., editors, Multiple Classifier Systems, Proceedings Second International Workshop MCS 2001, Lecture Notes in Computer Science, volume 2096, pages 299-308. Springer Verlag, Berlin. Tax, D. and Duin, R. (2001b). A generalized kernel approach to dissimilarity based classification. Journal of Machine Learning Research, pages 2(2): volume 155-173. Tax, D. and Duin, R. (2001c). Outliers and data descriptions. In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft. Torres, J., Willems, P., Kappen, H., and Koopman, W. (2001a). Hysteresis and bistability in a realistic model for ip3-driven ca2+ oscillations. Europhys. Lett., volume 55(5): pages 746-752. Torres, J., Willems, P., Kappen, H., and Koopman, W. (2001b). Mathematical model for calcium oscillations in non-excitable cells. Biophysical Journal, volume 80(1): 613A Part 2. Veelen van, M., ter Brugge, M., Nijhuis, J., and Spaanenburg, L. (2001a). Speech-driven dialing. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Veelen van, M., terBrugge, M., Nijhuis, J., and Spaanenburg, L. (2001b). Embedding fixedpoint intelligence. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Venema, R. and Spaanenburg, L. (2001). Learning feed-forward multi-nets. In Proceedings ICANNGA’01, pages 102-105. Verbeek, J. J., Vlassis, N., and Kröse, B. (2001). A soft k-segments algorithm for principal curves. In Proc. Int. Conf. on Artificial Neural Networks, pages 450-456, Vienna, Austria. Verbeek, J. J., Vlassis, N., and Kröse, B. (2002). A k-segments algorithm for finding principal curves. Pattern Recognition Letters, volume 23(8). To appear. Vlassis, N. and Likas, A. (2002). A greedy EM algorithm for Gaussian mixture learning. Neural Processing Letters, volume 15(1). In print. Vlassis, N., Motomura, Y., Hara, I., Asoh, H., and Matsui, T. (2001). Edge-based features from omnidirectional images for robot localization. In Proc. IEEE Int. Conf. on Robotics and Automation, pages 1579-1584, Korea. Werf, E. v. d., Uiterwijk, J., and Herik, H. v. d. (2001). Learning connectedness in binary images. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 459-466. Westerdijk, M. (2001). Hidden Variable Models for Knowledge Discovery. PhD thesis, University of Nijmegen. 83 WXYZ Westerdijk, M., Barber, D., and Wiegerinck, W. (2001). Deterministic generative models for fast feature discovery. Data Mining and Knowledge discovery, volume 5, pages 337-363. Wezel van, M., Kosters, W., van der Putten, P., and Kok, J. (2001). Nonmetric multidimensional scaling with neural networks. In Advances in Intelligent Data Analysis, The Fourth International Conference on Intelligent Data Analysis (IDA’01), volume 2189 of LNCS, pages 145-155, Cascais, Portugal. Springer-Verlag. Wichers, W., Graaf, J. D., Nijhuis, J., and Spaanenburg, L. (2001). Animated marketing and sales: a case of hidden intelligence. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht. Wiegerinck, W. and Heskes, T. (2001a). Probability assessment with maximum entropy in bayesian networks. In Lucas, P., van der Gaag, L., and Abu-Hanna, A., editors, AIME’01 Workshop: Bayesian Models in Medicine, pages 71-80. AIME’01. Wiegerinck, W. and Heskes, T. (2001b). Probability assessment with maximum entropy in bayesian networks. In Goodman, A. and Smyth, P., editors, Interface 2001. Interface. in press. Wiesman, F., Roos, N., and Vogt, P. (2001). Automatic ontology mapping for agent communication. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 291-298. Winands, M., Uiterwijk, J., and Herik, H. v. d. (2001a). Combining proof-number search with alpha-beta search. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 299-306. Winands, M., Uiterwijk, J., and Herik, H. v. d. (2001b). The quad heuristic in lines of action. ICGA Journal, volume 24(1): pages 3-14. Ypma, A. (2001a). Learning methods for machine vibration analysis and health monitoring. PhD thesis, Pattern recognition group, TU Delft, The Netherlands. ISBN 90-9015310-1. Ypma, A. (2001b). Learning methods for machine vibration analysis and health monitoring. PhD thesis, Delft University of Technology. Ypma, A., Baunbaek-Jensen, O., Melissant, C., and Duin, R. (2001a). Health monitoring with learning methods. In Bischof, H. and Hornik, K., editors, Proc. ICANN 2001, Int. Conf. on Artificial Neural Networks, pages 541-546. Ypma, A., Kleiman, R. J., Valk, J., and Duin, R. (2001b). Monisom-a system for machine health monitoring with neural networks. In Proceedings BNAIC 2001, pages 495-496. Ypma, A., Kleiman, R. J., Valk, J., and Duin, R. P. W. (2001c). MONISOM-a system for machine health monitoring with neural networks. In Proceedings of 13th Belgian-Dutch Conference on Artificial Intelligence BNAIC’01, October 25-26, Amsterdam, pages 495-496. Ypma, A. and Koopmans, H. (2001). Automatic fault detection in rotating equipment with neural networks (in Dutch). Drive and control, pages 32-36. Ypma, A., Melissant, C., Baunbaek-Jensen, O., and Duin, R. (2001d). Health monitoring with learning methods. In Proc. of ICANN’01, Vienna, August 22-24, pages 541-546. Ypma, A., Speekenbrink, O., and Duin, R. (2001e). The use of context in dynamic pattern recognition applications. In Proceedings BNAIC 2001, pages 307-314. Ypma, A., Speekenbrink, O., and Duin, R. P.W. (2001f). The use of context in dynamic pattern recognition applications. In Proceedings of 13th Belgian-Dutch Conference on Artificial Intelligence BNAIC’01, October 25-26, Amsterdam, pages 307-314. Zoeter, O. and Heskes, T. (2001). Hierarchical visualization of dynamic data using constrained switching kalman filters. In Kröse, B., Rijke, M. d., Schreiber, G., and Someren, M. v., editors, BNAIC 2001, pages 323-324. Zwaag van der, B. (2001). Handwritten digit recognition: a neural network demo. In Reusch, B., editor, Computational Intelligence: Theory and Applications, volume 2206, pages 762-771. Springer LNCS. 84