Download Annual report - SNN Adaptive Intelligence

Annual report 2001
SNN
P.O.Box 9101, interncode 231, 6500 HB Nijmegen, The Netherlands
tel: +31 24 3614245, fax: +31 24 3541435, e-mail: [email protected]
1
Colofon:
Editors: research directors
Final editor: Stan Gielen
Designer cover: ARGANTE-ARGANTE
Lay-out: Annet Wanders
Press: Trioprint
2
Contents
Contacts
4
Addresses
5
Introduction
1
7
Research Groups
2A
Intelligent Autonomous Systems Group
University of Amsterdam
13
2B
Laboratory for Biophysics
University of Nijmegen
19
2C
Algorithmics and Program Methodology
Leiden University
33
2D
Pattern Recognition Group
Delft University of Technology
41
2E
System Technology Cluster, Section Intelligent Modelling
University of Groningen
47
2F
Civil Engineering Informatics Group
Delft University of Technology
53
2G
Institute for Knowledge and Agent Technology (IKAT)
University Maastricht
57
2H
Evolutionary Systems and Applied Algorithmics
CWI, Amsterdam
61
3A
Industrial Partners
70
3B
Commercial Spin-off activities of SNN
72
Commercial Applications
Publications
4
75
3
Addresses
SNN
Prof.dr. C.C.A.M. Gielen (Director), Dr. H.J. Kappen (Vice Director), A. Wanders (Accounting).
University of Nijmegen, PO Box 9101, intern code 231, 6500 HB Nijmegen, The Netherlands
Tel: +31 24 3614245, fax: +31 24 3541435, e-mail: [email protected], http://www.snn.kun.nl
BOARD of SNN
Prof.drs. M. Boasson, Kuipers-Rietberglaan 17, 7271 EJ Borculo
Ir. G. Hiemstra, Van der Meer & van Tilburg, Zeistoever 11, 3704 GB Zeist
Dr.ir. G. van Oortmerssen, CWI, Kruislaan 413, PB 94079, 1090 GB Amsterdam
Drs. P.J.M. Timmermans (treasurer), Faculteit Natuurwetenschappen,Wiskunde en Informatica,
Toernooiveld 1, 6525 ED Nijmegen
Dhr. J.H.M. Uylings, Twinning Center Amsterdam, Kruislaan 400, 1098 SM Amsterdam
Drs. J.P. Veen (until June, 2001)
Dr.ir. P. Zuidema (chairman), CMG, Prof. Meyerslaan 2, 1183 AV Amstelveen
RESEARCH GROUPS
Intelligent Autonomous Systems Group, University of Amsterdam
Prof.dr.ir. F.C.A. Groen, Dr. B. Kröse
Kruislaan 403, 1098 SJ Amsterdam, The Netherlands
Tel:+31-20-5257463, fax: +31-20-5257490, e-mail: [email protected]
http://www.science.uva.nl/research/ias/
Laboratory for Biophysics, University of Nijmegen
Prof.dr. C.C.A.M. Gielen, Dr. H.J. Kappen
Geert Grooteplein 21, Intern 231, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands.
Tel: +31-24-3614245, fax: +31-24-3541435, e-mail:[email protected], http://www.snn.kun.nl
Algorithmics and Program Methodology, Leiden University
Prof.dr. J.N. Kok
P.O. Box 9512, 2300 RA Leiden, The Netherlands
Tel: +31-71-5277057, fax: +31-71-5276985, e-mail: [email protected]
http://www.wi.leidenuniv.nl/CS/ALP
Neural Network Research, Pattern Recognition Group, Delft University of Technology
Dr.ir. R.P.W. Duin
Department of Applied Physics, Lorentzweg 1, 2628 CJ Delft, The Netherlands
Tel: +31-15-2786143, fax: +31-15-2786740, e-mail: [email protected]
http://www.ph.tn.tudelft.nl/Research/neural/index.html
4
System Technology Cluster, University of Groningen, Section Intelligent Modelling
Prof.dr.ir. L. Spaanenburg
P.O. Box 800, 9700 AV Groningen, The Netherlands
Tel: +31-50-3633925, fax: +31-50-3633800 e-mail: [email protected], http://www.cs.rug.nl/Research/St/
Civil Engineering Informatics Group, Delft University of Technology
Prof.dr.ir. P. van der Veer
P.O. Box 5048, 2600 GA Delft, The Netherlands
Tel: +31-15-2781642, fax: +31-15-2787700, e-mail: [email protected], http://www.cti.ct.tudelft.nl/
Institute for Knowledge and Agent Technology (IKAT), Universiteit Maastricht
Prof. H.J. van den Herik
Universiteit Maastricht, Fac. Alg. Wetenschappen IKAT, Postbus 616, 6200 MD Maastricht,
The Netherlands
Tel: +31-43-3883485, fax: +31-43-3884897, email: [email protected], http://www.cs.unimaas.nl
Evolutionary Systems and Applied Algorithmics; CWI, Amsterdam
Prof.dr.ir. J.A. La Poutré
P.O. Box 94079, 1090 GB Amsterdam, The Netherlands
Tel: +31-20-5924082, fax: +31-20-5924199, email: [email protected], http://www.cwi.nl
INDUSTRIAL PARTNERS
Unit Energy Efficiency, Energy Research Foundation (ECN)
Drs.Ing. J.J. Kok, Ir. J.J. Saurwalt
P.O. Box 1, 1755 ZG Petten, The Netherlands
Tel: +31-224-564116, fax: +31-224-561407, e-mail: [email protected], http://www.ecn.nl/
COMMERCIAL SPIN-OFF COMPANIES
SMART Research BV
Dr. T.M. Heskes
P.O. Box 31070, 6503 CB Nijmegen, The Netherlands.
Tel: +31-24-3615039, fax: +31-24-3541435, email: [email protected], http://www.smart-research.nl
Nutech Solutions GmbH
Dr. Thomas Bäck, Managing Director
Leiden University, LIACS, Niels Bohrweg 1
NL-2333 CA Leiden, The Netherlands
emaill: baeck@ liacs.nl
5
Contacts
SNN had collaborations with the following companies and organisations in 2001
Aegon schadeverzekering NV
Ahold
ASTRON
BrandmarC
Buma Stemra
Cap Gemeni Ernst & Young
CLB
Coda Music Technology
CORUS Group Plc
Dacolian
De Telegraaf
DSM
ECN
Emagic Soft- und Hardware GmbH
Fokker
Hogeschool voor de Kunsten
IBM Watson Center New York
Kennis Centrum Papier en Karton KiQ
Korg Italy
KPN
Landustrie Sneek BV
MatchCare Data Compare
Midesa
NATO C3 Agency
Nederlandse Groeistichting
NLR
Noldus Information Technology BV
OPG Groothandel BV
Pfizer BV
Philips
Philips Research Labs
RaInteray
Rand Corporation
RIVM
Sappi
SaurECN
Schuitema NV
Shell SIEP BV
SKF
SMART Research BV
SoundPalette
Space Hiscom BV
Technische Unie
Technofysica
TNO-FEL
Tomandandy Music Inc
TVA Developments
Tweehuysen Consultancy BV
UMC Maastricht
UMC Nijmegen
UMC Utrecht
Unilever
Unilever Research Vlaardingen
Witteveen+Bos
Yamaha Corporation
SNN Research is funded in part by: European community, Japanese Ministry of International Trade
and Industry (MITI), Technology Foundation (NWO), Princes Beatrix Fonds.
6
1 Introduction
7
On the 9-th of July, Drs. Jean-Pierre Veen, member of the Board of SNN, suddenly
passed away in a car accident. Jan Pierre has been a member of the SNN-board
since 1994, and as such, he was one of the “senior” members.
Jean-Pierre has played a very active and important role in SNN. He had a keen
eye for good applied research and for applications within the market. His knowledge about the world of applied scientific research was unsurpassed. For every
good research plan with a prospect for applications, he always managed to
suggest successful ways to funding by national or international funding agencies.
His advice always came with inspiring ideas about future applications.
We will miss Jean Pierre’s enthusiastic contributions in our board meetings.
We will remember him as a stimulating, intellectual and pleasant colleague.
Dr. ir. Peter Zuidema, chairman
8
Introduction
The human brain is one of the great mysteries of nature. It strongly appeals
to human curiosity trying to understand itself. In this context it is remarkable
that the human brain has often been compared with the most advanced
computers. This is remarkable since computers differ from the brain both in
the “hardware” and the “software” with obvious consequences for the different
principles, which underlie information processing in biological and artificial
systems.
One of the aims of the Foundation for Neural Networks (SNN) is to bridge this
gap between natural intelligence and artificial intelligence. SNN aims to
investigate the computational principles of biological and artificial systems
and to apply the results in novel applications. SNN plays a leading role on
the international scene in fundamental research and advanced applications.
This annual report presents an overview of the various research projects within
SNN. Some of the high-lights of projects, which were finished in 2001, will be
described in more detail.
Major achievements in 2001
The Japanese Real World Computing Partnership (RWCP) has been one of
the major funding agencies of SNN. In 2001 this project, which had a duration
of 10 years, entered its last year. It was concluded by a large symposium in
Tokyo, where all participating groups demonstrated the results of their
research efforts. One of the demonstrations was presented by the IAS group
of SNN at the University of Amsterdam, who demonstrated an autonomous
‘office robot’ which was built in collaboration with a number of Japanese
research groups (AIST, Fujitsu, NTT) and a Swedish group (SICS). This robot
is able to navigate in office-like environments, to have a conversation with
users, and to dock to a loading station when it is running out of power. The
IAS group developed the environment learning and localization module of the
system. The robot learns a model of the environment from its `omni-directional’
vision system and uses a probabilistic model to estimate its location within
the environment.
Figure 1:
Visitors and IAS-robot at the RWCP
symposium
Various industries have showed interest in the developed system and the
underlying computer science methods. Based on the results obtained in this
project, a European (ITEA) research proposal by a consortium of European
industrial and academic partners was accepted for funding. A ‘personal’ robot,
which operates in an intelligent home environment, will be developed in
collaboration with Philips Research, Epictoid (a CWI spin-off), University of
Leuven and the IAS group of the University of Amsterdam. Such a robot should
be able to communicate with users using speech dialogue and emotional
feedback and to navigate in a home environment.
In another project, entitled “Hybrid modeling techniques for data-base mining” by Machiel Westerdijk (KUN), various methods for data mining were
explored. The aim of this project was:
•
to find hidden information in large data bases.
•
to present the newly found information in terms of simple rules explaining the structure of the data and relations between data.
9
An explanation about which parameters are relevant and how parameters
combine is equally important. In such a case a solution should meet 2
constraints: to find simple rules to obtain the best possible performance.
The main scientific value of this project lies in the combination of various
techniques (using so-called ‘hidden variable models’) which provide an optimal
balance between optimizing performance and optimizing insight into the data
and the rules explaining relations between the data to human experts. Our
algorithm gave a much better performance than any other algorithm in the
literature (such as the well-known C4.5 algorithm) on well-known bench-mark
problems such as the diagnosis of heart diseases and diabetes, and in optimal
credit assessment in financial applications.
The results of this project are now used by one of the leading international
companies on IT services and business consultancy for real-world applications.
More information about the project can be optained at:
http://www.snn.kun.nl/nijmegen/hybrid.html
The group at the Technical University in Delft has completed a project on machine
diagnostics by neural networks. The main objective of this project was to develop
a diagnostic tool, based on neural network technology, for on-line analysis and
evaluation of data in a complex process to monitor changes within the process.
Typical analyses include the use in machine monitoring (on-line pump monitoring,
off-line analysis of ship engine vibration, monitoring of rotating equipment in
paper industry) and medical monitoring problems (depth of anesthesia during
surgery, detection of eyeblink rate in Tourette’s syndrome patients, analysis of
EEG/MEG signals for detection of epilepsy or Alzheimer’s disease).
This project received the 2001 SKBS Award for best demo/application at the
13th conference on Artificial Intelligence in Belgium and The Netherlands. More
information about the project can be obtained at:
http://www.mbfys.kun.nl/~ypma/project/monisom/index.html.
Some of the results have been implemented in a commercially available software
system for machine health monitoring with self-organizing maps, called
MONISOM. Further information about commercial use of the results can be
obtained from:
J. Valk, Landustrie Sneek BV (e-mail: [email protected])
Figure 2:
MONISOM: a SOM-based system for machine health monitoring, applied
to a dataset representing a progressively loose foundation in a
submersible pump.
10
2 Research Groups
11
12
2A
Intelligent Autonomous Systems Group, University of Amsterdam
Prof.dr.ir. F.C.A. Groen, Dr.ir. B.J.A. Kröse, Drs. R. Bunschoten, Dr.ir. S. ten Hagen,
Drs. J. Portegies Zwart, B. Terwijn, Drs. J. Verbeek, Dr. N. Vlassis, Drs. W. Zajdel
Description of the group
Objective
The IAS group is engaged in the development of theoretical understanding and
computing methods for autonomous systems, including methods for sensor
data processing, reasoning, learning and distributed systems. Here we report
only the work on probabilistic reasoning and learning by autonomous systems.
More information about the other topics of the group can be found at:
http://www.science.uva.nl/research/ias.
Probabilistic methods for data modeling
Drs. J. Verbeek, Dr. N. Vlassis
Objective
In many engineering problems, we are given a collection of data that correspond
to observations of a physical system. Then we are asked to make a model of
this system and to use it later to make predictions, take decisions, etc. Because
the data are normally corrupted by noise, probabilistic methods for data modeling
(probabilistic techniques) are playing an important role in the modeling process
of these data.
Approach
We have recently developed a statistical technique for the problem of probabilistic clustering. In this problem we want to group the set of data into a
number of clusters, while at the same time assigning to each point a degree of
‘certainty’ as to which cluster it belongs. We have proposed a novel algorithm
for fitting the parameters of a Gaussian mixture in a greedy fashion: adding
components to the mixture one after the other. Our algorithm is based on the
Expectation-Maximization algorithm, a wellknown statistical technique for learning
such models.
Our experiments have shown that such a
‘greedy’ method for learning Gaussian
mixtures can give results superior to other,
state-of-the-art methods. We have demonstrated our algorithm in synthetic and realistic experiments with image data, and in all
cases our algorithm gives better or at least
as good solutions as other existing methods
(Vlassis and Likas, 2002).
Figure 3:
A six-component mixture and the component allocation
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−1.5
−2
−1.5
−1
−0.5
0
0.5
1
−2
1.5
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−1
−0.5
0
0.5
1
1.5
−1.5
−1.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
−1.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
13
Tools for non-linear data analysis
Drs. J. Verbeek, Dr. N. Vlassis, Dr.ir. B.J.A. Kröse
Objective
Sensory, ‘real world data’ is generally high dimensional and noisy while the
underlying process generating this data may have a lower dimension. The generated measurements can be considered as noisy points lying on a lowdimensional manifold embedded in a high-dimensional space (the space of
measurements). In this project we study methods that map high dimensional
data to a low dimensional space that preserves the structure of the data (i.e. it
keeps nearby points nearby). Such methods are of interest for data visualization,
where one needs to map data to two dimensional coordinates, and for regression,
inter- and extrapolation tasks, where it is helpful to map the data to a low
dimensional space. Linear projection techniques are the standard tool to perform
these tasks. However, if there is a non-linear relationship between the intrinsic
dimensions of the data and the observed dimensions, linear techniques may
fail to produce sensible mappings.
Approach
The approach we consider in our research is a combination of several local
linear mappings. Each local linear map is responsible for a region in the data
space and produces a linear map of the data to the low dimensional space. A
probabilistic framework allows us to find appropriate locations for the local linear
maps and the regions for which they are responsible. Robust and fast methods
have been developed (Verbeek et al., 2001; Verbeek et al., 2002).
Figure 4:
The one-dimensional structure (the curve) in a two-dimensional data set
(the dots) discovered by the method developed in this project.
The project is a collaboration with the Pattern Recognition Group of the TU Delft
and is funded by STW.
Classification of radar range profiles
Drs. J. Portegies Zwart, Dr.ir. B.J.A. Kröse, Prof.dr.ir. F.C.A. Groen
Objective
The goal of this project is to automatically recognize aircrafts by examining
how they reflect radar waves. Traditional methods of aircraft identification rely
on the cooperation of the target aircraft. These methods are prone to both human
and technical errors. Since misidentifications can have severe consequences,
there is a demand for non-cooperative identification techniques.
Approach
In this project we investigate the use of High Resolution Radar (HRR) range
14
profiles as features for a non-cooperative classification algorithm for aircrafts.
HRR range profiles are measurements of the reflectivity of aircrafts, measured
along the line of sight between the radar and the target. In figure 5 an example
of a HRR range profile is shown. As can be seen in the figure, the shape of the
range profile depends on the shape of the aircraft. Therefore, it might be possible
to determine the type of aircraft by measuring a range profile, and comparing it
with a database of previously measured profiles of known aircrafts.
HRR range profiles depend heavily on the pose of the aircraft at the time of
measurement. Estimates of this pose are available through the use of a secondary tracking radar. Our research has resulted in a method of using these
estimates to construct an estimate of the statistical distribution of HRR profiles.
We have been able to obtain a more accurate estimate of the pose by combining
information from the pose estimates and the behavior of certain peaks in the
HRR range profiles. Other research included automatic alignment of HRR range
profiles, and the use of linear feature extraction to improve classification
accuracy. The project is funded by TNO-FEL.
radar line
of sight
Figure 5:
Example of a HRR range profile. The radar is located at the left, and looking to
the right. Locations on the target aircraft, which reflect the radar signal highly,
result in distinctive peaks in the measured signal. Therefore, part of the
geometry of the target aircraft is encoded in the measured HRR range profile.
Probabilistic models for distributed surveillance
Drs. W. Zajdel, Dr.ir. B.J.A. Kröse
Objective
Surveillance systems often need to track a moving object through larger areas
or buildings. In case the cameras (or other sensors) do not overlap, the sys-
Figure 6:
Automatic tracking of moving
objects across different camera’s.
15
tem is faced with the problem whether an object observed with a camera at
some time is the same object as observed by some other camera some time
ago.
Approach
To deal with the uncertainty we use probabilistic networks. These methods are
able to model the belief in target trajectories. Hidden Markov models can be
seen as a special class. At the moment we adapt probabilistic methods developed for traffic surveillance to indoor tracking problems. An experimental set-up
will be built. The project is funded by STW, in collaboration with TNO-FEL.
Omnidirectional stereo
Drs. R. Bunschoten, Dr.ir. B. Kröse
Objective
Range estimation from multiple images is sensitive to noise and lack of texture.
We developed methods, which integrate information from multiple image pairs
to obtain a robust depth estimate.
Approach
Our robot is equipped with an ‘omnidirectional’ vision system (see figure 7).
While traversing the environment, images are captured. It is possible to make
a spatial reconstruction of the environment from the images. We have developed
a multi-baseline stereo vision algorithm for panoramic image data (Bunschoten
and Kröse, 2001a; Bunschoten and Kröse, 2001b). For increased robustness,
more than the two images required for reconstruction are used. Stereo data
obtained from multiple stereo pairs are fused in a probabilistic fashion. Some
results are displayed in figure 7.
Bayesian methods for mobile robot navigation
Drs. R. Bunschoten, Dr. N. Vlassis, B. Terwijn, Dr.ir. S. ten Hagen, Dr.ir. B. Kröse
Objective
Future ‘personal’ robots should be able to learn a representation of the world, in
which they operate, autonomously. We develop probabilistic methods for
environment learning, robot localization and navigation.
Figure 7:
The omnidirectional image from our vision system (left) and the depth estimates reconstructed from five images.
16
Approach
For the environment representation we developed a method, which, instead of
building a ‘geometric’ representation of the environment, builds an ‘appearance
model’, consisting of all sensory representations. With this model the robot can
localize itself after a new observation using a Markov procedure (Kröse et al.,
2001; Vlassis et al., 2001). The implementation of the Markov localization
procedure was made using a Monte Carlo (particle filtering) method. The
robustness of this method to ‘kidnapping’ of the robot was shown at the RWCP
symposium (see also figure 8).
For the learning stage we developed a method which corrects odometric pose
estimates (Hagen, ten and Kröse, 2001). The RWCP funding has ended, but the
work is continued within a European (ITEA) project ‘Ambience’, in collaboration
with Philips Research.
400
400
Monte Carlo
Estimated Position
350
350
300
300
250
250
200
200
150
150
100
100
Monte Carlo
Estimated Position
50
−150
50
−150
−100
−50
0
50
100
150
200
−100
−50
0
50
100
150
200
250
250
Figure 8:
The influence of kidnapping the robot on the particle filter representation. In the middle we see the distribution of samples
(representing the position of the robot) before kidnapping. The particles are condensed to the real location. In the right figure we
see the distribution of particles indicating the robot’s position estimate after the kidnapping: the robot is uncertain about its position.
It takes about five novel observations to reach again a condensed estimation.
Reinforcement learning and neurocontrol
Dr.ir. S. ten Hagen, Dr.ir. B. Kröse
Objective
In a joint project with the Control Group from the Delft University of Technology
we investigate Reinforcement Learning (RL) for control applications, in particular
for the continuous state domain and the applicability for real-world control
problems.
Approach
We investigated how much exploration (excitation) we have to add to the control
signal in order to obtain good learning performance. For non-linear systems we
developed ‘Neural-Q’ learning which requires far fewer trials then other neural
reinforcement learning approaches. The resulting feedback controller can be
regarded as a linear feedback with a non-linear correction that compensates for
the non-linearities in the system. The non-linear correction allows this feedback
to operate in a larger region than conventional linear feedback, based on local
linearities of the non-linear system. A Ph.D. thesis was defended in February
2001 (Hagen, ten, 2001). This project was funded by STW.
17
Kat holieke U n i v e r s i t e i t Ni j me g e n
18
2B
Laboratory for Biophysics, University of Nijmegen
Prof.dr. C.C.A.M. Gielen, Dr. H.J. Kappen, Dr. T.M. Heskes, Dr. W.A.J.J. Wiegerinck, Dr.
A. Ypma, Drs. M.J.D. Westerdijk, Drs. M. Leisink, Drs. B. Bakker, Ir. T.A. Cemgil, Drs.
L. Pantic, Drs. O. Zoeter, Ir. M.J. Nijman, Drs. J.J. Spanjers, Drs. N. Keyzer
Description of the group
The laboratory of Biophysics of the KUN is engaged in research on the theory
of neural networks since 1988, with links to computational issues in neurobiology
as well as applications in industry. Important research themes are stochastic
learning processes; Learning of structure in large databases using probability
models based on Bayes networks and Boltzmann Machines; Active decision
processes with application in medical diagnostics; Confidence estimation for
neural network prediction and relevance detection; Bayesian statistics;
Recognition of musical rhythms; The functional role of dynamical synapses
and pattern formation in neural dynamics. The group receives significant funding
from the Japanese Real World Computing program and from various STW project
subsidies. More information on the group can be found at:
http:/www.snn.kun.nl/nijmegen
Long term research projects
Graphical models
Wim Wiegerinck, Bert Kappen, Marcel Nijman, Martijn Leisink
Funding: STW Pionier, RWC
The aims of this project are to develop novel theory, methods and implementations
for learning and reasoning in a complex dynamic multi-sensory environment.
The approach to reasoning and learning is based on the axioms of probability
theory. It encompasses directed graphical models and Boltzmann Machines.
The research aims at the design of algorithms to enable learning and reasoning
involving up to the order of 1000 variables. This allows for applications, which
are order of magnitude larger than currently possible.
The fundamental operation that must be performed during learning or reasoning
tasks is to compute marginal probabilities for subsets of variables. In general,
these computations are intractable since they require O(2 n) operations, with n
the number of variables. Approximate schemes are therefore of great importance
for real world computation. An introduction to this research can be found in
(Kappen et al., 2001a; Kappen et al., 2001b; Kappen, 2001).
In 2001, we have obtained the following results:
•
Standard mean field theory can be derived from Jensen’s inequality yielding
a first-order lower-bound of the partition function. The quality of the bound is of
crucial importance for the quality of the approximation. For directed graphical
models, optimization of this lower bound provides an approximate EM-learning
scheme with guaranteed convergence. We derived a systematic sequence of
odd-order lower-bounds of the partition function that are guaranteed to improve
upon the first-order lower-bound (Leisink and Kappen, 2001b; Leisink and
19
Kappen, 2001c). We applied this approach to compute bounds on the means
and correlation’s in Boltzmann Machines (Leisink and Kappen, 2001a).
• For (symmetric) Boltzmann Machines, the mean field equations can be
derived as the first order approximation in a Taylor expansion of the marginals.
Second order corrections in this expansion are known as the TAP correction.
We generalized this expansion for asymmetric Boltzmann machines and
graphical models of general form. First and second order approximations were
derived and studied numerically. For sigmoid belief networks, the expansion
method gives an extremely fast alternative to the standard approximation
schemes (Kappen and Wiegerinck, 2001b; Kappen and Wiegerinck, 2001a).
• A very recent development is the application of the Cluster Variation method
(CVM) to probabilistic inference. CVM is a method that has been developed in
the physics community to approximately compute the properties of the Ising
model. The CVM approximates the probability distribution by a number of
(overlapping) marginal distributions (clusters). The quality of the approximation
is determined by the size and number of clusters. We derive novel iteration
schemes for the Cluster Variation method (Kappen and Wiegerinck, 2002;
Kappen, 2002).
Figure 9:
BayesBuilder graphical software environment, showing part
of the Anemia network. The network consists of 91 variables
and models some specific Anemia’s. The software implements
some of the approximation techniques described above.
Computational neuroscience
Associative memory with dynamic synapses
Lovorka Pantic, Joaquin J. Torres, Bert Kappen
Funding: University of Nijmegen
Recent biological findings show that the electrical properties of synapses depend
strongly on recent presynaptic neuron activity. This phenomenon is known as
synaptic depression and shows that the synapse has to recover after
transmission of an action potential (Abbott at al., 1997, Tsodyks and Markram,
1997). In our previous studies (Pantic et al., 2000; Pantic et al., 2001a) we
have shown that synaptic depression improves the ability of neurons to detect
coincident spikes. We have also studied the effects of synaptic depression on
associative memory. Our results reveal, in addition to the standard memory
20
states, a new type of behavior where the network resides in one memory state
for some time and then rapidly switches to another memory state (see figure
10). Using linear stability analysis we obtain a phase portrait defining the
parameter values for which this new phase occurs. We conclude that the
functionality of synaptic depression affects the stability of memory attractors
and enables flexible transitions among them (Pantic et al., 2001b).
s
20
20
15
τrec
neurons
Paramagnetic phase
40
Oscillatory phase
10
60
5
Ferromagnetic phase
80
0
100
20
40
60
time
80
100
0
2
4
β
6
8
10
Figure 10:
The rapid switching behavior of an associative memory network consisting of 100 neurons (vertical axis)
with dynamic synapses and 10 stored patterns during 100 msec (horizontal axis). Blue indicates neuron
firing. The network settles in a combination of one or more patterns for some time and then rapidly switches
to another activation. This behavior may be relevant for sequential processing of perceptual information.
Calcium signaling in non-excitable cells
Dr. J.J. Torres, University of Nijmegen and University of Granada
Dr. H.J. Kappen, Dr. P.H.G.M. Willems and Dr. W.J.H. Koopman, University of Nijmegen
Funding: STW Pionier, MEC, MCyT and FEDER (Ramon y Cajal program)
Ionized calcium (Ca2+) not only represents the most common signal transduction
element relaying information within cells to control a wide array of activities,
including secretion, contraction and cell proliferation, but also is invariably
involved in cell death. To coordinate all of these functions, cytosolic Ca 2+ needs
to be precisely regulated in space, time and amplitude. In this project we aim
to model calcium signaling in different types of non-excitable cells and study
calcium wave propagation in monolayers of those cells. As a first biological
example we started with the pancreatic acinar cell. This cell has been used
extensively as a biological model for studying the relationship between the
spatio-temporal aspect of cystosolic calcium signaling and cellular activities
such as fluid and protein secretion. Following this motivation, we have developed
a mathematical model that includes realistic assumptions about different
calcium fluxes described in living cells (see figure 11). The model reproduces
qualitatively and quantitatively the dynamics of the experimentally observed
calcium signals (Torres et al., 2001a; Torres et al., 2001b). Secondly and using
this mathematical model, we have modeled an extended medium consisting of
a monolayer of these non-excitable cells to explore the regions in the space of
parameters where calcium waves can propagate in the medium (Torres et al.,
2001a). The study has been useful to find regions in the space of parameters
21
where different oscillatory regimes appear. The ability of our model to generate
traveling waves, allows to investigate the mechanism that underlies the highly
complex spatio-temporal Ca2+ signals between and within non-excitable cells.
B
2mM
100nM
V leakER
V leakPM
V IP
3
V PMCA
V CRAC
[Ca2+]
ER
[Ca2+]
cyt
IP
3
DAG
V SERCA
PLC
PIP
2
Receptor
Hormone
Figure 11:
Schematic representation of the Ca 2+ fluxes present in
the model for calcium oscillations in pancreatic acinar
cells, and the diverse regulatory interactions of Ca 2+ :
Grey background: compartments with high [Ca 2+]; white
background: compartments with low [Ca 2+ ].
Modeling electrical activity in normal rat kidney fibroblast
Dr. J.J. Torres, University of Granada and University of Nijmegen. Drs. N.L.N. Cornelisse,
Dr. L. Theuvenet, Dr. E. Harks, Prof.dr. C.C.A.M. Gielen, University of Nijmegen. Prof.dr.
D.L. Ypey, University of Leiden and University of Nijmegen
Funding: NWO, MEC, MCyT and FEDER (Ramon y Cajal program)
Membrane voltage (mV)
40
20
0
-20
-40
-60
-80
-//5 min
Using standard Hodgkin-Huxley (HH) and Goldman-Hodgkin-Katz (GHK)
formalisms we have developed a mathematical model to study excitability and
action potential propagation in quiescent normal rat kidney fibroblasts (NRK)
(Torres et al., 2002). One can induce artificially growing of NRK fibroblasts forming
a two dimensional monolayer of such cells. In the monolayer the cells are
connected electrically by gap-junctions. Using standard techniques it is possible
to record the electrical activity of the cells when they are isolated, when they
are growing normally and when the cell are growing anomalously (see
figure 12). Theoretically, we can build two-dimensional networks of HH units to investigate what the conditions are
50 µM BAPTA-AM
in which an artificial mono layer is able to reproduce the
observed behavior in the recordings. Using standard techniques we were able to describe the main ionic currents
responsible for the excitability of NRK fibroblasts. We have
introduced this information in the model and we were able
to reproduce the voltage-clamp and current-clamp experiments in NRK fibroblasts in both quiescent and densityarrested NRK fibroblasts (see, for instance figure 12).
ICl(Ca)=0 pA
40
Membrane voltage (mV)
20
0
-20
-40
//
-60
-80
22
4 min
Figure 12:
Electrical activity in density-arrested NRK fibroblasts. Top layer:
Electrical signals recorded in a growing monolayer of NRK fibroblast
before and after cytosolic Ca 2+ is buffered by BAPTA-AM. Erasing
calcium from cytoplasm has the effect of reducing the calcium
dependent chloride conductance. Bottom panel: Model simulations
for the same conditions as the experiments in the top panel.
Regulation of melanotrope cell activity in Xenopus laevis
Drs. N. Cornelisse, Prof.dr. C. Gielen, Prof.dr. E. Roubos, Dr. W. Scheenen, Prof.dr. D.
Ypey
One of the mysteries in neuronal communication deals with the problem how
extracellular information, which arrives from many other cells onto a cell, is
coded into intracellular signals to initiate new cell activity (for example hormone
secretion, DNA expression, protein synthesis). One aspect of this problem is
that a large number of extracellular neurotransmitters is known to be involved.
However, only very few intracellular messengers are available to transport the
extracellular infor- mation from the cell membrane to the cell nucleus. In this
project we investigate this problem in the melanotrope cell of Xenopus laevis,
which is known to produce a hormone, which adapts the color of the skin to the
background of the animal to camouflage it for predators.
A minimal model is presented to explain changes in frequency, shape, and
amplitude of intracellular Ca 2+ (second messenger) oscillations in the
melanotrope cell (see Cornelisse et al., 2001 and figure 13). Local spatiotemporal changes in intracellular Ca2+ concentration are known to transport the
extracellular signals near the membrane of the cell to the cell nucleus. The
model describes the cell as a plasma-membrane oscillator (corresponding to
the well-known Hodgkin-Huxley model) with influx of extracellular Ca 2+ via voltagegated channels in the plasma membrane. Figure 13 shows an example of
changes in intracellular Ca 2+ concentration (top panel) as a function of the action
potentials of the cell membrane (lower panel).
Different extracellular neurotransmitters each have a different effect on the
dynamics of various ion-channels in the cell membrane, which affect the
membrane potential of a cell. Therefore, we have characterized the time
constants, which determine opening and closure of the Ca 2+, Na+ , and K+ ionchannels. The relationship between the concentration of extracellular
neurotransmitters and the time constants of ion-channel dynamics allow us to
provide an accurate description of the communication between cells and of the
intracellular Ca2+ oscillations.
[Ca ]i (mM)
0.35
0.3
2+
0.25
0.2
0.15
0.1
10
15
20
25
15
20
25
30
35
40
45
30
35
40
45
Figure 13:
Simulation of calcium oscillations
coupled to electrical bursting
Voltage(mV)
20
0
-20
-40
-60
10
time(s)
23
Applied research projects
Graphical models for data mining
Alexander Ypma and Tom Heskes
Funding: Technology Foundation STW, since May 1 2001
Goal
In this project, we focus on the unsupervised modeling of dynamic processes
with graphical models. We want to know how prior knowledge about a particular
problem can be used to find meaningful models, while at the same time allowing
for unsupervised ‘discovery’ of underlying patterns. In case studies we will explore
which approximations are valid and necessary for obtaining a practically usable
method.
Example: modeling of heterogeneous time series
Practical applications may be found, for example, in modeling of consumer
behavior using sequences of phone calls or web navigation patterns. Here, we
have prior knowledge about the origin of the data like e.g. the hierarchical
structure of a website and possibly other information about a user. Since many
different user types will visit a website, one should account for heterogeneity.
One way to take heterogeneity into account is by adopting a probabilistic
generative model for description of individuals.
Learning of the individual model parameters and the labels of each data item
(e.g. a sequence of web-pages or phone-numbers) can be done in an
unsupervised manner using the EM algorithm. Two examples of this approach
are shown in figure 14. The left figure illustrates the simultaneous learning of
parameters of different regression models. The right figures show two transition
matrices that describe the surfing behavior at a large commercial website, using
a mixture of Markov chains. Each entry in a transition matrix yields the probability
of clicking on a certain page-type given the currently visited page-type (e.g.
‘sports’ or ‘weather’ in a news-website). We assume that a certain surfer-type
can be characterized by such a page-type transition matrix. Mixture models
and (hidden) Markov models are instances of the broader class of graphical
models.
Outlook
Other applications include the prediction of brand market shares as a function
State transition matrix for user type 9
2
2
4
4
6
6
From state
From state
State transition matrix for user type 7
8
8
10
10
12
12
14
14
2
4
6
8
To state
10
12
14
2
4
6
8
10
12
14
To state
Figure 14:
Describing heterogeneity with mixture models. Left figure: simultaneous learning of parameters from three regression models using
noisy realizations of these models; right two figures: modeling web surfing behavior with mixtures of Markov chains; frequently
visited entries in page-to-page transition matrix are shown as bright (yellow) values; each surfer-type is modeled by a separate
transition matrix.
24
of several macro- and microeconomic variables and prediction of option prices.
Industrial partners in this project are currently: KPN Research and BrandmarC.
Ultimately, we intend to develop a data mining tool based on graphical models
that can be used in a practical setting.
Multitask learning
Bart Bakker, Bert Kappen, and Tom Heskes
Funding: STW Pionier
Approach
Many real-world problems can be seen as a series of similar, yet self contained
tasks. An example is newspaper sales, where we predict sales at different
locations within the country. In neural networks this learning process can be
implemented as follows. We build a feed-forward model (see figure 15) to
represent the series of parallel tasks. In this model the inputs correspond to
the inputs of the parallel tasks, e.g. weather figures and previous sales (both
long term and short term) for the newspaper example. Each output corresponds
to the output of one of the tasks (where for simplicity we have taken this output
to be one-dimensional). All weights W connected to the inputs in this model
are the same for all tasks, whereas the weights Ai connected to the outputs
are different for each task. This architecture allows the model to detect ‘features’
in the inputs, and to use these features for regression.
We use the same architecture for our multitask learning project, and add to it
a Bayesian prior over the task specific parameters Ai . This prior introduces a
‘soft sharing’ of the parameters Ai between the tasks. In our approach, all
shared parameters, including but not restricted to the hyperparameters
specifying the prior, are inferred through a maximum likelihood procedure. The
remaining parameters specific to each task are treated in a Bayesian manner.
Results
We apply the model to the prediction of newspaper sales for ‘De Telegraaf’.
The data that we use to train the model consist of three years of Saturday
newspaper sales figures at 343 outlets in The Netherlands, and corresponding
inputs. Inputs include recent sales (-4 to -6 weeks), last year’s sales (-51 to 53 weeks), weather information (temperature, wind, sunshine, precipitation
quantity and duration) and season (cosine and sine of scaled week number).
We compared our Bayesian approach to ‘single task learning’, where we trained
µ
outputs yi
A
i
µ
bias
hik
W
Figure 15:
Neural network model.
inputs xµ
ij
bias
25
one feedforward model for each task (so all parameters were task specific),
and ‘non-Bayesian multitask learning’, where we used the model architecture
shown in figure 16, and inferred all parameters through maximum likelihood
optimization. We found that our approach yielded predictions that were more
accurate than predictions made through non-Bayesian multitask learning, which
in turn performed much better than single task learning.
The task clustering aspect of our approach revealed hidden structure within the
data. When we allowed two tasks clusters, we obtained one cluster
corresponding to a model that was very sensitive to long term sales and season,
whereas the other was most sensitive to short term sales. A closer study of the
outlets assigned to each of the clusters (see figure 16) revealed a sensible
division. Outlets, that were assigned to the long term/seasonal cluster, were
located in the more touristic areas of Holland (where sales are strongly influenced
by the holidays), whereas those in the short term cluster were located in the
larger cities. This distinction between touristic and urban locations was detected
by the model, even though it was not explicitly coded in the data.
Figure 16:
Clustering of Dutch outlets. Circles mark outlets
assigned to either the ‘seasonal’ cluster (left
panel) or the ‘short term’ cluster (right panel).
Analysis and classification of dyskinesia in Parkinson Disease
Drs. N. Keijser, Dr. M. Horstink, Prof.dr. C.C.A.M. Gielen
Parkinson patients usually receive Levodopa medication. However, after using
Levodopa for about 10 years, it becomes more difficult to adjust the proper
dose of Levodopa. As a result patients either develop Parkinson symptoms, or
(more frequently) develop involuntary movements (dykinesia) which cannot be
suppressed by the patients. This is a very well known and hard problem in the
treatment of Parkinson Disease. The aim of this project is to develop an
automatic and objective procedure to detect dyskinesia and to find the proper
dose and timing of Levodopa administration. Although many groups are working
to solve this problem, the main difficulty so far has been to distinguish involuntary
dyskinesia from voluntary movements.
Movements of patients were measured using accelerometers on the wrist, upper
arm, trunk and leg, which measure 3-D movements. Accelerometer signals are
recorded on a small portable recorder and analyzed offline. After preprocessing
of the accelerometer signals to obtain specific movement parameters, a neural
26
network was trained with these parameters as input and the rating of the
movements of patients by experienced neurologists as output. After training
the neural network was able to classify movements correctly into normal
movements and dyskinesia (score 95%). The neural network also correctly
rated the amount of dyskinesia on the usual m-AIMS score (scale between 0
and 4) which is used by neurologists to rate dyskinesia. The performance
obtained in this study is by far the best in the literature reported so far. Moreover,
this is the first study, which is able to distinguish between normal voluntary
movements and dyskinesia.
Quite interestingly, the neural networks also provide detailed information about
the parameters, which are used to rate dyskinesia and to distinguish dyskinesia
from voluntary movements. For example, it appears that movements of different
body segments move in a much more coordinated way in voluntary movements
than in dyskinesia. Information about the parameters, which distinguish between
voluntary movements and dyskinesia, provide valuable information to assist
neurologists in the diagnosis and treatment of dyskinesia.
2
physicians rating
neural network rating
AIMS score
1.5
1
0.5
eat
sweep
enter a door
wash hands
enter a door
change clothes
Sit & mental
stand & mental
walk
clean the sheets
serve & drink
coffee
wipe off the
dishes
knit
wash the dishes
Prepare &
eat luncg
set the table
make coffee
sit
telephone
pack−board
stand
take off shoes
write a letter
0
Figure 17:
Example of the dyskinesia rating given by the physicians
(circles) and predicted by the neural network (dots with
error bars) for the trunk for eighty-one one-minute intervals
of various activities.
one−minute intervals and the tasks performed in these one−minute intervals
Neural networks for the paper industry
Onno Zoeter, Stan Gielen, and Tom Heskes
Funding: Kenniscentrum Papier en Karton
Approach
Papermaking is a difficult and only partially understood process. Traditionally,
the production process is controlled and optimized by human experts that have
gathered insights and rules of thumb through years of experience. The increase
in the number of sensors and the amount of process data stored facilitates a
more quantitative analysis. The goal is to obtain insight in the production process and to use this insight to help the machine operator improve the production.
Our first attempt in this direction is the development of a model for the visualization of highly dimensional dynamic data, a typical example being the sensor
readings of a running paper mill. The model that we use is a dynamic extension
of a hierarchical mixture of principal component analyzers. The generative model
27
for each level of the hierarchy corresponds to a particular kind of switching
dynamical system. Its graphical structure is given in figure 18.
Results
An example of the visualization obtained with our model is given in figure 19.
The top level visualizes the complete data set, with cluster and subclusters of
data points visualized at deeper levels. In a “parent” plot, the user can select
interactively the number of “children” and their approximate center locations. In
this way, the visualization focuses on particular aspects of the data.
A problem with switching linear dynamical systems is that exact inference (computation of the probabilities of switches and visualizations given all observations) is intractable, i.e. it scales exponentially with the length of the time
sequence. We have developed a new method for approximate inference, which
scales linearly with the length of the sequence and can be generalized to inference in general dynamic Bayesian networks.
Case study with Sappi Nijmegen
In collaboration with Joost Dercksen and Tim Mulder (Sappi) and Dario Lo Cascio
(TNO), we have been involved in a case study for the paper mill Sappi Nijmegen.
Figure 18:
Graphical structure corresponding to the third level of the hierarchical
switching Kalman filter. Square nodes refer to switches (open:
corresponding to subplots at the third level; shaded: subplots at the
second level), elliptical nodes to continuous variables (open: latent
variables corresponding to the location on each subplot; shaded:
observations, e.g. sensor readings).
The goal of this study was to search for relationships between pulp and process
parameters and off-line paper quality parameters such as internal bond. Using
self-organizing maps for data mining, we found interesting relationships, some
of them novel to the process engineers.
The quality of the data appears to be insufficient (many missing variables) to
apply more quantitative supervised models. This will be a topic for further
research.
Decision support systems for medical diagnosis
Wim Wiegerinck, Bert Kappen, Marcel Nijman, Ender Akay, SNN Nijmegen
Jan Neijt, Eelco de Koning, UMC Utrecht
Funding: STW Companies: Hiscom, Pfizer, Rand Corporation
Computer-based diagnostic decision support systems (DSS) will play an
increasingly important role in health care. Due to the inherent probabilistic
nature of medical diagnosis, a DSS should be preferably based on a probabilistic
28
Paper
Paper
Paper
Paper
type
type
type
type
1
2
3
4
Figure 19:
Ten hours of production data from a paper mill projected using a hierarchy of switching linear dynamical
systems. The labels encode different paper types.
Bars below the subplots visualize the probabilities
of the switches as a function of time. Note that the
algorithm does not use the labels themselves.
model. In particular Bayesian networks provide a powerful and conceptually
transparent formalism for probabilistic modeling. A system modeled with a
Bayesian network can typically generate a differential diagnosis based on the
patient findings, a suggestion of additional tests that are expected to be most
relevant to refine the diagnosis, and some insight in the disease mechanisms of
the patient.
In this joint project with the UMC Utrecht, we aim to build a large and detailed
Bayesian network for diagnostic support in internal medicine.
The practical application of Bayesian networks in large, complex domains such
as internal medicine requires that several major obstructions are taken. One
obstruction is that complex probabilistic models are intractable for exact
computation (i.e. exact computation will cost too much CPU time). For this
reason, we need fast and accurate approximate inference methods. Recently,
cluster variation techniques gained much interest in this context. We developed
methods to improve convergence of this approximation technique. Simulation
results on large networks show that cluster variation can be considerably more
accurate than other state-of-the-art methods such as the mean field
approximation (Kappen and Wiegerinck, 2002). Another obstruction is that in
internal medicine, patient-data is insufficient to train the model. In such a case
it is common practice that model parameters are set by hand by human domain
experts. Often, however, human domain experts are not able to express their
knowledge in such a way that it can directly be used to set the model parameters.
To overcome this problem we have developed methods that can use available
expert knowledge to train the model (Wiegerinck and Heskes, 2001a; Wiegerinck
and Heskes, 2001b).
Furthermore, we develop software tools for modeling and inferencing large
Bayesian networks. One of these tools is BayesBuilder, which is software for
construction of Bayesian networks and for efficient computation. A public version
of BayesBuilder can be downloaded from:
http://www. mbfys.kun.nl/snn/Research/bayesbuilder/.
29
Hidden variable models for data mining
Machiel Westerdijk, Wim Wiegerinck, Stan Gielen
Funding: STW
Hidden variable models are well suited to describe complicated data distributions
in a transparent and compact manner. Therefore they provide an important tool
for data mining. In data mining, however, one is often also interested in
classification. However, usually hidden variable models are optimized by
maximizing the joint (input-output) data likelihood. Therefore, they are suboptimal
for classification tasks. On the other hand, models that are directly optimized
for classification, e.g. support vector machines and multi-layer perceptrons,
often do not provide insight in the underlying joint data distribution.
In this project we aim to combine the advantages of hidden variable models and
models that are optimized for classification. We developed methods to increase
classification performance of hidden variable models, while keeping the likelihood
as large as possible. In addition, we have developed methods to induce
classification rules. These are if-then-else rules that explain to a user why
classifications are reached in particular cases. The methods were tested on a
generative vector quantizer and on a principal component analysis model on a
number of real world data sets. Compared to maximum likelihood models, the
results showed a significant increase in classification performances while the
data likelihood remained close to that of their maximum likelihood counterpart.
Furthermore, induced classification rules are found to be short and accurate
(Westerdijk et al., 2000; Westerdijk and Wiegerinck, 2001; Westerdijk et al.,
2001; Westerdijk, 2001).
Genetic linkage analysis
Martijn Leisink, Bert Kappen, Stan Gielen. Han Brunner, Antropogenetica KUN
Funding: STW Pionier
Localization of genes involved in a genetic disease is a promising real world
application for several techniques discussed above. Currently we are developing
a general scheme to find multilocus diseases in third party databases. In short,
the procedure is as follows: The DNA of the individuals in several pedigrees is
sampled at about 200 known locations. Furthermore the status (affected or
unaffected) is known for each individual. Using a Hidden Markov Model we
reconstruct the mode of inheritance based on the available data (see figure 20).
Grandfather
Grandmother
Grandfather
Grandmother
A B
C D
B E
B C
0
Selector bit
1
Phase bit
A
1
Selector bit
0
Selector bit
Father
1
0
Phase bit
B
0
Selector bit
Child
A B
B
B B
Phase bit
A
0
Selector bit
Mother
0
D A
Selector bit
30
D
B
Figure 20:
All parents-child relations are similarly modeled. This
results in a graphical model for the whole available
pedigree. Each child has two selector bits, which indicate
whether the paternal or the maternal chromosome is
inherited, and one phase bit, which holds the order of
the alleles.
Once we have this information, a Bayesian model gives us the likelihood that a
specific location (or gene) is ‘linked’ to the disease. An extension to multiple
genes is straightforward. The first experiments show that for single locus
diseases our approach yields better results than the standard techniques in
this field.
Quantization of temporal patterns by neural networks
Dr. H.J. Kappen, Ir. T.A. Cemgil, Biophysics, University of Nijmegen.
Dr. P. Desain, Dr. H.J. Honing, Ing. P. Trilsbeek, NICI, University of Nijmegen.
J. de Haan, D. Weiermans, SoundPalette, Utrecht. H. Timmermans, Hogeschool voor de
Kunsten, Utrecht. Leigh Smith, Tomandandy, New York. R. Boulanger, Berklee College of
Music, Boston. M. Dunn, Coda Music. G. Lengeling, Emagic
Funding: STW
Automatic music transcription refers to a computer generation of a high level
description of musical performance, for example in the form of western music
notation. To generate a meaningful transcription, one has to model timing
deviations from a strict mechanical template (i.e. music notation) (Desain et
al., 1999). We can identify deviations roughly in two time scales: in the short
scale due to expressive timing deviations and in the long scale due to tempo
fluctuations. A robust transcription system has to track tempo fluctuations and
filter out the expressive timing deviations to produce an acceptable score. The
former task is tempo tracking and the latter is rhythm quantization.
We formulate tempo tracking as a hidden state estimation problem in a linear
dynamical model. The hidden states of the model represent the beats of the
tempo track. In absence of data, this model describes likely tempo fluctuations
(Cemgil et al., 2000b; Cemgil et al., 2000c).
We use a quantizer as a measurement model. Quantization plays an important
role in identification of the correct tempo interpretation by defining it as the one
that results in simpler quantization. We use the quantization model to infer the
likelihood of a beat given an onset sequence (Cemgil et al., 1999; Cemgil et al.,
2000a; Cemgil and Kappen, 2001c).
We have also extended our model where model parameters are adapted online
and in real-time using an EM algorithm (Cemgil and Kappen, 2001b). The model
is implemented in real-time and demonstrated in (Cemgil and Kappen, 2001a).
Tempo
Tempo
>
Score
Figure 21:
When a musical score is performed, expressive
timing and tempo changes cause deviations
from exact timing implied by the score notation.
The figure displays a graphical model of such
deviations. When a performance is observed,
the most likely score and tempo are estimated
by application of the Bayes rule.
>
Score
Perf
Perf
31
32
2C
Algorithms and Program Methodology, Leiden University
Prof.dr. J.N. Kok, Prof.dr. G. Rozenberg, Prof.dr. H. Spaink, Prof.dr. A. Ijzerman, Dr.
Th. Bäck, Dr. C. Schmidt, Dr. M.W. Beukers, Dr. W. Kosters, Drs. E.-W. Lameijer
Description of the Group
The research within the section Algorithms and Program methodology (ALP) is
concentrated around the topics Bioinformatics, Coordination, Optimization and
Data Mining.
•
Bioinformatics. The application of computer science technology for solving
problems in biochemistry and genomics. The expertise in molecular biology,
medicinal chemistry and computer science with particular aspects on data
mining and adaptive search and optimization comes together in a way that
allows for the exploitation of interdisciplinary synergies between these disciplines.
• Coordination. Software systems are difficult to maintain due to their
inherent complexity. Coordination of components in Software Systems studies
how complex systems can be constructed from components using a clear
distinction between individual components and their coordinated interaction.
•
Optimization. Many problems can be described as an optimization problem
in which, within certain constraints, optimal values have to be found for the
variables. The techniques that are used are stemming from Simulated Evolution,
in particular from Evolutionary Computation. Research is being done on the
development of new algorithms, on the applicability on benchmark problems
and on concrete “real-life” cases.
•
Data Mining. Data Mining looks for hidden patterns in large amounts of
data. Within ALP the focus is on association rules and on techniques from the
area of Natural Computation.
Below we report on all topics, except for the Coordination topic. More information
on the group can be found at: http://www.wi.leidenuniv.nl/CS/ALP
Long-term Research Projects
Evolutionary Computation
Today, evolutionary computation is one of the key technologies for implementing
adaptive and self-adaptive algorithms useful for a variety of tasks in fields such
as optimization, business intelligence, and machine learning.
Evolutionary algorithms, encompassing methods such as genetic algorithms
and genetic programming as well as evolution strategies, are gleaned from the
model of organic evolution. They adapt to an unknown environment and search
for optimal solutions by means of an evolutionary process, improving a set of
candidate solutions by selection and by variation operators such as mutation
and recombination. The state-of-the-art in evolutionary computation is now
33
characterized by a mature understanding of the basic working principles of
these algorithms and a lack of an in-depth understanding of many of the more
specific features of their functioning. Concerning practical applications,
evolutionary algorithms have proven their capabilities and their economic relevance
by many successful applications to industrial problems. Our research focuses
on practical as well as theoretical aspects of evolutionary algorithms, including
among others
their behavior in dynamic environments, with a particular interest in exploiting
the self-adaptive capabilities of evolution strategies and genetic programming
for such kind of problems,
their convergence velocity analysis in case of dynamic/bimodal test problems,
their efficiency in comparison to classical optimization methods,
the deeper understanding of variation operators (mutation, multi-parent recom
bination), and
the general principles of adaptable and self-adaptable software.
Concerning their applications, current topics of research include the development
of an evolutionary DNA-computing approach implementing an evolutionary
algorithm in vitro, with the goal of exploiting massive population sizes, the
exploitation of evolutionary algorithms and data mining techniques in bioinformatics applications, and the application to technical design problems occurring
in industry.
Although it is a common assumption that a problem and its quality measure
never change during the run of an optimization algorithm, this restriction is too
simplistic in many real-world situations. As an important aspect of research,
the consequences on evolutionary optimization when the problem is allowed to
change over time are studied, in contexts such as control problems as well as
adaptive and self-adaptive properties of software.
In some cases, every problem dealing with changing situations can be considered
as a new optimization problem, which has to be solved independently of previous
and future problems. Research can then be concentrated on the exploitation of
previous solutions to speed-up finding new solutions.
More complex situations arise when solutions and problems interact with each
other. This is typically the case with dynamic scheduling problems, such as
dynamic job shop scheduling problems. Here, new jobs can arrive which have
not been taken into account when the optimization started. If some old jobs are
already being processed, the set of new schedules is limited. A schedule should
therefore not only be good, but also sufficiently robust to allow for efficient
solutions of yet unknown situations, which are likely to arise later on. The target
is to obtain a better overall solution than can be obtained by local optimization
only.
The application of robust optimization techniques is investigated both from a
theoretical point of view and from a practical point of view. In cooperation with
the Dutch Air Traffic Control, these techniques will be applied to problems
Figure 22:
Abstract sketch of an evolutionary algorithm operating in a dynamic environment.
34
occurring in air traffic control. This year we published on evolutionary art (Hemert
and Jansen, 2001), dynamic environments (Hemert van et al., 2001), constraint
satisfaction (Hemert van, 2001), regression problems (Eggermont and van Hemert,
2001), case-based memory (Eggermont et al., 2001), overview of Evolution Strategies
(Bäck, 2001), theory of genetic algorithms (Bäck et al., 2001a), noisy Evolution
Strategies (Markon et al., 2001), distributed direct optimization (Emmerich et al.,
2001), engineering applications (Giotis et al., 2001), the shape of evolutionary
search (Jelasity, 2001) and genetic programming (Banzhaf et al., 2001a, and 2001b.
Self-Adaptive Software
Self-adaptivity is one of the key discoveries in the field of evolutionary computation, originally invented in the framework of the so-called evolution strategies
in Germany. The concept enables the algorithm to dynamically adapt to the
problem characteristics and even to cope with changing environmental conditions
- as they occur in unforeseeable ways in many real-world business applications.
In evolution strategies, self-adaptivity is generated by means of an evolutionary
search process that operates on the solutions generated by the method, as
well as on the evolution strategy parameters and configuration, i.e. components
of the algorithm itself. This concept therefore enables the algorithm to adapt
itself to varying conditions of its environment, which is a desirable property for
software in general. Therefore, a deeper understanding of the principles
underlying self-adaptation is desirable and will have an influence to fields outside
evolutionary computation as well.
Genetic Programming complements the self-adaptive principles of evolution
strategies from the automatic programming point of view. Genetic Programming
is a label for that set of Evolutionary Algorithms, which, for the purpose of
quality evaluation, represent a potential solution to a decision problem as a
computer program.
The long-term objective of our research is an autonomous Genetic Programming
paradigm, which accepts an arbitrary problem representation and does not
require manual interference. We currently pursue the following theoretical and
applied approaches to this objective. The use of a Turing-complete target
language implies that Genetic Programming is applicable to an arbitrary problem
domain. In particular, Genetic Programming is increasingly being applied to
real-world decision problems, and especially to data mining. A generic key
task in data mining is the identification of problem-relevant decision variables
and their relationships. Often, traditional approaches require manual analysis
and processing prior to the employment of a standard solver. Thus, we focus
on an autonomous identification of the variables and their relationships by coevolving genotype-phenotype mappings with the genotypes such that beneficial
mappings encode and exploit highly relevant variables.
Agent-based strategy design in dynamic environments with incomplete knowledge is a field in its own right with significant application potential to a wide
range of domains such as economics, robotics, and sociology. We focus on
the automatic production of autonomous decision makers which exhibit problemoriented behavior under the control of an individual Machine Learning paradigm,
such as Genetic Programming. Autonomy of Genetic Programming requires
35
the effectiveness of solely implicit principles in order to turn manual interference
obsolete. Natural phenomena imply the emergence of autonomous agents, such
as biological organisms. We focus on the identification of algorithmic metaphors
of such phenomena in order to design a Genetic Program-ming system that
autonomously deals with generic tasks. Ultimately, we focus on a phenomenon
we call autopoietic programming, i.e. the emergence of self-maintaining
algorithms in a noisy working memory which exhibit problem-oriented behavior.
In 2001 we published on self-adaptation (Bäck et al., 2001b), dynamic
optimization (Nijssen, 2001), the evolution of genetic code (Keller and Banzhaf,
2001), and autopoietic solutions (Keller, 2001a).
Data-Mining
Data mining is concerned with the analysis of large amounts of data. Usually
these data are gathered in some automated fashion, for instance in the case of
market basket analysis. In Leiden we have a special interest in the application
of techniques from natural computing, such as neural networks and evolutionary
programming.
In many cases the programs find there own route through the problem space,
but for most applications it turns out to be important to have human guidance
during the process. Human experts are always necessary to raise the questions
- and to interpret the answers.
The main goal of data mining is to extract (possibly unanticipated) information
from the data. Interesting topics are classification, including clustering and
prediction. We are in particular interested in association rules: these rules try
to find statistically significant if-then rules in large datasets. The rules can be
used to understand what is happening, and provide, e.g., the management with
proper information about business processes such as purchase behavior. These
association rules may connect different databases; in this case inductive logic
programming is a good tool to fasten the queries.
We have performed several projects for the industry, including insurance
companies, wholesale departments, supermarkets and food producers. Briefly
speaking, such a project starts from existing (but sometimes unused as yet)
databases, where several of the above mentioned techniques are used to provide
the information needed. In the case of association rules, the result may be an
ordered list of if-then rules that are present, combined with information about
statistical significance, and coupled with data about pricing, season and so
on. On the one hand, known rules should be present (their occurrence is a
proof of principle for the method), on the other hand new rules should appear otherwise the method would not be necessary.
In 2001 we published on knowledge discovery and evolutionary computation
(Eggermont, 2001), evolutionary algorithms in data-mining (Bäck and Schütz,
2001; Keller, 2001b), association rules for multiple relations (Nijssen and Kok,
2001), fuzzy association rules (Graaf de et al., 2001), natural data mining
techniques (Kok and Kosters, 2001), intelligent data analysis (Kok, 2001),
mining E-markets (Putten van der and den Uyl, 2001), multidimensional scaling
(Wezel van et al., 2001) and data fusion (Smith et al., 2001).
36
Bioinformatics
Deriving knowledge from large databases and identifying structure in highdimensional spaces are some of the key problems that play a strong role in
bioinformatics, i.e. the application of computer science technology for solving
problems in biochemistry and genomics. At Leiden University, the expertise in
molecular biology, medicinal chemistry and computer science with particular
aspects on data mining and adaptive search and optimization come together
in a way that allows for the exploitation of interdisciplinary synergies between
these disciplines.
At present, research is conducted on projects related to using molecular biology
to implement evolutionary algorithms (Evolutionary DNA Computing), using data
mining techniques for drug design (intelligent data analysis for drug design),
and using optimization methods to support the drug design process (optimization
of drug structures). A brief overview of these activities is given in the next section.
In 2001 we published on Environmental Epidemiology (Lamers, 2001) and DNA
Computing (Schmidt et al., 2001).
Applied Research Projects
Evolutionary DNA Computing
Prof.dr. J.N. Kok, Prof.dr. G. Rozenberg, Prof.dr. H. Spaink, Dr. Th. Bäck, Dr. C. Schmidt,
Drs. C. Henkel, Leiden University
Funding: NWO Project
DNA computing studies the use of nucleic acids (the molecules that store and
transmit information in living cells) for attacking various computational problems.
DNA is especially suitable for dealing with NP complete optimization problems,
which require huge search spaces but relatively simple operations. Since billions
of DNA molecules can act as billions of parallel processors, the potential to
reduce computing time is enormous. Current limitations in the implementation
of DNA computing are scalability and the rather high error-rates of biochemical
operations. In order to solve problems of realistic sizes, laboratory procedures
must become both less error-prone and much faster. The latter is being achieved
by increased automation and miniaturization in molecular biology, including
advanced liquid handling robots, DNA arraying techniques and single molecule
Figure 23:
The evolutionary DNA computing loop illustrated, with emphasis on using amplification and diversification to model variation operator, and a
selection operator such as (µ,)-selection which eliminates a large portion of unfit molecules.
37
detection. Errors are harder to avoid, since many biochemical procedures are
intrinsically ‘noisy’.
At Leiden University, we try to exploit this fuzziness by implementation of
evolutionary algorithms in DNA. Pools of potential solutions can be generated
by mutation and recombination, and screened for ‘fitter’ molecules. Iteration of
this process will provide DNA computing with enormous search spaces, yielding
a potentially powerful application. In addition, the similarity of evolutionary DNA
computing to natural molecular evolution may inspire new abstractions to be
used in evolutionary computation in general, as well as computationally inspired
biotechnologies. Our current research focuses on both the adaptation of
molecular biological and biophysical techniques and the development of
algorithms.
More information at www.lcnc.nl and wwwimp.leidenuniv.nl/~henkel.
Intelligent Data Analysis for Drug Design
Prof.dr. A. IJzerman, Prof.dr. J.N. Kok, Dr. Th. Bäck, Dr. M.W. Beukers, Dr. W. Kosters,
Leiden University. Prof.dr. G. Vriend, Dr. E. Bettler, Nijmegen University
Funding: NWO-BMI Project
Data mining and evolutionary computation have a strong potential in analyzing
molecular biology databases and supporting the drug design process in many
ways. In cooperation with the Leiden / Amsterdam Center for Drug Research
(LACDR) and the Center for Molecular Bioinformatics (CMBI) in Nijmegen, the
ALP group is involved in a bioinformatics project focusing on so-called G proteincoupled receptors (GPCRs), as they are the major target of the majority of
today’s medicines. These receptors are membrane-bound proteins that serve
as anchor points for hormones and neurotransmitters, such as the classical
biogenic amines (nor)adrenaline, dopamine and serotonin. It is estimated that
the human genome consists of at least 1000 different receptors, far more than
the 200-300 presently known. This poses formidable questions and challenges,
e.g.: What are the endogenous ligands for these orphan receptors, what is
their function, are these validated targets for novel drugs etc.?
Within this project, data mining and evolutionary computation techniques,
developed for intelligent data analysis and efficient search processes in vast
Figure 24:
Left: Charge distribution on VdW surface of CGS15943 (red zone: negative; blue zone: positive); middle: same as previous, but
now as ‘fingerprint’; right: new derivative with good receptor affinity. The transition from middle to right should generate many
structures in an “automated” way by means of an evolutionary optimization rather than the trial-and-error approach used so far.
38
search spaces - such as biological sequence spaces - are introduced to the
fundamental problems in determining function and used to finally design novel
chemical entities that may be the lead structures for new drugs. In particular,
the work aims at using these methods for directly linking sequence with function,
thereby avoiding the diffcult structural element mentioned above.
Optimization of Drug Structures
Prof.dr. A. Ijzerman, Prof.dr. J.N. Kok, Dr. Th. Bäck, Drs. E.-W. Lameijer, Leiden University
In addition to the data mining elements, the aspect of search and optimization
in chemical structure space is also relevant within the framework of drug design
related to GPCRs as outlined in the previous project. Direct (i.e. with known
protein structures) and indirect molecular modeling approaches (i.e. in the
absence of a protein structure, and based solely on the structures of compounds
binding to such proteins) have been developed over the years.
Today the indirect approach is most feasible for GPCRs in view of the absence
of experimentally determined atomic coordinates for any member of this protein
family. However, it is anticipated that rather soon the first realistic structures
will be available given the rumors that abound. Genetic and evolutionary
algorithms are powerful search methods gleaned from the model of organic
evolution that can be used for such tasks. The basic idea is to use such an
evolutionary algorithm to optimize molecules composed of components taken
from a structure database so as to fit a given target shape as precisely as
possible, i.e. the algorithm creates a chemical structure that matches the target
so as to yield a certain desired functionality.
In the absence of an experimentally determined receptor protein structure, the
target in this case can be a so called supermolecule or receptor fingerprint,
composed of the characteristics of one (see figure 25) for more ligands known
to bind to a given receptor, in fact constituting a perfect example of the indirect
approach. The mapping (not the generation) of a chemical structure to the shape
and electronic characteristics of such a target can be performed by means of
existing software tools (CORINA, SPARTAN, Sybyl, INSIGHT/Discover, etc most of them available at Leiden University), while the generation (i.e.
optimization) of the structure is done by the evolutionary algorithm.
Figure 25:
Examples of applications of evolution strategies: Traffic light control optimization (left), elevator control optimization (middle), metal
stamping process optimization in automobile industry (right).
39
40
2D
Pattern Recognition Group, Delft Technical University
Dr.ir. R.P.W. Duin, Dr. A. Ypma, Dr.ir. D. de Ridder, Dr. D.M.J. Tax, Dr. M. Skurichina, Drs.
E. Pekalska, Ing. P. Paclik, Ir. P. Juszczak
Description of the group
The topic of statistical pattern recognition has been studied for more than 30 years
within the Delft Pattern Recognition Group. Since 1988 this research has been
focused on an understanding of neural network classifiers and a comparison with
traditional classification techniques. Recently, methods for novelty detection and
combining classifiers have been studied. In addition the question of how real world
objects can be optimally represented for recognition purposes receives more and
more attention. As the research is focused on sensor based applications observing natural scenes it is called Sensory Pattern Analysis (Duin, 2001; Jain et al.,
2001; Kuncheva et al., 2001; Loog et al., 2001). More information on the group can
be found at: http://www.ph.tn.tudelft.nl/Reesearch/neural/index.html
Research projects
Machine diagnostics by neural networks
Dr. A. Ypma, Dr.ir. R.P.W. Duin, Delft University of Technology.
Partners: Landustrie (Sneek), TechnoFysica (Barendrecht)
Funding: STW
This project was finished in 2001. Some of the results have been implemented in a
commercially available software system for machine health monitoring with selforganizing maps, called MONISOM.
MONISOM consists of tools for reading and processing datasets, (re)training a
SOM (a neural network realizing a Self Organizing Map) on a dataset, analyzing
the trained map and evaluation of a new dataset onto a trained map. Typical analyses
include: map cluster analysis, novelty analysis of a new dataset compared to a
known dataset, novelty and cluster analysis per feature, setting of thresholds,
inspecting regions of the map where a new dataset has many ‘hits’ and generation
of a trajectory plot over the map along with monitoring of the novelty through time.
The user group is investigating the use of MONISOM in practical machine monitoring (on-line pump monitoring, off-line analysis of ship engine vibration, monitoring
of rotating equipment in paper industry). Moreover, medical monitoring problems
(depth of anesthesia during surgery, detection of eyeblink rate in Tourette’s syndrome
patients, analysis of EEG/MEG signals for detection of epilepsy or Alzheimers
disease) may be approached with MONISOM.MONISOM has received the 2001
SKBS Award for Best demo/application at the 13th conference on AI in Belgium
and The Netherlands. More information can be obtained at: http://www.mbfys.kun.nl/
~ypma/project/monisom/index.html, http://www.mbfys.kun.nl/~ypma/papers/
list_of_papers.html, and in (Ypma, 2001b; Ypma et al., 2001a; Ypma et al., 2001b;
Ypma et al., 2001e). Further information about commercial use of MONISOM can
be obtained from J. Valk, Landustrie Sneek b.v. P.O.Box 199, 8600 AD Sneek, The
Netherlands, Phone: +31 515 486888 E-mail: [email protected].
41
Nonlinear-feature extraction from image data for classification tasks
Dr.ir D. de Ridder, Dr.ir. R.P.W. Duin, Delft University of Technology
Contacts: Prof.dr. J. Kittler, University of Surrey, UK
Funding: NWO
Feature extraction in pattern recognition is related to three goals: a better
understanding of the objects to be classified, more accurate classifiers and cheaper
classifiers. This topic has been extensively studied in the past for feature selection
and for linear feature extraction. Nonlinear feature extraction, however, has hardly
been studied until now due to the lack of a well defined restriction of the infinite
universe of possible nonlinearities and due to insufficient tools. This project aims
to investigate nonlinear feature extraction capabilities of adaptive methods, such
as neural networks. The project was ended in 2001 with a thesis. It focused on
three main questions:
1. Can adaptive methods (among others neural networks) be trained to solve image
processing problems?
2. How can one use prior knowledge about the problem as well as possible in
designing and training neural networks?
3. To what extent can one learn more about problems by studying neural networks
trained to solve them? Three techniques were applied: supervised feed-forward
networks for both object recognition (classification), image filtering (regression),
and unsupervised clustering methods for segmentation and object recognition.
The main conclusion is that neural networks are mostly applicable to complex
problems which cannot easily be split into sub-problems, but for which a good
performance criterion can be given. Inspection of neural networks after training is
difficult, as the solution found is hard to express in terms of image processing
primitives. If sufficient prior knowledge of the problem is present, it is pre-ferable to
specify a simpler adaptive model, as besides good performance this will give
solutions, which provide more insight (Ridder de, 2001; Ridder de et al., 2001;
Musa et al., 2001a; Musa et al., 2001b).
Dissimilarity-based Pattern Recognition
Drs. E. Pekalska, Dr.ir. R.P.W. Duin, Delft University of Technology
Funding: NWO
For learning purposes, objects are usually represented by features. Defining welldiscriminating features for a given task is not trivial, and sometimes not even
µ − 0.5 λ1 e 1
µ
µ + 0.5 λ1 e 1
e1
P1
P2
P3
P4
P5
P6
P7
P8
P9
42
Figure 26:
This illustrates a set of 9 subspaces found for the class of the handwritten digit 5. In
each row the mean of the subspace is shown and the deviations in the direction of
the first basis vector, shown on the right.
possible. An alternative is a representation built by using the concept of dissimilarity
(distance). An object is then characterized in a relative way, i.e. by its dissimilarities
to a set of prototypes. The use of dissimilarities is especially appealing when
features can hardly be defined in a natural manner, e.g. when some particular
characteristics of objects or measurements, like curves or shapes, are considered.
This approach not only offers a good alternative for the traditional feature base
approach. It may asymptotically have a very good performance: under rather general conditions the classification error for dissimilarity-based classifiers goes to
zero while in feature based approaches an intrinsic class overlap exists that cannot
be removed by more training examples, see figure 27 (Pekalska and Duin, 2001a;
Pekalska et al., 2001; Duin and Pekalska, 2001; Pekalska and Duin, 2001c;
Pekalska and Duin, 2001b).
Combining classifiers
Drs. M. Skurichina, Dr.ir. R.P.W.Duin
When a pattern recognition problem is complex and the amount of training data to
learn a classifier is limited, it may be difficult to construct a good single classification
rule to solve the problem. A well-known approach that allows us to improve the
performance of a weak (bad performing) classifier is to use the combined decision
of an ensemble of weak classifiers instead of a single decision of one weak classifier.
It is known that the ensembles often outperform single classifiers. In classifier
combination, it is believed that diverse ensembles have a better potential for
improvement on the accuracy than non-diverse ensembles. We put this hypothesis
to test for two methods for building the ensembles: Bagging and Boosting, renowned
for their success, as reported in numerous experimental studies. Our study showed
a vague support for the hypothesis of the beneficial diversity for Boosting and
virtually no support for Bagging. However, the performance of ensembles in bagging
and boosting is affected by the choice of the classification rule, by the training
sample size used to train classifiers and by the data distribution. Therefore, these
factors should also be involved in future studies in order to test the hypothesis
properly (Skurichina and Duin, 2001; Skurichina, 2001).
One-class classification
Dr. D.M.J. Tax, Ir. P. Juszczak, Dr.ir. R.P.W. Duin
Funding: NWO
This new project is inspired on the project ‘Confidence levels in neural networks’,
where the problem of one-class classification was posed. In that research project,
A
B
Dissimilarity matrix X?
D
Training set
Is there a feature space X
x2
for which Dis
Rk
Euclidean distances
D
x1
Figure 27:
A possible approach to dissimilarity based pattern recognition:
embedding the dissimilarities in a hypothetical feature space.
43
three types of one-class classifiers were distinguished, (1) the density estimators,
(2) the boundary estimators and (3) the reconstruction methods. A very important
issue that should be considered, is the representativity of the training set. When
the data is represented well, in the sense that the distribution of the training set is
very comparable to the distribution which will occur in real applications, the density
methods will perform well (also assuming reasonable sample sizes). In the case of
an unknown outlier distribution, the density methods completely fail and the
boundary methods are to be preferred. Finally, when the generating model behind
the data is known (or can be approximated with reasonable precision), it is usually
better to use this model and apply a reconstruction model to detect outliers.
These one-class classifiers were applied to several artificial and real world problems,
like machine diagnostics, handwritten digit recognition and image database retrieval.
In all cases satisfactory performances were achieved, but it appears that the results
depended on the chosen representation of the objects to be classified. The oneclass classifiers suffer significantly more from a poor representation, which ignores
important characteristics of the data, than a normal classifier. Poor representation
will result immediately in a poor distinction between target and outlier objects,
while in normal classification it might be still improved by feature selection or
feature extraction.
One of the major tasks in this research is therefore to find automatic procedures to
perform feature selection in the one-class classification problem. When only
example objects of the target class are present, it is not clear which features are
useful and which are useless. When some example objects from the outlier class
are present, it may be possible to apply the classical feature selection tools. It
should be investigated if these well-known selection tools suffice, and what the
best methods are for obtaining outlier objects (Tax, 2001; Tax and Duin, 2001b;
Tax and Duin, 2001a; Tax and Duin, 2001c).
Tools for nonlinear data analysis
Drs. J.J. Verbeek, Dr.ir. B.J.A. Kröse, University of Amsterdam
Dr.ir. D. de Ridder, Dr.ir. R.P.W. Duin, Delft University of Technology
Partners: Noldus BV, Shell EPT/RF, NATO, TNO-FEL, KIQ, Unilever Research
Funding: STW
Current computerized measurement systems and data acquisition systems deliver
a huge amount of data. Because several sensors are often measuring the same
physical phenomenon, the intrinsic dimensionality of the data will in many cases
be lower than the dimensionality of the data itself and only depend on the degrees
of freedom of the observed phenomenon. If the dimensionality of the measurement
space is not reduced correspondingly by some mapping, the outcome of any
analysis of the measurements may suffer from an increased noise resulting from
more sensor signals, instead of taking advantage of the increased information or
resolution. Feature extraction and feature reduction thereby become more and
more important in relation with increasing sensor capabilities. However, standard
analysis packages are often limited to linear projections, while the data not
necessarily reside on a linear manifold.
Recently, a number of novels promising techniques for nonlinear projections have
been developed. These techniques show to be useful if the number of datapoints is
larger than the dimensionality of the data, but may perform sub-optimally if the
44
number of datapoints is small. In this project we focus on situations in which the
number of datapoints is in the order of the dimensionality of the data. Depending on
the application (visualization, compression or classification) we will define criteria
to assess the performance. Novel methods will be tested on these criteria and on
speed.
Beside several theoretical studies, we worked on a real-life data set supplied by
Noldus BV. This set contains movies of observed animal behavior and sets of
derived features. In the first instance, prior work on this set has been successfully
repeated. Application of mixture-of-subspace models was not yet successful.
Figure 28:
Some profiles used
for the study on
automatic analysis
of animal behavior.
Multi-spectral image segmentation
Ing. P. Paclik, Dr.ir. R.P.W. Duin
Partner: Dr.ir. G.M.P. van Kempen, Dr. R. Kohlus, URV, Vlaardingen
Backscatter images (BSE) acquired by scanning electron microscopy (SEM) play
an important role in structural analysis of laundry detergents. Segmentation of
BSE images is an essential step in the analysis of powder properties. We have
developed a supervised algorithm for the segmentation and we proposed a novel
way to shorten feature computation time by feature selection. Currently, the product
analyst segments these images interactively, which is both time consuming and
an inaccurate operation. Moreover, the results delivered by different experts are
considerably varying. One of the reasons is that the analyst bases her decisions
on the single-band BSE that lacks information about underlying chemical
composition. The segmentation algorithm may be split into two parts. In the first
one, a classifier is trained using multi-band images. In the second, new backscatter
images are labeled by the trained classifier (Paclik et al., 2001a; Paclik et al.,
2001b). The segmentation process is shown in figure 29.
training backscatter classifier:
backscatter image
aligning
images
labeling
backscatter
image
computing
features
dataset with
all features
multi−spectral image
aligned images
feature
selection
training
classifier
dataset with
selected features
trained
classifier
labeled backscatter
labels
segmenting new image:
backscatter image
computing
features
labeling
backscatter
image
dataset with
selected features
labeled backscatter
Figure 29:
The procedure for automatic segmentation of backscatter images
trained by multi-spectral electronmircroscope images
45
46
2E
System Technology IWI-Cluster, University of Groningen
Prof.dr.ir. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, Ir. S. Achterop, Dr.ir. R. Moddemeijer,
Drs. M. van Veelen. Associated Researcher: Ir. A.J.W.M. ten Berg (Philips), Drs. M.H. ter
Brugge, Ir. W.J. Jansen (Dacolian), Drs. M. Diepenhorst (vanVliet), Ir. B.J. van der
Zwaag (University Twente), Dr. R.S. Venema (Boise State University), Ir. F.W. Greuter
(Rohill)
Description of the group
The System Technology Cluster of the RuG Institute for Mathematics and
Computing Science (IWI) is engaged in research on the systematic design of
computational intelligent units and their integration in large, distributed systems.
The Cluster is part of the KNAW Institute for Programming and Algorithmic
Research (IPA) and co-operates with the Chair in Signals & Systems of the
University of Twente. More information on the group can be found at:
http//www.cs.rug.nl/Research/St/
Aims and scope
Computational Intelligence (CI) has rapidly become an accepted technology
within a broadening area of applications. From the many different CI disciplines
we will primarily target on a neural executionable model to fuse a more
heterogeneous set of knowledge. Such a model will be a multi-net, abundantly
present in nature and so far little used in artificial neural systems. The popular
monolithic neural networks have been studied in the past decennium to find the
principles of operations and most importantly the reasons for not operating. It
appeared that an elaborate feature identification was required to facilitate the
neural capability by principle, while further developments in network topology
and learning algorithms were demanded to realize the promised quality. The
more flexible alternative is the multi-net: a deliberate composition of neural
nets and other algorithms, that have the promise to be easier to design, mold
and understand. They have a special relevance in distributed environments,
where the neural parts may easily be implemented as agents in a telematic
meaning.
Research
“Module”, “object” and “agent” are different names for the single purpose of
describing complex problems using concepts of local autonomy. The words
reflect the differences in development of the respective fields of technology.
Neural networks are at the start of such a development, where we only are
starting to devise more complex systems with neural behavior. We are pursuing
such developments in three related directions. In the past, neural nets have
largely been cast into software, and hardware realizations have been few and
only moderately successful. This is expected to change with the coming of
Polymorph Computing. Character recognition (or the super class of text mining)
is based on the intensive and large-scale application of clustering and
classification techniques.
Especially in mobile environments, neural networks will play a major role. Lastly
we devote our attention to process identification and the early detection of
abnormalities.
47
Neural Networks in Polymorph Computing
Prof.dr.ir. L. Spaanenburg, Ir. S. Achterop
The history of micro-electronic hardware is governed by the quest to find products
that create a new and larger market for the new technology. From the gate
came the adder, followed by the arithmetic unit, the micro-processor, the
computer and now the system-on-a-chip. Steadily the devices became larger
and more programmable. Meanwhile more ways to program are invented,
bridging the gap between hardwired and soft programmable functions. The
recently coined name for creating functions in an arbitrarily programmable
realization is Polymorph Computing, indication that a single function may have
many SW/HW realizations.
Typical polymorph architectures are based on the assembly of computer cores,
memory fields and field-programmable parts. Functions can be executed in
software on the cores, soft wired in the Field-Programmable Gate Arrays
(FPGAs) or stored in the memory. This gives the flexibility of a general-purpose
device and the speed of a dedicated one. Personalization is facilitated during
fabrication, during product definition and during application.
In other words, a neural network can be soft wired when needed and still adapted
later in the product life cycle. Polymorph devices are not yet widely available.
But existing parts mimic such capability already on board-level or within microcontrollers. This provides for a testing ground, preparing for the future. Notably
we aim for the use of modular structures, also in neural networks.
Structured design of neural networks uses the concepts of modularity and
hierarchy. Modularity involves the partitioning of the net into independent cooperating parts; hierarchy shows how an abstract building block can be created
as a net of neurons each with a lower-level function. It has been shown that
hierarchy supports the piecewise linear composition, thereby removing the need
for non-linear basic elements, and therefore eases the realization. Together
with earlier results on minimum word width requirements of digital realizations
this allows to create neural nets on a small footprint. Sizable neural nets can
already be created on 8-bit, 2k micro-controllers in a networked environment,
as forecasted for Home Automation (the Virtual Network). Special attention
has been given to the extension of learning rules for handling modular structures.
(Spaanenburg, 2001e; Spaanenburg et al., 2001b; Spaanenburg, 2001b;
Spaanenburg et al., 2001a; Berg and Spaanenburg, 2001; Achterop et al., 2001;
Spaanenburg and ten Berg, 2001; Veelen van et al., 2001b; Spaanenburg and
Stevens, 2001; Barakova and Spaanenburg, 2001; Bijma et al., 2001;
Spaanenburg, 2001a).
Industrial contacts
Polymorph computing is part of a co-operation with ASTRON (Dwingelo) as
part of the development of the LOFAR radio-telescope. LOFAR is a computing
network, that operates in the GHz range of signal transfer over a 400 km square
area. It requires highly programmable, high-speed nodes while handling terabytes
of data. Such nodes will not only be polymorph, but also need to be highly
autonomous and are therefore susceptible to the type of network catastrophes
that make self-healing by neural intelligence necessary.
48
Understanding printed text
Prof.dr. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, Drs. B.J. van der Zwaag
Most of the non-pictorial information comes to us as printed text. Getting the
information (or even knowledge) out of text is by no means a trivial task. This is
the challenge of text mining, where a range of clustering and classification
methods are applied to create global information on the content of the
communication. A typical example is in the automated maintenance of search
structures on the Net. It has been shown how text mining over the messages
exchanged between users of the Clubs on the KPN Net may reveal the content
need and therefore indicates the proper arrangement of Clubs to optimally
facilitate searches.
Optical Character Reading has developed into a mature technology. Various
high-quality, low price software tools are available and support the scanning of
printed text into a computer file with an acceptable small amount of errors. A
notable area, where the standard OCR is not sufficient, is in license-plate reading.
The amount of pixels in the image is too small to easily separate the characters.
Nevertheless reading such license plates has been achieved for an impressive
98%. This architecture as developed at the Rijks University Groningen uses a
judicious mix of neural networks, template matchers and voters to reach this
figure at a lower than 0.02% false readings.
Figure 30:
Site-test for license plate recognition systems at Woerden
It has been noted that even a small amount of occlusion will drastically reduce
the performance. Occlusion of characters on a license plate can easily occur
when part of the characters is obscured by utilities such as a bicycle rack.
Most of the practical problems are caused by top occlusion, i.e. where the top
of the characters is blocked from vision. Different characters have different
sensitivities for top occlusion; threshold figures range from 8 to 80%. The
essence of the reading problem is that a neural network trained partly with
occluded characters only deteriorates the performance. This is in contrast to
the popular belief that a net learns from its problems. The solution has been to
create a dedicated recognizer for characters with 10 to 35% top occlusion and
49
Figure 31:
The intelligent reading and writing pen of C-Technology
merge that in the existing architecture. Such multiple classifiers have been
widely studied in literature. A basic requirement is it’s “Plug-and-Play”
characteristic. So occlusion is tested when no ideal character has been read
with sufficiently high confidence. If, then again, no occluded character has been
read with sufficient confidence we do not take a vote but strictly decide on an
error. As a result, the overall classification system takes care of both occluded
and non-occluded characters.
Other OCR-like problems occur with the message entry devices that are being
popularized in the mobile environment. With the PDA comes the need for reading
handwriting. Both research results and professional offerings show a very mature
technology. The demands become even more stretched with devices like the
intelligent pen. The use on plain paper seems already advanced; reading from
different material such as blotting paper, curved screens and even wrapping
material seems possible when deformations like occlusion can be taken care
of (Oudshoff et al., 2001; Spaanenburg and Stevens, 2001; Bijma et al., 2001;
Wichers et al., 2001; Zwaag van der, 2001).
Industrial contacts
This research has been performed in combination with Dacolian, KPN Research, and MatchCare.
On novelties and abnormalities
Prof.dr. L. Spaanenburg, Dr.ir. J.A.G. Nijhuis, and Dr. M. van Veelen
In many real-world situations processes feature a time-dependent behavior with
unknown variations in time. Where such processes are part of a larger network,
small local failures may easily lead to large errors. The past decade has shown
a number of such dramatic situations, on the power grid, on the
telecommunications grid and on the transport grid. For instance, in 1996 a
small power dip near the Keeler-Alliston dam put 11 American states and 2
Canadian provinces in the dark within a couple of seconds. The central monitor
of the network came too late to block the spreading of the disaster; later one
concluded that a small local correction would have given ample time to control
the abnormal process behavior.
The classical Fault Diagnosis and Isolation (FDI) procedures do not allow for
the early detection of unknown abnormalities. Firstly, it requires all abnormalities
50
to be modeled in their effect by a dedicated fault model. Where faults are
unknown, such models do not exist. Secondly, faults must have a clear impact
to become distinguishable from other behavior. This makes it hard to label
small aberrations as leading to larger faults in the near future. Early detection
needs a look inside the process. Neural networks, and markedly the modular
ones, help to get a glimpse of the process internals. Training a neural knowledge
model helps to demarcate the input/output space and allows to detect
disturbances without the requirement of a pre-diagnosed fault model.
Performance of this method has outperformed existing methods and leads in a
number of benchmark tests to a high confidence level with no markedly false
warnings.
The ability to find a physically plausible structure within the measurement set
that can be explained through (and leads to) a better understanding of the
process is core to the identification of the process and therefore to the labeling
of distinct measurements as novelties or abnormalities. It is necessary to
document the knowledge within the neural network to validate abnormality
detection. Such may involve the automated extraction of rules in a manner and
order that allows for operator inspection. As rule extraction is a combinatorial
problem, the use of modular nets is called for (Spaanenburg, 2001f; Venema
and Spaanenburg, 2001; Spaanenburg, 2001d; Steen van der et al., 2001; Veelen
van et al., 2001a; Spaanenburg, 2001c).
Industrial contacts
The research on abnormality detection has been spurred by a contract with
ECN. Much of the experimental evidence has come forward on basis of data
from Hoogovens. Further research will be done in collaboration with Twente
University.
51
52
2F
Civil Engineering Informatics Group, Delft University of Technology
Prof.dr.ir. P.van der Veer, Prof.ir. F.Tolman, Dr.Ir. R. Beheshti, Ir. H. Boere, Dr.Ir. J.Cser,
Ir. D. Roozemond, Ir. L. Aarts, Ir. E.Dado, Ir. H. Havinga, Ir. M. Hooimeijer, Drs. L.
Oosterlaan, Ir. S. Öszariyildiz, Ir. S. Özmutlu, Ir. T. Rientjes, Ir. F. Schulze, Ir. T.
Botterhuis, Ir. H. van Lint, Ir. R. Wu, Ir. H. Schevers, Ir. M.Werner, Dipl.Ing. A. Zijderveld
Description of the group
The Civil Engineering Informatics group is engaged in the field of computational
intelligence for civil engineering systems. The research focuses on new
modeling methods, integrated method modeling and contributions to knowledge
of physical phenomena by data analysis. The main themes of research are
Hydro informatics and Building & Construction IT. The neural network research
is concentrated in the Hydro informatics theme and involves an integration of
computer science, hydro sciences and mathematics. It deals with bringing
physical knowledge into data oriented methods, and with integrating data
oriented methods with classical mathematical models. More information on
the group can be found at: http//www.cti.ct.tudelft.nl/
Research projects
Neural network method for solving partial differential equations
Ir. L.P. Aarts, Civil Engineering Informatics, Delft University of Technology
Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology
Users: Rijkswaterstaat, TNO-NITG
Funding: TU Delft
This project concerns the problem of the ‘black box’ nature of some neural
network models. In many situations this is a disadvantage besides the great
advantages like the ability to cope with strongly complex non-linear behavior of
systems, the adaptive properties, the performance as fast simulators and the
integrateability with various model systems. This project aims to introduce
physical knowledge into neural network modeling. The aim is to find neural
network architectures that represent directly partial differential equations. This
may lead to modeling methods based upon new types of neural networks that
can easily be integrated with measurement-data oriented neural networks. The
results about the basic concept are very promising, even in sets of nonlinear
partial differential equations. This research may lead to results with generally
applicable methods for all fields where physical phenomena are described by
partial differential equations. The very promising results led to funding for a
PhD-position.
Neural network method for solving potential flow
Prof.dr.ir. P. van der Veer, Civil Engineering Informatics, Delft University of Technology
Users: Rijkswaterstaat, TNO-NITG
Funding: TU Delft
This project concerns a neural network method for solving potential flow
problems, described by second order differential equations, known as Laplace
53
Equations. The method uses complex function theory, especially analytical
functions to describe basis components of the solutions. Problems of cyclic
behavior and many-valuedness of the basic components have to be coped with.
This method can be seen as a specific case of the generic method as mentioned
in the previously described project. The main advantage of this approach is that
the neural network needs only to be trained on boundary conditions, not on a
grid. In this sense the method has similarities with the family of boundary element
methods and integral equation methods.
Neural network methods for the simulation a complex water system.
Ir. A. Zijderveld, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil Engineering Informatics,
Delft University of Technology. Ir. P.Heinen, Rijkswaterstaat, Ministery of Public Works
Users: Rijkswaterstaat, TNO-NITG
Funding: Rijkswaterstaat
Research on neural network models for the prediction of water levels along the
coast and in the main Dutch rivers resulted in an integrated approach with
classical numerical methods and expert system technology. This combines
data analysis of measurement data, analysis and classification of storm
situations, physical knowledge representation in numerical models and human
knowledge and experience representation. Results are scientifically interesting,
as this integrated method modeling is able to cope with extreme values in water
level predictions much better than classical methods. For practical applications,
the prediction accuracy of extreme water level values is important as it is directly
related to human safety. Rijkswaterstaat (Ministry of Public Works) has
implemented the results of the project. A dissertation on this subject will be
finalized in 2002.
The use of neural networks in simulation of complex morphological systems
Ir. L.Oosterlaan, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil Engineering Informatics.
Dr. J. van de Graaff, Hydraulic Engineering.
Prof.dr.ir. A.W. Heemink, Information Technology and Systems, Delft University of Technology
Users: Rijkswaterstaat
Funding: LWI
The morphodynamics of a coastal system consists of complex, not yet fully
understood, relationships between hydrodynamics (waves and currents),
sediment transport processes, morphology and environmental conditions.
Hydrodynamics induces sediment transport, which is reflected in a change of
morphology. These interactions however, are non-linear and often display a time
lag between the initial hydrodynamic conditions and morphological response.
In addition, in many cases a hydrodynamic threshold value has to be exceeded
before sediment movement starts. Here we deal with a situation where all
research had to be done on the basis of measurement data.
The research is focussed on integration of neural networks with other methods.
An integration of an empirical orthogonal function approach with neural networks
appeared to provide promising results.
54
Integrated Method Modeling in water management systems
Ir. H.N.J.Havinga, Ir. M.G.F.Werner, Dr.ir. J.Cser, Prof.dr.ir. P. van der Veer, Civil
Engineering Informatics, Delft University of Technology
Users: Rijkswaterstaat, WL Delft
Funding: LWI and TU Delft
In many fields of civil engineering practical situations are too complex to be
modeled analytically. Often there is a large gap between measurement data
and the output of a mathematical model. In this research activities method
integration is investigated to achieve models with greater predictive properties.
Integration of neural network technology with fuzzy logic, genetic algorithms is
studied. Combinations with Geo Informatic Systems (GIS) are investigated, in
order to obtain infrastructural decision support systems that can better cope
with uncertain or incomplete data (Aarts and van der Veer, 2001b; Aarts and
van der Veer, 2001a; Aarts and van der Veer, 2001c; Cser and e.a, 2001;
Hooimeijer, 2001; Khu et al., 2000).
55
56
2G
Institute for Knowledge and Agent Technology (IKAT), University
Maastricht
Prof. H.J. van den Herik, Prof. A.J. van Zanten, Prof. L.A.A.M. Coolen, Prof. H. Visser,
Prof. J.M.J. Murre, Dr. E.O. Postma, Dr. J.W.H.M. Uiterwijk, Dr. S. Etalle, Dr. N. Roos, Dr.
I. Sprinkhuizen-Kuyper, Dr. F.J. Wiesman, Dr. P.A. Vogt, Dr. E.N. Smirnov, M.Sc, G.W.
Boers, M.Sc., N.I. Bourdonskaia, M.Sc, H.H.L.M. Donkers, M.Sc., M.F. van Dartel, M.Sc.,
C.A. van Dorp, M.Sc., L. Kocsis, M.Sc., L.J. Kortmann, M.Sc., P.A.M. van der Krogt,
M.Sc., E.H.N. Mathijsen, M.Sc., Y.P. Ran, M.Sc., A. Sprinkhuizen, M.Sc., J.P.G.M. Verbeek
LLM, M.H.M. Winands, M.Sc.
Description of the group
IKAT’s research is organized in five main research groups: agent technology,
neural networks and adaptive behavior, search and games, public services and
knowledge management. Neuro-computational and machine learning methods
are mainly employed in the neural networks and adaptive behavior group, and
to a lesser extent in the agent technology and search and games groups. More
information on the group can be found at: http//www.cs.unimaas.nl
Learning to recognize (painted) images
Recent developments in image classification have focused on efficient preprocessing of visual data to improve the performance of neural networks and
other machine learning algorithms when dealing with content-based classification
tasks. Pre-processing techniques are used to bring out the visual characteristics
relevant for the classification. This project studies modern neural pre-processing
techniques to enhance the performance on image classification tasks. The
performance of the techniques is assessed on the difficult classification task
of recognizing a painter from the contents of a painting. The basic assumption
is that the maker of a painting can be recognized from his/her “hidden signature”,
i.e., brushstroke, spatial and color composition, and texture. Such painterspecific features are to be retained in the pre-processed representation to ensure
successful classification. Domain-specific knowledge provides a rough albeit
indispensable guideline for determining the appropriate type of pre-processing.
Figure 32:
Illustration of the application of
neuro-computational techniques to
the analysis of paintings. Left:
digital reproduction of the painting
Supper at Emmaus by Rembrandt
(1648). Right: brush-stroke representation of the same painting
obtained by applying neuralnetwork techniques. The shades of
white indicate the degree to which
the corresponding parts contain
brush stroke patterns that are
typical for Rembrandt.
57
For this reason we collaborate with the Rembrandt Research Project and the
Amsterdam Rijksmuseum (as part of the NWO funded ToKeN 2000 project). In
2001, special learning algorithms have been developed to mimic the eye
movements of observers. These active-vision algorithms have been successfully
applied to the recognition of faces (Postma et al., 2001a) and will be applied to
select the relevant parts of paintings for further analysis.
Neural robots
The neural robots research deals with the study of autonomous robots that are
controlled by neural networks. More specifically, the research focuses on the
automatic learning of elementary behavioral tasks such as target following
(András et al., 2001; Kortmann et al., 2001) and object pushing (Spronck et al.,
2001a; Spronck et al., 2001b; Spronck et al., 2001c). By studying how robot
models learn to perform these tasks, we hope to gain a deeper understanding
of the way more complex behaviors can be learned in an automatic way.
The task of pushing an object (i.e. a circular box) between two walls is relevant
for robot soccer and underlies many more complex behaviors such as target
following, navigation, and object manipulation. Despite the apparent simplicity
of the task, training a robot to perform the object-pushing task turns out to be
rather difficult. In 2001 our experiments focused on the evolutionary optimization
of the neural networks that control robot behavior. In particular, the effects of
various evolutionary strategies have been explored (Spronck et al., 2001a).
Figure 33:
Top view of pushing behavior for nine different starting positions by an artificially evolved robot. The figures illustrate the nine initial
positions of the robot (bottom circle) and circular object (circle just above the robot) and the final positions (top two circles).
Symbol grounding in multi-agent systems
The symbol grounding problem concerns the question of how semantic
interpretations of symbolic representations can be made intrinsic to the system,
rather than being described in a formal way (e.g. as a set of rules). The project
focuses on the study of systems composed of two or more robots that solve
the symbol grounding problem through interaction with an environment of other
robots and objects. The robots solve the symbol grounding problem by engaging
in a series of so-called language games. In a language game the robots try to
58
name an object by categorizing the sensed object and subsequently assigning
a name to the categorized object. When robots fail in categorizing or naming
an object, they adapt their memory to improve their performance on future
occasions. At the start of each experiment, the robots have no categories or
names at all. During the experiment categories and names emerge from the
linguistic and physical interactions with the environment. The experiments
performed thus far reveal that the robots solve the symbol grounding problem
through the co-evolution of meaning and lexicon. In addition, the automatic
learning of ontology mapping through language games has been studied
(Wiesman et al., 2001)
ToKeN 2000
The ToKeN 2000 project is concerned with two fundamental problems: (1) How
can the accessibility of information and knowledge be improved for the user in
such a way that knowledge acquisition for the user is optimal? So as to make
the information and knowledge available and accessible for large groups of
users, the problem is: (2) How can the retrieved information and knowledge be
upgraded and enriched? These problems give rise to the following research
themes: (1) Control, (2) Navigation, Adaptation, and Learning, (3) Language
Technology, (4) Delivery Techniques, and (5) Knowledge Enrichment.
The research is applied and tested in three domains: Education & Culture,
Police & Law, and Health Care. In the first year of the project an experimental
platform will be built and tested. The Rijksmuseum’s collection database serves
as a testbed. In the second phase of the project the experiment platform serves
as a basis for further research.
ToKeN 2000 is a joint research program of IKAT, IPO (TUE Eindhoven), NICI
(KU Nijmegen), CWI, TU Delft, Universiteit Leiden Department of Psychology.
IKAT provides three parts: (1) The Metabrowser (2) Automatic Image Recognition,
and (3) Intelligent Information Filtering.
Figure 34:
Two robots in their environment of
three objects (black cylinders).
59
60
2H
Evolutionary Systems and Applied Algorithmics; CWI, Amsterdam
Prof.dr.ir. J.A. La Poutré, CWI and Eindhoven University of Technology. Drs. F. Alkemade,
Drs. S.M. Bohté, Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Dr.ir. P.J. ‘t Hoen, Drs. M.B. de
Jong, Dipl. Math. E. Kutschinski, Dr. D.J.A. Somefun, CWI, Amsterdam
Description of the group
The research group focuses on the combination of two components, consisting
of computer science techniques and application fields:
•
The techniques concern intelligent computation: evolutionary and multiagent systems, adaptive algorithms, and neural networks;
•
The application fields concern economics, management, and e-societies
(like for instance e-commerce); important topics are markets and market
mechanisms (economics), negotiations, auctions, and social aspects
(game theory), and optimization and classification (management).
Agent systems: e-business and economics.
The concept of software agents in computer science, as well as the concept of
societies of (human) agents in economics and social sciences, yield important
areas of research. In both cases, adaptive behavior of agents is essential. An
important aspect of adaptivity for an agent is the skill of learning. This is a
growing field of research, important for both agent technology (how to build a
really-learning agent) and economics (how to simulate learning agents).
In order to allow learning in agent systems, machine learning techniques are
necessary. In our research group, we focus on the investigation of evolutionary
systems and neural networks in order to build the internals of learning agents
for e-commerce applications as well as to simulate markets and market
mechanisms in economics and in e-commerce agent systems.
Focus areas are the following:
•
Adaptive strategies for trading, like negotiations, auctions, and dynamic
pricing; this concerns learning behavior, especially for micro-transactions
(a micro-transaction has a small value compared to the total value for
one agent; this allows for learning and experimenting agents) (Bragt van
et al., 2001; Bragt van and La Poutré, 2001a and 2001b).
•
The design and simulation of market mechanisms for e-commerce
applications (Bohté et al., 2001a).
•
Agent-based computational economics (ACE), in particular the simulation
of economic markets; this addresses emergent properties of markets,
seen as complex adaptive systems (CAS) (Alkemade and Poutré, 2001).
Optimization and Classification
Neural networks as well as evolutionary algorithms are particularly investigated
with respect to classification problems and active learning. Special attention is
given to spiking neural networks, which are novel types of neural networks
61
(Bohté et al., 2001b). Application areas are e.g. remote sensing, data mining,
and agent implementations.
Adaptive discrete algorithms (Stee van and Poutré, 2001b; Stee van and Poutré,
2001a) as well as evolutionary algorithms are designed for e.g. decision making
in dynamic environments, like on-line process management and quality of service
in information technology.
Other activities
Based on the above research activities, the research group also participates
(as initiator and coordinator) in the ICES/KIS-III expression of interest “Adaptive
Intelligent Systems for Health Care (I-CARE)”, for designing agent systems
and intelligent systems for e-health applications. More information on the group
can be found at: http://www. cwi.nl
Trade agents
Prof.dr.ir. J.A. La Poutré, Drs. S.M. Bohté, Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Dr.
D.J.A. Somefun, Dr.ir. P.J. ‘t Hoen
The Trade Agents project (“Autonomous Systems of Trade Agents in E-Commerce”) concerns research on systems of trading software agents in electronic
commerce (see http://www.cwi.nl/projects/ASTA). Partners in the project are
CWI, TNO, ING, and KPN. The project is funded by the Telematics Institute.
The major line of the CWI research addresses the development of models and
algorithmic software solutions for new business applications that are based on
interacting software components (agents), that are owned and controlled by
different, autonomous market parties. Specific examples include market
mechanisms, dynamic pricing, negotiation, bidding and buying, and (fast)
profiling in e-business. Our focus is on the evolutionary simulation and
development of trade agent systems; the development of adaptive, learning
trade agents; and the creation of new e-business concepts.
The amount of attention space available for recommending suppliers to consumers
on e-commerce sites is typically limited. Within the Trade Agents project, we
developed a competitive market-based recommendation mechanism (based on
adaptive software agents) for efficiently allocating the “consumer attention
space”, or banners (Bohté et al., 2001a). A patent application was filed (together
with KPN Research) for this system. In our approach, agents get information
about a consumer’s interest. Subsequently, each agent bids in an auction for
Figure 35:
The winning agents display their advertisement to the customer,
where the attention space is sufficiently large to display three
banners. Two lists of stores are shown for 2 types of consumers
(both searching for jeans).
62
the momentary attention of that consumer. Winning agents may display their
advertisement or banner in the available attention space to the consumer.
Successive auctions allow agents to rapidly adapt their bidding strategy to
focus on consumers interested in their offerings. The feasibility of this system,
for variety of customer behavior models, was demonstrated by evolutionary
simulation (as in agent-based computational economics).
A scalable and extensible agent architecture was developed for the abovedescribed system. This architecture supports agents in a distributed bidding
application, where the agents run on dedicated machines for maximum
computational resources. Furthermore, as an extension, the agents can operate
in multiple independent markets concurrently.
bid
1.5
1
0.5
0
x
y
Figure 36:
An example of an evolved bidding strategy for a two-dimensional
consumer profile.
We further studied the problem of applying price discrimination with an online
adaptive algorithm. In particular, we considered the case where the seller of a
product or service applies price discrimination by distinguishing between different
delivery times. For this situation, we developed an efficient and robust algorithm
that dynamically adjusts both the price and the size of the discount for delayed
delivery of the product. The algorithm consists of a (multi-variable) derivative
follower algorithm with an adaptive step-size. We showed that this algorithm
has attractive (convergence) properties when operating in static and dynamic
pro fit landscapes (unlike previously proposed algorithms). Compu-tational
experiments show that the algorithm is able to generate high profit levels in a
dynamic pricing setup with price discrimination.
Consumer
Consumer
Mall
Manager
Agent
Consumer
W eb
Site
Supplier
Agent
Supplier
Supplier
Agent
Supplier
Supplier
Agent
Supplier
Figure 37:
The architecture of the competitive market-based
distribution system for consumer attention space.
63
To explore the potential of active profiling by example in information filtering,
selective sampling for text classification was further investigated. This work
focused on so-called “committee-based” sampling methods. In particular, a
technique to create more robust homogeneous committees was developed,
and a novel approach of using several different classifier types within one
committee for selective sampling was examined. These mixed committees
perform significantly better in terms of learning speed and Classification
performance than any of the individual classifiers.
Generating successful negotiation strategies by evolutionary computation
Dr. D.D.B. van Bragt, Drs. E.H. Gerding, Prof.dr.ir. J.A. La Poutré
The rapid growth of a global electronic market place, together with the
establishment of standard negotiation protocols, currently leads to the
development of multi-agent architectures in which artificial agents can negotiate
on behalf of their users. Our long-term goal in this project is to develop artificial
bargaining agents, which are able to deal successfully with a variety of opponents
in such an electronic market place. To reach this goal, we combine and further
refine techniques from the fields of multi-agent systems, learning by co-evolution,
and evolutionary computing.
In recent work (Bragt van and La Poutré, 2001b), we observed that most of
today’s (prototype) systems for automated negotiations, like Kasbah or Têteá-Tête, use simple and static negotiation rules. We were able to show that
such “fixed” bidding agents can be exploited by more sophisticated “adaptive”
software agents (based upon evolving automata). These adaptive agents are
able to learn strategies, which perform (almost) optimally against a variety of
fixed opponents. Furthermore, they are able to adapt their strategies online to
deal with changing opponents and open environments.
We also developed an abstract model of multiple adaptive agents who are
updating their strategies over time (Bragt van and La Poutré, 2001a). The
bargaining strategies are represented in this model by special kind of finite
automata, which require only two transitions per state. We show that such
automata (with a limited complexity) are a suitable choice in a computational
setting. We furthermore describe an evolutionary algorithm (EA) which generates
s0
s1
τacc = 0.97
o0
= 1.0
o1
= 1.0
s*init
τacc = 0.84
o0
= 0.81
o1
= 0.97
else
o0 >= 0.42
o1 >= 0.90
o0 >= 0.74
o1 >= 0.35
else
τacc = 0.48
o0 = 0.65
o1 = 0.0
s2
64
o0 >= 0.55
o1 >= 0.48
else
τacc = 0.10
o0 = 1.0
o1 = 0.03
o0 >= 0.26
o1 >= 0.32
sinit
s3
Figure 38:
A finite automaton (with four states) which is capable of negotiating about two
issues. An evolutionary algorithm has optimized the structure of this automaton
has been optimized by an evolutionary algorithm.
highly efficient bargaining automata in the course of time. A series of
computational experiments shows that co-evolving automata are able to
discriminate successfully between different opponents, although they receive
no explicit information about the identity or preferences of their opponents.
These results are important for the further development of evolving automata for
real-life (agent system) applications.
As part of the project on efficient bargaining strategies we studied a negotiation
game with “out-side opportunities” for the players. In this model, players can
negotiate with other opponents if negotiations with the current opponent are
not successful. The emergent behavior in this model was studied by evolutionary
simulation. These experiments show that “fair” agreements can evolve if the
players have the opportunity to negotiate at a low cost with multiple opponents.
Evolutionary exploration systems for electronic markets
Drs. F. Alkemade, Prof.dr.ir. J.A. La Poutré, Prof.dr. H.M. Amman, Eindhoven University
of Technology
Funding: NWO-EW project
The rapid growth of the Internet affects markets and commerce in a substantial
way. For example, the rise of electronic markets occurs together with an
important increase of scale due to globalization. The opportunities for interaction
and information gathering also increase substantially for the participants (agents)
in an electronic market.
Evolutionary techniques appear to be very appropriate to model and study such
electronic markets. We therefore develop (computational) evolutionary systems,
which model electronic markets. This project falls within the novel field of “agentbased computational economics” (ACE) and combines research topics from
computer science, cognitive science and evolutionary economics.
We developed alternative evolutionary techniques, which yield more robust
results in comparison with previous studies. We also started to develop a model
of network economics, in which adaptive agents buy and sell goods.
Transactions can only take place in this model if there is a link (network
connection) between the two parties. Our goal is to investigate the role of
intermediaries in such a setting. Because the complexity of our model is
significant we use genetic algorithms to determine highly efficient outcomes.
We use these outcomes as a benchmark to evaluate the performance of agentbased models.
Evolutionary
Economics
Agent-Based
Computational
Economics
Cognitive
Science
Computer
Science
65
Computation in networks of spiking neurons
Drs. S.M. Bohté, Prof.dr.ir. J.A. La Poutré. Prof.dr. J.N. Kok, Leiden University
Artificial neural networks (ANNs) have received much attention in the last
decades as powerful methods for adaptive classification and function
approximation. Implicitly, such neural networks also claim to model the type of
information processing and learning as takes place in systems of real neurons
(e.g. the human brain). A particularly important assumption is that the artificial
neurons as used in traditional neural networks sufficiently capture the information
processing capabilities of real neurons. This assumption is increasingly being
questioned, as more information on the actual behavior of cortical neurons and
their connectivity becomes available.
In addition, there are important computational questions like the manipulation
of structured information by neural networks (e.g. the representation of visual
objects and the construction and expression of language). The exploration of
the computational properties of a more biologically realistic neuron model, spiking
neurons, is the subject of this project (Bohté et al., 2001b). In particular, the
(additional) computational abilities offered by the careful timing of individual
spikes emitted by such neurons is being researched. Results so far include the
development of unsupervised and supervised learning rules. Current work is
focused on the structured representation of information with spiking neurons,
where we have been exploring the use of distributed representations in individual
neural nodes. This has resulted in architectures that are able to perform dynamic
variable binding, and, in a vision application, the position invariant detection of
feature-conjunctions (e.g. finding {yellow-and-rectangle} anywhere in a picture).
Dynamic algorithms for on-line optimization
Drs. R. van Stee, Prof.dr.ir. J.A. La Poutré. L. Epstein, The Interdisciplinary Center, Herzliya,
Israel. S. Seiden, Louisiana State University, USA.
Funding: NWO-EW project
The project “Dynamic Algorithms for On-Line Optimization” addresses the design
and analysis of efficient algorithms for on-line optimization problems that are
fundamental to various management and design problems in computer systems
and networks.
Within this project, research on online bin packing was performed (Seiden and
van Stee, 2001). New upper and lower bounds were presented for a multidimensional generalization of bin packing, called box packing. Several variants
of this problem, including bounded space box packing, square packing, variable
sized box packing and resource augmented box packing are also studied.
Furthermore, work was done on on-line scheduling (Epstein and van Stee, 2001a;
Epstein and van Stee, 2001c). In particular, the scheduling problem of minimizing
the maximum starting time on parallel identical machines on-line was studied
(Epstein and van Stee, 2001b). The goal is to minimize the last time that a job
starts. We designed an algorithm with constant competitive algorithm for this
problem. We also showed that a “greedy” algorithm is optimal for resource
augmentation.
66
We also developed an algorithm to minimize the total completion time on-line
on a single machine, using restarts, with a low, constant competitive ratio of
only 3/2. This is the first restarting algorithm to minimize the total completion
time that has a better competitive ratio than an algorithm that does not restart.
Figure 39:
(a) A complete remote-sensing image. Inset: image cutout that is actually clustered with
various methods. (b) Classification of the cutout as obtained by clustering the entire image
with the UNSUP algorithm. (c) Classification by Kohonen’s SOM algorithm. (d) Classification
by a spiking neural network.
Quality of service for multimedia systems
Dr.ir. J. Korst, Ir. J. Aerts, Ir. W. Michiels, Philips Research
Funding: NWO-EW project
“Quality of Service” (QoS) is a typical feature of multimedia servicing. It yields
important perspectives of providing services to clients in a flexible and meaningful
way. The primary problem with QoS is resource management: i.e. the assignment
of activities to scarce resources. In this NWO-EW project, we investigate how
to use, handle, and optimize QoS for servicing requests in on-line multimedia
systems, in order to obtain a good system performance.
In the project, research was performed on reinforcement learning with multiple
objectives. In co-operation with researchers from Philips research, work was
also performed on scheduling strategies for near video-on-demand.
67
68
3
Commercial Applications
69
3A
Participating Industrial Partner
Energy research Centre of the Netherlands (ECN)
Unit Technological Services and Consultancy
Description of the Group
Ir. J.J. Saurwalt, Drs.ing. J.K. Kok, Ir. G.P. Leendertse, Ir. J.J. Kaandorp, Ing. F.J.
Kuijper
The Netherlands Energy Research Foundation ECN is the leading institute for
energy research in the Netherlands. Research at ECN is carried out under
contract from the government and from national and foreign organizations and
industries. Intelligent systems, incorporating neural networks and intelligent
agent technology are developed as part of ECN’s activities in the following areas:
solar energy, wind energy, clean fossil fuels, energy efficiency in industry, and
renewable energy in the built environment. The department Technological
Services and Consultancy (TSC) is engaged in the development, engineering
and implementation of these intelligent systems, as well as those of industrial
parties. Moreover, ECN - TSC applies neural networks as an advanced tool for
data mining studies. More information on the group can be found at:
http//www.ecn.nl
Co-operation with SNN
Projects
ECN started a strategic-alliance with the SNN in 1999. The focus of the alliance
has been on knowledge transfer in two directions. ECN contributes to
technological innovation by developing and transferring specific knowledge and
technologies for its target groups and clients. A further dissemination of artificial
intelligence techniques towards energy-intensive sectors will enhance energy
efficiency and will strengthen the position of sustainable energy sources. By
co-operation with the SNN, ECN is able to utilize new technologies directly
after they have reached maturity.
Short-Term Forecasting of Sustainable Energy Production
Sponsored by: Foundation for Sustainable Energy (SDE), System Integration ·ENGINE (ENergy
Generation In the Natural Environment)
Partners: KNMI, Meteo Consult, WEOM
The contribution of wind and solar energy systems to the overall electrical power
supply has a varying character. Due to liberalization of the energy market, short-
70
term supply and demand matching will gain importance. A reliable power output
prediction system will contribute to the cost-effectiveness of sustainable energy
sources. ECN units co-operate in a research project in which the technical and
economical feasibility of a Power Output Prediction System is investigated.
Further, a proof-of-principle prototype is designed and implemented. Neural
networks operate next to physical models in the heart of the prediction system.
Intelligent Data Analysis for High Speed Screening of Catalysts
Sponsored by: European Union
Partners: DSM Research, AMTEC, and others
High Speed Screening (HSS) for discovery of catalysts is a relatively new field
of research. By running tens or hundreds of experiments in parallel, the average
time for an experiment is drastically reduced. In catalysis research the number
of possible combinations is virtually infinite. It is not possible to try all of them.
So, in addition to performing experiments, the need remains for being very
smart in the selection of experiments. ECN - TSC is developing intelligent
techniques for the optimal selection of these experiments as well as for
knowledge extracfrom the database of experimental results.
Figure 40:
ECN investigates the application of neural networks for power output prediction of wind
turbines. (Photo: A.A. Homan, ECN).
71
3B
Commercial Spin-off activities of SNN
SMART Research BV
SMART Research BV is the commercial outlet of SNN Nijmegen. Within SMART,
we conduct pilot and feasibility studies, but we also develop actual working
applications. The techniques that we apply include advanced neural networks
and graphical models like Bayesian networks. We have worked, among others,
for companies and organizations in the consumer branch (Schuitema, Riedel,
Vendex, Centraal Bureau Levensmiddelen), publishers (De Telegraaf, BauerVerlag, Edipresse), production industry (Shell, SKF, Kenniscentrum Papier en
Karton), and finance (Simplex Credit Advisory). We will highlight some of the
projects we have worked on in 2001. SKF, market leader in the production of all
kinds of bearings, has chosen BayesBuilder as their development tool to build
a probabilistic rule-based system for the analysis of bearing failures. Better
than its competitors, BayesBuilder facilitates the implementation of huge
Bayesian networks within a user-friendly interface and environment. Within this
project SMART provides support and maintenance.
In 1996, De Telegraaf started using JED, for “Just Enough Delivery”, to optimize
the distribution of single-copy newspapers. JED is a software package based
on neural-network technology: it learns from the past sales, to predict the sales
in the future. For De Telegraaf, we have developed and implemented a completely
new version. As shown in comparative studies for Bauer-Verlag (Germany) and
Edipresse (Switzerland), this new version makes even better predictions, clearly
outperforming alternative approaches. Furthermore, it contains many new
features: tools to select and target all outlets in specific regions and/or
branches,algorithms to calculate and visualize different scenarios, graphics
displaying the characteristics and explaining the predictions for individual outlets,
and so on. The new version is fully automatic that is predictions are continuously
updated based on the latest available information.
We expect JED 2.0 to be up and running at De Telegraaf early 2002. After
having built a successful prototype, we will in 2002 also start implementing JED
for Midesa, the distributor of the Portuguese newspaper Público.
72
50
40
30
20
sales
prediction
delivery (actual)
delivery (suggested)
10
0
2000−14
2000−27
2000−40
2001−1
2001−14
2001−27
2001−40
2002−1
Figure 41:
Just Enough Delivery: optimizes the trade-off between returns and sellouts
More information can be obtained at:
SMART Research BV
Dr. T.M. Heskes
P.O. Box 31070
6503 CB Nijmegen
The Netherlands.
Tel: +31-24-3615039
Fax:+31-24-3541435
email: [email protected]
http://www.smart-research.nl
73
NuTech Solutions GmbH
NuTech Solutions’ core business is in the field of adaptive business intelligence
based on modern computer science methods. These methods are gleaned from
the model of information processing in natural systems and enable industry to
enter a new economic dimension of effectiveness and adaptability in applications
in the field of business process optimization and data mining.
Data Mining indicates the process of deriving hidden knowledge from large
databases, like they occur in many industrial branches such as e.g. banks,
health industry, automobile industry, chemical industry etc. Methods such as
classical statistics, genetic programming, neural networks, rough sets and
fuzzy logic are applied to such databases to derive compact, readable and
interpretable pieces of knowledge from massive amounts of data. This knowledge
is then used to derive optimized management recommendations for high-level
decision support within companies, leading to huge savings, profitability
increases or quality improvements.
Business process optimization refers to the problem of optimizing complex
industrial systems according to given criteria (e.g. quality, profitability, value).
Applications are given e.g. in the domain of scheduling problems (e.g. trucks,
production processes on machines, crews for airlines), transport optimization
(e.g. lengths and duration’s of routes, cost of transport), technical design
optimization (many different domains), management process optimization
(modeling, simulation and optimization of decision making processes) and
experimental optimization (e.g. in biotechnology, food and cosmetics industry).
Many staff members of NuTech Solutions are among the leading experts worldwide in the above mentioned fields. In total, NuTech employs about 20 staff
members in the field of computational intelligence. Software products in adaptive
business intelligence as well as client-specific consulting activities are the
main focus of the business of NuTech Solutions. Evolution strategies as a key
technology give NuTech access to technologies which no other company world
wide can offer to its clients.
Contact and further information:
Dr. Thomas Bäck
Managing Director
Leiden University, LIACS
Niels Bohrweg 1
NL-2333 CA Leiden
emaill: baeck@ liacs.nl
http://www.liacs.nl
74
4 Publications
75
A B CD
Aarts, L. and van der Veer, P. (2001a). Neural network method for solving parital differential
equations. Neural Processing Letters,volume 17(3): pages 261-271.
Aarts, L. and van der Veer, P. (2001b). Solving nonlinear differential equations by a neural
network method. In Computational Science-ICCS 2001, volume Part II, pages 181-188.
Aarts, L. and van der Veer, P. (2001c). Solving nonlinear differential equations by a neural
network method. Lecture notes in Computer Science, volume 2074(2): pages 181-189.
Achterop, S., DeVos, M., v.d.Schaaf, K., and Spaanenburg, L. (2001). Architectural requirements for a lofar generic node. In Proceedings ProRISC01, pages 234-239.
Alkemade, F. and Poutré, J. L. (2001). Heterogeneous, boundedly rational agents in the Cournot
duopoly. In Proceedings of the 13th Belgian-Dutch Conference on Artificial Intelligence (BNAIC
‘2001), pages 21-22, Amsterdam.
András, P., Postma, E., and van den Herik, H. (2001). Natural dynamics and neural networks:
Searching for efficient preying dynamics in a virtual world. Journal of Intelligent Systems, 11(3):173202. ISSN 0334-1860.
Bäck, T. (2001). Evolution strategies: Overview and a CFD application. In Periaux, J., Joly, P.,
Pironneau, O., and Onate, E., editors, Innovative Tools for Scientific Computation in Aeronautical Engineering. CIMNE, Barcelona.
Bäck, T., de Graaf, J. M., Kok, J. N., and Kosters, W. A. (2001a). Theory of genetic algorithms.
In Paun, G., Rozenberg, G., and Salomaa, A., editors, Current Trends in Theoretical Computer
Science: Entering the 21st Century, pages 546-578. World Scientific, Singapore.
Bäck, T., Emmerich, M., and Schallmo, M. (2001b). Industrial applications of evolutionary
algorithms: A comparison to traditional methods. In Parmee, I., editor, Proceedings of the International Conference on Optimisation in Industry III. Springer, Berlin.
Bäck, T. and Schütz, M. (2001). Evolutionäre algorithmen im data-mining. In Hippner, H., Küsters,
U., Meyer, M., and Wilde, K., editors, Handbuch Data-Mining im Marketing, chapter 10, pages 221244. Vieweg Verlag, Wiesbaden.
Bakker, B. and Heskes, T. (2001b). Task clustering for learning to learn. In Kröse, B., Rijke, M. d.,
Schreiber, G., and Someren, M. v., editors, BNAIC 2001, pages 33-40.
Bakker, B., Heskes, T., Neijt, J., and Kappen, B. (2001). Improving cox survival analysis with
a neural-Bayesian approach. Statistics in Medicine (in press).
Banzhaf, W., Nordin, P., Keller, R., and Francone, F. (2001a). Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications (3rd edition).
Morgan Kaufmann, dpunkt.verlag.
Banzhaf, W., Nordin, P., Keller, R., and Francone, F. (2001b). Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications (Japanese
version). SciTech Press.
Barakova, E. and Spaanenburg, L. (2001). Learning and reproducing. In van Noort, G. and
Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht.
Barber, D. and Heskes, T. (2002). An introduction to neural networks. In Encyclopedia of Life
Support Systems. In press.
Berg, A. B. t. and Spaanenburg, L. (2001). On the compositionality of neural networks. In
Proceedings ECCTD, volume 3, pages 405-408.
Bijma, M., Haseborg, H. T., Diepenhorst, M., and Nijhuis, J. (2001). Neural hardware implementations: an overview. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ.,
Maastricht.
Boers, E. and I.G. Sprinkhuizen-Kuyper, I. (2001). Combined biological metaphors. In Patel, M.,
Honavar, V., and Balakrishnan, K., editors, Advances in the Evolutionary Synthesis of Neural
Systems, pages 153-183. MIT Press, Cambridge, MA. SBN 0-262-16201-6.
Bohté, S., Gerding, E., and Poutré, J. L. (2001a). Competitive market-based allocation of consumer attention space. In Proceedings of the 3rd ACM Conference on Electronic Commerce (EC
76
’01), pages 202-205, USA.
Bohté, S., Poutré, J. L., and Kok, J. (2001b). Learning in spike-time encoded neural networks.
In Proceedings of the 5th International Conference on Cognitive and Neural Systems, Boston,
USA.
Bossi, A., Etalle, S., and Rossi, S. (2001a). Semantics of well-moded input-consuming programs. Computer Languages, volume 26(1): pages 1-25. ISSN 0096-0551.
Bossi, A., Etalle, S., Rossi, S., and Smaus, J.-G. (2001b). ). semantics and termination of
simply-moded logic programs with dynamic scheduling. In Sands, D., editor, Proceedings of the
European Symposium on Programming, pages 402-416. LNCS 2028, Springer-Verlag, Berlin,
Germany. ISBN 3 540 418628.
Bragt van, D., van Kemenade, C., and Poutré, J. L. (2001). The influence of evolutionary
selection schemes on the iterated prisoner’s dilemma. Computational Economics, volume 17(2/3):
pages 253-263.
Bragt van, D. and La Poutré, J. A. (2001a). Co-evolving automata negotiate with a variety of
opponents. In Proceedings of the Eleventh Dutch-Belgian Conference on Machine Learning
(BENELEARN-2001), pages 77-84.
Bragt van, D. and La Poutré, J. A. (2001b). Generating efficient automata for negotiations - An
exploration with evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), page 1093, San Francisco, California, USA. Morgan Kaufmann.
Breuker, D., Uiterwijk, J., and Herik, H. v. d. (2001a). The pn2 -search algorithm. In van den
Herik, H. and Monien, B., editors, Advances in Computer Games, pages 115-132. IKAT, Universiteit
Maastricht, Maastricht. ISBN 90 6216 5761.
Breuker, D., van den Herik, H., Uiterwijk, J., and Allis, L. (2001b). A solution to the ghi problem
for best-first search. Theoretical Computer Science, volume 252(1-2): pages 121-149. ISSN
0304-3975.
Bunschoten, R. and Kröse, B. (2001a). 3-d scene reconstruction from cylindrical panoramic
images. In Proceedings of the 9th International Symposium on Intelligent Robotic Systems
(SIRS’2001), pages 199-205, LAAS-CNRS, Toulouse, France.
Bunschoten, R. and Kröse, B. (2001b). 3-d scene reconstruction from multiple panoramic
images. In Proceedings of 7th annual conference of the Advanced School for Computing and
Imaging (ASCI 2001), pages 49-54, Heijen, The Netherlands. ASCI.
Cemgil, A. and Kappen, B. (2001a). A dynamic belief network implementation for realtime music
transcription. In Proceedings of the Belgian-Dutch Conference on Artificial Intelligence 2001,
Amsterdam.
Cemgil, A., Kappen, B., Desain, P., and Honing, H. (2001c). On tempo tracking: Tempogram
representation and kalman filtering. Journal of New Music Research. In press.
Cemgil, A. and Kappen, H. (2001b). Bayesian real-time adaptation for interactive performance
sytems. In Proceedings ICMC, Habana/Cuba. In press.
Cemgil, A. and Kappen, H. (2001c). Tempo tracking and rhythm quantization by sequential
monte carlo.In Proceedings NIPS. In press.
Cliteur, P., Herik, H. v. d., Huls, N., and Schmidt, A. (2001). It ain’t necessarily so. E.M. Meijers
Instituut, Leiden & Kluwer, Deventer, The Netherlands. ISBN 90 268 3854 9.
Cornelisse, L., Scheenen, W. J., Koopmani, W. J., Roubos, E. W., and Gielen, S. C. (2001).
Minimal model for intracellular calcium oscillations and electrical bursting in melanotrope cells of
xenopus laevis. Neural Computation, volume 13: pages 113-137.
Cser, J. and e.a, L. O. (2001). A neural network approach to predict nearshore morphology along
the dutch coast. Journal of Coastal Research, 17:1-11.
Donkers, H., Uiterwijk, J., and Herik, H. v. d. (2001a). Admissibility in opponent-model search.
In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th
Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 373-380.
77
DEFG
H I J K
Universiteit van Amsterdam, Amsterdam.
Donkers, H., Uiterwijk, J., and Herik, H. v. d. (2001b). Probabilistic opponent-model search.
Information Scien-ces, volume 135(3-4): pages 123-149. ISSN 0020-0255.
Dorp, C. v., Beulens, A., and Beers, G. (2001). Erp in semi-process: Requirements capture for
tracking and tracing. In Strong, D., Straub, D., and DeGross, J., editors, Proceedings of the 7th
Americas Conference on Information Systems (AMCIS 2001). Association for Information Systems, Boston, Massachusetts, USA.
Duin, R. (2001). Four scientific approaches to pattern recognition. In Vossepoel, A. and Vos, F.,
editors, Fourth Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image
Processing, pages 331-337. NVPHBV, Delft.
Duin, R. and Pekalska, E. (2001). Complexity of dissimilarity based pattern classes. In Austvoll,
I., editor, Proceedings of the 12th Scandinavian Conference on Image Analysis, SCIA 2001,
pages 663-670. NOBIM, Norway.
Eggermont, J. (2001). Knowledge discovery and evolutionary computation - an partial abstract
of my ph.d. In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van Remortel, P., editors,
Proceedings of the Brussels Evolutionary Algorithms Day ( bead-2001). Vrije Universiteit Brussel
(vub).
Eggermont, J., Lenaerts, T., Poyhonen, S., and Termier, A. (2001). Raising the dead; extending evolutionary algorithms with a case-based memory. In Miller, J., Tomassini, M., Lanzi, P.,
Ryan, C., Tetamanzi, A., and Langdon, W., editors, Proceedings on the Fourth European Conference on Genetic Programming (EuroGP’01), volume 2038 of LNCS, pages 280-290, Lake Como,
Milan, Italy. Springer-Verlag.
Eggermont, J. and van Hemert, J. I. (2001). Adaptive genetic programming applied to new and
existing simple regression problems. In Miller, J., Tomassini, M., Lanzi, P., Ryan, C., Tetamanzi, A.,
and Langdon, W., editors, Proceedings on the Fourth European Conference on Genetic Programming (EuroGP’01), volume 2038 of LNCS, pages 23-35, Lake Como, Milan, Italy. Springer-Verlag.
Emmerich, M., Willmes, L., and Bäck, T. (2001). Asynchronous evolution strategies for distributed direct optimisation. In Giannakoglu, K., Tsahalis, D., Periaux, J., Papailiou, K., and Fogarty,
T., editors, Proceedings of the International Conference on Industrial Applications of Evolutionary Algorithms, EUROGEN. CIMNE, Barcelona.
Epstein, L. and van Stee, R. (2001a). Lower bounds for on-line single-machine scheduling. In
Proceedings of the 26th International Symposium on Mathematical Foundations of Computer
Science (MFCS ‘2001), Lecture Notes in Computer Science, LNCS 2136, pages 338-350, Mariánské
Lázne, Czech Republic. Springer Verlag.
Epstein, L. and van Stee, R. (2001b). Minimizing the maximum starting time on-line. Technical
Report SEN-R00133, CWI, Amsterdam.
Epstein, L. and van Stee, R. (2001c). Optimal on-line flow time with resource augmentation. In
Proceedings of the 13th International Symposium on the Fundamentals of Computation Theory
(FCT ‘2001), Lecture Notes in Computer Science, LNCS 2138, pages 472-482, Riga, Latvia.
Springer Verlag.
Gao, X., Iida, H., Uiterwijk, J., and Herik, H. v. d. (2001). Strategies anticipating a difference in
search depth using opponent-model search. Theoretical Computer Science, volume 252(1-2):
pages 83-104. ISSN 0304-3975.
Giotis, A. P., Emmerich, M., Naujoks, B., Bäck, T., and Giannakoglu, K. (2001). Low-cost
stochastic optimisation for engineering applications. In Giannakoglu, K., Tsahalis, D., Periaux, J.,
Papailiou, K., and Fogarty, T., editors, Proceedings of the International Conference on Industrial
Applications of Evolutionary Algorithms, EUROGEN. CIMNE, Barcelona.
Graaf de, J., Kosters, W., and Witteman, J. (2001). Interesting fuzzy association rules in
quantitative databases. In de Raedt, L. and Siebes, A., editors, 5th European Conference on
Principles and Practice of Knowledge Discovery in Databases (PKDD’01), volume 2168 of LNAI,
78
pages 140-151, Freiburg, Germany. Springer Verlag.
Hagen ten, S. (2001). Continuous State Space Q-Learning for Control of Nonlinear Systems. PhD
thesis, Computer Science Institute, University of Amsterdam, The Netherlands.
Hagen ten, S. and Kröse, B. (2001). Autonomous map building by mobile robots. In Kröse, B.,
Rijke, M. d., Schreiber, G., and Someren, M. v., editors, Proceedings of the 13th Dutch-Belgian
Artificial Intelligence Conference, BNAIC’01, pages 117-124.
Hemert, J.I., v. and Jansen, M. (2001). An engineering approach to evolutionary art. In Spector,
L., Goodman, E. D., Wu, A., Langdon, W., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S.,
Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation
Conference (gecco-2001), page 177. Morgan Kaufmann Publishers.
Hemert van, J. (2001). Evolutionary computation in constraint satisfaction and machine learning
– an abstract of my ph.d. In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van
Remortel, P., editors, Proceedings of the Brussels Evolutionary Algorithms Day ( bead-2001).
Vrije Universiteit Brussel (vub).
Hemert van, J., Hoyweghen, C. V., Lukschandl, E., and Verbeeck, K. (2001). A “futurist”
approach to dynamic environments. In Branke, F. D. O. P. J. and Bäck, T., editors, Proceedings of
the Work-shops at the Genetic and Evolutionary Computation Conference (gecco-2001), pages
35-38. Morgan Kaufmann.
Herik, H. v. d. (2001). . it ain’t necessarily so. In Cliteur, P., van den Herik, H., Huls, N., and Schmidt,
A., editors, Grensoverschrijdende Communicatie, pages 331-352. E.M. Meijers Institituut, Leiden
& Kluwer, Deventer, The Netherlands. ISBN 90 268 3854 9.
Herik, H. v. d. and Iida, H. (2001). Computer and games. Theoretical Computer Science, volume
252(1-2):ISSN 0304-3975.
Heskes, T. (2001). Self-organizing maps, vector quantization, and mixture modeling. IEEE Transactions on Neural Networks, volume 12: pages1299-1305.
Heskes, T. (2002). Approximate algorithms for neural-Bayesian approaches. Theoretical Computer Science. In press.
Hooimeijer, M. (2001). Reduction of complex computational models. In Phd-dissertatie. Technical University Delft.
Jain, A., Duin, R., and Mao, J. (2001). Statistical pattern recognition: A review. In Vossepoel, A.
and Vos, F., editors, Fourth Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image Processing, pages 1-34. NVPHBV, Delft.
Jelasity, M. (2001). The Shape of Evolutionary Search, discovering and representing search
space structure. PhD thesis, Universiteit Leiden.
Kappen, H. (2001). An introduction to stochastic neural networks. In Moss, F. and Gielen, S.,
editors, Neuro-Informatics and Neural Modelling, pages 517-552. North-Holland.
Kappen, H. (2002). The cluster variation method for approximate reasoning in medical diagnosis.
In Nardulli, G. and Stramaglia, S., editors, Modeling Bio-medical signals. World-Scientific. In press.
Kappen, H., Gielen, C., Wiegerinck, W., Cemgil, A., Heskes, T., Nijman, M., and Leisink,
M. (2001a). Approximate reasoning: Real world applications of graphical models. In Uesaka, Y.,
Kanerva, P., and Asoh, H., editors, Foundations of Real-World Intelligence, pages 73-121. CSLI,
California, USA. Submitted to RWC in October 2001.
Kappen, H. and Wiegerinck, W. (2001a). Mean field theory for graphical models. In Saad, D. and
Opper, M., editors, Advanced mean field theory, pages 37-49. MIT Press.
Kappen, H. and Wiegerinck, W. (2001b). Second order approximations for probability models. In
Leen, T., Dietterich, T., Caruana, R., and Sa, V. d., editors, Advances in Neural Information Processing Systems 13, pages 238-244. MIT Press.
Kappen, H. and Wiegerinck, W. (2002). A novel iteration scheme for the cluster variation
method. In Dieterich, T., Becker, S., and Ghahramani, Z., editors, Advances in Neural Information
Processing Systems, volume 14. In press.
79
KLMN
OPQR
Kappen, H., Wiegerinck, W., and Braak, E. t. (2001b). Decision support for medical diagnosis.
In STT, editor, The future of data mining. In press.
Keller, R. E. (2001a). Autopoietic solutions for real-world problems. In EvoDebates: The future of
Evolutionary Computing. EvoNet-The European Network of Excellence in Evolutionary Computing.
In press.
Keller, R. E. and Banzhaf, W. (2001). The evolution of genetic code on a hard problem. In Spector,
L., Goodman, E. D., Wu, A., Langdon, W., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S.,
Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation
Conference (gecco-2001). In press.
Khu, S., van der Veer, P., and Liong, S. (2001). Improving the performance of artificial neural
network and its application to rainfall-runoff simulation. Canadian Journal of Civil Engineering,
pages 1-21.
Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001a). Learning time allocation using neural
networks. In Marsland, T. and Frank, I., editors, Computers and Games, Proceedings of the Second
International Conference, CG 2000: Lecture Notes in Computer Science 2063, pages 170-185.
Springer Verlag, Berlin. ISBN 3-540-43080-6.
Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001b). Move ordering using neural networks. In
Monostori, L., Váncza, J., and Ali, M., editors, Engineering of Intelligent Systems: Lecture Notes in
Artificial Intelligence, pages 45-50. Springer Verlag, Berlin, Germany. ISBN 3-540-42219-6.
Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001c). Move ordering using neural networks. In
Monostori, L., Váncza, J., and Ali, M., editors, Engineering of Intelligent Systems, Lecture Notes in
Artificial Intelligence, pages 45-50. Springer Verlag, Berlin, Germany. ISBN 3-540-42219-6.
Kocsis, L., Uiterwijk, J., and Herik, H. v. d. (2001d). Search-independent forward pruning. In
Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th
Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 159-166.
Universiteit van Amsterdam, Amsterdam, The Netherlands.
Kok, J. (2001). Intelligent data analysis. In Kent, A., editor, Encyclopedia of Library and Information Science, volume 70, pages 218-224. Marcel Dekker Inc., New York.
Kok, J. and Kosters, W. (2001). Natural data mining techniques. In Paun, G., Rozenberg, G., and
Salomaa, A., editors, Current Trends in Theoretical Computer Science: Entering the 21st Century,
pages 603-613. World Scientific, Singapore.
Kortmann, R., Postma, E., and Herik, H. v. d. (2001). Evolution of visual resolution constrained
by a trade-off. Artificial Life, volume 7(2): pages 125-145. ISSN 1064-5462.
Kröse, B., Vlassis, N., Bunschoten, R., and Motomura, Y. (2001). A probabilistic model for
appearance-based robot localization. Image and Vision Computing, volume 19(6): pages 381391.
Kuncheva, L., Bezdek, J., and Duin, R. (2001). Decision templates for multiple classifier fusion:
An experimental comparison. Pattern Recognition, volume 34(2): pages 229-314.
Lamers, M. (2001). Neural Networks for Analysis of Data in Environmental Epidemiology - A Casestudy into Acute Effects of Air Pollution Episodes. PhD thesis, Universiteit Leiden.
Leisink, M. and Kappen, H. (2001a). Means, correlations and bounds. In Proceedings NIPS. In
press.
Leisink, M. A. R. and Kappen, H. J. (2001b). A tighter bound for graphical models. In Leen, T.,
Dietterich, T., Caruana, R., and Sa, V. d., editors, Advances in Neural Information Processing
Systems 13. MIT Press. In press.
Leisink, M. A. R. and Kappen, H. J. (2001c). A tighter bound for graphical models. Neural
computation, volume 13. In press.
Loog, M., Duin, R., and Haeb-Umbach, R. (2001). Multiclass linear dimension reduction by
weighted pairwise fisher criteria. IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 23(7): pages 762-766.
80
Mathijsen, E., Lipsch, M., Dorp, C. v., and Herik, H. v. d. (2001). A model for exchanging
knowledge among virtual teams. In Proceedings of the 2nd European Conference on Knowledge
Management. The IEDC-Bled School of Management, Bled, Slovenia.
Musa, M., Duin, R., and de Ridder, D. (2001a). An enhanced em algorithm for principal component analyzers. In Leonardis, A. and Bischof, H., editors, Proceedings ICANN 2001, Workshop on
Kernel and Subspace Methods for Computer Vision, pages 25-35.
Musa, M., Duin, R., and de Ridder, D. (2001b). Modelling handwritten digit data using probabilistic
principal component analysis. In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI
2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft.
Nijssen, S. (2001). Dynamic optimisation, ec theory and data mining - a partial abstract of my ph.d.
In Defaweux, A., Manderick, B., Lenearts, T., Parent, J., and van Remortel, P., editors, Proceedings
of the Brussels Evolutionary Algorithms Day ( bead-2001). Vrije Universiteit Brussel (vub).
Nijssen, S. and Kok, J. (2001). Faster association rules for multiple relations. In Proceedings of
the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI’01), pages 891-896.
Oudshoff, A., I.E. Bosloper, Klos, T., and Spaanenburg, L. (2001). Categorizing virtual communities by text-mining. In Proceedings BNAIC’01, pages 195-202.
Paclik, P., Duin, R., and van Kempen, G. (2001a). Multi-spectral image segmentation algorithm
combining spatial and spectral information. In Austvoll, I., editor, Proceedings of the 12th
Scandinavian Conference on Image Analysis, SCIA 2001, pages 230-235. NOBIM, Norway.
Paclik, P., Duin, R., and van Kempen, G. (2001b). Segmentation of multi-spectral images using
a combined classifier approach. In Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc.
ASCI 2001, 7th Annual Conf. of the Advanced School for Computing and Imaging. ASCI, Delft.
Pantic, L., Torres, J., and Kappen, H. (2001). On the role of dynamical synapses in coincidence
detection. In Proceedings CNS. In Press.
Pekalska, E. and Duin, R. (2001a). Automatic pattern recognition by similarity representations.
Electronics Letters, volume 37(3): pages 159-160.
Pekalska, E. and Duin, R. (2001b). Is combining useful for dissimilarity representations? In
Lagendijk, R., Pimentel, A., and Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the
Advanced School for Computing and Imaging. ASCI, Delft.
Pekalska, E. and Duin, R. (2001c). On combining dissimilarity representations. In Kittler, J. and
Roli, F., editors, Multiple Classifier Systems, Proceedings Second International Workshop MCS
2001, Lecture Notes in Computer Science, volume 2096, pages 359-368. Springer Verlag, Berlin.
Pekalska, E., Paclik, P., and Duin, R. (2001). A generalized kernel approach to dissimilarity
based classification. Journal of Machine Learning Research, volume 2(2): pages 175-211.
Postma, E., Dartel, M. v., and Kortmann, R. (2001a). Recognition by fixation. In Kröse, . B., de
Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 425-432. Universiteit van Amsterdam, Amsterdam, The Netherlands.
Postma, E., Roebroeck, A., and Lacroix, J. (2001b). The dynamics of semantic network
development. In French, R. and Sougné, J., editors, Connectionist Models of Learning, Development and Evolution, pages 293-302. Springer-Verlag, London, U.K.
Putten van der, P. and den Uyl, M. (2001). Mining e-markets. In IT Monitor, volume 3. In press.
Ridder de, D. (2001). Adaptive methods of image processing. PhD thesis, Delft University of
Technology. ASCI Dissertation Series, 70.
Ridder de, D., Lemmers, O., Duin, R., and Kittler, J. (2001). The adaptive subspace map for
image description and image database retrieval. In Vossepoel, A. and Vos, F., editors, Fourth
Quinquennial Review 1996-2001 Dutch Society for Pattern Recognition and Image Processing,
pages 291-299. NVPHBV, Delft.
Roos, N., Teije, A. t., Bos, A., and C., W. (2001). Multi-agent diagnosis: an analysis. In Kröse, B.,
de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-
81
STUV
Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 221-228. Universiteit van
Amsterdam, Amsterdam, The Netherlands.
Sasaki, N., Hirasawa, M., Hashimoto, T., Iida, H., Uiterwijk, J., and Herik, H. v. d. (2001).
Hira, a four-handed shogi program. In van den Herik, H. and Monien, B., editors, Advances in
Computer Games 9, pages 247-259. IKAT, Universiteit Maastricht, Maastricht, The Netherlands.
ISBN 90 6216 5761.
Schmidt, K., Henkel, C., Rozenberg, G., and Spaink, H. (2001). Experimental aspects of dna
computing by blocking: Use of fluorescence techniques for detection. In Kraayenhof, R., Visser,
A., and Gerritsen, H., editors, Fluorescence Spectroscopy, Imaging and Probes. Springer-Verlag.
Seiden, S. and van Stee, R. (2001). New bounds for multi-dimensional packing. Technical
Report SEN-R00132, CWI, Amsterdam.
Skurichina, M. (2001). Stabilizing weak classifiers. PhD thesis, Delft University of Technology.
Skurichina, M. and Duin, R. (2001). Bagging and the random subspace method for redundant
feature spaces. In Kittler, J. and Roli, F., editors, Proceedings Second International Workshop
MCS 2001, Lecture Notes in Computer Science, volume 2096, pages 1-10. Springer Verlag, Berlin.
Smirnov, E. (2001). Conjunctive and Disjunctive Version Spaces with Instance-Based Boundary
Sets. PhD thesis, IKAT, Universiteit Maastricht, Maastricht, The Netherlands.
Smirnov, E., Herik, H. v. d., and Sprinkhuizen-Kuyper, I. (2001). An adaptable version-space
representation. In Kröse, B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages
233-240.
Smith, K., Chuan, S., and van der Putten, P. (2001). Determining the validity of clustering for
data fusion. In International Workshop on Hybrid Intelligent Systems 2001. In press.
Spaanenburg, L. (2001a). Embedded intelligence glues the home together. In PT-ES Research
Dossier 2002, pages 24-27. Ten Hagen & Stam.
Spaanenburg, L. (2001b). Embedding virtual products by generic components. In Proceedings
2nd Embedded Systems Workshop, pages 237-242.
Spaanenburg, L. (2001c). Intelligente systemen: wantrouwen is op zijn plaats. In Automatisering
Gids, page 19. Ten Hagen & Stam.
Spaanenburg, L. (2001d). Intelligentie robots stagneert door anti-techniek stemming. In Intelligentie
robots stagneert door anti-techniek stemming, page 17. Ten Hagen & Stam Uitg.
Spaanenburg, L. (2001e). Neural networks in fixed-point hardware. In On-line Symposium for
Electronics Engineers. In Press.
Spaanenburg, L. (2001f). Unlearning in feed-forward multi-nets. In Proceedings ICANNGA’01,
pages 106-109.
Spaanenburg, L., Diepenhorst, M., Haseborg, H. T., and Wang, P. (2001a). Training neural
nets for small word width. In Proceedings AmiRA’01, pages 171-180.
Spaanenburg, L., Haseborg, H. T., and Peng, W. (2001b). Bringing neural intelligence to 8-bit
virtual peripherals. Int. Journal of Knowledge-based Intelligent Engineering Systems, volume
5(3): pages 171-178.
Spaanenburg, L. and Stevens, H. (2001). Embedded intelligence to understand images. In van
Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht.
Spaanenburg, L. and ten Berg, A. (2001). Neural networks with a small foot-print: a lesson in
trading programs for data. In Proceedings AmiRA’01, pages 291-300.
Spronck, P. (2001). Palm game design. In Mehdi, Q., Gough, N., and Al-Dabass, D., editors,
GAME-ON 2001 2nd International Conference on Intelligent Games and Simulation, pages 9599. SCS Europe, Ghent, Belgium.
Spronck, P. and Schilstra, K. (2001). Boks: A knowledge based system in support of the Dutch
building materials regulations. In Kowalczyk, R., Loke, S.-W., Reed, N., and Williams, G., editors,
Advances in Artificial Intelligence. PRICAI 2000 Workshop Reader: Lecture Notes in Computer
82
Science, volume 2112, pages 93-102. Springer-Verlag, Berlin, Germany. ISBN 3-540-42597-7.
Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001a). Evolutionary learning of a
neural robot controller. In Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation-CIMCA 2001, pages 510-518. ISBN: 0-858-89847-0.
Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001b). Infused evolutionary learning.
In Hoste, . V. and de Pauw, G., editors, Proceedings of the Eleventh Belgian-Dutch Conference on
Machine Learning, pages 61-68. University of Antwerp, Antwerp, Belgium. ISBN: 0-858-89847-0.
Spronck, P., Sprinkhuizen-Kuyper, I., and Postma, E. (2001c). Island-based evolutionary
learning. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the
13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 441-448.
Stee van, R. and Poutré, J. L. (2001a). Partial servicing of on-line jobs. Journal of Scheduling,
volume 4(6): pages 379-396.
Stee van, R. and Poutré, J. L. (2001b). Running a job on a collection of dynamic machines, with
on-line restarts. Acta Informatica, volume 37(10): pages 727-742.
Steen van der, E., Nijhuis, J., Spaanenburg, L., and van Veelen, M. (2001). Sampling casts
plasticity in adaptive batch control. In Proceedings ProRISC01, pages 646-651.
Tax, D. (2001). One-class classification; Concept-learning in the absence of counter-examples.
PhD thesis, Delft University of Technology. ASCI Dissertation Series, 65.
Tax, D. and Duin, R. (2001a). Combining one-class classifiers. In Kittler, J. and Roli, F., editors,
Multiple Classifier Systems, Proceedings Second International Workshop MCS 2001, Lecture
Notes in Computer Science, volume 2096, pages 299-308. Springer Verlag, Berlin.
Tax, D. and Duin, R. (2001b). A generalized kernel approach to dissimilarity based classification.
Journal of Machine Learning Research, pages 2(2): volume 155-173.
Tax, D. and Duin, R. (2001c). Outliers and data descriptions. In Lagendijk, R., Pimentel, A., and
Wilkinson, M., editors, Proc. ASCI 2001, 7th Annual Conf. of the Advanced School for Computing
and Imaging. ASCI, Delft.
Torres, J., Willems, P., Kappen, H., and Koopman, W. (2001a). Hysteresis and bistability in a
realistic model for ip3-driven ca2+ oscillations. Europhys. Lett., volume 55(5): pages 746-752.
Torres, J., Willems, P., Kappen, H., and Koopman, W. (2001b). Mathematical model for calcium
oscillations in non-excitable cells. Biophysical Journal, volume 80(1): 613A Part 2.
Veelen van, M., ter Brugge, M., Nijhuis, J., and Spaanenburg, L. (2001a). Speech-driven
dialing. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht.
Veelen van, M., terBrugge, M., Nijhuis, J., and Spaanenburg, L. (2001b). Embedding fixedpoint intelligence. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2. Shaker Publ., Maastricht.
Venema, R. and Spaanenburg, L. (2001). Learning feed-forward multi-nets. In Proceedings
ICANNGA’01, pages 102-105.
Verbeek, J. J., Vlassis, N., and Kröse, B. (2001). A soft k-segments algorithm for principal
curves. In Proc. Int. Conf. on Artificial Neural Networks, pages 450-456, Vienna, Austria.
Verbeek, J. J., Vlassis, N., and Kröse, B. (2002). A k-segments algorithm for finding principal
curves. Pattern Recognition Letters, volume 23(8). To appear.
Vlassis, N. and Likas, A. (2002). A greedy EM algorithm for Gaussian mixture learning. Neural
Processing Letters, volume 15(1). In print.
Vlassis, N., Motomura, Y., Hara, I., Asoh, H., and Matsui, T. (2001). Edge-based features from
omnidirectional images for robot localization. In Proc. IEEE Int. Conf. on Robotics and Automation,
pages 1579-1584, Korea.
Werf, E. v. d., Uiterwijk, J., and Herik, H. v. d. (2001). Learning connectedness in binary images.
In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th
Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 459-466.
Westerdijk, M. (2001). Hidden Variable Models for Knowledge Discovery. PhD thesis, University
of Nijmegen.
83
WXYZ
Westerdijk, M., Barber, D., and Wiegerinck, W. (2001). Deterministic generative models for
fast feature discovery. Data Mining and Knowledge discovery, volume 5, pages 337-363.
Wezel van, M., Kosters, W., van der Putten, P., and Kok, J. (2001). Nonmetric multidimensional
scaling with neural networks. In Advances in Intelligent Data Analysis, The Fourth International
Conference on Intelligent Data Analysis (IDA’01), volume 2189 of LNCS, pages 145-155, Cascais,
Portugal. Springer-Verlag.
Wichers, W., Graaf, J. D., Nijhuis, J., and Spaanenburg, L. (2001). Animated marketing and
sales: a case of hidden intelligence. In van Noort, G. and Spaanenburg, L., editors, V-Annals 2.
Shaker Publ., Maastricht.
Wiegerinck, W. and Heskes, T. (2001a). Probability assessment with maximum entropy in
bayesian networks. In Lucas, P., van der Gaag, L., and Abu-Hanna, A., editors, AIME’01 Workshop: Bayesian Models in Medicine, pages 71-80. AIME’01.
Wiegerinck, W. and Heskes, T. (2001b). Probability assessment with maximum entropy in
bayesian networks. In Goodman, A. and Smyth, P., editors, Interface 2001. Interface. in press.
Wiesman, F., Roos, N., and Vogt, P. (2001). Automatic ontology mapping for agent communication. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors, Proceedings of the 13th
Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001), pages 291-298.
Winands, M., Uiterwijk, J., and Herik, H. v. d. (2001a). Combining proof-number search with
alpha-beta search. In Kröse, . B., de Rijke, M., Schreiber, G., and van Someren, M., editors,
Proceedings of the 13th Belgium-Netherlands Conference on Artificial Intelligence (BNAIC 2001),
pages 299-306.
Winands, M., Uiterwijk, J., and Herik, H. v. d. (2001b). The quad heuristic in lines of action.
ICGA Journal, volume 24(1): pages 3-14.
Ypma, A. (2001a). Learning methods for machine vibration analysis and health monitoring. PhD
thesis, Pattern recognition group, TU Delft, The Netherlands. ISBN 90-9015310-1.
Ypma, A. (2001b). Learning methods for machine vibration analysis and health monitoring. PhD
thesis, Delft University of Technology.
Ypma, A., Baunbaek-Jensen, O., Melissant, C., and Duin, R. (2001a). Health monitoring with
learning methods. In Bischof, H. and Hornik, K., editors, Proc. ICANN 2001, Int. Conf. on Artificial
Neural Networks, pages 541-546.
Ypma, A., Kleiman, R. J., Valk, J., and Duin, R. (2001b). Monisom-a system for machine health
monitoring with neural networks. In Proceedings BNAIC 2001, pages 495-496.
Ypma, A., Kleiman, R. J., Valk, J., and Duin, R. P. W. (2001c). MONISOM-a system for machine
health monitoring with neural networks. In Proceedings of 13th Belgian-Dutch Conference on
Artificial Intelligence BNAIC’01, October 25-26, Amsterdam, pages 495-496.
Ypma, A. and Koopmans, H. (2001). Automatic fault detection in rotating equipment with neural
networks (in Dutch). Drive and control, pages 32-36.
Ypma, A., Melissant, C., Baunbaek-Jensen, O., and Duin, R. (2001d). Health monitoring with
learning methods. In Proc. of ICANN’01, Vienna, August 22-24, pages 541-546.
Ypma, A., Speekenbrink, O., and Duin, R. (2001e). The use of context in dynamic pattern
recognition applications. In Proceedings BNAIC 2001, pages 307-314.
Ypma, A., Speekenbrink, O., and Duin, R. P.W. (2001f). The use of context in dynamic pattern
recognition applications. In Proceedings of 13th Belgian-Dutch Conference on Artificial Intelligence BNAIC’01, October 25-26, Amsterdam, pages 307-314.
Zoeter, O. and Heskes, T. (2001). Hierarchical visualization of dynamic data using constrained
switching kalman filters. In Kröse, B., Rijke, M. d., Schreiber, G., and Someren, M. v., editors, BNAIC
2001, pages 323-324.
Zwaag van der, B. (2001). Handwritten digit recognition: a neural network demo. In Reusch, B.,
editor, Computational Intelligence: Theory and Applications, volume 2206, pages 762-771. Springer
LNCS.
84