Download Keynotes - IEEE Computer Society

Keynote I
How Deep Is Deep and What’s Next
in Computational Intelligence?
Prof. Lambert Schomaker
University of Groningen, Netherlands
Abstract
We are currently experiencing the third wave of neural-network research. For a researcher who joined
in on the second wave, i.e., post Rosenblatt, about 1987, one would expect happiness with the current
successes in deep learning ('See? We told you so!'). Indeed, the presence of big data and sufficient
computing resources have resulted in exciting progress. However, it is also time for a critical evaluation.
Rather than constituting the long-heralded computational intelligence, deep learning is unfortunately
again and still mainly concerned with intelligent humans spending expensive labor hours on training,
retraining and tinkering network architectures. Experiments are concerned with closed data sets, such that
even the results of a thorough k-fold evaluation are not a good predictor for performance in the real
world. At the same time, human cognition can handle many problems that represent 'one-shot learning',
solving new puzzles without any training sample. Also, with proper feature schemes and distance
functions, even nearest-neighbor and nearest-mean classifiers have an attractive performance level in big
data, with the additional advantage that training is trivial. I will illustrate these insights on the basis of our
experience with a 24/7 learning system for retrieval of words in massive historical manuscript collections:
Monk.
Biography: Lambert Schomaker (1957) is a full professor in Artificial Intelligence at the University of
Groningen since 2001. His main interest is in pattern recognition and machine learning problems. He has
contributed to over 150 peer-reviewed publications in journals and books (h=39/Google Citations). His
work is cited in 23 patents. In recent years his focus is on continuous learning systems and bootstrapping
problems, where learning starts with very few examples. Professor Schomaker is a senior member of
IEEE, member of the IAPR and is a member of a number of Dutch research programme commitees in eScience (NWO), Computational Humanities (KNAW), Computational science and energy
(Shell/NWO/FOM). He is the chairman of the international Unipen foundation for benchmarking of
pattern-recognition systems and director of DeepLearn24 BV, a company that has as its ambition to
translate current insights from computational intelligence to real-world applications.
xx
Keynote II
Handwriting and Speech Recognition:
From Bayes Decision Rule to Deep Neural Networks
Prof. Hermann Ney
RWTH Aachen University, Germany
Abstract
The last 40 years have seen a dramatic progress in machine learning and in statistical methods for
handwriting and speech recognition (and for other tasks in human language technology like machine
translation). Most of the key statistical concepts for human language technology had originally been
developed for handwriting and speech recognition. Examples of such key concepts are the Bayes decision
rule for minimum error rate and probabilistic approaches to acoustic modelling (e.g. hidden Markov
models) and language modelling. Recently the accuracy of handwriting and speech recognition could be
improved significantly by the use of artificial neural networks (ANN), such as deep feedforward multilayer perceptrons and recurrent neural networks (incl. long short-term memory extension). In addition,
from the architectural point of view, there are a number of novel concepts like CTC (connectionist
temporal classification) and the attention based mechanism, both of which try to replace the conventional
HMM by ANN based structures. We will discuss these issues in detail and how they fit into the statistical
approach.
Biography: Hermann Ney is a full professor of computer science at RWTH Aachen University,
Germany. His main research interests lie in the area of statistical classification, machine learning and
human language technology with specific applications to speech recognition, machine translation and
handwriting recognition. In particular, he has worked on dynamic programming and discriminative
training for speech recognition, on language modelling and on phrase-based approaches to machine
translation. His work has resulted in more than 700 conference and journal papers (h-index 83, 35000
citations; estimated using Google scholar). He and his team contributed to a large number of European
(e.g. TC-STAR, QUAERO, TRANSLECTURES, EU-BRIDGE) and American (e.g. GALE, BOLT,
BABEL) joint projects.
Hermann Ney is a fellow of both IEEE and ISCA (Int. Speech Communication Association). In 2005,
he was the recipient of the Technical Achievement Award of the IEEE Signal Processing Society. In
2010, he was awarded a senior DIGITEO chair at LIMIS/CNRS in Paris, France. In 2013, he received the
award of honour of the International Association for Machine Translation. In 2016, he was awarded an
advanced grant of the European Research Council (ERC).
xxi
Keynote III
Online Handwriting Recognition: Past, Present and Future
Prof. Masaki Nakagawa
Tokyo University of Agriculture and Technology, Japan
Abstract
It is about 50 years since RAND tablet was invented. Methods to recognize online handwriting started
from heuristic methods, then proceeded to DP-matching, Time-delayed NN, HMM, MRF, etc. and then
moving to Deep NN. The combination with offline recognition and linguistic and geometric contexts has
made the recognition more robust. We experienced two small peaks of online recognition in the 80s and
the 90s but did not expand to a larger success. Due to a steady progress in handwriting recognition and the
success of touch-sensitive smart phones/tablets, however, online recognition has now established
platforms for direct pointing and direct manipulation. In this talk, I will first summarize the history of
online handwriting recognition, the merits of handwriting input, and the requirements for recognition and
user interfaces. Then, I will present the common architecture of recent online handwriting recognition
systems on smart phones/tablets. Finally, I will present the future applications of computer-assisted and
automated marking of handwritten answers, which is challenging for pattern recognition, human interface
and artificial intelligence.
Biography: Masaki Nakagawa is a professor with the Department of Computer and Information
Sciences, Tokyo University of Agriculture and Technology. He has been working on handwriting
recognition, pen-based user interfaces and applications — especially educational applications. Since
1980s, he has been collaborating with many companies and has contributed to develop handwriting
recognizers for real commercial use. In 2011, he founded a start-up iLabo, which now sells the best
handwriting recognizers for touch-based smart phones, tablets and so on in Japan. In 2012, iLabo was
selected as one of the 100 most promising ventures in Japan by Nikkei Business. In 1990, he also
introduced User Interfaces for tablet devices and developed several educational applications using various
sizes of tablets. His U.S. patents to scroll the window in proportion to the pen speed, called “touch scroll”,
were sold from his university to a company for the highest amount among all the Japanese universities in
the fiscal year 2010. He is also working on historical document processing to read excavated documents
from the Heijo palace (the capital in the 7th century) in Nara, Japan, and to read Chu Nom documents in
Vietnam. He received the Minister of Education and Science award of Japan this year. He is a fellow of
IAPR (International Association of Pattern Recognition), IEICE (Institute of Electronics, Information and
Communication Engineers, Japan) and IPSJ (Information Processing Society of Japan).
xxii