Download New Trends in Intelligent Systems and Soft Computing Towards and

New Trends in Intelligent
Systems and Soft Computing
Towards an Increased Role of Natural
Language
Janusz Kacprzyk
Systems Research Institute,Polish Academy of Sciences
Ul. Newelska 6
01-447 Warsaw, Poland
Email: [email protected]
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003,
Granada, Spain
1
Contents
 What is intelligence, from the perspective of




psychology and cognitive science, and of information
technology,
What are „intelligent systems”,
Why is soft computing important,
Why is the use of (quasi)natural language so crucial
in intelligent systems,
Why is Zadeh’s computing with words and
perceptions paradigm viable, intuitive and
constructive,
Some implementation (linguistic database summaries)
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
2
What is intelligence?
Initially: psychology, cognitive science
Different views (an exact definition of intelligence is
probably impossible), for instance:
 an ability to handle complexity and solve problems in
some useful context as, e.g., reaching an agreement,
finding a solution to the quadratic equation,
 an ability to protect the organism from bodily risks
and to satisfy its wants with the least possible chance
of failure,...
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
3
Nature of intelligence
Two basic schools of thought on the nature of
intelligence:
 One general intelligence (Eysenck, Galton, Jensen,
Spearman, ...)
 Multiple intelligences (Gardner, Sternberg,
Thurstone,...)
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
4
One general intelligence
For instance, Eysenck (1982):
There is one general factor governing the level of
intelligence of an individual
„Proof”:
 a high positive correlation (positive manifold)
between tests of cognitive abilities (Spearman, 1904),
e.g., good verbal abilities are usually linked to
mathematical abilities,
 A high correlation of reaction time and IQ, ...
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
5
Multiple intelligence
More than a single type of intelligence, but how many?
For instance:
Gardner (1983): 7 different forms of intelligence:
Linguistic, musical, spatial, bodily, interpersonal,
intrapersonal, and logico-mathematical
Solid biological basis (seven different areas of brain)!
Brain as a major determinant of intelligence!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
6
Multiple intelligence, continued
Sternberg (1985): Social and contextual factors apart
from human abilities!
Two different types of intelligence:
 Analytic (academic), rather formulated by others and
well defined, with all information needed for solution,
have a single answer, ...
 Practical: problems poorly defined, require problem
recognition and formulation, have various acceptable
solutions, require experience, motivation, etc.
„Practical”: very relevant for us!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
7
Multiple intelligence, continued
Thurstone (1938) – 13 different factors:
 Spatial, perceptual, numerical, verbal, memory,
arithmetic reasoning, deductive abilities
Also (e.g. Thurstone, 1938; Guildford, 1967):
 Intelligence is composed of 4 contents, 5 operations
and 6 processes (= 120 combinations of abilities)
Many aspects very relevant to „intelligent systems”!!!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
8
Intelligence in a „machine context”
Turing’s (1950) test:
We have a person, a machine, and an interrogator, in
different rooms. The interrogator is to determine which
of the other two is the person, and which is the
machine.
Turing (1950):
„I believe that in about fifty years' time it will be possible
to programme computers, with a storage capacity of
about 109, to make them play the imitation game so well
that an average interrogator will not have more than 70
% chance of making the right identification after five
minutes of questioning”
So far, no success
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
9
More pragmatic definition
Wiener’s (1894-1964) pragmatic definition:
Intelligence is a process of acquisition and processing
of information for attaining goals
Serves our purpose!
A point of departure for „constructive”, implementable
intelligent systems!
Basically: the perspective adopted here
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
10
Towards intelligent machines
So, let us build a „machine” (hardware,
software,...) that will exhibit such intelligence!
→ Artificial intelligence (term coined by John
McCarthy in 1956)
→ Computational intelligence
→ Machine intelligence
→ Intelligent systems
→ Intelligent systems + soft computing
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
11
Artificial intelligence
What is artificial
intelligence?
A modern book:
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
12
Russell and Norvik’s book on AI

PART I: Artificial Intelligence ... 1

1 Introduction ... 3

2 Intelligent Agents ... 31

PART II: Problem-Solving ... 53

3 Solving Problems By Searching
Problem-Solving Agents, Formulating Problems,Searching For Solutions, Search
Strategies,Constraint Satisfaction Search ...

4 Informed Search Methods ... 92
Best-First Search, Heuristic Functions, Memory Bounded Search, Iterative Improvement,...

5 Game Playing ... 122
Games As Search Problems, Perfect Decisions In Two-Person Games, Imperfect Decisions,
Alpha-Beta Pruning, Games That Include An Element Of Chance, State-Of-The-Art Game
Programs ...
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
13
Russell and Norvik’s book on AI

PART III: Knowledge And Reasoning ... 149

6 Agents That Reason Logically ... 151

7 First-Order Logic ... 185
Syntax And Semantics, Extensions And Notational Variations,Using First-Order
Logic, Representing Change In The World, Preferences Among Actions,

8 Building a Knowledge Base ... 217
Properties Of Good And Bad Knowledge Bases, Knowledge Engineering,
General Ontology

9 Inference In First-Order Logic ... 265
Inference Rules Involving Quantifiers, Generalized Modus Ponens, Forward And
Backward Chaining, Completeness, Resolution: Complete Inference Procedure,

10 Logical Reasoning Systems ... 297
Introduction,Indexing, Retrieval, Unification, Logic Programming Systems,
Theorem Provers, Forward-Chaining Production Systems, Frame Systems,
Semantic Networks, Description Logics,
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
14
Russell and Norvik’s book on AI

PART IV: Acting Logically ... 335

11 Planning ... 337
A Simple Planning Agent,From Problem Solving To Planning, Planning In
Situation Calculus, Basic Representations For Planning,Planning With Partially
Instantiated Operators, Knowledge Engineering For Planning

12 Practical Planning ... 367
Practical Planners, Hierarchical Decomposition, Resource Constraints

13 Planning And Acting ... 392
Conditional Planning, A Simple Replanning Agent, Fully Integrated Planning And
Execution

PART V: Uncertain Knowledge And Reasoning ... 413

14 Uncertainty ... 415
Acting Under Uncertainty,Basic Probability Notation, The Axioms Of Probability,
Bayes' Rule And Its Use, Where Do Probabilities Come From?
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
15
Russell and Norvik’s book on AI
 15 Probabilistic Reasoning Systems ... 436
Representing Knowledge In An Uncertain Domain, The Semantics Of
Belief Networks, Inference In Belief Networks,inference In Multiply
Connected Belief Networks, Knowledge Engineering For Uncertain
Reasoning, Other Approaches To Uncertain Reasoning
 16 Making Simple Decisions ... 471
Combining Beliefs And Desires Under Uncertainty, The Basis Of Utility
Theory, Utility Functions, Multiattribute Utility Functions, decision
Networks, The Value Of Information, Decision-Theoretic Expert
Systems
 17 Making Complex Decisions ... 498
Sequential Decision Problems, Value Iteration, Policy Iteration,
Decision-Theoretic Agent Design, Dynamic Belief Networks, Dynamic
Decision Networks...
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
16
Russell and Norvik’s book on AI
 PART VI: Learning ... 523
 18 Learning From Observations ... 525
A General Model Of Learning Agents, Inductive Learning, Learning
Decision Trees, Using Information Theory, Learning General Logical
Descriptions ...
 19 Learning In Neural And Belief Networks ... 563
How The Brain Works, Neural Networks, Perceptrons, Multilayer FeedForward Networks,
 20 Reinforcement Learning ... 598
Passive Learning In A Known Environment, Passive Learning In An
Unknown Environment, Active Learning In An Unknown Environment,
Exploration, Learning An Action-Value Function, Generalization In
Reinforcement Learning, Genetic and Evolutionary Programming
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
17
Russell and Norvik’s book on AI

21 Knowledge In Learning ... 625
Knowledge In Learning, Explanation-Based Learning, Learning Using Relevance
Information, Inductive Logic Programming,

PART VII: Communicating, Perceiving, And Acting ... 649


22 Agents That Communicate ... 651
Communication As Action, A Formal Grammar For A Subset Of English,
Syntactic Analysis (Parsing), Semantic Interpretation, Ambiguity And
Disambiguation ...

23 Practical Natural Language Processing ... 691
Efficient Parsing, Scaling Up The Lexicon, Scaling Up The Grammar, Ambiguity,
Discourse Understanding ...

24 Perception ... 724
Image Formation, Image-Processing Operations For Early Vision, Extracting 3-D
Information Using Vision, Using Vision For Manipulation And Navigation, Object
Representation And Recognition, Speech Recognition ...
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
18
Russell and Norvik’s book on AI
 25 Robotics ... 773
Tasks: What Are Robots Good For?, What Are Robots Made Of?
Navigation And Motion Planning ...
 PART VIII: Conclusions ... 815
 26 Philosophical Foundations ... 817
The Big Questions, Foundations Of Reasoning And Perception, On The
Possibility Of Achieving Intelligent Behavior, Intentionality And
Consciousness,
 27 AI: Present And Future ... 842
Have We Succeeded Yet?, What Exactly Are We Trying To Do?, What
If We Do Succeed?
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
19
Russell and Norvik’s book on AI
Still in the spirit of traditional AI:
 Emphasis on symbolic computation but (finally!)






some numerical computations too,
Based on strict logical calculi,
Based on an idealized approach to natural language,
Limited use of uncertainty/imprecision calculi,
Little relation to foundational works on intelligence,
Little relation to real needs for useful „intelligent
systems”, in particular for decision support,
Is spite of claims by traditionalists, questionable
„great successes” in terms of implementable
systems.
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
20
What can we („soft people”) do?
So, let us try to:
 Support decision making – crucial, is still a „bottleneck” in
virtually all real life situations.
 Take into account specifics of human beings (notably natural
language!),
 Use most adequate and best tools to solve the problem.
Build an implementable, intelligent decision support system!
Note: Zadeh’s recent papers on a need to a new approach to
decison analysis/support using computing with
words/perceptions
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
21
Zadeh (ca. 1995): a paradigm of computing with
words (CWW), and perceptions (CWP)
Books by Zadeha and Kacprzyk (1999a, b)
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
22
Computing with words and percepttions
 Essence:
For a human being, the only fully natural means of
articulation and communication is natural language
Maybe, in many situations:
instead of traditional computing with numbers (from
measurements) it would be better to compute with words
(from perceptions)?
 a paradigm of computing with words and
perceptions (CWP)
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
23
Computing with words and precisiated
natural language
What is needed for computing with words and perceptions?
A formal representation of linguistic descriptions, relations, etc.
Zadeh (early 1990s?): PNL (precisiated natural language)
PNL: a subset of natural language that is equipped with a
constraint-centered semantics, and associated with a
generalized constraint language
A set of so-called generalized constraints corresponding to
linguistic statements
X isr R
assigns a value
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
24
Various forms of generalized constraints
Examples of constraints:





Equality: X is= R (X=R)
Possibilistic constraint: X is R (R is a possibilistic distribution)
Probabilistic constraint: X isp R (R is a probabilistic distribution)
Usuality constraints: X isu R [usually(X is R)]
Veristic, rough set, etc.
Here: mainly the usuality constraint
 in most, almost all, much more than 50%, ... cases
because we seek some „regularities”, „normal/typical” relations
i.e. those which „usually happen”
For instance, commonsense (world) knowldege
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
25
Decision making and decision support
Point of departure: decision making
Omnipresent!
First formal attempts: a structured problem:



Set of options,
A preference structure (utility function),
A best decision is chosen (optimization)
All this well defined!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
26
Recent trends:
Modern, good, ... decision making (good decisions!)
Decision making process (DMP):





Use of own and external knowledge,
Involvement of various „actors”, aspects, etc.
Individual habitual domains (P.L. Yu),
Non-trivial rationality,
Different paradigms when appropriate.
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
27
Soft approach to systems analysis
For instance:
Peter Checkland’s (1975-99) soft approach to systems
analysis
Deliberative (soft) decision making:
 To perceive the whole picture,
 To observe it from all angles (actors, criteria,...)
 To find a good decision using knowledge and intuition.
Intelligent systems + soft computing!
Most elements may only be expressed in natural language!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
28
Decision making process:
modern approaches
Modern decision making process (involves creative,
strategic, deliberative, etc. decision making):





Recognition,
Deliberation and analysis,
Gestation and enligtenment („eureka!”, „aha”),
Rationalization,
Implementation.
Involves much intelligence!
Implementable only through intelligent systems!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
29
Modern decision making paradigm
(continued):
 Heavily based on data, information and knowledge, and
human specific characteristics (intuition, attitude, natural
language for communication and articulation,...)
 Need number crunching, but also more „delicate” and
sophisticated „intelligent” analyses,
 Heavily relying on computer systems, and capable of a
synergistic human-computer interaction, notably using
(quasi)natural language.
So:
Intelligent decision support systems!
+ soft computing
+ (quasi)natural language
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
30
DSSs – characteristic features:
Emphasis on:








Ill/semi/un-structured questions and problems,
Non-routine, one of a kind answers,
A flexible combination of analytical models and data,
Various kinds of data, e.g. numeric, textual, verbal,...
Interactive interface (e.g. GUI, LUI),
Iterative operation („what if”),
Supporting various decision making styles,
Supporting alternate decision making passes, ...
Intelligent systems!!!
Knowledge based!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
31
Data, information and knowledge
Various definitions:
 data - raw facts;
 information - data in a context relevant to an individual,
team or organization,
 knowledge - an individual’s utilization of information and
data complemented by an unarticulated expertise, skills,
competencies, intuitions, experience and motivations.
So:
 knowledge resides in an individual person and not in a
collection of information (Churchman, 1970s)
 We gain knowledge through communication, personal
interactions and “bouncing ideas” off other people.
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
32
Knowledge:
 Explicit, expressed in words or numbers, and shared as
data, equations, specifications, documents, and reports;
can be transmitted individuals and formally recorded,
 Tacit, highly personal, hard to formalize, and difficult to
communicate or share with others; technical (skills or
crafts), and cognitive (perceptions, values, beliefs, and
mental models).
Both extremely relevant for intelligent systems (e.g. decision
support)!
Notice: Zadeh’s computing with words and perceptions!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
33
Brief history of DSSs:
Starting point:
Mid-1960s: development of IBM 360 and a wider use of
distributed, time-sharing computing
Mid-1960s: MISs (management information systems) first to
provide managers with structured, periodic reports,
Late 1960s-early 1970s: attempts to use analytical models,
first attempts at interactive systems
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
34
Brief history of DSSs:
Early 1980s:
 EISs (executive information systems) that use relational
database, and use predefined screans, and are made by
analysts for executives,
 knowledge-oriented DSSs (use of AI tools),
 group DSSs,
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
35
A technology shift:
Early 1990s:
 Use of relational DBMS techniques,
 Shift from mainframe based to client-server based
solutions,
 Object oriented technology for builing „reusable” systems.
Mid-1990s
:
 Data warehouses and on line analytical processing
(OLAP) tools,
 Web based and Web enabled systems
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
36
Basic types of DSSs:
 Data driven,
 Communication driven and group DSSs,
 Document driven,
 Model driven,
 Knowledge driven,
 Web based and interorganizational.
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
37
Specifics of DSSs:







Data Driven DSSs - emphasize access to and manipulation of internal company
data and sometimes external data. Low level: simple file systems with query and
retrieval tools, then data warehouses, finally with On-line Analytical Processing
(OLAP) or data mining tools.
Communications Driven DSSs - use network and communications technologies to
facilitate collaboration and communication,
Group GDSSs - interactive, computer-based systems that facilitate solution of
unstructured problems by a set of decision-makers working together as a group.
Document Driven DSSs - integrate a variety of storage and processing technologies
for a complete document retrieval and analysis; documents may contain numbers,
text, multimedia.
Model Driven DSSs -emphasize access to and manipulation of a model, e.g.,
statistical, financial, optimization and/or simulation; use data and parameters, but are
not usually data intensive.
Knowledge Driven DSSs – interactive systems with specialized problem-solving
expertise consisting of knowledge about a particular domain, understanding of
problems within that domain, and "skill" at solving some of these problems.
Web based DSSs – computerized system that deliver decision support related
information and/or tools to a manager/analyst using a "thin-client" Web browser
(Explorer); TCP/IP protocol!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
38
Types of DSSs and the role of Web
mining (3):
 Data Driven DSSs - emphasize access to and manipulation of
internal company data and sometimes external data. Low
level: simple file systems with query and retrieval tools, then
data warehouses, finally with On-line Analytical Processing
(OLAP) or data mining tools.
Tendency: „intelligent”, soft computing and natural language
Examples:
Fuzzy querying, also over the Internet: FQUERY for Access
(Kacprzyk and Zadrożny, 1996 - ...),
Linguistic data summaries (Kacprzyk and Zadrożny, 1998 -...)
Fuzzy logic + OLAP (Laurent, 2001)
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
39
Concept of a linguistic data(base) summary
Yager’s approach [Yager(1982, ...), ...., Kacprzyk and
Yager(2001)]:
- V - a quality of interest, e.g. salary in a database of workers,
- Y={y1,...,yn} – a set of objects (records) that manifest V, e.g.
the set of workers; V(yi) are values of quality V for object yi.
A linguistic summary of a data set consists of:

a summarizer S (e.g. young),
 a quantity in agreement Q (e.g. most),
 truth (validity) T - e.g. 0.7,
e.g.,
T(most of employees are young)=0.7
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
40
Yager’s linguistic data(base) summaries:
Yager’s concept of a linguistic summary:
 Intuitive,
 Simple,
 Extendable (Kacprzyk and Yager, 2001),
 Implementable, ...
But:
 Concerns relational (numerical) databases!
 structured data!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
41
Example of implementation
A small computer retailer in South Poland:
Owner:
must make sophisticated decisions concerning:
 number of employees on Saturday,
 type of advaertisement,
 Commisions
But: very busy
 Simple summaries, in natural language!
 Inexpensive technology, add-in without any „touching” his
database!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
42
Example:...
Relations between commision and type of product:
About 1/2 of sales of network elements is with a high ommission
Much sales of accessories is with a high commission
Much sales of elements is with a low commission
About 1/2 sales of software is with a low commission
About 1/2 sales of computers is with a low commision
So:
 No problem with accessories and network elements,
 Critical are: elements, software and computers!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
43
Extensions (external data from WWW)
Own database only!
But: a company operates in an environment (e.g. weather)
So, e.g., lrelations between group of products, time of sale,
temperature, precipitacion, and type of customers:
Very few sales of software in hot days to individual customers
About 1/2 of sales of accessories in rainy days on weekends by the
end of the year
About 1/3 of sales of computers in rainy days to individual
customers
Next step: semi-structured weather info (text forecasts from a
local newspaper, SMS messages from a local provider) – local
info!
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
44
Conclusions
My purpose:
 To present and advocate an urgent need for intelligent




systems,
To advocate the need to employ a broader perspective on
intelligence and intelligent systems than it is customary in
traditional artificial intelligence,
To advocate the need to more adequately, effectively and
efficiently deal with natural language,
To sketch Zadeh’s computing with words and perceptions
as a viable, simple and intuitive alternative for the above,
To show an ideaq of an implemented „intelligent” system
for supporting decisions in a company.
Seminar on New Trends in Intelligent Systems
and Soft Computing October 2-3.2003, Granada,
Spain
45