Download Expert systems

Prof.dr.sc. Bojana Dalbelo Bašić
Faculty of Electrical Engineering and Computing
Department of Electronics, Microelectronics, Computer and
Intelligent systems
www.zemris.fer.hr/~bojana
[email protected]
Expert systems
© Bojana Dalbelo Bašić
FER
travanj, 2008.
1. INTRODUCTION

In the early days of AI (50s and 60s of the 20th century),
people have tried to develop sophisticated inference
techniques

These techniques did not rely on knowledge. The goal in
this period was to develop software systems for general
problem solving (GPS)

The failure to develop a general problem solving system
gave rise to a different approach: the developement of
knowledge based systems
2. RULE-BASED SYSTEMS
How can one most easily represent human knowledge?
 A vast majority of human knowledge (expert knowledge)
can be represented using if – then rules

If the temperature of the patient is greater than 38°C
then medications that lower the temperature should be
prescribed

If the traffic light is red then stop

These rules are called production rules
Production systems
2. RULE BASED SYSTEMS

An important step in successful solving of AI problems →
reducing the scope or domain of the problem
Expert
knowledge

General
knowledge
Expert knowledge (domain knowledge) is knowledge
specific for a given domain (e.g., medicine, finances,
chess etc.), unlike general problem solving knowledge
Expert systems
3. BASIC SCHEMA OF RULE-BASED SYSTEMS
Expert systems are a branch of AI that uses knowledge
specialized for a certain problem domain to solve a
problem at the level of a human expert
 Basic schema of a rule-based system – expert
system.

Facts
Knowledge base
Expert
knowledge
Inference engine
User
User
IMPORTANT – separating
domain knowledge from
the inference procedure
Expert system
3. BASIC SCHEMA OF RULE-BASED SYSTEMS
Facts
Knowledge base
Expert
knowledge
Inference engine
User
User
Inference engine ->
EXPERT SYSTEM SHELL
is the expert system
without the rules
Expert system
4. INSPIRATION – THE COGNITION PROCESS
Beginning of production systems:

Logician E. Post 1943. suggested production rules in
formal theoretical computer science. Strength of PRs is
equivalent to the Turing machine: production systems can
be used for all solvable problems

1972. Paper «Human problem Solving», Newell and
Simon, 920 pages. Human cognition can be modeled by if
then rules. One rule – a chunk of knowledge.
4. INSPIRATION – THE COGNITION PROCESS

A model of the human cognition process is imitated in the
production system

Senses stimulate the brain with stimuli. Such stimuli
activate knowledge in our long term memory (in a prod.
system those are if – then rules)

Short term memory – temporary storage of knowledge
for the period of when then inference procedure is
executed – represents the number of «chunks of
knowledge» that can be considered simultaneously (rules which can be activated at the same time)
4. INSPIRATION – THE COGNITION PROCESS

Cognition process – finds rules that will be activated by
stimuli – (in a prod. system this is the inference engine, it
finds rules and resolves conflicts)

Newell i Simon – set up the basic model of modern
production systems

1970 – the paradigm of knowledge based systems is
widely accepted
5. PRODUCTION SYSTEMS - NOMENCLATURE
Synonyms:
 Rule based systems
 Production systems
 Expert systems (In a classical sense these are systems
that contain expert knowledge, obtained through a series
of conversations between the knowledge-base engineer
and a domain expert)
Humanexpert
Knowledge
-base
engineer
Expert
system
knowledgebase
This procedure is the “bottleneck” in building an expert
system
6. Application

Production systems - an alternative to conventional
algorithmic programming
Some areas of application for expert systems:
 various kinds of medicial diagnoses (internal medicine,
pulmology, infectious diseases etc.)
 diagnoses of faults in complex electronic and
electromechanical devices
 many applications in legal systems (e.g., planning income
and assets to minimize taxes)
 applications in finances and banking (e.g., evaluation of
credit standing for an individual or a company)
6. Application
An example of application: medicine
 Diagnosing and determining therapy based on symptoms

knowledge acquired
in medical school
Knowledge acquired through
experience – heuristic knowledge
doctor inference
Expert knowledge – human knowledge:
 human – expert
 influence of fatigue and emotions on an assesment
 duration of the knowledge is limited
7. Building a system
attempt of formalising
expert knowledge
using a computer

EXPERT
SYSTEMS
Specialized (expert) knowledge gathered and organised for
specialized, domain limited use
Humanexpert

Knowledge
-base
engineer
Expert
system
knowledgebase
This step can be automated
8. Examples of early ES
Examples of early successful expert systems:

DENDRAL – discovering molecular structure of
substances based on spectrometry

MYCIN – diagnosing infectuous blood diseases

PROSPECTOR – estimating existence of ore pased on
geographical characteristics

XCON/R1 – DEC a system for choosing components
when building a complex computer system
9. PRODUCTION SYSTEMS
Expert systems – advantages:
 built based on the knowledge of more than one expert expert knowledge repository for a domain
 can be easily multiplied an widely used
Rule based systems are based on:
 production rules (prodrules)
If ……………. then …………………
Condition (state)
premise, antecedent
Action
conclusion,consequent
9. PRODUCTION SYSTEMS
condition-action
If it’s raining then take an umbrella
 If car motor doesn’t start and
the lights don’t work
then the problem is the battery or the cables

If the patient is in a chronically generally bad state and
gender is female and
premise age < 30 and
conclusion
patient has symptom A and
biochemical test shows value C
 then the diagnosis is autoimmune chronic hepatitis

9. PRODUCTION SYSTEMS

stateconsequence
If John is a student of FER
then John is a student of the University in Zagreb
Rule based
based systems
systems are
are used
used very
very
Rule
often because
because the
the majority
majority of
of human
human
often
knowledge can
can be
be represented
represented using
using
knowledge
production rules
rules (if
(if –– then).
then).
production
10. PRODUCTION SYSTEMS - LOGIC

In logic: implication A → B (If A is true then B is true)
Logical formulae

In production rules: If A then B
actions, commands, equations, programs …
Production systems are a combination of two kinds of
knowledge:
MODUS
[1] factual (A is true)
A
PONENS
[2] conditional (rule exists)
A→B
(B is true) B

11. Definition of the PRODUCTION SYSTEM
Definition – PRODUCTION SYSTEM is defined by:
1. SET OF PRODUCTION RULES
[2]
(production is a pair condition - action)
Knowledge –
1. WORKING MEMORY (list of facts
base
containing the description of the current
[1]
state of the world in the inference
procedure)
1. MATCH – ACT CYCLE
This is a control mechanism consisting of a sequence of
actions: pattern matching – choosing the rule to apply by
resolving conflicts – applying the rule.
 In a more specific sense the knowledge base can refer
only to the set of production rules.
Knowledge base is also called the rule base.
12. SCHEMA OF PRODUCTION SYSTEM
FUNCTION

Schema of production system function
2. Set of conflicted
rules
PRODUCTION
RULES
WORKING
MEMORY
3. Choosing a
rule
1. Matching
4. Applying the rule
adding new facts
and rules
12. SCHEMA OF PRODUCTION SYSTEM
FUNCTION
Description of a production system
1.
SET OF PRODUCTION RULES –
1 PR (also called a condition-action pair) -> 1 chunk of
knowledge from the problem domain.
1.
WORKING MEMORY contains a set of facts .
Facts (samples) from WM -> current state in the
procedure of solving the problem.
12. SCHEMA OF PRODUCTION SYSTEM
FUNCTION
1.
MATCH – ACT cycle – is the control structure ofof a
production system
Samples from the working memory are compared with
the conditional parts of the rules from the rule base.
When a conditional part of a rule matches samples from
the working memory, then that rules is enabled, and it is
put into the set of conflicted rules. At the start of the
cycle the set of conflicted rules is empty.
12. SCHEMA OF PRODUCTION SYSTEM
FUNCTION
After the matching procedure the set of conflicted rules
can contain one or more enabled rules.
If the set of conflicted rules is empty then the procedure is
finished.
If there is more than one enabled rule, then ONE rule is
chosen and applied (the rule fires).
Choice of the rule that fires is called conflict resolution.
Firing of a rule means applying the right hand side of the
rule – changing the contents of the working memory.
After a rule has fired the control cycle repeats with the
modified contents of the working memory.
12. SCHEMA OF PRODUCTION SYSTEM
FUNCTION
Conflict resolution can be very simple – e.g. the first
enabled rule fires, but it can also be complex, i.e. it can
include a heuristic to choose which rule fires
 Note: When a production rule is applied it can also be
said that the rule fires.
Example
 A simple production system for sorting strings which
consist of letters a,b and c
SET OF PRODUCTION RULES:
1. ba → ab
2. ca → ac
3. cb → bc

Example
1.
2.
3.
Iteration
ba → ab
ca → ac
cb → bc
WORKING
MEMORY
0
cbaca
1
cabca
2
acbca
3
acbac
4
M
a
t
c
h
i
n
g
Set of
conflicted
rules
The rule that is
chosen to
fire
3, 1, 2
1
2
2
3, 2
2
1, 3
1
acabc
2
2
5
aacbc
3
3
6
aabcc
∅
Stop
Example
1.
2.
3.
Iteration
ba → ab
ca → ac
cb → bc
WORKING
MEMORY
0
cbaca
1
cabca
2
acbca
3
acbac
4
M
a
t
c
h
i
n
g
Set of
conflicting
rules
The rule that is
chosen to
fire
3, 1, 2
1
2
2
3, 2
2
1, 3
1
acabc
2
2
5
aacbc
3
3
6
aabcc
∅
Stop
14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

A sequence made of several conclusions that link the
starting description of the problem with the solution is
called a CHAIN
Forward chaining
SOLUTION/
GOAL/
CONCLUSION
KNOWN
DATA
Backward chaining

Inference procedure, i.e. automatic advancement along
the chain is called chaining.
14. INFERENCE PROCESS IN RULE-BASED SYSTEMS
There are two main ways of advancing towards
conclusions:
1. FORWARD CHAINING

→ starting with known data and advancing toward a
conclusion (also called forchaining, data driven processing,
event driven, bottom-up, antecedent, pattern directed processing ⇔
reasoning)
1.

BACKWARD CHAINING
→ choosing a possible conclusion (hypothesis) and trying
to prove that it is valid by finding valid evidence.
(also called backchaining, goal driven processing, goal driven, topdown, consequent, expectation driven processing)
14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

Forward chaining - when there is a small amount of data
and a large number of possible solutions
(for problem domains including synthesis :
for designing, planning, scheduling, supervision and
diagnostics of real time systems)
KNOWN
DATA
GOAL
14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

Backward chaining a reasonable choice whene there
aren’t many possible conclusions and the amount of data
known is large (Probleme dijagnosing, classification)
KNOWN
DATA
GOAL
The choice of inference method depends on the
properties of the problem domain and the way the expert
reasons
 It is also possible to implement bidirectional inference.

14. INFERENCE PROCESS

Production rule:
CONDITION → ACTION
LEFT
HAND SIDE
of the rule
(LHS)
RIGHT
HAND SIDE
of the rule
(RHS)
15. FORWARD CHAINING
Forward inference starts with known data, an initial
problem description (e.g., a set of logical axioms,
disease symptoms, data that needs to be interpreted etc.)
which is stored in working memory.
 For each rule it is checked if the given data satisfy the
premises of the rule. If the rule is satisfied it can be
applied, and new data can be derived, which can be used
to satisfy other rules. The procedure of checking the rule
is called rule interpretation.

Nomenclature:
 rule interpretation ≡ forward inference ≡ forward chaining
15. FORWARD CHAINING
INFERENCE ENGINE is a contorol mechanism that
performs the following steps when executing the forward
chaining procedure.
1.
Matching – the control cycle starts by matching the state
in working memory with LEFT sides of production rules.
1.
Conflict resolution – if during matching more than one
enabled rule was found , then the rule with the higher
priority is chosen.
15. FORWARD CHAINING
1.
Application (firing) of a rule – applying the right hand
side of a production rule results in :
 a new fact added to working memory (a new current
state of the world) or,
 A new rule added to the rule base which can be
considered in future cycles
15. FORWARD CHAINING
RULE BASE
MATCHING
enabled
rules
CONFLICT
RESOLUTION
WORKING
MEMORY
chosen
rules
APPLYING
RULES
15. FORWARD CHAINING
Example


Rule base for determining type of fruit
Parametres (i.e., variables) and their values are:
Shape:
elongated
Diameter:
circular
rounded
Surface:
Color:
Seed_number
Seed_type
< 10 cm
Fruit_type:
smooth
coarse
> 10 cm
vine
tree
Fruit:
banana
green
watermelon
yellow
melon
brown-yellow
cantaloupe
red
apple
blue
apricot
orange
cherry
>1
peach
=1
plum
multiple
orange
15. FORWARD CHAINING
RULES
1
IF
Shape = elongated & Color = green or yellow
THEN
2
IF
Fruit = banana
Shape= circular or rounded & Diameter > 10 cm
THEN
3
IF
Fruit_type = vine
Shape= circular & Diameter < 10 cm
THEN
4
IF
Fruit_type = tree
Seed_number = 1
THEN
5
IF
THEN
6
IF
THEN
Seed_type = bony
Seed_number > 1
Seed_type = multiple
Fruit_type = vine & Color = green
Fruit = watermelon
15. FORWARD CHAINING
7
IF
Fruit_type = vine & Surface = smooth & Color= yellow
THEN
8
IF
THEN
9
IF
Fruit = melon
Fruit_type = vine & Surface = coarse & Color= brown-yellow
Fruit = cantaloupe
Fruit_type = tree & Color= orange & Seed_type = bony
THEN
10
IF
Fruit = apricot
Fruit_type = tree & Color= orange & Seed_type = multiple
THEN
11
IF
THEN
12
IF
Fruit = orange
Fruit_type = tree & Color = red & Seed_type = bony
Fruit = cherry
Fruit_type = tree & Color = orange & Seed_type = bony
THEN
13
IF
Fruit = peach
Fruit_type = tree & Color = yellow or green or red & Seed_type =
multiple
THEN
14
IF
THEN
Fruit = apple
Fruit_type = tree & Color = blue & Seed_type = bony
Fruit = plum
15. FORWARD CHAINING
Known data: Diameter = 2 cm
Shape = circular
Seed_number= 1
Color = red
Conflict resolution strategy: the rule that is listed first

Working memory
Set of conflicting Rule that
rules
fires
0
Diameter = 2 cm
Shape = circular
Seed_number = 1
Color = red
3,4
3
1
Fruit_type: tree
3, 4
4
2
Seed_type = bony
3, 4, 11
11
3
Fruit = cherry
3, 4, 11
STOP
18. BACKWARD CHAINING
Backward chaining
 fundamentally different from forward chaining, even
though both inference methods match and apply rules
 starts with a desired goal (hypothesis) and determines if
the existing facts support proving the goal

The system starts with an empty list of facts. A list of
goals is given for which the system tries to determine the
values.
18. BACKWARD CHAINING
Steps:
1. Form a stack containing the goals (hypotheses) that we
want to prove.
1.
2. On top of the stack is the goal which needs to be proven.
If the stack is empty STOP
3. Find all rules which can derive the current goal (find rules
which have the desired goal on their right hand side)
18. BACKWARD CHAINING
4. For each of such rules do the following:
4.a. If all premises of the rule are satisfied (each premise
parameter has the correct value in the working memory)
then apply the rule, i.e. The RIGHT hand side of the rule
i.e. add the conclusion to the working memory. Do
not consider any more rules for that goal – its value
has just been derived when the rule fired.
If the goal was a top goal then remove the goal from
the stack & go back to step 2.
If the goal was an intermediate goal then remove the
goal from the stack & return to the temporarily
suspended goal
18. BACKWARD CHAINING
4.b. If the value of the parameter in working memory is not
the same as the one in the premise
then do not apply that rule
4.c. If premises of the rule are not satisfied because one of
the parameter values of the premise is not in working
memory
then find a rule whose right hand side derives the value
of that parameter
If at least one such rule exists
then set that parameter as an intermediate goal, i.e.
set that parameter on top of the stack &
go to step 2
18. BACKWARD CHAINING
4.d. If step(c) cannot find a rule which derives the required
value of the parameter
then ask the user for the value of the parameter & add
the value to working memory.
Go to step 4a and consider the next premise of the
current rule
5. If all rules that could derive the current goal have been
checked and if none of them could derive the goal
then goal remains underived. Remove the goal from the
stack and go to step 2.
18. BACKWARD CHAINING
Example of backward chaining
 Let our starting hypothesis be that we are seeking a fruit.

hypothesis = (fruit). Knowing that it is a cherry let’s
follow the backward chaining procedure to see if it is
possible to derive that the fruit is a cherry.

In the second step we consider ALL rules which derive
the value of fruit
18. BACKWARD CHAINING
Goal (stack)
WORKING
MEMORY
(fruit)
(fruit)
shape
circular
=
(R1)
(fruit_type
Is the current goal;
the goal fruit is
temporarily
suspended)
(fruit_type
fruit)
diameter <
10 cm
(fruit)
fruit_type =
stablo
(R2)
SET OF CONFLICTING
RULES
Parameter which
being checked
is
1,6,7,8,9,10,11,12,
13,14
shape is not in WM or
on the RHS of a rule - ?
shape?
6,7,8,9,10,11, 12,13,14
fruit_type – RHS of rules
2i3
2,3
Shape – contained in
WM – OK
Diameter – not in WM or
on the RHS of a rule
?diameter?
3-FIRES,
6,7,8,9,10,11, 2,13,14
both premises of R3 are
in WM
6,7,8,9,10,11, 12,13,14
first premise of R6 is not
satisfied
(fruit)
(R6)
7,8,9,10,11, 12,13,14
first premise of R7 is not
satisfied
(fruit)
(R7)
8,9,10,11, 12,13,14
first premise of R8 is not
satisfied
18. BACKWARD CHAINING
(fruit)
(fruit)
color = red
(fruit)
….
(fruit)
…
(R8)
9,10,11, 12,13,14
fruit_type is in WM;
2nd premise is not in
WM and not at RHS of
any of the rules
? color ?
(R9)
10,11, 12,13,14
Second premise of R10
is not satisfied
(R10)
11, 12,13,14
..
…
…
11 - FIRES
fruit = cherry
? … ? – query to the user. All rules removed from the top of the stack for which it is
not specified that they fire were not satisfied. Chaining goes backward until until the
right hand side of the rule appears on the left hand side of another rule.
19. EXAMPLE OF FORWARD CHAINING
Production rule set:
1.
2.
3.
4.
5.
6.

p ∧ q → goal
r∧s→p
w∧r→ q
t∧u→q
v→s
start → v ∧ r ∧ q
Conflict resolution strategy: choose the rule from the set
of conflicted rules which fired first or never fired
19. EXAMPLE OF FORWARD CHAINING

Execution
Working memory
Set of conflicting Rule that
rules
fires
0
start
6
6
1
start, v, r, q
6,5
5
2
start, v, r, q, s
6,5,2
2
3
start,v, r, q, s ,p
6,5,2,1
1
4
start,v, r, q, s ,p cilj
6,5,2,1
STOP
19. EXAMPLE OF FORWARD CHAINING

The space which is searched
arc
e
S
rt
a
t
s
q
r
v
goal
s
p
h
e
r
i
d
o
ct i
n
19. EXAMPLE OF FORWARD CHAINING

The space which is searched
start
r
v
t
Search direc
q
s
p
goal
ion
20. EXAMPLE OF BACKWARD CHAINING
Set of production rules
1.
2.
3.
4.
5.
6.
p ∧ q → goal
r∧s→p
w∧r→ q
t∧u→q
v→s
start → v ∧ r ∧ q
20. EXAMPLE OF BACKWARD CHAINING
1. p ∧ q → goal
2. r ∧ s → p
3. w ∧ r → q
4. t ∧ u → q
5. v → s
6. Start → v ∧ r ∧ q
SET OF
CONFLICTING
RULES
Parametar currently
being checked
goal
1
p–not in WM
p, goal
2
r–not in WM
GOAL
(STACK)
WORKING MEMORY
r, p, goal
start
6 – FIRES
?start?
p, goal
start, v, r, q
2
r is in WM, s is not
s, p, goal
start, v, r, q
5 - FIRES
v is in WM
p, goal
start, v, r, q, s
2 - FIRES
r, s – in WM
goal
start, v, r, q, s, p
1 – FIRES
p, q in WM
start, v, r, q, s, p, goal
STOP
20. EXAMPLE OF BACKWARD CHAINING

The space which is searched
goal
Search
direction
q
p
u
w
r
t
s
v
start
When backward chaining the systems searches for
evidence (support) for the goal and intermediate goals.
This kind of search is suitable for diagnostics and
classification.
21. SUMMARY
backward chaining – depth first search
forward chaining – breadth first search

MYCIN (Shortliffe, 1976) – example of backward chaining

XCON/R1 (MCDermott, DEC, 1978) – example of forward
chaining
Bibliography and links
Jackson, P., Introduction to Expert Systems – 3rd
Ed., Addison-Wesley, 1999.
Giarratano, J., Expert Systems: Principals and
Programming, PSW-Kent, 1989.
Examples of ES and ES shells
http://www.zemris.fer.hr/predmeti/is/StudentskiRadovi.
html
Download Jess:
http://herzberg.ca.sandia.gov/jess/index.shtml
CLIPS Expert System Shell
http://www.ghg.net/clips/CLIPS.html
FuzzyJ Website
http://www.iit.nrc.ca/IR_public/fuzzy/fuzzyJToolkit2.html