Download finalReport - Suraj @ LUMS

Teaching Computer Science – CS501
State Space Search – Final Report
Submitted to:
Dr. Ashraf Iqbal
Teaching Assistant:
Mohammad Sadiq
Submitted by:
Talha Bhatty
2003-02-0201
Contents
Contents .............................................................................................................................. 2
1. Abstract ....................................................................................................................... 3
2. Panoramic view ........................................................................................................... 4
3. Critical Stages ............................................................................................................. 5
4. Critical operations ....................................................................................................... 6
5. Feedback mechanism .................................................................................................. 7
5. Compared to what/ connected to what ........................................................................ 9
6. How not to teach this class........................................................................................ 10
7. Peer evaluation and feedback.................................................................................... 11
8. Slides/material used in the lecture ............................................................................ 12
2
1. Abstract
The lecture will include some basics of artificial intelligence, which shall be approached
in a historical framework, identifying the basics of the development of Boolean logic and
Graph theory. The lecture shall be about strategies for State Space Search, which are
techniques used to represent data (Graph theory), and apply either data-driven (forward
chaining) or goal-driven (backward chaining) searches to obtain a path to the desired
result. These algorithms will consist of Backtrack, Depth-First and Breadth-First
searches. Heuristics, and algorithms used for heuristic searching, namely the “Best-First”
search will also be introduced.
3
2. Panoramic view
Artificial Intelligence is a field that is aimed to make a machine capable to imitate
intelligent human behavior. AI draws its roots from Philosophy and particularly the
philosopher Aristotle, and scientists believe that his quantifying of things he saw in a
terms of logic was the birth-place of AI. He said that certain propositions can be said to
be “true” because they are related to other things that are known to be true. So for
instance if we know that “all men are mortal”, and we know that “Socrates is a man”,
then we can conclude that “Socrates is mortal”. After this first philosophical encounter AI
lost most of its charm and only resurfaced with the advent of Boolean logic or more
specifically the vehicle of the modern computer as we knew it was created, man was once
again mesmerized by the possibilities of being able to create (even if just in theory) a
being that would be capable of imitating his behavior. From here on AI split from modern
computer science as we knew it and structures and representations were created to allow
for the storage and use of the data that would enable these machines to think. AI is but a
piece of a very large picture that includes many hardware and software constructs.
4
3. Critical Stages
The critical stages will be addressed progressively thru the course of the lecture, the
foundation will be built in a building-blocks fashion and will comprise of the following
concepts to give the students a proper feel of the subject at hand:
1.
2.
3.
4.
5.
6.
What is Artificial Intelligence?
What are the applications of AI?
What is Knowledge Representation?
What is a State Space?
How do we search a state space?
What techniques can be use to search the state space intelligently?
5
4. Critical operations
The critical operations involved follow the critical stages very closely and will require the
students to grasp the problems involved in the following sections:
1.
What is Knowledge Representation?
The students will be shown by means of various examples how situations varying
from everyday occurrences to complex games can be captured by knowledge
representation in the following structures:
a. Decision trees
b. Predicate calculus. (brief introduction only)
2.
What is a State Space?
Through the use of examples the students will be introduced to the concepts of
state space and will learn how to apply the following to the knowledge they have
previously represented.
a. Goal states, and
b. Solution paths
3.
How do we search a state space?
The students will be shown how the previously created trees can be searched by
two broad categories of search algorithms which will only slightly be elaborated
on:
a. Forward Chaining (Data driven)
b. Backward Chaining (Goal driven)
They will then be shown why the following algorithms that they have previously
studied are not the best solution for AI:
c. Breadth-first search
d. Depth-first search
4.
What techniques can be use to search the state space intelligently?
At this point the students will be in a better position to grasp the subtleties of the
concept of “Heuristics” which is an invaluable aid to solving most AI problems,
after which the lecture shall be concluded.
6
5. Feedback mechanism
Feedback mechanisms shall be inserted at almost every critical point in the lecture to
facilitate teacher-student interaction. Most of these mechanisms will take the form of
questions which the students will be asked to answer while paying full attention to
whatever is being taught on the board. These questions thus might ask about the attributes
of a particular object or the may require the class to perform some calculations or
inferences.
1. What is Artificial Intelligence?
a.
What do we mean by artificial?
(expected answer = Humanly contrived often on a natural model)
b.
What do we mean by Intelligence?
(expected answer = the skilled use of reason)
c.
What does AI then mean?
(expected answer = automation of intelligent behavior)
d.
What is logic, where is this word most often used?
(expected answer = Boolean logic)
2. What are the applications of AI?
(expected answer = expert systems, fuzzy logic, neural networks)
a.
What does a computer game exhibit?
(expected answer = Artificial Intelligence)
b.
How many people use mobile phones or voice dialing?
(expected answer = nod or hand raising)
c.
All these examples are governed by what?
(expected answer = Rules)
3. What is Knowledge Representation?
a.
How many states could there be in a game of knots-&-crosses?
(expected answer = 9x8 = 72, 72x7 = 504 … 362880)
4. What is a State Space?
a.
What good is knowledge if we can’t use it?
b.
What is a goal?
(expected answer = favorable outcome)
c.
What would the goal state be in a game of knots-&-crosses?
(expected answer = Winning board)
d.
In a game of chess?
(expected answer = Checkmate)
e.
In the traveling sales-person problem?
(expected answer = “shortest” path has been found)
f.
When would a parsing problem terminate?
(expected answer = sentence has been successfully analyzed)
7
5. How do we search a state space?
a.
How many ways are there to solve a maze?
(expected answer = 2)
b.
What is Breadth-First and Depth-First search?
(expected answer = class will attempt to describe what they know)
6. What techniques can be use to search the state space intelligently?
(expected answer = evaluate possibilities without having to
investigate every single possibility )
a.
Why would an acceptable solution do?
(expected answer = cost of finding a solution is too high )
b.
How can we use a heuristic to make the state space smaller?
(expected answer = class will attempt to shrink the space)
c.
Which will be the path that has the most wins now?
(expected answer = class will identify on the board)
d.
Why do we use heuristics?
(expected answer = to prune vast state spaces to make searching
possible)
e.
What practical applications can we use heuristics for?
(expected answer = Virus detection, computer problems, games)
8
5. Compared to what/ connected to what
Throughout the course of the lecture the students will continually be reminded of big
picture and how AI is related to their lives by means of examples (for instance the
applications of AI in games such as Counter Strike or Chess, and other software and
hardware products) connecting what they are learning to what occurs in their daily lives
but they don’t realize it. They will also be shown a few things that they may remember
from other courses that they have taken and may use this knowledge to see not only the
similarities but also the differences that exist in the way that artificial intelligence differs
from other forms of computer science. This will be done by constantly reminding the
students that a conceptual understanding is very important before jumping to hardcode a
rule-base and this will keep them interested in the applications of the field to their
personal lives as well as a completely new outlook on data representation and searching
methods which most students would not have seen previously in other courses such as
databases or software engineering.
9
6. How not to teach this class
I think that this topic would not have been taught well, if the teacher tried teaching the
actual algorithms that were used here. Or perhaps tried using a programming language or
pseudo code to talk about the these algorithms, such an endeavor would have been lost on
the students, because they have seen these algorithms numerous times before and seeing
code is never exciting. The teacher should give the students a conceptual understanding
and not bog them down with coding; he should show them how and where these things
are useful by means of examples, and how and where these concepts fit in the big picture,
none of which would surface if he directly taught just the algorithms, and which are
integral to good understanding.
10
7. Peer evaluation and feedback
After the lecture I realized that perhaps my biggest fault was that I had not properly timed
the lecture and was thus unable to complete the lecture in its entirety, this meant the that
conclusion could not be delivered and thus there were certain ambiguities that couldn’t be
tied in. This could have been remedied by cutting down the lecture by omitting a few
concepts that could have been covered in subsequent lectures.
Another thing that was pointed out was that perhaps I hadn’t provided enough of a
challenge to the students and ‘spoon-fed’ them by making the lecture too simple and not
perhaps sparking the interest of some of the students by giving them a taste of what was
to come later on in the class or the rest of the course. This might have be solved by
perhaps showing the class just a small example of a complex AI structure and telling
them that by the end of the course they would also be able to make something like it, or
perhaps better yet, not making something everything as clear as it may have come across
by telling them half the truth and telling them to solve the rest at home.
Some of the students also noted that although the pace of the lecture was not too fast, the
content of the lecture was slightly overwhelming and was too much for the first class.
Which again would have been beneficial to cut the lecture time down by a few minutes so
as not to saturate the students completely.
11
8. Slides/material used in the lecture
The following is the speech that was prepared prior to the lecture and was used by the
lecturer to maintain proper time and a focused class, conducive to a favorable working
environment.
What we must remember is that today’s lecture will provide a conceptual understanding
of SSS and will not be hindered by the subtleties of a programming language or pseudo
code.
What we shall be talking about today is a topic in AI,
these 2 words independently.
What do we mean by artificial?
What is AI? We can analyze
Humanly contrived often on a natural model.
Ok, what do we mean by Intelligence?
The ability to learn or understand or
to deal with new or trying situations: the skilled use of reason (2): the ability to apply
knowledge to manipulate one's environment or to think abstractly as measured by
objective criteria
So the 2 of them together would mean? Scientists have not been able to formalize a
definition exactly. They call it the automation of intelligent behavior (the
capability of a machine to imitate intelligent human behavior). AI draws its roots from
Philosophy
and particularly the philosopher Aristotle, and scientists believe that
his quantifying of things he saw in a terms of logic was the birth-place of AI. He said that
certain propositions can be said to be “true” because they are related to other things that
are known to be true. So for instance if we know that “all men are mortal”, and we
know that “Socrates
is a man”, then we can conclude that “Socrates is mortal”.
What is logic?
Where do we hear that word so often used today in CS?
Yes perhaps the most heard use is in Boolean
logic.
George Boole developed a mathematical symbolic logic (later called Boolean algebra) for
reasoning about categories (i.e., sets) of objects, which is also applicable to manipulating
and simplifying logical propositions.
What are the applications of AI?
12
Think about your everyday life, how many things do see around you that are AI in
progress but you don’t realize it.
Most mobile phones today feature voice dialing, that’s AI isn’t it? So we’ll put
recognition or parsing down.
How many people here have played or perhaps even created a computer game?
Everyone right? What does a computer game exhibit? “Signs of automated
intelligent behavior”! The computer will know exactly where you are and will get quite
unbeatable at higher levels, why? Do your gaming abilities get worse at higher levels? No.
The computer gets more intelligent!
What other examples of AI are there? How many people have heard of expert
systems, fuzzy logic, neural networks, they’re all parts of AI.
All these examples are governed by what? (Yes, they’re governed by rules).
Therefore the most important part about AI will be what? (Capturing these rules by some
means of representation), this makes up one very important part of AI is called
knowledge representation. But how you may ask do we capture these rules, there
are many techniques and we computer scientists would instantly start thinking along the
familiar if-else lines. There are therefore programming languages, for instance the
familiar C++ or Java and then there are languages made specifically for KR such as LISP
or PROLOG. Almost a precursor to this as is taught before any coding is done is to look
at the analysis and design of the problem at hand, which is done in the form of either
What is this knowledge representation?
It is when we interpret physical or conceptual realities in a form that can then be
communicated to a computer. It is thus in essence constructing reality for a computer, we
are in effect telling the computer what is and is not true, what can and cannot be done.
Predicate calculus or by means of
ontop(A,B)
check(B7,A2)
where we represent a relationship or a property between 2 or more related
entities
Decision trees otherwise known as graphs.
This is perhaps best illustrated by an example.
“Where is the problem?”
“Does the car start?”
“Does it respond to a ‘self’?”
“Do lights come on?”
13
Battery dead
For instance think of a trying to start your car one morning and finding that it won’t start.
We now have a situation where we can represent the problem as a state space. Look at
this figure, the first node represents a stage in our problem or a state of partial knowledge,
and asks us “where is the problem?”, at this stage we are faced with a choice this choice
is in the form of an inference and lends its truth value to the final outcome, thus
depending on the choice we make we will get a different outcome. This figure was a state
space and the search we applied given our goal and initial states enabled us to identify a
path through the tree indicating what our problem was and what a possible solution might
be.
To clarify the concept of a state space further lets think of a simple game of
knots-&-crosses.
If we start with an initially empty grid, how many states can there be in the first level,
that is to say how many places can the first person put a cross? (Yes, 9 places.) Once the
first person has moved how many places can the second player put a knot in? (Yes, 8
places.) But the subtlety that must be noted here is that the person 2 can put a knot in 8
places for each of the 9 places that person 1 put his cross in. Now you’re thinking this is
a simple problem,
How many states could there be?
Anyway, we move on to the second level which has possible 9 x 8 = 72 legal
combinations. And so the third will have 72 x 7 = 504 combinations and so on, this entire
set is the State space, and is what we limit our searching algorithms to look in. This does
not mean that the problem has been trivialized, because even a simple game like knots-&crosses has approximately 362880 different combinations, imagine how complex it
would be to define the state space alone for a game like chess for example! Chess has
states spanning more than 10120 Solving the problem is a process of searching the state
space graph for a path to a solution. By describing the entire space of problem solutions,
state space graphs provide a powerful tool for measuring the structure and complexity of
problems and analyzing the efficiency, correctness, and generality of solution strategies.
Thus if you play against a computer that has a state space at its disposal, the
computer will try for a path that has the most number of wins and
the least number of losses and will in effect force you down that path,
Without going into the subtleties of programming languages and
pseudo code does everyone see how artificial intelligence is in
effect automating intelligent behavior?
14
What then is a state space?
The nodes of a state space search represent possible stages of a problem solution; the arcs
of the graph represent inferences, moves in a game, or other steps in a problem solution.
Solving the problem is a process of searching the state space graph for a path to a
solution.
Now that we have our knowledge represented in a way that it can be communicated to
our computer what should we do next?
What good is knowledge if we can’t use it?
In order to be able to efficiently use knowledge we must have some way of searching
through it for a desired result. This then makes up the second most important branch of
AI, i.e. using the knowledge meaningfully, or searching for a desired outcome.
Now that we have an understanding of what AI is and what are the major components,
lets us examine where today’s topic figures in the big picture. Today we will be studying
about State space search.
state space search can now be given a formal definition, it is said to
process of finding a solution path from the start state to a goal.
A
be the
We may now ask, what is a goal?
The goal varies from problem to problem. What do you think? If I tell you that it could
represent a state,
What would the goal state be in a game of knots-&-crosses?
(Winning board)
In a game of chess?
(Checkmate). Alternatively a goal could describe some
property of the solution itself, for instance in the
traveling sales-person problem, the search would terminate when the “shortest”
path has been found.
When would a parsing problem terminate?
(When the sentence has been
successfully analyzed).
Now that we have understood what we mean when we say state space search, we can
start applying algorithms to aid us in our quest to reach a goal. When we talk about
searching there are 2 techniques which may be adopted. The first being
15
forward chaining, and the second being
backward chaining. I’m quiet sure everyone in this class has at some point in his
life tried solving a maze.
How many ways are there to solve a maze?
There are two ways of solving a maze, we either work from one of the start points
towards our goal, or? (We work from our goal towards a start point). This then is the
distinction between data-driven and goal driven search. So we shall now refine
our crude terminology, what would backward chaining be keeping the example of the
maze in your mind? Goal or data driven? (Goal) Exactly. And so a forward chain would
be what driven? (Data)
Get slide of pg 95.
Data-driven, is when the problem solver
problem and a set of legal moves or rules
begins with the given
facts of the
for changing state. Search proceeds by
applying rules to facts to produce new facts, which are in turn used by the rules to
generate more new facts. This process continues until it generates a path that satisfies
the goal condition.
Goal-driven,
is when we take the goal that we want to solve, see what rules or legal
moves could be used to generate this goal and determine what
conditions must be
true to use them. These conditions become the new goals, or sub goals, for the search.
Search continues, working backward through successive sub goals until it works back to
the facts of the problem. This finds the chain of moves or rules leading from data to a
goal, although it does so in backward order. Mathematical theorem proving,
doctor’s diagnosis.
The preferred strategy is determined by the properties of the problem itself. These include
the complexity of the rules, the “shape” of the state space, and the nature and availability
of the problem data.
An example may be for instance, may be the problem of
confirming or denying
the statement “I am a descendant of Quaid-e-Azam” we can examine this problem
by either looking at the “I” and then looking at the ancestors all the way to Quaid-e-Azam,
or another approach would be to look at Quaid-e-Azam and follow all his descendants.
16
Now that we have identified how we can search our state space we
can conceptually examine some search algorithms and use them to further clarify our
understanding of how exactly AI automates intelligent behavior. I’m quite sure
everyone is familiar with the basic operations involved in
Depth-First
Breadth-First
and
search? As most of you also know from similar algorithms in other
very large branching factor. And
not suitable for large state spaces as is commonly encountered in AI,
courses, is that both these algorithms have a
our
because they either go too deep or too wide and thus do not efficiently solve the problem.
For example it would even take a super-computer a considerable amount of time to make
the best move in a game of chess.
search "intelligently". After all we are constructing
an “intelligent” machine aren’t we? How can we achieve this?
What we therefore need is to
We need somehow to evaluate
possibilities without having to investigate
every single possibility. The practical
answer to this is perhaps the creation of
heuristics.
What do you think heuristics are?
I’ll give you a little hint here, the word
heuristic itself come from the Greek that means
find
or discover. Heuristics in effect
are then like rules of thumb, they are for instance what we as humans would use to
solve a problem, heuristics are thus formalized as rules for choosing those branches in a
state space that are most likely to lead to an acceptable solution. Here we must note
that they will lead to an acceptable problem solution, a heuristic does not guarantee an
optimal solution. An example of this might be taken from our lives, when we go to a
Doctor, with some illness he will diagnose us and then the medication he recommends
may not be the optimal cure, because what he is doing is picking the most likely
diagnosis and is using that as a plan of treatment.
Why would an acceptable solution do?
Perhaps because the state space growth increases exponentially, such that although a
solution may exist, the computational cost of finding it may be unaffordable. Again the
example of chess, finding the best goal state out of 10 may fail to find one in a practical
length of time.
We shall illustrate this by means of an example. Take the game of
knots-&-crosses
again, if we use the symmetry of the game to reduce some states, we have eliminated
a few of the states.
17
What can we do now? How can we use a heuristic to make the
state space smaller?
Lets look at it this way, now that we have seen that the first player can only put a square
either in the center (4) of the grid, in a corner (3), or in the middle (2) of a side.
We shall analyze these 3 states and
“guess”
which one perhaps has the most wins
associated with it. As we can see the center has the most 4 wins associated with it, so
we’ll use that as our first move. Once we’ve made this move the opponent in now
restricted in putting a knot either in a corner or in the middle of a side.
What can we do at this point?
We can again use are heuristic to narrow down our search. How will we do this?
Do you notice how we don’t need to perform an exhaustive search anymore?
Which will be the path that has the most wins now?
Does everyone see how we used our heuristic to shrink the state space?
This then is the basis for algorithms that incorporate this “rule
of thumb” to
automate intelligent behavior. Here now we see the utility of the bestFirst search. The best-first search works in such a way as to make use of the
heuristics available to it, and these are made on the basis of how close a state is from
the goal state, this in effect is then the most promising of the states presented to it. The
best-first works such that the assumptions it will make at any given point in time will be
greedy, and if the state it has moved to does not find a suitable child goal it will
backtrack to the next best sibling.
Why do we use heuristics?
We use them to prune vast state spaces so that the
algorithms used to search them can do so in a practical length of time.
What practical applications can we use heuristics for? Virus detection,
omputer problems, games etc.
CONCLUSION
18
Before we end lets look once again at what we learnt today. The first thing we saw was
the
structure,
history and
content
of the discipline of AI, we then moved on to recognize some of the
applications
of AI and used them to identify the key components of AI, which were
Knowledge representation
Searching.
We then used these components to understand how and what comprised a
state space
and how we performed
searches
on these spaces using a variety of techniques the most important of which perhaps were
the use of
heuristics
We then used our knowledge of heuristics to understand another algorithm which made
use of them and provided faster perhaps slightly fallible results.
19