Download Evaluation Functions

Implementing the
MinMax Algorithm
With
Alpha Beta Pruning
For playing
The
Tic Tac Toe Game
By Sriram Rajan
Team Members
Cody Kutac
Harish Vuyurru
TABLE OF CONTENTS
INTRODUCTION TO AI .............................................................................................. 1
TECHNIQUES TO SOLVE AI PROBLEMS........................................................... 2
PROBLEM DESCRIPTION......................................................................................... 3
THE TIC TAC TOE GAME .............................................................................................. 3
PROBLEM STATEMENT .................................................................................................. 3
THE ALGORITHM ........................................................................................................ 4
MINMAX WITH ALPHA-BETA CUTOFFS ......................................................................... 4
PRUNING ......................................................................................................................... 4
HEURISTIC EVALUATING TECHNIQUES .......................................................... 6
EVALUATION FUNCTIONS .............................................................................................. 6
MY EVALUATION FUNCTIONS .......................................................................................... 7
Function 1 ................................................................................................................... 7
Function 2 ................................................................................................................... 8
Function 3 ................................................................................................................... 9
SPECIAL TECHNIQUES ................................................................................................. 10
IMPLEMENTATION TECHNIQUE ........................................................................ 11
DATA STRUCTURES ........................................................................................................ 11
FUNCTIONS .................................................................................................................... 11
OBSERVATION ........................................................................................................... 12
ANALYSIS...................................................................................................................... 13
APPENDIX ..................................................................................................................... 14
WORK DISTRIBUTION .................................................................................................. 14
SOURCE CODE ............................................................................................................. 15
SAMPLE OUTPUT .......................................................................................................... 16
REFERENCES ............................................................................................................. 23
Introduction to AI
“Artificial Intelligence is the study of how to make computers do things, which at the
moment; people do better” [Rich & Knight]. Computers can do many wonderful things.
They can perform calculations millions or billions of times faster than human beings. In
the past, computer scientists have created a great many programs that could perform
tasks that people wouldn't have otherwise believed a computer could do. This is not
limited to only playing chess, or proving theorems, but also programs that can hold a
regular conversation with humans, understand stories and perform many other humanlike tasks. Yet, there have also been very many legitimate questions whether or not the
intelligence that these programs exhibit can be comparable to human intelligence. The
problem lies with architecture, the way our programs are structured. Alan Turing, one of
the fathers of AI, once created a theorem that stipulated that all computers (he uses the
term Turing machines, which can be likened to digital computers) could compute anything
that is computable [1].
AI addresses one of the ultimate puzzles. How is it possible for a slow, tiny brain, whether
biological or electronic, to perceive, understand, predict, and manipulate a world far larger
and more complicated than itself? How do we go about making something with those
properties? These are hard questions, but unlike the search for faster-than-light travel or
an antigravity device, the researcher in AI has solid evidence that the quest is possible.
All the researcher has to do is look in the mirror to see an example of an intelligent
system.
Artificial Intelligence is a very dynamic field. Although lots of research has been done in
the field of AI, It is a field of study that is very far from saturation with a huge area left
unexplored and waiting to be studied and analyzed.
1
Techniques to solve AI problems
Broadly AI systems can be divided into the following categories




Systems that think like humans: The cognitive modeling approach
Systems that think rationally: The laws of thought approach
Systems that act like humans: The rational agent approach
Systems that act rationally: The rational agent approach
Almost all AI problems have the following characteristics
 It is Voluminous
 It is hard to characterize accurately
 It is constantly changing
 It differs from data by being organized in a way that corresponds to the
way it will be used
The three most important AI techniques that help to solve the above problems are
Search
Provide a way of solving problems for which no more direct approach is available
as well as a framework into which any direct techniques that are available can be
embedded. This is the field of heuristic (intelligent) searching. The idea is to
apply human intelligence in programs to search in unconventional ways
Use of Knowledge and Reasoning
Provides an intelligent way of solving complex problems by exploiting the data
structures of the objects that are involved. Make the programs think like humans.
Analyze how humans solve a particular problem and apply similar human
reasoning in programs.
Abstraction
The idea of abstraction is to separate the important features from the unimportant features that would otherwise overwhelm the process and
unnecessarily increase the complexity. Study the problem to detect areas that
need not be processed by the program thus saving time and memory.
2
Problem Description
The Tic Tac Toe Game
Tic-Tac_Toe or X & O’s is a simple two player game where players put eirher X or or O
on the grid. Any player who can achieve three of his symbols in the same line wins.
Empty Grid
Player 1(X) moves
Player 2(X) moves
Player1 Wins
X
X
X
0
0
X
X
Problem Statement
Our project will build an intelligent agent that plays the game of Tic-Tac-Toe against a
human adversary. We will implement this intelligent agent using the MinMax algorithm
with alpha-beta cut-offs (explained in the next section).
Three evaluation functions used with the Minmax algorithm will also be implemented
and their analysis in terms of the number of nodes generated and effectiveness will be
presented in the analysis section.
3
The Algorithm
MinMax with alpha-beta cutoffs
MinMax_A_B(Position,Depth,Player,Use-Thresh,Pass-Thres)
If DEEP_ENOUGH(position, depth)
Return value=STATIC(position,player);
PATH=NIL
Else
Call MOVE_GEN and generate all the SUCCESSORS
If
there are no successors then return the same as in DEEP_ENOUGH
else
Do till all SUCCESSORS are checked
Result-SUCC = MinMax_A_B(Position,Depth+1, other Player, -pass_thresh,Use-Thresh)
NEW_VALUE=-VALUE(RESULT_SUCC)
IF NEW_VALUE> pass_thresh
Set Pass thresh to NEW VALUE
Set Best Path from the CURRENT to SUCC
IF pass thresh >= use thresh
Then stop (this is cut-off)
Return value=pass thresh & BEST PATH
Return VALUE=pass_ threh
Path=BEST_PATH
Pruning
Pruning a search tree is the process of eliminating, a branch of the search tree. One
such example of pruning is the alpha-beta pruning, which is used in the implementation
of this project. Pruning is a very good method of controlling the amount of searching is
to have a limit on the depth of searching. In many games the tree is very huge and
pruning helps to reduce it considerably thus saving time and space.
Difficulties with Pruning
In certain examples there might be a situation where, if it had searched one depth extra,
it might have been able to do a better analysis of the state, which might affect the future
of the game. Another problem is where a move by the opponent causes serious
damage, but there is nothing, which the program can do in the end to prevent it. The
problem which occurs here, is that it will always think that it will be able to prevent this
4
from happening, and has no way of detecting its position. Possible solutions include if
we get a big change in the utility value then we check one level more or using futility
cutoff techniques.
Alpha-Beta pruning
Alpha-Beta pruning is where when applied, to a minmax search tree, it returns the
identical move as minmax, but prunes away the branches, which cannot influence the
future of the game. The effectiveness of alpha-beta pruning depends, on the ordering in
which the future moves are analyzed.
5
Heuristic Evaluating Techniques
Evaluation Functions
It is not always suitable to search all the possible nodes of a search tree, in order to find
a way to the terminal state, as there are things like the time limit. Therefore heuristic
evaluations functions, help to determine the state of the game from a given position. For
example in games such as chess, each piece is given a material value. In tic tac toe we
will determine which player’s position is in advantage.
There are three main actions, which it must perform:

The evaluation must agree with the utility function on the terminal state.

It must be able to perform within a specified time limit; this might result in loss of
accuracy.

The function should be able to accurately tell the chances of winning the game.
For Tic-Tac-Toe we can develop evaluation function that evaluate the state of the game
and return values that indicate which player is winning or which player is in advantage.
Since we don’t have a large node generation factor as in chess time will not be a big
issue. Also since tic tac toe is a simple game we need not search at a level greater than
4 although it is quite possible to do so.
How the Evaluation Function calculates the utility of a move.
The Evaluation Function calculates the next move by assuming that the opposition will
take the best possible move in its current situation. The utility or worth of a position is
calculated by taking the difference between the players score and the opponents score.
6
My Evaluation Functions
Function 1
The first function uses the programmer’s game knowledge to evaluate the state of the
game. Since tic tac toe is a simple game that can be easily analyzed on the basis of the
positions and also is not a very long game (it can have a maximum of nine moves from
each player).
This function checks the state if the opponent is winning the game then it returns a large
negative value else if the player is winning then it returns a large positive value.
If neither is winning then it checks the advantage position squares such as center then
the corners etc and returns a value based on their priority.
X
0
X
0
0
0
X
0
X
0
X
Return large negative value
X
0
Return large positive value
Return negative since
center occupied by O
Analysis
This function works very well for most of the states in tic tac toe. One state it fails is
when
O
O
X
Should chose this
X
Here the function should return the boldfaced value but because of the way it is
designed it will pick the first corner (since the opponent is not winning). It fails to foresee
the threat resulted by the placing of X in that square when the opponent moves again.
The 3rd evaluation function explained in this section will handle this situation.
7
Function 2
This evaluation function is more like the one minmax algorithm would use. It analyzes
the whole grid and doesn’t stop at one pattern. It checks all the positions and calculates
the value based on it.
f(n) = [number of 3-lengths open for Min] - [number of 3-lengths open for Max]
where a 3-length is a complete row, column, or diagonal.
At every state it finds the value of f(n) and returns it . This is a considerably good
evaluation since the aim of the game is to produce a 3-length i.e. a complete row,
column, or diagonal.
No of 3-lenghts for Min =1
No of 3 lenghts for Max = 1
Hence returns 0
X
X
No of 3-lenghts for Min(O) =0
No of 3 lenghts for Max = 2
Hence returns 2
0
X
X
0
X
0
0
Analysis
This function doesn’t solve the problem posed by the first function. Though it does an as
better estimate of many states as compared to the first one.
8
Function 3
This function was designed keeping in mind the problem posed by the first two
evaluation functions. This function does a special check on that case and instead of
picking on any corner it picks the right square. Other than the special case it calculates
the values in the same way as the first function.
Chooses this
O
X
X
Analysis
This function works very much similar to the previous two function except for the special
case above. It is kinds of extension too check special cases of the games. In a game
like tic tac toe it is possible to check all the special cases in order to ensure that the
computer never loses. This function has to do a lot of checking and computation and
hence is time consuming if we use it for depths more that four because then we will
have to apply our evaluation functions on more number of terminal nodes
9
Special Techniques
This technique is used to improve node generation and more importantly pruning. In
most cases of Tic tac toe if the player captures the center square than he is in
advantage unless the opponent has a possible 3-length row, column or diagonal. The
2nd and 3rd evaluation functions described above check the occupant of the center
square and produce a value according to it. This value is usually high (unless the
player has a position that results in his winning).
The main idea is that the “movegen” function should generate the node with the center
square first. Since the left most node will be evaluated we can easily prune most of the
remaining nodes because the left most node is the best. This technique results in a lot
of pruning some many of the start moves of the games.
For ex: Let assume the start of the game i.e. all squares are blank and MAX is playing.
If we generate the center square with the MIN symbol in it and then apply evaluation
function at the next depth for MAX we can prune all the nodes generated from the
center square since MIN is in advantage and MAX wont choose them.
This technique helps in pruning at the initial stages of the game. Also it doesn’t add any
extra complexity since we still generate all the nodes (only we generate the center one
first)
10
Implementation Technique
Data Structures
Successors :The successors for each node are generated by the MOVE_GEN
and stored in a linked list so that it can be traversed one bye one
Best Path : The best path storage is also implemented using a link list.
Functions
DEEP_ENOUGH : It is implemented as a part of the recursive function
MinMax_A_B since it is just an IF-CLAUSE
Move_Gen : It is implemented as a part of the recursive function MinMax_A_B in
order to reduce the complexity arising due to the recursive calls
Other: Returns the other player
Play_Sym : returns the player symbol
Isgridfull : to check if the grid is full and there are no more possible moves
Three Evaluation functions
Various other functions in order to draw the grid, fill it with symbols, get player
options insert symbols in the grid and error checking functions.
11
Observation
Note: The number of nodes depends on the how long the games lasts. If we finish the
game quickly we might end up with less number of nodes. Also it depends a lot on the
player moves, which might lead to different degrees of pruning
Evaluation
Function
Total
Length of
Game
Path
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
1
1
1
2
2
2
3
3
3
4
4
4
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
Evaluation 1
Evaluation 2
Evaluation 3
1
1
1
2
2
2
3
3
3
4
4
4
Number of nodes
Generated
Sample Run 1
20
18
38
154
145
141
321
290
344
1131
930
1144
Sample Run 2
20
18
34
162
138
175
333
284
340
1737
1312
1643
12
Analysis
The first function works ok for almost all cases except for the special case (explained in
the evaluation function section). It does a good job at pruning but doesn’t prune as
much as second but more importantly gives a correct estimate
If we play the game and use the second evaluation the computer makes a lot of
mistakes. The node generation and pruning is very good but the estimate is not
accurate in terms of making a player win which is more important if we want to build a
really intelligent agent.
The third function is the best in terms of the estimate because if we use the third the
computer we will never lose. It prunes as well as the first because it is just an extension
of the first covering the special case. This evaluation function produces an unbeatable
agent.
13
Appendix
Work Distribution
Sriram Rajan
Cody Kutac
Harish Vuyurru
Tic tac toe interface
Functions that help to
play the game
The state of each min
max node.
Functions to check the
grid status
The MinMax Node.
The other and play_sym
& insert functions
Worked together in implementing the minmax algorithm and the alpha-beta
cutoffs
14
Source code
15
Sample Output
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒TIC TAC TOE▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒1.Play Game▒▒▒▒▒▒
▒▒▒▒2.Game Options▒▒▒
▒▒▒▒3.Use Minmax▒▒▒▒▒
▒▒▒▒0.Exit▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Enter Choice:1
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0]
▒[1]
▒[2]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3]
▒[4]
▒[5]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6]
▒[7]
▒[8]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Player1( X ) : Enter cell no:0
player :MAX Depth:0 Max Depth:1
player :MIN Depth:1 Max Depth:1
New Value:20player :MIN
New Value:20player :MIN
New Value:20player :MIN
UT:10000
UT:10000
[0] X
[1]
[2]
[3]
[4] 0
[5]
[6]
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1] 0
[2]
[3]
[4]
[5]
[6]
[7]
[8]
Depth:1
PT:-10000
PT:-10000
Max Depth:1
[0] X
[1]
[2] 0
[3]
[4]
[5]
[6]
[7]
[8]
Depth:1
Max Depth:1
UT:-20
PT:-10000
UT:-20
PT:-10000
UT:-20
PT:-10000
16
New Value:20player :MIN
New Value:20player :MIN
New Value:20player :MIN
New Value:20player :MIN
[0] X
[1]
[2]
[3] 0
[4]
[5]
[6]
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2]
[3]
[4]
[5] 0
[6]
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2]
[3]
[4]
[5]
[6] 0
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2]
[3]
[4]
[5]
[6]
[7] 0
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8] 0
UT:-20
PT:-10000
UT:-20
PT:-10000
UT:-20
PT:-10000
UT:-20
PT:-10000
New Value:20
Total Number of Nodes Generated:8
17
BEST PATH
[0] X
[1]
[2]
[3]
[4] 0
[5]
[6]
[7]
[8]
[0] X
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
Value=4
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0] X ▒[1]
▒[2]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3]
▒[4] 0 ▒[5]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6]
▒[7]
▒[8]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Player1( X ) : Enter cell no:2
player :MAX Depth:0 Max Depth:1
player :MIN Depth:1 Max Depth:1
New Value:20player :MIN
New Value:20player :MIN
New Value:20player :MIN
UT:10000
UT:10000
PT:-10000
PT:-10000
[0] X
[1] 0
[2] X
[3]
[4] 0
[5]
[6]
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2] X
[3] 0
[4] 0
[5]
[6]
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2] X
[3]
[4] 0
[5] 0
[6]
[7]
[8]
Depth:1
Max Depth:1
UT:-20
PT:-10000
UT:-20
PT:-10000
UT:-20
PT:-10000
18
New Value:-15player :MIN
New Value:-15player :MIN
[0] X
[1]
[2] X
[3]
[4] 0
[5]
[6] 0
[7]
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2] X
[3]
[4] 0
[5]
[6]
[7] 0
[8]
Depth:1
Max Depth:1
[0] X
[1]
[2] X
[3]
[4] 0
[5]
[6]
[7]
[8] 0
UT:-20
PT:-10000
UT:-20
PT:-10000
New Value:-15
Total Number of Nodes Generated:6
BEST PATH
[0] X
[1] 0
[2] X
[3]
[4] 0
[5]
[6]
[7]
[8]
[0] X
[1]
[2] X
[3]
[4] 0
[5]
[6]
[7]
[8]
Value=1
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0] X ▒[1] 0 ▒[2] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3]
▒[4] 0 ▒[5]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6]
▒[7]
▒[8]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Player1( X ) : Enter cell no:7
player :MAX Depth:0 Max Depth:1
player :MIN Depth:1 Max Depth:1
UT:10000
UT:10000
PT:-10000
PT:-10000
19
New Value:-15player :MIN
New Value:-15player :MIN
New Value:-15player :MIN
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5]
[6]
[7] X
[8]
Depth:1
Max Depth:1
[0] X
[1] 0
[2] X
[3]
[4] 0
[5] 0
[6]
[7] X
[8]
Depth:1
Max Depth:1
[0] X
[1] 0
[2] X
[3]
[4] 0
[5]
[6] 0
[7] X
[8]
Depth:1
Max Depth:1
[0] X
[1] 0
[2] X
[3]
[4] 0
[5]
[6]
[7] X
[8] 0
UT:15
PT:-10000
UT:15
PT:-10000
UT:15
PT:-10000
New Value:-15
Total Number of Nodes Generated:4
BEST PATH
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5]
[6]
[7] X
[8]
[0] X
[1] 0
[2] X
[3]
[4] 0
[5]
[6]
[7] X
[8]
20
Value=3
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0] X ▒[1] 0 ▒[2] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3] 0 ▒[4] 0 ▒[5]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6]
▒[7] X ▒[8]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Player1( X ) : Enter cell no:8
player :MAX Depth:0 Max Depth:1
player :MIN Depth:1 Max Depth:1
New Value:-20player :MIN
UT:10000
UT:10000
PT:-10000
PT:-10000
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5] 0
[6]
[7] X
[8] X
Depth:1
Max Depth:1
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5]
[6] 0
[7] X
[8] X
UT:20
PT:-10000
New Value:-15
Total Number of Nodes Generated:2
BEST PATH
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5]
[6] 0
[7] X
[8] X
[0] X
[1] 0
[2] X
[3] 0
[4] 0
[5]
[6]
[7] X
[8] X
Value=6
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0] X ▒[1] 0 ▒[2] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3] 0 ▒[4] 0 ▒[5]
▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6] 0 ▒[7] X ▒[8] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
21
Player1( X ) : Enter cell
no:5
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[0] X ▒[1] 0 ▒[2] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[3] 0 ▒[4] 0 ▒[5] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒[6] 0 ▒[7] X ▒[8] X ▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Player1 WON !!!
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
Total Nodes in the entire game:20
Press Enter
to continue....
22
References
 Artificial Intelligence By Elaine Rich & Kevin Knight
 Artificial Intelligence A Modern Approach By Stuart Russel & Peter
Norweigh
 AI by Lugger & Stubblefield
 http://www.aaai.org/
23