Download MS PowerPoint format

Lecture 10 of 41
Game Tree Search (Concluded)
Introduction to Logical Agents
Monday, 13 September 2004
William H. Hsu
Department of Computing and Information Sciences, KSU
http://www.kddresearch.org
http://www.cis.ksu.edu/~bhsu
Reading:
Sections 6.4-6.8, Russell and Norvig 2e
Next class (Wednesday): Sections 7.1-7.4
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Lecture Outline
•
Today’s Reading
– Sections 6.4-6.8, Russell and Norvig 2e
– Recommended references: Rich and Knight, Winston
•
Reading for Next Class: Chapter 6, Russell and Norvig
•
More on Game Tree Search
– Review: framework; minimax algorithm; quiescence detection, horizon effect
– Alpha-Beta pruning
– Expectiminimax
•
Applications: Current “Hot” Problems
– “Harder” games: Go
– Interactive agents and games
• Economically-founded multiagent systems, negotiation
• Simulated societies
•
Next Week: Knowledge Representation – Logics and Production Systems
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Review:
Minimax Algorithm
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 5.2 p. 125 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Definitions
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Example [1]
≥3
MAX
MIN
Warning: Gratuitous use of
Microsoft® PowerPoint ® XP 
≤2
3
≤ 14 ≤5
2
MAX
3
12
8
2
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
14
5
2
Figure 5.6 p. 131 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Example [2]
What are ,  values here?
≥3
MAX
MIN
≤2
3
≤ 14 ≤5
2
MAX
3
12
8
2
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
14
5
2
Figure 5.6 p. 131 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Modified Minimax Algorithm
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Expectiminimax
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Properties
•
Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply
•
Assumptions about
– … game?
– … opponent?
•
Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
•
Practical Issues
– Effects of depth cutoff
– Need for pruning
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Digression:
Learning Evaluation Functions [1]
•
Learning = Improving with Experience at Some Task
– Improve over task T,
– with respect to performance measure P,
– based on experience E.
•
Example: Learning to Play Checkers
– T: play games of checkers
– P: percent of games won in world tournament
– E: opportunity to play against self
•
Refining the Problem Specification: Issues
– What experience?
– What exactly should be learned?
– How shall it be represented?
– What specific algorithm to learn it?
•
Defining the Problem Milieu
– Performance element: How shall the results of learning be applied?
– How shall the performance element be evaluated? The learning system?
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Digression:
Learning Evaluation Functions [2]
•
Possible Definition
– If b is a final board state that is won, then V(b) = 100
– If b is a final board state that is lost, then V(b) = -100
– If b is a final board state that is drawn, then V(b) = 0
– If b is not a final board state in the game, then V(b) = V(b’) where b’ is the best
final board state that can be achieved starting from b and playing optimally
until the end of the game
– Correct values for minimax / alpha-beta, but not operational
•
Choosing a Representation for the Static Evaluation Function
– Collection of rules?
– Neural network?
– Polynomial function (e.g., linear, quadratic combination) of board features?
– Other?
•
A Representation for Learned Function
– Vˆ b   w 0  w 1bp b   w 2 rp b   w 3 bk b   w 4 rk b   w 5 bt b   w 6 rt b 
– bp/rp = number of black/red pieces; bk/rk = number of black/red kings;
bt/rt = number of black/red pieces threatened (can be taken on next turn)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
State of The Art:
Learning to Play Games
•
“Easy” Games: Nim, Tic-Tac-Toe
– Nim: known Grundy function
– TTT: ditto (easy to play, prove; hard to store game tree)
•
“Cracked” Games: Go-Moku, Checkers, Go Subgames
– Go-Moku aka Connect Five
– Some limited Go (smaller than 19x19) or specialized games (Guess – see SciAm)
•
Games where Best Human Players Have Been Defeated by AI
– Reversi / Othello (1988, Lee et al)
– Backgammon (1992, Tesauro)
– Checkers (1994)
– Chess (1997, Campbell et al)
•
“Harder” Games
– Go (see Computer Go, Ing Prize pages)
– Others
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Current Research Topics in Games and AI
•
Game Theory
– Common ramework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply
•
Negotiation
– … game?
– … opponent?
•
Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
•
Practical Issues
– Effects of depth cutoff
– Need for pruning
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Resources for Term Project
•
Print Resources on The Web (You May Cite Some as Print References)
– CiteSeer (largest online citation database): http://citeseer.nj.nec.com
– See also: Google (author search), E-journals (JAIR, JMLR), societies (AAAI, AUAI)
•
Resources for AI Projects
– USENET: many AI-related groups
• comp.ai, comp.ai.edu, comp.ai.{neural-nets | genetic | fuzzy}
• Other groups on vision, optical character recognition (OCR)
– Watch CIS730 Yahoo! Group (see Part 3 of 3, project guidelines)
•
Don’t Forget Interlibrary Loan!
– K-State: off-campus borrowing facilities (http://www.lib.ksu.edu)
– Arrangements for remote (NTU) students: contact instructor
•
Instructor Contact Info
– ICQ #28651394, cis730ta, [email protected]
– Voice: (785) 532-6350 x29 [Office], (785) 539-7180 [Home 18:00 – 20:00 CDT]
– Mobile phone contact: call Delores Winfough (785) 532-6350
– Schedule: http://www.kddresearch.org/Personal/bhsu/schedule.txt
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Ideas for Term Project
•
Rogue versus Angband
– Idea: discrete domain, single-agent, control and target acquisition problem
– Rog-O-Matic: http://www.cs.princeton.edu/~appel/papers/rogomatic.html
– Angband Borg: http://www.phial.com/angborg/
•
Empire versus Intergalactics
– Idea: continuous domain, multi-agent, competitive, control and target acquisition
– Empire: http://gamespot.com/gamespot/features/pc/history_spaceempire/
– Intergalactics: http://www.intergalactics.net
•
Robocode versus Space Pirates
– Idea: continuous domain, multi-agent, competitive, control and target acquisition
– Robocode: http://www.alphaworks.ibm.com/tech/robocode
– Space Pirates: http://www.lostlogic.com
•
Hey! Are Computer Games All This Course Is About?
– No!
– Diagnostic expert systems, data mining, intelligent IR
– Many many more… (see web group)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Advice About Term Project [1]
•
Start Early!
– Recall “How to Get an A in This Course” (Lecture 0)
– See below: start ball rolling ASAP
•
Ask Questions on Course Web Group (ksu-cis730-fall2001)
– On online fora
• Course web group (ksu-cis730-fall2001) – check progress with classmates!
• USENET (caveat: know what you are asking, what you will do with answers)
• Mailing lists (ditto) – data mining; Uncertainty in AI; many, many others
– Directly question instructor
• By remote: see contacts, previous page
• in person: before class, Wednesday office hours (with project partner, by self)
– Do footwork
• Not everything is on web (Inter-Library Loan – excruciatingly slow)
• Not everything on web is well-indexed, easily accessible
•
Stay Coordinated with Project Partner
•
Don’t Waste Too Much Time on Nonessentials (Display/GUI)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Advice About Term Project [2]
•
Evaluation Criteria
– Originality, significance, scope
• Nontriviality of problem: very important!
• Originality of solution: reimplementation / rehashing of common “applet”?
• Generality of results
– Functionality
– Writing quality
• Proposal is not end of writing!
• Conference/workshop-style report much more important
•
How to Focus on Essentials
– Keep list of milestones, track schedule slippage carefully
– Report to instructor and get feedback (no formal interim reports, but post)
– Know what is essential
• Ounce of prevention: good planning, design, specification, antibugging
• Don’t lose time on interfaces, et cetera: scoring not based on “KLOC”
• Leave time for debugging and testing!
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Knowledge Bases
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Simple Knowledge-Based Agent
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.1 p. 152 R&N
Kansas State University
Department of Computing and Information Sciences
Representing Intelligent Agent (IA) Problems:
PAGE Formulation
•
Percepts
– What can agent observe?
– What can sensors tell it?
•
Actions
– What actuators does agent have?
– In what context are they applicable?
•
Goals
– What are agents goals? Preferences (utilities)?
– How does agent evaluate them (check environment, deliberate, etc.)?
•
Environment
– What are “rules of the world”?
– How can these be represented, simulated?
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Wumpus World PAGE Description
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.2 p. 154 R&N
Kansas State University
Department of Computing and Information Sciences
Wumpus World Characterization
•
Deterministic?
– In general: is there any uncertainty about
• Outcomes of actions?
• Initial conditions? (Can still be deterministic domain; uncertainty is agent’s)
– Yes, deterministic… outcomes exactly specified
•
Fully Accessible?
– Global perception?
– No, not fully accessible… can only observe locally
•
Static?
– Obstacles, other “Non-Player Character (NPC)” agents can move?
– Yes, static… wumpus, pits cannot move
•
Discrete?
– NB: not representation; actual domain (why important?)
– Yes, discrete
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Exploring A Wumpus World
OK

OK
OK
KSU
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figures 6.3-6.4 p. 156 R&N
Kansas State University
Department of Computing and Information Sciences
Wumpus World Reasoning Problems:
Uncertainty and Coercion
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Logic in General
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Ontology and Epistemology:
Describing Types of Logic
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.7 p. 166 R&N
Kansas State University
Department of Computing and Information Sciences
Summary Points
•
Games as Search Problems
– Frameworks
– Concepts: utility, reinforcements, game trees
– Static evaluation under resource limitations
•
Family of Algorithms for Game Trees: Minimax
– Static evaluation algorithm
• To abritrary ply
• To fixed ply
• Sophistications: iterative deepening, pruning
– Credit propagation
• Intuitive concept
• Basis for simple (delta-rule) learning algorithms
•
State of The Field
•
Uncertainty in Games: Expectiminimax and Other Algorithms
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Terminology
•
Game Graph Search
– Frameworks: zero-sum, multi-player
– Concepts
• Utility and representations (e.g., static evaluation function)
• Reinforcements: possible role for machine learning
• Game tree: node/move correspondence, search ply
• Important distinction: perfect play versus perfect information
•
Family of Algorithms for Game Trees: Minimax
– Propagation of credit
– Imperfect decisions
– Issues
• Quiescence
• Horizon effect
•
Alpha-Beta Pruning
•
Expectiminimax: Extension to Uncertain Games
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences