Download Artificial Intelligence in Games

Artificial Intelligence in Games
Videogame Design and Programming
Prof. Pier Luca Lanzi
References
2
•  Artificial Intelligence for Games
Ian Millington and John Funge
http://ai4g.com/
•  Unity 4.x Game AI Programming
Aung Sithu Kyaw, Clifford Peters, & Thet Naing Swe
Prof. Pier Luca Lanzi
what is the role of artificial intelligence?
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Resident Evil 2002
https://www.youtube.com/watch?v=wSmYSZGMZj0
Prof. Pier Luca Lanzi
think
“rationally”
systems that think
like humans
systems that think
rationally
action
“humanly”
systems that act
like humans
systems that act
rationally
Prof. Pier Luca Lanzi
Acting Like a Human
•  A system is “intelligent” if it acts like a human
•  Not only thinking but also acting
•  Turing test (1950)
•  Ability to match § Human-like performance
§ Sufficient to disguise a human operator
Prof. Pier Luca Lanzi
what is the role of artificial
intelligence in video games?
Prof. Pier Luca Lanzi
“good artificial intelligence”
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
“good artificial intelligence”
vs
“fun artificial intelligence”
Prof. Pier Luca Lanzi
Good AI vs Fun AI
14
•  “Good” AI
§ Beat player at own game
§ Essentially a human substitute
•  “Fun” AI
§ Algorithms are the content
§ Focus on the player’s experience
Prof. Pier Luca Lanzi
“good” artificial intelligence
“fun” artificial intelligence
rules arefixed
symmetrical
multi-player
unlimited strategies
objective evaluation
coherent with the Turing test
rules evolve
asymmetrical
single player
limited strategies
subjective evaluation
irrelevant
play to lose
play to win
Prof. Pier Luca Lanzi
Far Cry 2
http://www.youtube.com/watch?v=sDiGYDNMsZU
Prof. Pier Luca Lanzi
intelligenza artificiale nei videogiochi?
personaggi non giocanti
generazione del contenuto
animazioni
game design, analisi dei giocatori,
gameplay adattativo, ecc.
Prof. Pier Luca Lanzi
Artificial Intelligence and Games
Prof. Pier Luca Lanzi
18
artificial intelligence in video games?
non player characters
content generation
procedural animation
analysis of player behavior
adaptive gameplay, …
Prof. Pier Luca Lanzi
What is Game AI?
20
•  Makes the characters in the game move
•  Decide what, where, how, when
•  Opposing/allied player AI
•  Path planning & collision avoidance
•  Animation selection
Prof. Pier Luca Lanzi
Game AI in High-End Videogames
• 
• 
• 
Nascar 2002 – each driver has his own
personality and driving style (controlled
by a series of attributes)
Warcraft 3 – NPCs form alliances,
scout surrounding areas and devise
battle plans to take enemies out
Sims – each user controlled character
interacts with a series of NPCs each
one with his own characteristics, traits
and needs
Prof. Pier Luca Lanzi
21
Artificial Intelligence Control of NPCs
AI control of NPCs
•  Common model
§  Sense
§  Reason
§  Act
Sense
22
Reason
Act
• Common model:
•  Evaluating AI
• Sense
• Reason
Solve problems (robustly)
• Act
§ 
§  Time requirements §  Space requirements
§  Realistic enough
•  Time requirements
§  30 fps, 33ms per frame, 1-3 ms for AI (~10%), 100ms per unit planning
§  Split long computations over many frames
Prof. Pier Luca Lanzi
“Game AI is largely an unsolved problem”
Prof. Pier Luca Lanzi
How Game AI Should Be?
24
•  The noble goal is to require that the AI system follows the same
rules and constraints as the human so that it plays fairly
•  In a strategy game, the human can only see parts of a map that
they have explored, so should the game AI
•  In practice, this is still very difficult and many AI systems are still
very limited in certain circumstances (in Jedi Academy, NPCs
often fall to their deaths)
•  To get around this the AI system “cheats” a little but not so
much that it is obvious to the player
Prof. Pier Luca Lanzi
How can the AI cheat?
Using the full perfect knowledge of the world
It can dodge bullets, it knows where your units are
hidden, what cards and how much money you have
But also …
Prof. Pier Luca Lanzi
26
Prof. Pier Luca Lanzi
Mainstream AI Techniques Applied to Games
27
•  Finite State Machines
§  States, inputs and transitions to represent a system
§  Used in most of the games in the past
•  Production Systems
§  Essentially a set of if-then-else rules that match a set of input
§  Generate a new set of “input” that may cause other rules to fire
§  Useful in adventure games when a combination of inputs and states can
infer something about a situation
•  Search
§  Permuting all possible states of a game board to attempt to predict
outcome (e.g., chess, pathfinding, etc.)
Prof. Pier Luca Lanzi
Mainstream AI techniques that have been applied
to Games
• 
Planning Systems
§  Used to determine the best set of tasks needed to go from a start state to an end
state. Useful for organizing a collection of troops for a unified goal.
• 
Agents and mult-agent systems
• 
Artificial Life
• 
28
§  Used to model single and groups of NPCs
§  Multi-agent systems that attempt to use living systems to simulate AI agents. Used
in games like SimCity, SimAnt, Spore.
Flocking
§  Techniques to get agents to flock like birds or in army simulations, form and move
•  …
in particular battle formations.
Prof. Pier Luca Lanzi
Space Invaders
http://www.youtube.com/watch?v=QObneYZIdKI
Prof. Pier Luca Lanzi
In the Beginning…
30
•  Initially the characteristics of Non-Playing Charactes (NPCs) were
hard-coded in the game
•  In Space Invaders, the bots move on a predefined level velocity
towards one of the two available directions •  They randomly shoot towards the user controlled spaceship
•  There is no AI in this game!
Prof. Pier Luca Lanzi
What was the first videogame with AI?
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Pong!
33
•  ”Pong” was the first game to introduce a form of artificial
• 
• 
• 
• 
• 
intelligence in gameplay
The goal of the game was to score against the computer or
another player in a game of ping pong
Each player has a paddle that reflects the ball that may be moving
either up or down
The NPC paddle movement was computed using a simple
formula predicting the height the ball would cross the objective
line
The paddle was assigned an allowed velocity and a possibility of
failure based on the level of difficulty
Thus, the paddle may fail to reach the exit point of the ball in the
objective line
Prof. Pier Luca Lanzi
Pac-Man
http://www.youtube.com/watch?v=jiJ0PkU8t3c
Prof. Pier Luca Lanzi
Pac-Man
35
•  One Hero, four ghosts (Blinky, Pinky, Inky, Clyde)
•  244 dots and infinite levels
•  Three ghost modes
§ CHASE: The Ghosts try to capture Pacman
§ SCATTER: The Ghosts head for their home corners
§ FRIGHTENED: The Ghosts feel a little “blue”
•  Four different ghost behaviors
Prof. Pier Luca Lanzi
• Everything you ever
want to know about
Pacman:
Pac-Man
36
• http://
home.comcast.net/
~jpittman2/pacman/
pacmandossier.html
•  The ghosts are prohibited from changing • 
• 
• 
directions when inside a mode
They only change directions when their mode changes
More specifically, chase-to-scatter, chase-to-frightened, scatter-to-chase, and scatter-to-frightened
Time limits for each mode
Modes change on predefined Mode
Level 1
Levels 2–4
Scatter
7
7
times depending on the level,
Chase
20
20
and the time the user Scatter
7
7
has spent on each
Chase
20
20
Prof. Pier Luca Lanzi
Levels 5+
5
20
5
20
Scatter
5
5
5
Chase
20
1033
1037
Scatter
5
1/60
1/60
Chase
indefinite
indefinite
indefinite
Ghost Behavior: The Dots
37
•  Blinky starts always outside
•  Pinky is the first to follow
•  Inky follows after Pacman eats some dots
•  Clyde follows after Pacman eats some more dots
Prof. Pier Luca Lanzi
Ghost Behaviors: Scatter Mode
Prof. Pier Luca Lanzi
38
Ghost Behaviors: Chase Mode - Blinky
•  Blinky
§  It will always target Pacman's position
§  Increases his rate of speed twice each round based on the number of dots
§ 
remaining in the maze
Follows Pacman even in scatter mode when few dots remain
•  Inky §  Needs Pac-Man's current tile/orientation and Blinky's current tile to
§ 
calculate his final target, by drawing a vector with the distance between
the two projected over Pacman's current point
Targeting logic will keep him away from Pac-Man when Blinky is far away
from Pac-Man, but as Blinky draws closer, so will Inky's target tile.
Prof. Pier Luca Lanzi
Ghost Behaviors: Chase Mode - Blinky
•  Clyde
§  First calculates the Euclidean distance between his tile and Pac-Man's tile. If
§ 
the distance between them is eight tiles or more, Clyde targets Pac-Man
directly just as Blinky does. If the distance between them is less than eight tiles, however, Clyde
switches his target to the tile he normally uses during scatter mode and
heads for his corner until he gets far enough away to start targeting PacMan again
•  Pinky
§  Always selects an offset four tiles away from Pac-Man in the direction PacMan is currently moving
Prof. Pier Luca Lanzi
How could we implement the game
intelligence behind Pac-Man?
Prof. Pier Luca Lanzi
Example of Behavior for a First Person Shooter
42
Got Ammo
Get Health
Got Away,
Low on Health
Low on Ammo
Low on Health
Attack
Get Ammo
Run Away
Prof. Pier Luca Lanzi
Got Away,
Low on Ammo
Got Health
Enemy
Lost
Enemy
Found
Search For Enemy
Finite State Machines (FSM)
• 
• 
• 
• 
• 
• 
• 
43
Widely used in the past to implement simple AI behaviors
States represent a high-level view of what the AI is trying to do
Each state has custom code
Transitions occur when the situation changes
Advantages
Easy to conceptualize
Easy to implement
Middleware tools exist
Disadvantages
Quickly becomes large and unwieldy
Difficult to debug
Difficult to expand
Rampant duplication of code
§ 
§ 
§ 
§ 
§ 
§ 
Prof. Pier Luca Lanzi
Finite State Machines (FSM)
44
•  Hard-Coded
§  This process is known as hard-coding the state machine — since it results
§ 
§ 
in static machine code
Most common way of building a small FSM
The FSM is entirely written in the programming language of the game as
static machine code
•  Interpreted
§  Use an interpreter that loads a file specifying the state graph and its
§ 
transitions. It’s often called data-driven.
The state machine is no longer fixed at runtime
•  Compiled
§  Hard-coded approach is fast and down to the metal, the interpreted
§ 
approach is flexible but slow
An FSM compiler can bridge the gap between the two approaches
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
F.E.A.R. States
• Goto
46
• Physical movement to a new location
•  Goto
• Animate
•  Animate
• Use Smart Object
• In-place
animation
§  Physical movement
to a localtion
§  In place animation
• Special-case of animate
•  Use Smart-Object
§  Special case of animate
•  People perceive the AI as smarter if they know what it is doing §  If one unit remains, say “I need reinforcements”
§  Introduce conversations instead of talking
§ 
“I am shot!” vs “what’s your status?”
• “A gamer posting to an internet forum expressed that they
If the AI is stuck he
saywas
“I have
nowhere
impressed
that to
thego!”
A.I. seem to actually understand
each other’s verbal communication. ‘Not only do they give
each other orders, but they actually DO what they’re told!’
Of course
is that it’s all smoke and mirrors, and
Prof. Pierthe
Lucareality
Lanzi
10 Reasons the Age of Finite State Machines is
Over
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
47
They’re Unorthodox
They’re Low-Level
Their Logic is Limited
They Require Custom Extensions
They Are Hard to Standardize
They Are Not Deliberative
They Have Concurrency Nightmares
They Scale Poorly
They Are Labor Intensive
Industry is Moving On
“FSMs, like any other technology, have their place in the game development process.
However, the age of finite state machines, where developers pick them by default to
implement their AI, is coming to an end. Scripts with coroutines are the most popular these
days, and hierarchical planners are increasingly making their way into games and
middleware.”
Prof. Pier Luca Lanzi
Evolution of Game AI
48
•  Fighting games were the first games to introduce complex AI
•  These were based on user action and position as well as the state
of the game
•  Approach 1: the user action and position was searched in a
lookup table and the best appropriate action (corresponding
reaction to user action)
•  Approach 2: minimax or minmax algorithm to search on the
possible statespace and determine the best possible action
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Black & White (2001, Richard Evans)
• 
• 
• 
• 
Peter Molineux’s famous “God Game”
Player determines fate of
villagers as their “God” (seen as a hand)
Creature can be taught
complex behaviour
Good and Evil - actions have
consequence
Prof. Pier Luca Lanzi
50
Black and White
51
•  Intelligent Villagers
§  Use large tables and situation calculus
§  If left alone they would do what they consider best for the village
§  You can bypass this by assigning a villager next to a tree
•  Intelligent Creatures AI
§  Given a variety of AI techniques, § 
the one most suitable for a task will be used for that task.
The creature uses respective perception for the user input and builds
decision trees that allow it to act
•  Symbolic Attributes
§  object.man-made.fence = 1.0
§  object.natural.body-of-water.shallow-river = 0.5
§  object.natural.rock = 0.1
Prof. Pier Luca Lanzi
pathfinding
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Pathfinding in Videogames
•  Four main representations
•  Grids
•  Waypoint graphs
•  Navigation meshes
•  Delaunay Triangulation
Prof. Pier Luca Lanzi
54
For Each Representation …
•  Localization
55
§  How do you find where you are in the representation?
•  Generation
§  How is the representation generated?
•  Dynamic changes
§  How are dynamic changes handled?
•  Planning
§  How are possible actions computed?
•  Memory
§  How much memory is required?
Prof. Pier Luca Lanzi
Grids
56
•  Very simple method to implement
•  Partially depends on your world representation
•  Used by Dragon Age series
•  Used in early Starcraft/Warcraft games
•  May not be able to represent 2.5/3d worlds
Prof. Pier Luca Lanzi
Waypoint Graphs
57
•  Set down pathfinding “nodes” in the world
•  Can only travel between these nodes
•  Referred to also as “Dirichlet domains” or points of visibility
Prof. Pier Luca Lanzi
Navigation Mesh 58
•  Break the world up into convex polygons
•  All area inside a polygon is considered passable
•  Constrained Delaunay Triangulation
§  Somewhat like a navigation mesh using only triangles
§  Maintain the properties of the Delaunay Triangulation
§  Constrained, because not all edges can be controlled
Prof. Pier Luca Lanzi
Some kinds of game worlds require
a ridiculous number of waypoints.
Large open areas usually require tons of waypoints sprinkled
throughout the game world to achieve adequate movement
Navigation meshes usually require few big polygons for these
kinds of areas which you can pathfind through quickly
Prof. Pier Luca Lanzi
Example of Pathfinding with Waypoints Prof. Pier Luca Lanzi
60
Example of Pathfinding with Waypoints Prof. Pier Luca Lanzi
61
Other Issues with Waypoints:
Obstacles
62
•  They don't allow path correction. That makes robust dynamic
obstacle avoidance difficult, if not impossible.
Prof. Pier Luca Lanzi
Other Issues with Waypoints:
Characters That Move Differently
• 
• 
• 
• 
63
They don't work robustly for characters that move differently
Suppose that you have a panzer (blue) and a soldier (red)
With a waypoint approach, you can't handle this robustly with a single
representation. You need to use totally separate waypoint networks for the
tanks and the soldiers. Add a new "motorcycle" unit, and now you need a third network
Prof. Pier Luca Lanzi
Combining Navigation Mesh
with Other Markers Prof. Pier Luca Lanzi
64
Questions About Navigation Meshes
• 
• 
• 
• 
Isn't it slower to do pathfinding
on a navigation mesh?
Not at all. A navigation mesh is a graph, just like a waypoint
graph is The core pathfinding routines are very similar
The difference is that a navigation mesh has a polygon associated with each graph node.
Prof. Pier Luca Lanzi
65
Some Games Using Navigation Mesh •  Halo 2
•  Halo 3
•  First Encounter Assault Recon (F.E.A.R.)
•  Counter-Strike: Source
•  Metroid Prime
•  Metroid Prime 2: Echoes
•  Metroid Prime 3: Corruption
•  Jak and Daxter: The Precursor Legacy
•  Jak II
•  Jak 3
•  Uncharted: Drake's Fortune
•  Scarface: The World is Yours
Prof. Pier Luca Lanzi
66
Tips on Path Finding
67
•  Computing shortest path in real-time is costly
•  Instead you can pre-compute all the costs for going from any
• 
• 
• 
node to any other node
Then during runtime, you can just look it up
You can also create a hierarchy of maps too
Question: How do you encode “nodes” into a terrain like in a
typical first-person-shooter?
Prof. Pier Luca Lanzi
Patroling AI
• 
• 
• 
• 
68
Pre-defined patrol routes.
When agent is in idle-mode, it can follow a pre-defined patrol path.
Multiple patrol paths can reduce
repetitiveness of agent.
Common in stealth games like Tenchu, Splinter Cell.
Prof. Pier Luca Lanzi
behavior trees
Prof. Pier Luca Lanzi
Behavior Trees are Everywhere
• 
• 
In Systems
Character AI
Squad Logic
Strategy AI
In-Game Tutor
AI Directors
Built-in in engines
CryEngine
Unity
Evolution Engine
§ 
§ 
§ 
§ 
§ 
• 
§ 
§ 
§ 
Prof. Pier Luca Lanzi
70
In Games
Halo 3 & ODST
[PROTOTYPE]
Spore
GTA: Chinatown Wars
The Bourne Conspiracy
SWAT 4, Bioshock
Dark Sector
§ 
§ 
§ 
§ 
§ 
§ 
§ 
Behavior Trees: Node Selection
71
•  Priority
§ Child nodes are evaluated in order until one validates
•  Sequential
§ First child is validated and executed
§ When it is finished, the next one is validated
•  Stochastic
§ All children are validated
§ A random node is selected among the valid ones
Prof. Pier Luca Lanzi
A Behavior Tree
72
RELAXED
SURVIVAL
COMBAT
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Behavior Trees
•  Behavior trees replace the often intangible growing mess of state
transitions of finite state machines (FSMs) with a more restrictive
but also more structured traversal defining approach
•  Behavior trees are formed by hierarchically organizing behavior
sub-trees consisting of nodes. Visiting a node, respectively the
sub-tree it roots, means running it according to its semantics.
•  Execution of a node or a sub behavior tree results in an
(aggregated) return state, e.g.: sequence, loops, random,
concurrent, decorator
Prof. Pier Luca Lanzi
Behavior Trees
•  Priority
§ Child nodes are evaluated in order until one validates
•  Sequential
§ First child is validated and executed
§ When it is finished, the next one is validated
•  Stochastic
§ All children are validated
§ A random node is selected among the valid ones
Prof. Pier Luca Lanzi
Leaf Nodes
• 
76
Actions Implement an actors or game world state changes, for example to plan a
path and move on it, to sense for the nearest enemies, to show certain
animations, switch weapons, or run a specified sound. Actions will typically coordinate and call into different game systems. They
might run for one simulation tick – one frame – or might need to be ticked
for multiple frames to finish their work.
Conditions Check that certain actor or game world states hold true. If a sequence node has a condition as one of its children then the failing of
the condition will prevent the following nodes from being traversed during
the update. When placed below a concurrent node, conditions become a kind of
invariant check that prevents its sibling nodes from running if a necessary
state becomes invalid.
§ 
§ 
• 
§ 
§ 
§ 
Prof. Pier Luca Lanzi
emotions, interaction, & pace
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
AI of Uncharted 3
•  "Yes, the enemy AIs have a set of rules that they wish to operate
by. They have a hierarchical sets of goals – they'll analyse the
terrain and make different choices each time about how to
approach the player character. It took a lot of work, we had to
disassemble the whole AI system – before, it was much more
scripted, we'd tell the enemies explicitly, go over here, flank the
player… now they have a choice."
Prof. Pier Luca Lanzi
Psychology – Emotional Modelling
80
•  Joy
§  Positive feedback, achieving instant peak, decreasing with time
•  Fear
§  Negative, gradual peak, happening mostly in safe environments
•  Other feelings
§  Success/failure, danger/safety, expectations, trust, surprise, disgust and
anticipation
•  Diversity
§  Two individuals never have the same emotional responce (they need to be randomised)
•  Stereotypical animation
§  Fear: rapid head motion, low posture, bulged eyes
§  Happiness: smiling straightened body, slow and graceful movement
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
AI for Dramatic Pacing in Left4Dead
• 
• 
• 
• 
• 
82
Coop game involving other three players
Algorithmically adjust game pacing on the fly to maximize drama
Based on the experience in counter strike
Constant, unchanging combat is fatiguing
Long period of inactivity are boring
§ 
§ 
Unpredictable peaks and valleys of intensity create a powerfully compelling and
replayable experience
Same scenario, almost same map, different experience
Prof. Pier Luca Lanzi
AI for Dramatic Pacing in Left4Dead
• 
• 
83
Algorithm
Estimate the “emotional intensity” of each survivor
Track the max intensity of all 4 survivor
If intensity is too high,
remove major threats for a while
!"#$%&!'(')#*'&+)#,#*-.&/#,'&0#.-(/&
1+#0*-2'&3)#,#*-.&4#.-(/
Otherwise, create an interesting
population of threats
§ 
§ 
§ 
§ 
5"6&*7'&18&3-)'.*")&,"+9$#*':&*7'&0"09$#*-"(&;#:'+&
The AI director creates the population based on the survivor team’s “emotional intensity”
Prof. Pier Luca Lanzi
Prof. Pier Luca Lanzi
Sometimes the AI is the Main Feature
• 
• 
• 
85
Evolutionary Algorithm – encode characteristics of a system as genes and uses
mutation and crossover in conjunction with a fitness function to help evolve to
a more “perfect system”
Can be used to evolve Artificial-Life creatures to make them “smarter.”
Behavior trees – encode the creaturebehavior as previously done in Halo 2
Prof. Pier Luca Lanzi