Download CS 430 Lecture 4

Lecture 4



SWE CECS Career Forum signup by Friday in
Dean's Office (KC-250)
Reminder: Homework 1 (4 problems) due next
Tuesday
Questions?
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
1
Outline

Chapter 2 - Intelligent Agents

Agents and Environments

Concept of Rationality

Nature of Environments

Structure of Agents
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
2
Agents and Environments

An agent is an entity that can be viewed as
perceiving its environment through sensors,
and acting upon that environment through
actuators.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
3
Agent Function and Agent Program



An agent's perceptual input at any given
instance is called a percept. The complete
history of everything an agent has seen (so far)
is its percept sequence.
An agent's behavior is described by a function
that receives a percept and returns an action.
In theory, this a table that maps all possible
percept sequences to actions.
An agent program is a concrete
implementation of the agent function.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
4
Vacuum-Cleaner World

Percepts


location (square A or square B), cleanliness state:
(Clean or Dirty)
Actions

move Left, move Right, Suck dirt
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
5
Concept of Rationality


A rational agent does the "right thing" for every
entry in the agent function table. The sequence
of actions results in environment states that are
desirable, a notion captured by a performance
measure.
Designing performance measures is hard.
Beware of unintended consequences.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
6
Concept of Rationality

Rationality depends on





Performance measure
Agent's prior knowledge of environment
Actions that can be performed
Percept sequence to date
Define rational agent as:

For each possible percept sequence, a rational
agent should select an action that is expected to
maximize its performance measure, given the
evidence provided by the percept sequence and
whatever built-in knowledge the agent has.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
7
Vacuum-Cleaner World

Does the table we generated represent a
rational agent?



First, we need a performance measure...
Assume the "geography" of environment is know a
priori, but not the distribution of dirt or initial location
of agent. Clean squares stay clean and sucking
cleans a square

Only available actions are Left, Right, Suck

Agent correctly perceives location and dirt.
If any of these "assumptions" are changed?
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
8
Concept of Rationality



Omniscience - knowing the actual outcome of actions is not part of rationality. With omniscience, can
achieve perfection and maximize actual performance
vs. expected performance.
Information gathering and exploration is part of
rationality, necessary for initially unknown
environments and to maximize performance.
Agents should learn from perception and become
autonomous - effectively independent of its a priori
knowledge.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
9
Nature of Environments


Task environments are specified by

Performance measure

Environment

Actuators

Sensors
Give a PEAS description for an automated taxi
driver. For a medical diagnosis system. For an
interactive English tutor.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
10
Characteristics of Environments

Fully observable vs. Partially observable


Can the agent "see" the entire environment relevant
to performance measure.
Single agent vs. Multi-agent

Are other participants agents or are they
environment that obey "laws"? Are the other
agents competitive or cooperative? Is there
communication between agents?
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
11
Characteristics of Environments

Deterministic vs. Stochastic


Is the next state of environment completely
determined by the current state and the agent
action? Environments that are not fully observable
or not deterministic are said to be uncertain.
Episodic vs. Sequential

In episodic environments, agent experience is
divided into atomic epsiodes that to not effect each
other. In sequential environments, current decision
can potentially effect all future decisions.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
12
Characteristics of Environments

Static vs. dynamic


Does the environment change while the agent is
making a decision? If the environment does not
change, but the performance score does, said to be
semidynamic.
Discrete vs. Continuous

Refers to the state of the environment, to the way
time is handled, and to percepts and actions of the
agent.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
13
Characteristics of Environments

Known vs. unknown


Does the agent know the "laws of physics" for the
environment? Note: not the same as fully
observable vs. partially observable.
Not too surprisingly, hardest case is partially
observable, multiagent, stochastic, sequential,
dynamic, continuous, and unknown
environment. E.g. driving a rented car in a new
country with unfamiliar geography and traffic
laws.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
14
Characteristics of Environments

What are the characteristics of the
environments for the following?

Solving crossword puzzles

Playing backgammon

Playing poker

Medical diagnosis

Interactive English tutor

Taxi driving
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
15
Structure of Agents


Agent function describes behavior. Job of AI
is to design agent program that implements
agent function on a particular architecture
(computing device with physical sensors and
actuators).
Architecture makes the percepts from the
sensors available to the program, runs the
program, and feeds the program's action
choices to the actuators as they are generated.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
16
Table-Driven-Agent Program
Receives: percept
Returns: action
Static data:


percepts, a sequence of percepts, initially empty
table, a table of actions, indexed by percepts,
initially fully specified
1. Append percept to percepts
2. action = Lookup (percepts, table)
3. Return action
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
17
Structure of Agents


Although Table-Driven-Agent is not feasible, it
provides the basis for understanding how well
possible programs implement the agent
function.
Goal is to produce rational behavior from a
smallish program rather than vast tables,
analogous to replacing math tables with
calculator algorithms.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
18
Structure of Agents


Basic kinds of agents

Simple reflex agents

Model-based reflex agents

Goal-based agents

Utility-based agents
The differences are in how much internal state
is stored and how much processing is done to
decide what the result action is.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
19
Simple Reflex Agent


Simplest kind of agent. Selects actions on the
basis of the current percept. Ignores the
percept history.
Example: Reflex-Vacuum-Agent Program
Receives: [location, status]
Returns: action
1. If status = Dirty then return Suck
2. else if location = A then return Right
3. else if location = B then return Left
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
20
Simple-Reflex-Agent Program
Receives: percept
Returns: action
Static data:

rules, a set of
condition-action rules
1. state = InterpretInput(percept)
2. rule = RuleMatch (state, rules)
3. action = rule.Action
4. Return action
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
21
Simple Reflex Agent

Issues


When can a simple reflex agent make a correct
decision?
What can happen when the simple reflex agent
cannot make a correct decision?
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
22
Model-Based Reflex Agent


Handle partially observability by keeping track
of what has been seen. Maintain an internal
state that depends on the percept history.
Requires two kinds of knowledge to be
encoded in agent program.

How the world evolves independently of the agent

How the agent's actions affect the world
Forms a model of the world.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
23
Model-Based-Reflex-Agent Program
Receives: percept
Returns: action
Static data:




state, current world state
model, how next state is related
to current state and action
rules, a set of condition-action
rules
action, most recent action,
initially none
1. state = UpdateState (state, action, percept, model)
2. rule = RuleMatch (state, rules)
3. action = rule.Action
4. Return action
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
24
Model-Based Reflex Agent

Issues


Even with model, agent cannot determine exact
state of partially observable environment. It is a
"best guess".
Model does not have to be literal.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
25
Goal-Based Agents


Knowing the current state of environment is not
always enough to decide what to do next.
Correct decision often depends on what state
the agent it trying to get to.
Add one or more goals that describe situations
that are desirable. May require searching and
planning to determine correct action.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
26
Model-Based, Goal-Based Agent

Though more complicated, making information
about the future explicit makes the agent more
flexible.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
27
Utility-Based Agents



Goals are not enough to produce high-quality
behavior. Often many action sequences will
result in achieving a goal, but some are better
than others.
Goals distinguish between "happy" and
"unhappy" states. Want to know exactly how
"happy" in making decisions. Use the term
utility to sound more scientific.
Utility function is an internalization of the
performance measure used to compare states.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
28
Utility-Based Agents

Not the only way to be rational, but provides
more flexibility. E.g., provides tradeoffs when
there are conflicting goals or uncertainty in
achieving goals.
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
29
Learning Agents


Preferred method of for creating systems in many
areas of AI. Allows agent to operate in an initially
unknown environment.
Four conceptual components




Learning element – responsible for making improvements
Performance element – responsible for selecting actions
(previous kinds of agents)
Critic – provides feedback by comparing against external
performance standard
Problem generator – suggests actions for exploration
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
30
Learning Agents

Learning element can change any of the
"knowledge" components. Goal is to bring
components in closer agreement with available
feedback, thus improving overall performance
Thursday, January 19
CS 430 Artificial Intelligence - Lecture 4
31