Download AI Chapter 3: Intelligent Agents - Foundations of Artificial Intelligence

Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Agenda
Artificial Intelligence
3. Intelligent Agents
1
Introduction
2
What is an Agent?
3
What is a Rational Agent?
4
Different Classes of Agents
5
Different Classes of Environments
6
Conclusion
Let’s Get a Little Orientation Here
Jörg Hoffmann
Wolfgang Wahlster
Summer Term 2017
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
1/34
References
Introduction
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
2/34
References
Disclaimer
→ A central aspect of intelligence (and one possible way to define it) is
the ability to act successfully in the world (cf. Chapter 1).
This chapter provides a broad categorization of, and terminology
for, talking about that:
Useful for a research field to establish a common language.
→ The concepts and explanations in this chapter are very broad and
superficial. In the remainder of the course, we will consider in detail
particular problems relating to decision-making, and specify algorithms
for solving these.
Useful for you to get a rough overview.
→ We’ll think a bit about what it means to “behave rationally”, what
are possible architectures for achieving such behavior with a computer,
and what are relevant properties of the world we need to act in.
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
4/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
5/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
What is an “Agent” in AI?
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Rational Agents . . .
Agents:
. . . do “the right thing”!
Perceive the environment through sensors (→ percepts).
Act upon the environment through actuators (→ actions).
Agent
Sensors
→ Meaning of “do the right thing”: Rational agents select their actions
so as to maximize a performance measure.
→ What’s the performance measure of an autonomous vacuum cleaner?
Percepts
Environment
?
Actions
Actuators
→ Examples?
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
7/34
References
Actually, Rational Agents . . .
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
9/34
References
So, What Is a Rational Agent?
. . . ATTEMPT to do “the right thing”!
Mapping your input to the best possible output:
→ The hypothetical best case (“the right thing”) is often unattainable.
→ The agent might not be able to perceive all relevant information. (Is
there dirt under this bed? Is this a hamster or a shoe?)
Rationality vs. Omniscience:
An omniscient agent knows everything about the enviroment, and
knows the actual effects of its actions.
A rational agent just makes the best of what it has at its disposal,
maximizing expected performance given its percepts and knowledge.
Performance measure × Percepts × Knowledge → Action
An agent has a performance measure M and a set A of possible
actions. Given a percept sequence P , as well as knowledge K about
the world, it selects an action a ∈ A.
The action a is optimal if it maximizes the expected value of M ,
given the evidence provided by P and K. The agent is rational if it
always chooses an optimal a.
→ If the vacuum cleaner bumps into the Hamster, then this can be
rational in case
→ Note: If observation actions are required, they are elements of A, i.e.,
the agent must perceive actively. Example:
→ Example?
Hoffmann and Wahlster
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
10/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
11/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
A Rational Agent is an Optimal Action Choice Function?
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Examples of Agents: PEAS Descriptions
We also need to realize the agent, through:
an agent program, executed on
an architecture which also provides an interface to the environment
(percepts, actions).
→ Agent = Architecture + Program.
Practical limitations:
Our definition captures limitations on percepts and knowledge.
It does not capture computational limitations (often, determining an
optimal choice would take too much time/memory).
→ In practice, we often merely approximate the rational decision.
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
12/34
References
Questionnaire
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
13/34
References
Questionnaire Answers
Question!
Which are agents?
(A): James Bond.
(C): Vacuum cleaner.
(B): Your dog.
(D): Thermometer.
Question!
Who is rational?
(A): James Bond, crossing the
street without looking.
(C): Vacuum cleaner, deciding
to clean under your bed.
(B): Your dog, crossing the
street without looking.
(D): Thermostat, deciding to
cool down your fridge.
Hoffmann and Wahlster
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
14/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
15/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Table-Driven Agents
Introduction
Agents
Rational Agents
Classes of Agents
Environments
More useful, but still very simple, method for choosing actions:
Condition-Action Rules (note: raw sensor data interpreted, using a
world model, prior to evaluating the rules)
Sensors
append percept to the end of percepts
action ← L OOKUP( percepts , table)
return action
State
What my actions do
→ Why is this not a good idea?
What action I
should do now
Condition-action rules
function R EFLEX -VACUUM -AGENT( [location,status ]) returns an action
Agent
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left
The agent
program
for a simple reflex Chapter
agent in3:the
two-state
vacuum environment.
Artificial
Intelligence
Intelligent
Agents
17/34 This
program implements the agent function tabulated in Figure ??.
Classes of Agents
Environments
Conclusion
References
function S IMPLE
-R EFLEX -AGENT( percept ) returns an action
Goal-Based
Agents
persistent: rules, a set of condition–action rules
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
18/34
References
Learning Agents
state ← I NTERPRET-I NPUT( percept )
Often, doing
the right thing requires considering the future:
rule ← RULE -M ATCH(state, rules)
action ← rule.ACTION
return action
Actuators
→ Example?
Figure
2.4
Hoffmann and
Wahlster
Rational Agents
Environment
What the world
is like now
How the world evolves
Figure 2.3 The TABLE -D RIVEN -AGENT program is invoked for each new percept and returns an
action each time. It retains the complete percept sequence in memory.
Agents
References
Reflex Agents
function TABLE -D RIVEN -AGENT( percept ) returns an action
persistent: percepts , a sequence, initially empty
table, a table of actions, indexed by percept sequences, initially fully specified
Introduction
Conclusion
It is typically useful to be able to learn: (extreme case: start with 0
knowledge in an unknown environment)
Sensors
Performance standard
What the world
Critic
State
What it will be like
if I do action A
2
Goals
Agent
Sensors
feedback
changes
Learning
element
What action I
should do now
knowledge
learning
goals
Problem
generator
Actuators
Actuators
Agent
→ Example?
Hoffmann and Wahlster
Performance
element
Environment
What my actions do
Environment
Figure 2.6
A How
simple
reflex
agent. It acts according
is like now to a rule whose condition matches the current
the world
evolves
state, as defined by the percept.
→ Performance element: Selects actions (exploitation). Learning element:
Learns new knowledge. Critic: Measures performance. Problem generator:
Suggests actions favoring informative learning experiences (exploration).
Artificial Intelligence
Chapter 3: Intelligent Agents
19/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
20/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Domain-Specific vs. General Agents
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Questionnaire
Question!
What kind of agent are you?
(A): Table-Driven
(C): Learning
vs.
Solver specific to a particular
problem (“domain”).
vs.
Solver based on description in a
general problem-description
language (e.g., the rules of any
board game).
More efficient.
vs.
More intelligent.
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
21/34
References
The Environment of Rational Agents
Hoffmann and Wahlster
Introduction
Agents
(B): Reflex Agent
(D): Domain-Specific
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
22/34
References
The Environment of Rational Agents, ctd.
Accessible vs. inaccessible (fully observable vs. partially observable)
Static vs. dynamic
Are the relevant aspects of the environment accessible to the
sensors?
Can the environment change while the agent is deliberating?
If the environment does not change, but the agent’s performance
score changes, the environment is called semi-dynamic.
Deterministic vs. stochastic
Is the next state of the environment completely determined by the
current state and the selected action?
Discrete vs. continuous
Is the environment discrete or continuous?
If the only non-determinism are actions of other agents, the
environment is called strategic.
Single agent vs. multi-agent
Is there just one agent, or several of them?
Episodic vs. sequential
There are competitive and cooperative multi-agent scenarios.
Can the quality of an action be evaluated within an episode
(perception + action), or are future developments decisive?
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
24/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
25/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Examples of Environments
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Classifying AI Areas
Many sub-areas of AI can be classified by:
Domain-specific vs. general.
The environment.
(Particular agent architectures sometimes also play a role, especially
in Robotics.)
→ The same is true of the sub-topics in this course. The focus is on
general methods (a bias in much of the AI field), and simple
environments (after all, it’s an introductory course only).
→ Up next: A rough classification of our topics, in these terms.
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
26/34
References
Classifying AI Areas: Our Topics
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
27/34
References
Classifying AI Areas: Our Topics
Planning
Environment:
Classical Search
Environment:
Fully observable vs. partially observable.
Fully observable vs. partially observable.
Deterministic vs. stochastic.
Deterministic vs. stochastic.
Episodic vs. sequential.
Episodic vs. sequential.
Static vs. dynamic.
Static vs. dynamic.
Discrete vs. continuous.
Discrete vs. continuous.
Single-agent vs. multi-agent.
Single-agent vs. multi-agent.
Approach:
Approach:
Domain-specific vs. general.
Domain-specific vs. general.
→ Planning formalisms and approaches exist also for any and all of partial
observability, and stochastic/dynamic/continuous/multi-agent settings.
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
28/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
28/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Classifying AI Areas: Our Topics
Introduction
Agents
Rational Agents
Classes of Agents
Environments
General Game Playing (not covered in this course)
Environment:
Fully observable vs. partially observable.
Fully observable vs. partially observable.
Deterministic vs. stochastic.
Deterministic vs. stochastic.
Episodic vs. sequential.
Episodic vs. sequential.
Static vs. dynamic.
Static vs. dynamic.
Discrete vs. continuous.
Discrete vs. continuous.
Single-agent vs. multi-agent.
Single-agent vs. multi-agent.
Approach:
Approach:
Domain-specific vs. general.
Domain-specific vs. general.
→ Adversarial search formalisms and approaches exist also for partial
observability and stochastic settings.
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
→ General game playing formalisms and approaches exist also for partial
observability and stochastic settings.
28/34
References
Classifying AI Areas: Our Topics
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Chapter 3: Intelligent Agents
Classes of Agents
Environments
Conclusion
28/34
References
Classifying AI Areas: Our Topics
Constraint Satisfaction & Reasoning
Environment:
Probabilistic Reasoning (not covered in this course)
Environment:
Fully observable vs. partially observable.
Fully observable vs. partially observable.
Deterministic vs. stochastic.
Deterministic vs. stochastic.
Episodic vs. sequential.
Episodic vs. sequential.
Static vs. dynamic.
Static vs. dynamic.
Discrete vs. continuous.
Discrete vs. continuous.
Single-agent vs. multi-agent.
Single-agent vs. multi-agent.
Approach:
Approach:
Domain-specific vs. general.
Hoffmann and Wahlster
References
Classifying AI Areas: Our Topics
Adversarial Search
Environment:
Hoffmann and Wahlster
Conclusion
Artificial Intelligence
Domain-specific vs. general.
Chapter 3: Intelligent Agents
28/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
28/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Questionnaire
(B): Episodic.
(D): Single-Agent.
Question!
Your own environment is?
(A): Fully Observable.
(C): Static.
(B): Episodic.
(D): Single-Agent.
Introduction
Agents
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
Questionnaire Answers
Question!
James Bond’s environment is?
(A): Fully Observable.
(C): Static.
Hoffmann and Wahlster
Introduction
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
29/34
References
Summary
Hoffmann and Wahlster
Introduction
Agents
Artificial Intelligence
Rational Agents
Classes of Agents
Chapter 3: Intelligent Agents
Environments
Conclusion
30/34
References
Reading
An agent is something that perceives and acts. It consists of an
architecture and an agent program.
A rational agent always takes the action that maximizes its expected
performance, subject to the percept sequence and its environment
knowledge.
There are a variety of agent designs:
Reflex agents respond to percepts by condition-action rules.
Goal-based agents work towards goals.
Utility-based agents make trade-offs using a utility function.
Learning agents improve their behavior over time.
Chapter 2: Intelligent Agents [Russell and Norvig (2010)].
Content: A much more detailed description of the subjects I
overviewed here (agents, agent architectures, environments). Just as
broad and superficial, though.
Some environments are more demanding than others . . .
. . . your own, and that of James Bond, are the most difficult.
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
32/34
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
33/34
Introduction
Agents
Rational Agents
Classes of Agents
Environments
Conclusion
References
References I
Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach (Third
Edition). Prentice-Hall, Englewood Cliffs, NJ, 2010.
Hoffmann and Wahlster
Artificial Intelligence
Chapter 3: Intelligent Agents
34/34