Download lecture03 - University of Virginia, Department of Computer Science

CS 416
Artificial Intelligence
Lecture 3
Agents
History of AI
Read the complete story in text
• Early to mid 50s
– Alan Turing, Marvin Minsky (student of von Neumann),
John McCarthy, Alan Newell, Herbert Simon
History of AI: 1952- 1969
Great successes!
• Solving hard math problems
• game playing
• LISP was invented by McCarthy (1958)
– second oldest language in existence
– could accept new axioms at runtime
• McCarthy went to MIT and Marvin Minsky started lab at Stanford
– Both powerhouses in AI to this day
History of AI: 1966 - 1973
A dose of reality – Overhyped
• Systems fail to play chess and translate Russian
– Computers were ignorant to context of their logic
– Problems were intractable
 algorithms that work in principle may not work in practice
 Combinatorial Explosion / Curse of Dimensionality
– Fatal flaw in neural networks was exposed
 though flaw was first resolved in 1969, neural networks did not
return to vogue until late 1980s
AI History: 1969 - 1979
Knowledge-based Systems (Expert systems)
• Problem: General logical algorithms could not be applied to
realistic problems
• Solution: accumulate specific logical algorithms
– DENDRAL – infer chemical structure
 knowledge of scientists boiled down to cookbook logic
 large number of special purpose rules worked well
AI History: 1980 - present
Let the good times roll
• The demonstrated success of AI invited investments
– from millions to billions of dollars in 10 years
• extravagant AI promises again led to “AI Winter” when
investments in technology dropped (1988)
Neural Networks came back from the dead (1986)
AI History: 1987 - present
AI becomes a science
• More repeatability of experiments
• More development of mathematical underpinnings
• Reuse of time-tested models
Intelligent Agents (1994)
• AI systems exist in real environments with real sensory inputs
• Niches of AI need to be reorganized
AI History: Where are We Now?
• Autonomous planning: scheduling operations aboard a robot
– Some notable failures (Dante falls in a crater after one
step) and shining successes (Mars Spirit Rover)
• Game playing: Kasparov lost to IBM’s Big Blue in chess
– Rules were changed to prevent computer from retraining
over night and to provide human players with more
examples of computerized play
AI History: Where are We Now?
• Autonomous Control: CMU’s NAVLAB drove from Pittsburgh
to San Francisco under computer control 98% of time
• Stanford vehicle wins 2006 DARPA Grand Challenge
CMU’s 2005 vehicle falls crashes at starting line
• Logistics: deployment of troops to Iraq
• Robotics: remote heart operations
• human genome, protein folding, drug discovery
• stock market
Review
We’ll study systems that act rationally
• They need not necessarily “think” or act like humans
• They need not “think” in rational ways
The domain of AI research changes over time
AI research draws from many fields
• Philosophy, psychology, neuroscience, mathematics,
economics, mechanical, linguistics
AI has had ups and downs since 1950
Outline
1. What is an agent?
2. What is rationality?
3. Where do agents operate?
4. What goes into an agent?
1. What is an agent?
Perception
• Sensors receive input from environment
– Keyboard clicks
– Camera data
– Bump sensor
Action
• Actuators impact the environment
– Move a robotic arm
– Generate output for computer display
Perception
Percept
• Perceptual inputs at an instant
• May include perception of internal state
Percept Sequence
• Complete history of all prior percepts
Do you need a percept sequence to play Chess?
An agent as a function
Agent maps percept sequence to action
• Agent:
– Set of all inputs known as state space
• Repeating loop:
We must construct f( ), our agent
• It must act rationally
The agent’s environment
What is known about percepts?
• Quantity, range, certainty…
– If percepts are finite, could a table store mapping?
What is known about environment?
• Is f (a, e) a known function and predictable?
– Do you know what effect your actions will have?
More on this later
2. Is your agent rational?
We agree on what an agent must do
Can we evaluate its quality?
Performance Metrics
• Very Important
• Frequently the hardest part of the research problem
• Design these to suit what you really want to happen
Performance vis-à-vis rationality
For each percept sequence, a rational agent
should select an action that maximizes its
performance measure
Example: autonomous vacuum cleaner
• What is the performance measure?
• Penalty for eating the cat? How much?
• Penalty for missing a spot?
• Reward for speed?
• Reward for conserving power?
Learning and Autonomy
Learning
• To update the agent function,
, in light of observed
performance of percept-sequence to action pairs
– Learn what you know better
 Learn from focused trial and error
– Learn to distinguish what you don’t know
 What parts of state space to explore?
Adding intelligence to agent
function
At design time
• Some agents are designed with clear procedure to improve
performance over time. Really the engineer’s intelligence.
– Camera-based user identification
At runtime
• With experience, agent changes its program (parameters)
3. Where do agents operate?
An effective way to classify agents
• What does “where” mean?
– What do inputs to agent look like?
 Discrete, repetitive, noisy, etc.
How big is your percept?
Dung Beetle
• Almost no perception (percept)
– Rational agents fine-tune actions based on feedback
Sphex Wasp
• Has percepts, but lacks percept sequence
– Rational agents change plans entirely when fine tuning fails
A Dog
• Equipped with percepts and percept sequences
– Reacts to environment and can significantly alter behavior
Qualities of a task environment (1)
Fully Observable
• Agent need not store any aspects of state
– Hansel and Gretel’s bread crumbs
– Volume of observables may be overwhelming
Partially Observable
• Some data is unavailable
– Maze
– Noisy sensors
Qualities of a task environment (2)
Deterministic
• Always the same outcome for environment/action pair
Stochastic
• Not always predictable – random
Partially Observable vs. Stochastic
• My cats think the world is stochastic
– It’s really only partially observable by them
• Physicists think the world is deterministic
– Somewhere there is a “god function” that explains it all
Qualities of a task environment (3)
Markovian
• Future environment depends only on current environment and action
Episodic
• Percept sequence can be segmented into independent temporal
categories
– Behavior at traffic light independent of previous traffic
Sequential
• Current decision could affect all future decisions
Qualities of a task environment (4)
Static
• Environment doesn’t change over time
– Crossword puzzle
Dynamic
• Environment changes over time
– Driving a car
Semi-dynamic
• Environment is static, but performance metrics are dynamic
– You never make a second first impression
Qualities of a task environment (5)
Discrete
• Values of a state space feature (dimension) are constrained
to distinct values from a finite set
– Blackjack:
Continuous
• Variable has infinite variation
– Antilock brakes:
– Are computers really continuous?
Qualities of a task environment
Towards a terse description of problem domains
• Environment: features, dimensionality, degrees of freedom
• Observable?
• Predictable?
• Dynamic?
• Continuous?
• Performance metric
4. Building Agent Programs
Example: the table approach
• Build a table mapping states to actions
– Chess has 10150 entries (1080 atoms in the universe)
– I’ve said memory is free, but keep it within the confines of
the boundable universe
• Still, tables have their place
Discuss four agent program principles
Simple Reflex Agents
• Sense environment
• Match sensations with rules in database
• Rule prescribes an action
Reflexes can be bad
• Don’t catch a falling iron.
Inaccurate information
• Misperception can trigger reflex when inappropriate
But rules databases can be made large and complex
Simple Reflex Agents w/ Incomplete
Sensing
How can you react to things you cannot see?
• Vacuum cleaning the room w/o any sensors
• Vacuum cleaning room w/ bump sensor
• Vacuum cleaning room w/ GPS and perfect map of static
environment
Randomized actions are very useful here
Model-based Reflex Agents
So when you can’t see something, you model it!
• Create an internal variable to store your expectation of
variables you can’t observe
• If I throw a ball to you and it falls short, do I know why?
– I don’t really know why…
 Aerodynamics, mass, my energy levels…
– I do have a model
 Ball falls short, throw harder
Model-based Reflex Agents
Admit it, you can’t see and understand everything
Models are very important!
• We all use models to get through our lives
– Psychologists have many names for these contextsensitive models
• Agents need models too
Goal-based Agents
Overall goal is known, but lacking moment-to-moment
performance measure
• Don’t exactly know what performance maximizing action is at each step
Example:
• How to get from A to B?
– Current actions have future consequences
– Search and Planning are used to explore paths through state space
from A to B
Utility-based Agents
Goal-directed agents that have a utility function
• Function that maps internal and external states into a scalar
– A scalar is a number used to make moment-to-moment
evaluations of candidate actions
Learning Agents
Desirable to build a system that “figures it out”
• Generalizable
• Compensates for absence of designer knowledge
• Reusable
• Learning by example isn’t easy to accomplish
– What exercises do you do to learn?
– What outcomes do you observe?
– What inputs to your alter?
Learning Agents
Performance Element
•
Selecting actions (this is the “agent” we’ve been discussing)
Problem Generator
•
Provides suggestions for new tasks to explore state space
Critic
•
Provides learning element with feedback about progress (are we doing
good things or should we try something else?)
Learning Element
•
Making improvements (how is agent changed based on experience)
A taxi driver
Performance Element
•
Knowledge of how to drive in traffic
Problem Generator
•
Proposes new routes to try to hopefully improve driving skills
Critic
•
Observes tips from customers and horn honking from other cars
Learning Element
•
Relates low tips to actions that may be the cause
Review
Outlined families of AI problems and solutions
I consider AI to be a problem of searching
• Countless things differentiate search problems
– Number of percepts, number of actions, amount of a priori
knowledge, predictability of world…
• Textbook is divided into sections based on these differences
Sections of book
• Problem solving: Searching through predictable, discrete environments
• Knowledge and Reasoning: Searching when a model of the world is
known
– a leads to b and b leads to c… so go to a to reach c
• Planning: Refining search techniques to take advantage of domain
knowledge
• Uncertainty: Using statistics and observations to collect knowledge
• Learning: Using observations to understand the way the world works
and to act rationally within it