Download Artificial Intelligence - KDD

Lecture 1
Artificial Intelligence:
Foundations Review
Wednesday, January 17, 2001
William H. Hsu
Department of Computing and Information Sciences, KSU
http://www.cis.ksu.edu/~bhsu
Reading for Next Class:
Chapter 2, Russell and Norvig
Handout, Artificial Intelligence (3e), P. H. Winston
Handout, Essentials of Artificial Intelligence, M. Ginsberg
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Course Outline
•
Overview: Intelligent Systems and Applications
•
Artificial Intelligence (AI) Software Development Topics
– Knowledge representation
• Logical
• Probabilistic
– Search
• Problem solving by (heuristic) state space search
• Game tree search
– Planning: classical, universal
– Machine learning
• Models (decision trees, version spaces, ANNs, genetic programming)
• Applications: pattern recognition, planning, data mining and decision support
– Topics in applied AI
• Computer vision fundamentals
• Natural language processing (NLP) and language learning survey
•
Practicum (Short Software Implementation Project)
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Questions Addressed
•
Problem Area
– What are intelligent systems and agents?
– Why are we interested in developing them?
•
Methodologies
– What kind of software is involved? What kind of math?
– How do we develop it (software, repertoire of techniques)?
– Who uses AI? (Who are practitioners in academia, industry, government?)
•
Artificial Intelligence as A Science
– What is AI?
– What does it have to do with intelligence? Learning? Problem solving?
– What are some interesting problems to which intelligent systems can be applied?
– Should I be interested in AI (and if so, why)?
•
Today: Brief Tour of AI History
– Study of intelligence (classical age to present), AI systems (1940-present)
– Viewpoints: philosophy, math, psychology, engineering, linguistics
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
What is AI? [1]
•
Four Categories of Systemic Definitions
–
–
–
–
•
1. Think like humans
2. Act like humans
3. Think rationally
4. Act rationally
Thinking Like Humans
– Machines with minds (Haugeland, 1985)
– Automation of “decision making, problem solving, learning…” (Bellman, 1978)
•
Acting Like Humans
– Functions that require intelligence when performed by people (Kurzweil, 1990)
– Making computers do things people currently do better (Rich and Knight, 1991)
•
Thinking Rationally
– Computational models of mental faculties (Charniak and McDermott, 1985)
– Computations that make it possible to perceive, reason, and act (Winston, 1992)
•
Acting Rationally
– Explaining, emulating intelligent behavior via computation (Schalkoff, 1990)
– Branch of CS concerned with automation of intelligent behavior
(Luger and Stubblefield, 1993)
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
What is AI? [2]
Thinking and Acting Like Humans
•
Concerns: Human Performance (Figure 1.1 R&N, Left-Hand Side)
– Top: thought processes and reasoning (learning and inference)
– Bottom: behavior (interacting with environment)
•
Machines With Minds
– Cognitive modelling
• Early historical examples: problem solvers (see R&N Section 1.1)
• Application (and one driving force) of cognitive science
– Deeper questions
• What is intelligence?
• What is consciousness?
•
Acting Humanly: The Turing Test Approach
– Capabilities required
• Natural language processing
• Knowledge representation
• Automated reasoning
• Machine learning
– Turing Test: can a machine appear indistinguishable from a human to an
experimenter?
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
What is AI? [3]
Viewpoints on Defining Intelligence
•
Genuine versus Illusory Intelligence
– Can we tell?
• If so, how?
• If not, what limitations do we postulate?
– The argument from disability (“a machine can never do X”)
•
Turing Test Specification
– Objective: develop intelligent system “indistiguishable from human”
• Blind interrogation scenario (no direct physical interaction – “teletype”)
• 1 AI system, 1 human subject, 1 interrogator
• Variant: total Turing Test (perceptual interaction: video, tactile interface)
– Is this a reasonable test of intelligence?
– Details: Section 26.3, R&N
– See also: Loebner Prize page
•
Searle’s Chinese Room
– Philosophical issue: is (human) intelligence a pure artifact of symbolic
manipulation?
– Details: Section 26.4, R&N
– See also: consciousness in AI resources
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
What is AI? [3]
Thinking and Acting Rationally
•
Concerns: Human Performance (Figure 1.1 R&N, Right-Hand Side)
– Top: thought processes and reasoning (learning and inference)
– Bottom: behavior (interacting with environment)
•
Computational Cognitive Modelling
– Rational ideal
• In this course: rational agents
• Advanced topics: learning, utility theory, decision theory
– Basic mathematical, computational models
• Decisions: automata (Chomsky hierarchy – FSA, PDA, LBA, Turing machine)
• Search
• Concept learning
•
Acting Rationally: The Rational Agent Approach
– Rational action: acting to achieve one’s goals, given one’s beliefs
– Agent: entity that perceives and acts
– Focus of next lecture
• “Laws of thought” approach to AI: correct inferences (reasoning)
• Rationality not limited to correct inference
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Intelligent Agents:
Overview
•
Agent: Definition
– Any entity that perceives its environment through sensors and acts upon that
environment through effectors
– Examples (class discussion): human, robotic, software agents
•
Perception
– Signal from environment
– May exceed sensory capacity
•
Percepts
Sensors
– Acquires percepts
– Possible limitations
•
Action
Sensors
?
Environment
Agent
– Attempts to affect environment
– Usually exceeds effector capacity
•
Effectors
Actions
Effectors
– Transmits actions
– Possible limitations
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Application:
Knowledge Discovery in Databases
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Rule and Decision Tree Learning
•
Example: Rule Acquisition from Historical Data
•
Data
– Customer 103 (visit = 1): Age 23, Previous-Purchase: no, Marital-Status: single,
Children: none, Annual-Income: 20000, Purchase-Interests: unknown, StoreCredit-Card: no, Homeowner: unknown
– Customer 103 (visit = 2): Age 23, Previous-Purchase: no, Marital-Status: married,
Children: none, Annual-Income: 20000: Purchase-Interests: car, Store-CreditCard: yes, Homeowner: no
– Customer 103 (visit = n): Age 24, Previous-Purchase: yes, Marital-Status: married,
Children: yes, Annual-Income: 75000, Purchase-Interests: television, Store-CreditCard: yes, Homeowner: no, Computer-Sales-Target: YES
•
Learned Rule
– IF customer has made a previous purchase, AND customer has an annual income
over $25000, AND customer is interested in buying home electronics
THEN probability of computer sale is 0.5
– Training set: 26/41 = 0.634, test set: 12/20 = 0.600
– Typical application: target marketing
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Text Mining:
Information Retrieval and Filtering
•
20 USENET Newsgroups
–
comp.graphics
misc.forsale
soc.religion.christian sci.space
–
comp.os.ms-windows.misc
rec.autos
talk.politics.guns
–
comp.sys.ibm.pc.hardware
rec.motorcycles
talk.politics.mideast sci.electronics
–
comp.sys.mac.hardware
rec.sports.baseball
talk.politics.misc
–
comp.windows.x
rec.sports.hockey
talk.religion.misc
–
•
sci.crypt
sci.med
alt.atheism
Problem Definition [Joachims, 1996]
– Given: 1000 training documents (posts) from each group
– Return: classifier for new documents that identifies the group it belongs to
•
Example: Recent Article from comp.graphics.algorithms
Hi all
I'm writing an adaptive marching cube algorithm, which must deal with cracks. I got the vertices of the
cracks in a list (one list per crack).
Does there exist an algorithm to triangulate a concave polygon ? Or how can I bisect the polygon so, that I
get a set of connected convex polygons.
The cases of occuring polygons are these:
...
•
Performance of Newsweeder (Naïve Bayes): 89% Accuracy
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Specifying A Learning Problem
•
Learning = Improving with Experience at Some Task
– Improve over task T,
– with respect to performance measure P,
– based on experience E.
•
Example: Learning to Filter Spam Articles
– T: analyze USENET newsgroup posts
– P: function of classification accuracy (discounted error function)
– E: training corpus of labeled news files (e.g., annotated from Deja.com)
•
Refining the Problem Specification: Issues
– What experience?
– What exactly should be learned?
– How shall it be represented?
– What specific algorithm to learn it?
•
Defining the Problem Milieu
– Performance element: How shall the results of learning be applied?
– How shall the performance element be evaluated? The learning system?
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Survey of Machine Learning Methodologies
•
Supervised (Focus of CIS730)
– What is learned? Classification function; other models
– Inputs and outputs? Learning: examples x,f x   approximat ion fˆx 
– How is it learned? Presentation of examples to learner (by teacher)
– Projects: MLC++ and NCSA D2K; wrapper, clickstream mining applications
•
Unsupervised (Surveyed in CIS730)
– Cluster definition, or vector quantization function (codebook)
– Learning: observatio ns x  distance metric d x , x   discrete codebook f x 
1
2
– Formation, segmentation, labeling of clusters based on observations, metric
– Projects: NCSA D2K; info retrieval (IR), Bayesian network learning applications
•
Reinforcement (Not Emphasized in CIS730)
– Control policy (function from states of the world to actions)


– Learning: state/reward sequence si ,ri : 1  i  n  policy p : s  a
– (Delayed) feedback of reward values to agent based on actions selected; model
updated based on reward, (partially) observable state
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Related Online Resources
•
Research
– KSU Laboratory for Knowledge Discovery in Databases
http://www.kddresearch.org (see especially Group Info, Web Resources)
– KD Nuggets: http://www.kdnuggets.com
•
Courses and Tutorials Online
– At KSU
• CIS798 Machine Learning and Pattern Recognition
http://www.kddresearch.org/Courses/Fall-1999/CIS798
• CIS730 Introduction to Artificial Intelligence
http:// www.kddresearch.org/Courses/Fall-2000/CIS730
• CIS690 Implementation of High-Performance Data Mining Systems
http:// www.kddresearch.org/Courses/Summer-2000/CIS690
– Other courses: see KD Nuggets, www.aaai.org, www.auai.org
•
Discussion Forums
– Newsgroups: comp.ai.*
– Recommended mailing lists: Data Mining, Uncertainty in AI
– KSU KDD Lab Discussion Board: http://www.kddresearch.org/Board
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Terminology
•
Artificial Intelligence (AI)
– Operational definition: study / development of systems capable of “thought
processes” (reasoning, learning, problem solving)
– Constructive definition: expressed in artifacts (design and implementation)
•
•
Intelligent Agents
Topics and Methodologies
– Knowledge representation
• Logical
• Uncertain (probabilistic)
• Other (rule-based, fuzzy, neural, genetic)
– Search
– Machine learning
– Planning
•
Applications
–
–
–
–
Problem solving, optimization, scheduling, design
Decision support, data mining
Natural language processing, conversational and information retrieval agents
Pattern recognition and robot vision
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Summary Points
•
Artificial Intelligence: Conceptual Definitions and Dichotomies
– Human cognitive modelling vs. rational inference
– Cognition (thought processes) versus behavior (performance)
– Some viewpoints on defining intelligence
•
Roles of Knowledge Representation, Search, Learning, Inference in AI
– Necessity of KR, problem solving capabilities in intelligent agents
– Ability to reason, learn
•
Applications and Automation Case Studies
–
–
–
–
•
Search: game-playing systems, problem solvers
Planning, design, scheduling systems
Control and optimization systems
Machine learning: pattern recognition, data mining (business decision support)
More Resources Online
– Home page for AIMA (R&N) textbook
– CMU AI repository
– KSU KDD Lab (Hsu): http:// www.kddresearch.org
– Comp.ai newsgroup (now moderated)
CIS 830: Advanced Topics in Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences