Download Intro-ANN - Computer Science

CS 4700:
Foundations of Artificial Intelligence
Prof. Carla P. Gomes
[email protected]
Module:
Intro Neural Networks
(Reading: Chapter 20.5)
Carla P. Gomes
CS4700
Neural Networks
Rich history, starting in the early forties with McCulloch and Pitts’s model
of artificial neurons (McCulloch and Pitts 1943).
Two views:
– Modeling the brain
– “Just” representation of complex functions
(Continuous; contrast decision trees)
Much progress on both fronts.
Drawn interest from: Neuroscience, Cognitive science, AI, Physics,
Statistics, and CS/EE.
Carla P. Gomes
CS4700
Computer processor speed (MIPS)
Computer vs. Brain
Circa 1997
Information or computer storage (Megabytes)
Increasing Compute Power:
Moore’s Law
The world's first
2-billion transistor microprocessor
delivered in next-generation
Intel® Itanium® processors
codenamed Tukwila
Tukwila is targeted for production
towards the end of 2008.
In 1965, Gordon Moore, Intel co-founder, predicted that the
number of transistors on a chip would double about every two years.
(popularly known as Moore's Law). Intel has kept that pace for nearly 40 years.
Carla P. Gomes
CS4700
Computer processor speed (MIPS)
Computer Power / Cost
Circa 1997
Neural Networks
Computational model inspired by the brain
Brain
based on the interaction of
multiple connected processing elements
(Connectionism, parallel distributed processing, neural computation)
.
When inputs reach some threshold
 an action potential
Inputs
(electric pulse)
is sent along
the axon to the outputs
Outputs
Brain made up
Brain’s information
11
of neurons (~10 ) and processing power
emerges from a
highly interconnected
network of neurons.
Around 1011 neurons, 1014 synapses; a cycle time of 1ms-10 ms.
Connection between cells
Excitatory or inhibitory
and may change over time
Carla P. Gomes
CS4700
Connectionist Models of Learning
Characterized by:
A large number of very simple neuron-like processing elements.
A large number of weighted connections between the elements.
Highly parallel, distributed control.
An emphasis on learning internal representations automatically.
But of course the interconnectivity is not really at the brain scale…
Carla P. Gomes
CS4700
Autonomous Learning Vehicle In a Neural Net
(ALVINN)
ALVINN learns to drive an autonomous vehicle at normal speeds on
public highways.
ALVINN is a perception
systems which learns to control
the NAVLAB vehicles by
watching a person drive.
Pomerleau et al, 1993
Carla P. Gomes
CS4700
ALVINN drives 70mph on highways
30 x 32 grid of pixel
intensities from camera
Each output unit correspond to a particular steering direction.
The most highly activated one gives the direction to steer.
What kinds of problems are suitable for
neural networks?
Have sufficient training data
Long training times are acceptable
Not necessary for humans to understand learned target function or
hypothesis
 neural networks are magic black boxes
Carla P. Gomes
CS4700
Tasks
– Function approximation, or regression analysis, including time
series prediction and modeling.
– Classification, including pattern and sequence recognition, novelty
detection and sequential decision making.
– Data processing, including filtering, clustering, blind signal
separation and compression.
Carla P. Gomes
CS4700
Example of Application Areas
Application areas include:
•
•
•
•
•
•
•
•
•
System identification and control (vehicle control, process control),
Game-playing and decision making (backgammon, chess, racing),
Pattern recognition (radar systems, face identification, object recognition, etc.)
Sequence recognition (gesture, speech, handwritten text recognition),
Medical diagnosis
Financial applications
Data mining (or knowledge discovery in databases, "KDD"),
Visualization
E-mail spam filtering.
Carla P. Gomes
CS4700