Download Lecture 1

S. Mandayam/ ANN/ECE Dept./Rowan University
Artificial Neural Networks
ECE.09.454/ECE.09.560
Fall 2006
Lecture 1
September 18, 2006
Shreekanth Mandayam
ECE Department
Rowan University
http://engineering.rowan.edu/~shreek/fall06/ann/
S. Mandayam/ ANN/ECE Dept./Rowan University
Japan's humanoid robots
Better than people
Dec 20th 2005 | TOKYO
From The Economist print edition
Why the Japanese want their robots to act more like humans
S. Mandayam/ ANN/ECE Dept./Rowan University
Why the Japanese want their robots to
act more like humans
HER name is MARIE, and her impressive set of skills
comes in handy in a nursing home.
MARIE can walk around under her own power. She can
distinguish among similar-looking objects, such as
different bottles of medicine, and has a delicate
enough touch to work with frail patients.
MARIE can interpret a range of facial expressions and
gestures, and respond in ways that suggest
compassion. Although her language skills are not
ideal, she can recognise speech and respond clearly.
Above all, she is inexpensive.
Unfortunately for MARIE, however, she has one glaring
trait that makes it hard for Japanese patients to accept
her:
S. Mandayam/ ANN/ECE Dept./Rowan University
Why the Japanese want their robots to
act more like humans
………………….she is a flesh-and-blood
human being from the Philippines. If only
she were a robot instead.
S. Mandayam/ ANN/ECE Dept./Rowan University
Harveian Oration
In celebration of cerebration
by Professor Colin Blakemore,
presented at the Royal College of
Physicians, London, UK,
on Oct 18, 2005
www.thelancet.com Vol 366 Dec 10, 2005
S. Mandayam/ ANN/ECE Dept./Rowan University
Plan
• What is artificial intelligence?
• Course introduction
• Historical development – the neuron
model
• The artificial neural network paradigm
• What is knowledge? What is learning?
• The Perceptron
• Widrow-Hoff Learning Rule
• The “Future”….?
S. Mandayam/ ANN/ECE Dept./Rowan University
Artificial Intelligence
Systems that think like humans
Systems that think rationally
• Cognitive modeling
• Logic
Systems that act like humans
• Natural language processing
• Knowledge representation
• Machine learning
Systems that act rationally
• Decision theoretic agents
S. Mandayam/ ANN/ECE Dept./Rowan University
Course Introduction
• Why should we take this course?
• PR, Applications
• What are we studying in this course?
• Course objectives/deliverables
• How are we conducting this course?
• Course logistics
• http://engineering.rowan.edu/shreek/fall06/ann/
S. Mandayam/ ANN/ECE Dept./Rowan University
Course Objectives
• At the conclusion of this course the
student will be able to:
• Identify and describe engineering
paradigms for knowledge and learning
• Identify, describe and design artificial
neural network architectures for simple
cognitive tasks
S. Mandayam/ ANN/ECE Dept./Rowan University
Biological Origins
S. Mandayam/ ANN/ECE Dept./Rowan University
Biological Origins
S. Mandayam/ ANN/ECE Dept./Rowan University
History/People
1940’s
Turing
General problem solver, “Turing test”
1940’s
Shannon
Information theory
1943
McCulloch and Pitts
Math of neural processes
1949
Hebb
Learning model
1959
Rosenblatt
The “Perceptron”
1960
Widrow
LMS training algorithm
1969
Minsky and Papert
Perceptron deficiency
1985
Rumelhart
Feedforward MLP, backprop
1988
Broomhead and Lowe
Radial basis function neural nets
1990’s
VLSI implementations
1997
IEEE 1451
S. Mandayam/ ANN/ECE Dept./Rowan University
Neural Network Paradigm
Stage 1: Network Training
Present Examples
Stage 2: Network Testing
New Data
Artificial
Neural
Network
Determine
Synaptic
Weights
Artificial
Neural
Network
Indicate Desired Outputs
“knowledge”
Predicted Outputs
S. Mandayam/ ANN/ECE Dept./Rowan University
ANN Model
x
Input
Vector
 x1 
x 
 2
 x3 
Artificial
Neural
Network
f
Complex
Nonlinear
Function
f(x) = y
“knowledge”
y
Output
Vector
 y1 
y 
 2
 y3 
S. Mandayam/ ANN/ECE Dept./Rowan University
Popular I/O Mappings
Single output
x
Coder
ANN
y
1-out-of-c selector
x
ANN
x
ANN
y1
y2
yc
Associator
y1
y2
yc
x
ANN
y
S. Mandayam/ ANN/ECE Dept./Rowan University
Inputs
The Perceptron
x1
wk1
x
wk2
2
x
wkm
m
Synaptic
weights
Bias,
bk
S
uk
Activation/
squashing
function
S
j(.)
Induced
field,
vk
Output,
yk
S. Mandayam/ ANN/ECE Dept./Rowan University
“Learning”
Mathematical Model of the Learning Process
Intitialize: Iteration (0)
ANN
x
x
[w]
[w]0
y(0)
[w]1
y(1)
y
Iteration (1)
x
desired
o/p
Iteration (n)
x
[w]n
y(n) = d
S. Mandayam/ ANN/ECE Dept./Rowan University
“Learning”
Mathematical Model of the Learning Process
Intitialize: Iteration (0)
ANN
x
x
[w]
[w]0
y(0)
[w]1
y(1)
y
Iteration (1)
x
desired
o/p
Iteration (n)
x
[w]n
y(n) = d
S. Mandayam/ ANN/ECE Dept./Rowan University
Error-Correction Learning
x1 (n)
Inputs
x
wk1(n)
wk2(n)
2
Synaptic
weights
x
m
wkm(n)
Bias,
bk
Desired
Output,
dk (n)
Activation/
squashing
function
+
S
j(.)
Induced
field,
vk(n)
Output,
yk (n)
-
S
Error
Signal
ek (n)
S. Mandayam/ ANN/ECE Dept./Rowan University
Learning Tasks
•
•
•
•
Pattern Association
Pattern Recognition
Function Approximation
Filtering
Classification
x2
x2
2
1
2
DB
1
x1
DB
x1
S. Mandayam/ ANN/ECE Dept./Rowan University
Perceptron Training
Widrow-Hoff Rule (LMS Algorithm)
w(0) = 0
n=0
y(n) = sgn [wT(n) x(n)]
w(n+1) = w(n) + h[d(n) – y(n)]x(n)
n = n+1
Matlab Demo
S. Mandayam/ ANN/ECE Dept./Rowan University
The Age of Spiritual Machines
When Computers Exceed
Human Intelligence
by Ray Kurzweil | Penguin
paperback | 0-14-028202-5 |
S. Mandayam/ ANN/ECE Dept./Rowan University
Summary