Download Neural Networks Overview - The Computer Engineers` Blog

Chapter0. Overview of the Neural Network
1. Brain = Neural Network in Topology
Cell
body
Neural Network
= Computational
Structure of the
Brain
= Distributed, Adaptive,
Nonlinear Learning
Machine built from
many Processing
Elements
Synonyms :
Artificial Neural
System,
Artificial Neural
Network,
Neuromorphic System,
Parallel Distributed
Processing,
Adaptive Network,
Connnectionism,
Neurocomputer.
1
2. Features of the Human Brain (Differences from Digital Computers)
●
Sensory Perception (Pattern Recognition) and motor control
– Vision, audition, olfaction, touch, temperature sensing
●
Planning and Reasoning
●
Learning By Examples – Non-algorithmic, Trainability, Self-Organization
● Learning and Adaptation by Generalization
● Reflex and Intuition (Similar to Table Lookup)
●
Processing Ill-defined (Unstructured, Inconsistent, Probabilistic, Noisy) Information
– fault-tolerant, flexible, robust
● As Computer: Massively parallel and distributed : Algorithm + Architecture
● As Computer: Wetware (Netware) vs. Software/Hardware
● As Math: Nonlinear and Adaptive Modeling Scheme
2
At birth, 10 11 neurons Innate, Tends to Decrease with
Time. However, their interconnections evolve and
new ones are created.
3
3. Intelligent [Learning] Machines
Intelligence ?
● Webster = Ability to learn or understand or to deal with new or trying situations
: REASON
●
Jang = Humanlike Expertise, adapt and learn to do better in changing env. And
explain its decisions/actions
●
Constituent Tech. = Synergy of NN, FL, EC
EC = Systematic random search = Biological genetics + Natural selection)
●
CI = Any methodology involving computing exhibiting ability to learn (do better)
with new situations by reasoning
(generalization, discovery, association, abstracting); also explain how it reasons.
4
History :
Intelligence to Machines  Freedom to Mankind !
Industrial Revolution (Machine)
 Information Technology Revolution
 Artificial Brain (Intelligence) Revolution :
Creative, human-friendly, autonomous (adaptive)
Brain = Final Frontier = Neural Nets
● Left Brain (Logic)
– Program
(Symbolic, Software, Structured)
 Machine Learning, Expert System
: AI
● Right Brain (Intuition, Emotion)
– Neural Network
(Numeric, Hardware, Unstructured)
 Cybernetics, Bionics
5
4. Computational Intelligence and Soft Computing
(Ref. Eberhart, Chap. 1 of Computational Intelligence, IEEE Press, 95)
Bezdek
Intelligence
Symbolic
Comp.
NN
AI
C
I
N
F
Intelligence
BI
CI
Numeric
Comp.
FL
N
NF E
E
FE
EC
Biology
BI
MI = AI or
CI
6
7
8
9
5. Neural Network Architecture
dendrite
x1
w1
cell (soma)
y
wm
xm
xm+1
wm+1
axon
 m 1

y     w x  S: Linear Combination
 j 1

Activation function
 : activation
ftn
j
j
10
11
12
(1) Feedforward Architecture
Input
General
....
....
Hidden
Output
Layered(3-2-2)
Input
Hidden
Output
13
14
(2) Feedback, Recurrent Architecture
15
16
17
18
6 . Usage of the Neural Network –
Function Approximation and Generalization
NN
Amorphous
◆ Training
◆ Training
NN
Word
Voice
NN
Object Name
Object Image
-
-
e
e
+
+
Teacher
Teacher
19
◆ Apply to Different Tasks After Training
NN
Voice
Object Image
Word
Object Name
20
7. Generalization By Nonlinear Interpolation
Learning Mode (weights change)  Performance Mode (weights fixed)
◆ Training
– Function Approximation
= Create Internal Representations Only through Examples
x
y  F ( x, w) @ f ( x)
Neural
Network
if w is fixed after training →
NN is a Model-Free Estimator
Training Data
y
x
Digital Computer vs. Neural Computer
Discrete Samples
y
• Digital – recalculate even for same inputs
f(x)
Underlying
Function
f (x): Normally Unknown
• Neural – can memorize and recall results of
previous calculation [previous answers].
21
◆ Generalization
NN
Function Approximation
F (x, w )
F ( x, w ")
x
y = F(x, w *)
①
②
F (x, w *)
②
F ( x , w ')
f (x)
①
x
① Approximation
② Generalization
22
23
24
25
8. Applications
1) Pattern Recognition & Character Recognition, Document Search
Face and Speech Recognition - Biometrics
Computer Vision : Image Understanding, Object Recognition, PCB Inspection
2) Model Building from Experimental Data: Function Mapping,
Regression, System Identification, Data Mining (Knowledge Discovery), Prediction
3) Image / Signal Processing and Communication
4) Optimization
5) Time Series Analysis and Financial Engineering
6) Medicine - Patient Care and Clinical Decision Support,
Biomedical Engineering, Bio-mimetics, Bio-informatics
7) Robotics and Automation
◆ Service Robots, Pet Robots, Surveillance Robots
26
◆ Process Control
Kodak - Film Making Amoco – Oil Exploration
◆ Aircraft Control
◆ Automotive Control
◆ Machine Control / Maintenance
◆ Machine Health Monitoring / Diagnosis
◆ Diagnostics and Quality Control
◆ Power System Control ( Canada Vancouver Island Power)
◆ Chemical Product Design ( AIWARE사의 CAD/Chem )
◆ Airline Luggage Inspection System
( -20 % cost + 50 % performance )
◆ Active Vibration Cancellation
8) Music Composition
9) Neural Network Products in Korea:
Green Technology – Counterfeit Recognizer ( W 66B for 5 yrs)
Korea Axis – Speech Recognition Toy (2000. 12)
Slip Processing Machine for Banks using Handwritten Recognition (2000.12)
Speech Recognition Chip Product which is Robust to Noise Applying Human
Auditory Model (2000. 7)
27
Industrial App of CI
GE
Imagination at work
Insurance underwriting, proactive maintenance Recom.
Paper Web time-to-break prediction with Fuzzy+nn
Equip Prognosis, anomally dete
CI model = domain knowledge + field data
Ford: CIS system
DOD Air Force Res. Lab.
Image Patterns below Clutter
Tracking & detection below clutter
NN appli – bioinfo, drug design, financial mkr pred, internet search engine,
medical app, 30 commercial componemts,
Future Dir.
Drug design (big) nat lang under, search eng, high leve sensor fusion, sensor-web,
neural & psycholinguistic study – working of mind, cog and emo, lang & music.28
9. NN Development Tools
◆ Types
General Purpose Computer
1) S/W Simulation
2) S/W Simulation with H/W Accelerators
3) S/W Simulation on a Parallel Computer
Special Purpose H/W
1) Neurocomputing Workstations
2) Electronic - VLSI
3) Optical - Laser Holography
10. Government Sponsored NN Research Worldwide
1990-2000 Decade of the Brain (US)
1990-2090 Century of the Brain (Japan)
1998-2007 Braintech 21(Korea) – Brain Research Promotion Act
29