Download No Slide Title - Ohio University

Hybrid Pipeline Structure for
Self-Organizing Learning Array
ISNN 2007: The 4th International Symposium on Neural Networks
Yinyin Liu1, Ding Mingwei2 , Janusz A. Starzyk1,
1 School
of Electrical Engineering & Computer Science
Ohio University, USA
2 Ross University
Outline
• RC systems design of
SOLAR
• Dimensionality reduction
• Input selection, weighting
• Pipeline structure
Motor cortex
Pars
opercularis
Sensory associative
cortex
Visual associative
cortex
• Experimental results
• Conclusions
Somatosensory cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
2
Intelligence
AI’s holy grail
From Pattie Maes MIT Media Lab
• “…Perhaps the last frontier of science – its ultimate challenge- is
to understand the biological basis of consciousness and the
mental process by which we perceive, act, learn and remember..”
from Principles of Neural Science by E. R. Kandel et al.
E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of
memory storage in neurons.
• “…The question of intelligence is the last great terrestrial
frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins
founded the Redwood Neuroscience Institute devoted to brain research. He
co-founded Palm Computing and Handspring Inc.
3
How can we design intelligence?
• We need to know how
• We need means to
implement it
• We need resources to build
and sustain its operation
4
Resources – Evolution of Electronics
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 20065
By Gordon E. Moore
6
7
Clock Speed (doubles every 2.7 years)
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 20068
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
9
Outline
• RC systems design of
SOLAR
• Dimensionality reduction
• Input selection, weighting
• Pipeline structure
Motor cortex
Pars
opercularis
• Experimental results
• Conclusions
Somatosensory cortex
Sensory associative
cortex
Visual associative
cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
10
Traditional ANN Hardware
information flow
– Limited routing
resource.
output
– Quadratic relationship
between the routing
and the number of
neuron makes classical
ANNs wire dominated.
Interconnect is
70% of chip area
input
hidden
11
Biological Neural Networks
Cell
body
From IFC’s webpage
Dowling, 1998, p. 17
12
Sparse Structure
• 1012 neurons in human brain are
sparsely connected
• On average, each neuron is
connected to other neurons through
about 104 synapses
• Sparse structure enables efficient
computation and saves energy and
cost
13
Why should we care?
Source: SEMATECH
14
Design Productivity Gap  Low-Value Designs?
Percent of die area that must be occupied by memory to
maintain SOC design productivity
100%
80%
60%
% Area Memory
40%
% Area Reused
Logic
20%
% Area New Logic
19
99
20
02
20
05
20
08
20
11
20
14
0%
Source = Japanese system-LSI industry
15
Outline
• RC systems design of
SOLAR
• Dimensionality reduction
• Input selection, weighting
• Pipeline structure
• Experimental results
Motor cortex
Pars
opercularis
• Conclusions
Somatosensory cortex
Sensory associative
cortex
Visual associative
cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
16
SOLAR System Design
•
SOLAR Introduction

–
–

Entropy based selforganization
data-driven
Local connection
Dynamical reconfiguration

Local and sparse
interconnections


Online inputs selection
Feature neurons and
merging neurons

Pattern recognition,
classification
17
Pipeline Overview
node computing ability → “soft” connections
Four
modes
1. Idle
2. Read
3. Process
4. Write
18
Pipeline Signal Flow 1
19
Pipeline Signal Flow 2
20
Pipeline Signal Flow 3
21
Node Operations
Implemented with Xilinx picoBlaze
Runs at higher frequency
22
Outline
• RC systems design of
SOLAR
• Dimensionality reduction
• Input selection, weighting
• Pipeline structure
Motor cortex
Pars
opercularis
• Experimental results
• Conclusions
Somatosensory cortex
Sensory associative
cortex
Visual associative
cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
23
Em(x) Simulation Results
24
Iris Data Processing
Linear growth of
HW cost
4x7 array processing Iris data
25
Chip Layout
26
Hardware Development
27
Future Work
- System SOLAR
28
Conclusions & Future work
• Sparse coding building in sparsely connected networks
• WTA scheme: local competition accomplish the global competition
using primary and secondary layers –efficient hardware
implementation
• OTA scheme: local competition produces neuronal activity reduction
• OTA – redundant coding: more reliable and robust
• WTA & OTA: learning memory for developing machine intelligence
Future work:
• Introducing temporal sequence learning
• Building motor pathway on such learning memory
• Combining with goal-creation pathway to build intelligent machine
29