Download Chap. 4 F

博士本審査 2006.01.04
Family of Self-Organized Network Inspired
by Immune Algorithm (SONIA) and
Their Various Applications
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
Muhammad R. Widyanto
03D35190
Dept. of Computational Intelligence & Systems Science
Tokyo Institute of Technology
1/56
Thesis Road Map
SONIA
SONIA-DNN
Chapter 1
Introduction
CMF-SONIA
F-SONIA
EF-SONIA
[Jx]: Journal Paper x-th
Chapter 2 [J1]
SONIA and Food Quality Prediction
Chapter 3 [J2]
SONIA-DNN and Preference Modeling
Chapter 4 [J3]
F-SONIA and Fragrance Recognition
Chapter 5 [J4]
CMF-SONIA and Overlapping Pat. Clas.
Chapter 6 [J5]
EF-SONIA and Unknown Odor Recog.
Chapter 7
Conclusions
2/56
Contents
SONIA
Chap. 1 Introduction
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
3/56
Chap. 1 Introduction
Problems
BPNN [Rumelhart, 86]
Back-Propagation Neural Network
Global Response
Overfitting
Low Generalization
4/56
Chap. 1 Introduction
Opportunity
Immune Algorithm [Timmis, 01]
Local Response
Characteristics only
Diverse Representation
5/56
Chap. 1 Introduction
Solution
BPNN [Rumelhart,86]
Immune Algorithm [Timmis,01]
A Self-Organized Network
inspired by Immune Algorithm
[proposed]
Better Recognition
SONIA
Better Generalization
6/56
Chap. 1 Introduction
Applications
Food Quality Prediction
Preference Modeling
SONIA
Fragrance Recognition
Overlapping Pat. Clas.
Unknwon Odor Recog.
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
7/56
Contents
SONIA
Chap. 1 Introduction
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
8/56
Chap. 2 SONIA
Self-Organized Network inspired by Immune Algorithm
[proposed]
Input
layer
Hidden
layer
Output
layer
・
・
・
・
・
・
・
・
・
BPNN :
[Rumelhart,86]
Input Vector
Hidden Unit
Immune
Algorithm :
[Timmis,01]
Antigen
Recognition Ball (RB)
9/56
Recognition Ball & Hidden Unit
Antigen
Epitope
Paratope
Antibody
B Cell
Recognition Ball (RB)
Chap. 2 SONIA
[proposed]
Input Vector
Euclidian
Distance
Unit
Centroid
Hidden Unit
10/56
B-Cell Construction & Mutation
Antibody Generation
[Timmis,01]
Chap. 2 SONIA
[proposed]
Hidden Unit Creation
of BPNN [proposed]
RB 1
Hidden Unit 1
RB 2
Hidden Unit 2
Hidden Unit i
Antigen
[1..m]
RB i
Mutated
RB n
Input
Vector
[1..m]
Mutated
Hidden Unit n
11/56
Chap. 2 SONIA
BPNN Regularization [MacKay, 92]
BPNN [Rumelhart, 86]
Approximation Error : 0.01994 Approximation Error : 0.00241
1
1
0.9
0.9
h(x)
h(x)
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
x
0.7
0.8
0.9
1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
SONIA without mutation SONIA with mutation [proposed]
Approximation Error : 0.01008 Approximation Error : 0.00118
1
1
0.9
h(x)
0.9
h(x)
0.8
0.8
0.7
0.7
0.6
0.6
Approximation
0.5
0.5
0.4
0.4
0.3
0.3
Training Data
0.2
0.2
0.1
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
x
0.7
0.8
0.9
1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x
0.8
0.9
1
12/56
Chap. 2 SONIA
Food Quality Prediction
Collaborative Project with Japan Ministry of Agriculture and CSD Inc.
Prediction Engine:
Neural Networks
Quality Control Server
Quality
Check
Production Area
Perishable Food
Frozen Truck
Market Area
Supermarket
Food Store
13/56
Chap. 2 SONIA
Prediction System
[proposed]
Collaborative Project with Japan Ministry of Agriculture and CSD Inc.
011016_t
Time-temperature Data
199
177
155
133
111
89
67
Series1
Series2
Channel
2
45
oC
Channel 1
23
Pre-Processing
Pre-Processing: :
Range
RangeSelection
Selection
30
25
20
15
10
5
0
-5
1
Data
DataCollection
Collection: :
Data
DataLodger
Lodger
Time ( X 5 Minutes )
Range Selected
Quality
Feature
Feature
Extraction
Extraction: :
Mean
Mean&&
Standard
StandardDeviation
Deviation
ch1:Mean
ch1:SD
ch2:Mean
ch2:SD
Neural
Networks
A
B
C
D
E
good
14/56
Chap. 2 SONIA
Recognition Accuracy
Collaborative Project with Japan Ministry of Agriculture and CSD Inc.
TOP
MIDDLE
BOTTOM
SONIA
BPNN
Recognition (%)
100
50
0
TOP
MIDDLE
BOTTOM
15/56
Contents
Chap. 1 Introduction
SONIA-DNN
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
16/56
Chap. 3 SONIA-DNN
Decision Maker (DM) Preference
JSPS Center Of Excellence Project
Modeling
DM Preference ???
Alternative1:
Nissan Fuga
Price: 5 million yen
Engine: 3000 cc
Preference
Value
Decision
Maker (DM)
Consumption: 10km/l
17/56
Chap. 3 SONIA-DNN
Preference Value by Comparisons
Alternative1:
Nissan Fuga
Comparison
Value
Decision
Maker (DM)
Alternative2:
Toyota Mark X
18/56
Chap. 3 SONIA-DNN
SONIA-based Decision Neural Network [proposed]
JSPS Center Of Excellence Project
Alternative 1
SONIA(1)
Alternative 2
Comparison
Value
Incomplete Comparisons
SONIA(2)
Better Generalization
19/56
Chap. 3 SONIA-DNN
Incomplete Comparisons
JSPS Center Of Excellence Project
Too many!
Alternative 1
Alternative 2
Alternative 3
・ ・ ・
Alternative n
Alternative 1
ー
1.2
0.8
1.3
0.9
Alternative 2
ー
ー
1.1
Alternative 3
ー
ー
ー
ー
ー
ー
ー
ー
ー
ー
ー
・
・
・
Decision
Maker (DM)
Alternative n
0.7
ー
Limited Training Data
20/56
Chap. 3 SONIA-DNN
Lp-metric Function Benchmark
[Sun, 1996]
Preference
Value
Alternative
Decision Maker (DM)
1/ p


V  L    i  ( zi*  zi ) p 
 i 1

I
V : Preference Value, L : Maximum Value
I : Number of Criteria,  : Weight Parameter
z * : Max Vector Value, z : Alternative Vector
p : Number of Dimension
21/56
Chap. 3 SONIA-DNN
Experimental Setting
Alternative
Vector
1
2
3
4
5
6
1
ー
2
ー
ー
3
ー
ー
ー
4
ー
ー
ー
ー
5
ー
ー
ー
ー
ー
6
ー
ー
ー
ー
ー
ー
7
ー
ー
ー
ー
ー
ー
7
ー
21 comparison values
7 discarded randomly, 14 training samples
22/56
Chap. 3 SONIA-DNN
Experimental Result
Average Error (%)
4
Excellent!
2
0
BPNN-DNN
SONIA-DNN
23/56
Chap. 3 SONIA-DNN
Experiments
JSPS Center Of Excellence Project
Average
Error (%)
8
BPNN-DNN
Wonderful!
4
[Chen, 2004]
SONIA-DNN
[proposed]
0
18
15
Number of Samples
12
24/56
Contents
Chap. 1 Introduction
F-SONIA
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
25/56
Chap. 4 F-SONIA
Odor Discrimination System
Human Experts
Perfume Industry
Three Mixture
Two Mixture
Artificial Odor
Discrimination System
Pure Perfume
Problem
Complexity
26/56
Chap. 4 F-SONIA
Artificial Odor Discrimination System
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Sensory
System
Frequency
Counter
System
Neural
Network
27/56
Chap. 4 F-SONIA
Fuzzy Similarity based SONIA (1/4) [proposed]
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Input Vector
Euclidean
Distance
Unit
Centroid
SONIA : Hidden Unit
Fuzzy Input Vector
Fuzzy
Similarity
Fuzzy Unit
Centroid
F-SONIA : Fuzzy Hidden Unit
28/56
Chap. 4 F-SONIA
Fuzzy Similarity based SONIA (2/4) [proposed]
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
SONIA :
Crisp Value
1
Membership
Value
F-SONIA :
[proposed]
Frequency
minimum
mean
maximum
Fuzzy Triangular Number
29/56
Chap. 4 F-SONIA
Fuzzy Similarity based SONIA (3/4) [proposed]
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Membership
Value
1
Input Vector
Hidden Unit Vector
Similarity
Value (μ)
Frequency
30/56
Chap. 4 F-SONIA
Fuzzy Similarity based SONIA (4/4) [proposed]
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Input Unit
SONIA :
Sensor 1
・
・
・
Sensor i
F-SONIA :
[proposed]
Euclidean
Distance
Sensor 1
・
・
・
Fuzzy
Similarity
Hidden Unit
Square Root of
Quadratic Distances
Arithmetic Mean of
Similarity Measures
Sensor i
31/56
Chap. 4 F-SONIA
Citrus-Canangga-Ethanol(%)
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Recognition (%)
100
50
0
F-SONIA
[proposed]
SONIA
FLVQ
LVQ
BPNN
[Sakuraba,91] [Kohonen,86] [Rumelhart,86]
32/56
Chap. 4 F-SONIA
Error Convergence
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Error
0.12
0.1
0.08
0.06
SONIA
0.04
0.02
0
F-SONIA
0
100
200
300
400
500
600
700
800
900
1000
Epoch
33/56
Chap. 4 F-SONIA
Dissimilarity Comparison
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Dissimilarity Definition
[Hastie,01]
SONIA
M1 M 2 N H
D( 1 ,  2 )    xaj  xbj 
2
a 1 b 1 j 1
F-SONIA
 NI

2
DSONIA ( 1 ,  2 )   ( g  (  xai  xai  )1/ 2 
a 1 b 1
 i 1

M1 M 2
 NI

2
 g  (  xbi  xai  )1/ 2 )2
 i 1

 NI
2 
M1 M 2    ai1  xai , xai  

DF  SONIA ( 1 ,  2 )    i 1
NI

a 1 b 1 




 NI

2
 ( g  (  xai  xbi  )1/ 2 
 i 1

 NI

2
 g  (  xbi  xbi  )1/ 2 )2
 i 1

 NI
2 
  bi 2  xbi , xbi  

  i 1
NI






DSONIA ( 1 ,  2 )  DF SONIA ( 1 ,  2 )
2
2
34/56
Contents
Chap. 1 Introduction
CMF-SONIA
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
35/56
Chap. 5 CMF-SONIA
Overlapping Data
Adaptive Clustering inspired by
B-Cell Construction of SONIA
Class A
Class B
Errors in
Classification
36/56
Chap. 5 CMF-SONIA
Class Majority F-SONIA [proposed]
Good Idea!
Class Majority for each Cluster
Class A
Class B
Reduce Errors in
Classification
37/56
Chap. 5 CMF-SONIA
Vowel Data [Lippmann,89]
F2 1
(Hz) 0.9
heed
head
hid
0.8
0.7
had
0.6
20000.5
0.4
heard
hood
who’d
0.3
0.2
hud
hod
0.1
hawed
0
0
0
0
0.1
0.2
0.3
0.4
0.5
750
0.6
0.7
0.8
0.9
1
F1(Hz)
38/56
Chap. 5 CMF-SONIA
Recognition Accuracy
Recognition (%)
80
Excellent!
40
0
CMF-SONIA F-SONIA
BPNN
[proposed]
[Rumelhart,86]
39/56
Chap. 5 CMF-SONIA
Classification Plane
F2
(Hz)
Wow!
2000
0
0
750
F1(Hz)
40/56
Contents
Chap. 1 Introduction
EF-SONIA
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
41/56
Unknown Odor Recognition
Chap. 6 EF-SONIA
Collaboration with University of Indonesia
Under Indonesia Ministry of Sciences & Technology Project
Input
Neural Nets
Known Odor
Unknown Odor
42/56
Chap. 6 EF-SONIA
Far with High Similarity
Arithmetic
Mean
High Similarity
No Similarity
High
Similarity
43/56
Chap. 6 EF-SONIA
Euclidean Fuzzy Similarity [proposed]
No
Similarity
44/56
Chap. 6 EF-SONIA
Similarity Measure
Membership
Value
1
Input Vector
Hidden Unit Vector
Similarity
Value (μ)
Euclidean
Dimension
??
??
45/56
Chap. 6 EF-SONIA
Fuzziness Region
Averaging Approach
Second Dimension
Elliptical Approach
??
??
First Dimension
46/56
Chap. 6 EF-SONIA
Elliptical Approach [proposed]
Brilliant Idea!
Θ
47/56
Chap. 6 EF-SONIA
Citrus-Canangga-Ethanol(%)
Excellent!
Method
Unknown
Category
Only (%)
Overall
Recognition
(%)
EF-SONIA with Elliptical
Approach [proposed]
96.17
98.33
EF-SONIA with Averaging
Approach
89.47
96.67
Fuzzy Learning Vector
Quantization (FLVQ)
[Sakuraba,91]
73.32
76.66
Learning Vector
Quantization (LVQ)
[Kohonen,86]
57.63
37.91
48/56
Contents
SONIA
Chap. 1 Introduction
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
Chap. 2 SONIA and Food Quality Prediction
Chap. 3 SONIA-DNN for Preference Modeling
Chap. 4 F-SONIA for Fragrance Recognition
Chap. 5 CMF-SONIA for Overlapping Pattern Class.
Chap. 6 EF-SONIA for Unknown Odor Recognition
Chap. 7 Conclusions
49/56
Chap. 7 Conclusions
Research Results
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
SONIA Family
- Proposed Methods -
- Applications -
SONIA
Food Quality Prediction
SONIA-DNN
Preference Modeling
F-SONIA
Fragrance Recognition
CMF-SONIA
Overlapping Patt. Class.
EF-SONIA
Unknown Odor Recog.
50/56
Chap. 7 Conclusions
Research Impacts
- Educational Institutes -
- Industrial Companies -
Univ. of Indonesia
CSD Inc.
Tokyo Inst. of Tech.
IURI
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
- Governments Japan Ministry of Agriculture
Indonesia Ministry of Sciences & Tech.
Japan Society for the Promotion of Science
51/56
Related Publications (1/5)
SONIA
Journal Papers
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
[J1] M. R. Widyanto et al., “Improving Recognition and Generalization Capability of "Back-
Propagation NN using a Self-Organized Network inspired by Immune Algorithm”, Applied
Soft Computing Journal, Elsevier Science Pub., Vol. 6, No. 1, 2005.
[J2] M. R. Widyanto et al., “SONIA based Decision Neural Networks for Preference Assessment
with Incomplete Comparisons”, International Journal of Advanced Computational
Intelligence & Intelligent Informatics, Vol. 9, No. 6, 2005.
[J3] M. R. Widyanto et al., “A Fuzzy Similarity based Self-Organized Network Inspired by
Immune Algorithm for Three Mixture Fragrances Recognition”, IEEE Transactions on
Industrial Electronics, Vol.53, No.1, 2006 (to appear).
[J4] M. R. Widyanto et al., “Class Majority in Designing Fuzzy Local Approximation NN for
Overlapping Data in Pattern Classification”, International Journal of Fuzzy Systems, Vol. 7,
No. 1, 2005.
[J5] M. R. Widyanto et al., “Unknown Odor Recognition using Euclidean Fuzzy Similarity-
based Self-Organized Network Inspired by Immune Algorithm”, Neural Computing and
Applications, Springer-Verlag Pub., (under review).
[J6] M. R. Widyanto et al., “Local Gas Holdup Measurement using SONIA-Ultrasonic
Noninvasive Method”, Sensors & Actuator – Part A: Physical, Elsevier Science Pub., Vol.
127, No.1, 2006 (to appear) .
52/56
Related Publications (2/5)
International Conference Papers (1/2)
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
[C1] M. R. Widyanto et al., “Improvement of Artificial Odor Discrimination System using FuzzyLVQ Neural Network”, in the proceedings of the 3rd International Conference on Computational
Intelligence and Multimedia Applications, New Delhi, India, IEEE Press, pp. 474-478, 1999.
[C2] M. R. Widyanto et al., “Clustering Analysis using a Self-Organized Network Inspired by
Immune Algorithm”, in the proceedings of the IASTED International Conference on Artificial
and Computational Intelligence, Tokyo, Japan, ACTA Press, pp. 197-202, 2002.
[C3] M. R. Widyanto et al., “A Time-temperature-based Food Quality Prediction using a SelfOrganized Network Inspired by Immune Algorithm”, in the proceedings of the 1st International
Conference on Soft Computing and Intelligent Systems, Tsukuba, Japan, 2002.
[C4] M. R. Widyanto et al., “Improvement of Three mixture Fragrances Recognition using Fuzzy
Similarity based Self-Organized Network Inspired by Immune Algorithm”, in the proceedings
of the 4th International Symposium on Advanced Intelligent Systems, Jeju, Island, Korea, 2003.
[C5] M. R. Widyanto et al., “Class Majority in Designing a Fuzzy Local Approximation NN”, in
the proceedings of the 2nd International Conference on Soft Computing and Intelligent
Systems, Yokohama, Japan, 2004.
53/56
Related Publications (3/5)
International Conference Papers (2/2)
SONIA
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
[C6] M. R. Widyanto et al., “Analysis of Fuzzy Local Approximation NN on Uncertainty Decision of
Frequency Measurements”, in the proceedings of the International Symposium on
Computational Intelligence and Industrial Applications, Hainan, China, 2004.
[C7] M. R. Widyanto et al., “Agent-based Decision Maker Preference Modeling Using SONIA-
DNN for Restaurant Work Assignment and Scheduling Problem”, in the proceedings of the
International Workshop on Agent-based Approaches in Economics and Social Complex
Systems, Tokyo, Japan, 2005.
[C8] M. R. Widyanto et al., “SONIA-based Decision Neural Network and Its Application to
Restaurant Work Assignment”, in the proceedings of the 6th International Symposium on
Advanced Intelligent Systems, Yeosu, Korea, 2005
[C9] M. R. Widyanto et al., “Unknown Odor Category Classification using EF-SONIA”, in the
proceedings of the 2nd International Symposium on Computational Intelligence and
Intelligence Informatics, Hammamet, Tunisia, 2005.
[C10] M. R. Widyanto et al., “SONIA-Ultrasonic Technique for Gas Holdup Measurement of
a Bubble Column”, in the proceedings of the 1st Daedeok International Conference on
Human-Centered Advanced Technology, Daedeok Science Town, Korea, 2005.
54/56
Related Publications (4/5)
SONIA
Domestic Conference Papers
SONIA-DNN
CMF-SONIA
F-SONIA
EF-SONIA
[D1] M. R. Widyanto et al., “Dealing with Incomplete Comparisons using SONIAbased Decision Neural Network”, in the proceedings of the 35-th Symposium on
System Engineering, Yokohama, Japan, 2005.
[D2] M. R. Widyanto et al., “Restaurant Work Assignment Modeling using
SONIA-DNN”, in the proceedings of the 2nd Tokyo Tech COE RA Forum,
Tokyo, Japan, 2005.
[D3] M. R. Widyanto et al., “Decision Preference Modeling using SONIA-DNN
and Its Application to Work Assignment Problem, in the proceedings of the
21-th Fuzzy System Symposium, Tokyo, Japan, 2005.
55/56
Related Publications (5/5)
SONIA
Awards
[A1]
CMF-SONIA
SONIA-DNN
F-SONIA
EF-SONIA
Excellent Presentation Award
The 1st International Conference on Soft Computing & Intelligent Systems,
Tsukuba, Japan, September 2002.
[A2]
Gold Prize, Best Poster Award, Master Thesis Presentation
Dept. of Computational Intelligence & Systems Science,
Tokyo Institute of Technology, Japan, February 2003.
[A3]
Outstanding Paper Award
The 6th International Conference on Advanced Intelligent Systems,
Yeosu, South Korea, September 2005.
56/56