Download fuzzy engineering expert systems with neural network applications

9»V*
Jf
FUZZY ENGINEERING
EXPERT SYSTEMS WITH
NEURAL NETWORK
APPLICATIONS
ADEDEJI B. BADIRU
Department of Industrial Engineering
University of Tennessee
Knoxvilfe, TN
JOHNfY. CHEUNG
School of Electrical and Computer Engineering
University of Oklahoma
Norman, OK
§)
JOHN WILEY & SONS, INC.
CONTENTS
Preface
Acknowledgments
xv
xvii
1 Artificial Intelligence
1.1 Origin of Artificial Intelligence 1
1.2 Human Intellligence versus Machine Intelligence
1.3 The First AI Conference 5
1.4 Evolution of Smart Programs 6
1.5 Branches of Artificial Intelligence 8
1.6 Neural Networks 9
1.7 Emergence of Expert Systems 10
/ f 1.7.1 Embedded Expert Systems 12
1
3
I'
2 Fundamentals of Expert Systems
2.1 Expert Systems Process 13
2.2 Expert Systems Characteristics 14
2.2.1 Domain Specificity 15
/
2.2.2 Special Programming Languages 16
2.3 Expert Systems Structure 16
2.3.1 The Need for Expert Systems 19
2.3.2 Benefits of Expert Systems 19
2.3.3 Transition from Data Processing to Knowledge
Processing 20
2.4 Heuristic Reasoning 21
2.4.1 Search Control Methods 22
2.4.2 Forward Chaining 22
2.4.3 Backward Chaining 22
2.5 User Interface 22
2.5.1 Natural Language 23
2.5.2 Explanations Facility in Expert Systems 23
2.5.3 Data Uncertainties 23
13
viii
CONTENTS
i ;
I i
'
i
!
!
!
j
2.5.4 Application Roadmap 23
2.5.5 Symbolic Processing 24
*
;
y
3 Problem Analysis
28
3.1 Problem Identification 28
3.2 Problem Analysis 29
3.2.1 Scope of the Problem 30
3.2.2 Symbolic Nature of the Problem 30
3.2.3 Solution Time 30
3.2.4 Frequency of Problem Occurrence 30
3.2.5 Optimization versus Satisficing 31
3.2.6 Data and Knowledge Availability 31
3.3 Data Requirement Analysis 34
3.4 Expert System Justification 36
3.4.1 Problem-Selection Guidelines 47
4 Knowledge Engineering
4.1
Knowledge-Acquisition Phases 49
;
4.1.1 The Knowledge Engineer 49
4.1.2 Knowledge Characteristics 50
4.1.3 Choosing the Expert 51
4.1.4 Knowledge Extraction versus Knowledge
Acquisition 53
4.2 Methods of Extracting Knowledge from Experts 54
4.2.1 Interviews 54
.--'
4.2.2 Open-Ended Interviews 55
4.2.3 Advantages and Disadvantages of Interviews 56
4.2.4 Task Performance and Protocols 56
4.2.5 Analyzing the Expert's Thought Process 57
4.2.6 Constrained Task 57
4.2.7 Tough Case Method 58
4.2.8 Questionnaires and Surveys 58
4.2.9 Documentation and Analysis of Acquired
Knowledge 58
4.2.10 Expert's Block 59
4.3 Knowledge-Acquisition Meetings 59
4.4 Group Knowledge Acquisition 61
4.4.1 Brainstorming 62
4.4.2 Delphi Method 63
4.4.3 Nominal Group Technique 64
49
CONTENTS
4.5
4.6
4.7
4.8
4.9
Knowledge-Acquisition Software 66
4.5.1 Knowledge Elicitation Tools 66
4.5.2 Induction-by-Example Tools 67
Characteristics of Knowledge 68
4.6.1 Types of Knowledge 68
Knowledge-Representation Models 69
4.7.1 Semantic Networks 70
4.7.2 Frames 71
4.7.3 Scripts 73
4.7.4 Rules 74
4.7.5 Predicate Logic 76
4.7.6 O-A-V Triplets 77
4.7.7 Hybrids 78
4.7.8 Specialized Representation Techniques
Concept of Knowledge Sets 87
4.8.1 Properties of Knowledge Sets 88
Reasoning Models 97
^
78
Probabilistic and Fuzzy Reasoning
k.'5.1
5.2
5.3
5.4
5.5
5.6
103
Human Reasoning and Probability 103
Bayesian Approach to Handling Uncertainty
5.2.1 Logical Relations 106
5.2.2 Plausible Relations 106
5.2.3 Contextual Relations 106
,
Decision Tables and Trees 106
•'
Dempster-Shafer Theory 113
5.4.1 Support Function 116
5.4.2 Plausibility 116
5.4.3 Uncertainty of A 116
5.4.4 Belief Interval 117
5.4.5 Focal Elements 117
5.4.6 Doubt Function 117
Certainty Factors 119
Fuzzy Logic 122
5.6.1 Definition of Fuzzy Set 124
Fuzzy Systems
6.1
Crisp Logic versus Fuzzy Logic
6.1.1 Crisp Sets 133
ix
104
133
133
CONTENTS
6.2
6.3
6.4
6.5
6.6
6.7
6.1.2 Fuzzy Sets 136
6.1.3 Fuzzy Set Construction 140
6.1.4 Fuzzy Set Operations 141
6.1.5 a-Cuts 143
6.1.6 Extension Principle 145
Fuzzy Operations 146
6.2.1 Fuzzy Complement 147
6.2.2 Fuzzy Intersection 148
6.2.3 Fuzzy Union 151
6.2.4 Duality 153
6.2.5 Fuzzy Implication 154
6.2.6 ' Fuzzy Aggregation 156
Fuzzy Numbers 157
6.3.1 Fuzzy Number Representation 157
6.3.2 Fuzzy Number Operation 158
6.3.3 Fuzzy Ordering 161
Fuzzy Relation 161
6.4.1 Definition of a Fuzzy Relation 161
6.4.2 Binary Relation 162
6.4.3 Operations with Relations 164
Evidence Theory 165
6.5.1 Believability and Plausibility 165
6.5.2 Uncertainty 168
6.5.3 Fuzziness 168
/
6.5.4 Nonspecificity 170
Fuzzy Logic 171
6.6.1 Multivalued Logic 172
6.6.2 Unconditional Fuzzy Propositions 172
6.6.3 Conditional Fuzzy Propositions 173
6.6.4 Selection of Implication Operator 176
6.6.5 Multiconditional Reasoning 177
Summary 179
7 Neural Networks
7.1
7.2
Introduction 180
7.1.1 Definition of a Neurode
7.1.2 Variations of a Neurode
Single Neurode 183
y
180
181
182
CONTENTS
7.2.1
7.2.2
7.3
7.4
'
t
7.5
7.6
7.7
7.8
7.9
The McCulloch-Pitts Neurode 183
McCulloch-Pitts Neurodes as Boolean
Components 183
7.2.3 Single Neurode as Binary Classifier 184
7.2.4 Single-Neurode Perceptron 186
Single-Layer Feedforward Network 187
7.3.1 Multicategory SLP 187
7.3.2 Associative Memory 187
7.3.3 Correlation Matrix Memory 189
7.3.4 Pseudoinverse Memory 190
' 7.3.5 Widrow-Hoff Approach 190
7.3.6 Least-Mean-Squares Approach 191
7.3.7 Adaptive Correlation Matrix Theory 191
7.3.8 Error-Correcting Pseudoinverse Method 192
Self-Organizing Networks 192
7.4.1 Principal Components 193
7.4.2 Clustering by Hebbian Learning 194
7.4.3 Clustering by Oja's Normalization 195
1AA Competitive Learning Network 197
Multiple-Layer Feedforward Network 199
7.5.1 Multiple-Layer Perceptron 199
7.5.2 XOR Example 200
7.5.3 Back-Error Propagation 201
7.5.4 Variations in the Back-Error Propagation
Algorithm 202
7.5.5 Learning Rate and Momentum 203
7.5.6 Other Back-Error Propagation Issues 205
7.5.7 Counterpropagation Network 205
Radial Basis Networks 206
7.6.1 Interpolation 207
7.6.2 Radial Basis Network 208
Single-Layer Feedback Network 211
7.7.1 Single-Layer Feedback Network 212
7.7.2 A Discrete Single-Layer Feedback Network 213
7.7.3 Bidirectional Associative Memory 215
7.7.4 Hopfield Network 216
Summary 219
References 220
y"
xi
xii
CONTENTS
8
Neural-Fuzzy Networks
8.1
8.2
8.3
8.4
9
/
;
y
221
Technology Comparisons 221
Neurons Performing Fuzzy Operations 223
8.2.1 Neurons Emulating Fuzzy Operations 224
8.2.2 Neurons Performing Fuzzy Operations 226
Neural Network Performing Fuzzy Inference 227
8.3.1 Regular Neural Network with Crisp Input and
Output 227
8.3.2 Regular Neural Network with Fuzzy Input and
Output 228
8.3.3 Fuzzy Inference Network 229
8.3.4- ANF1S 230
8.3.5 Applications 231
Clustering and Classification 232
8.4.1 Classification 233
8.4.2 Multilayer Fuzzy Perceptron 235
Evolutionary Computing
9.1 Introduction 237
9.2 Binary Genetic Algorithm 239
9.2.1 Genetic Representation 241
9.2.2 Population 242
9.2.3 Fitness Check and Cost Evaluation
/
9.2.4 Mating Pool 243
9.2.5 Pairing 244
9.2.6 Mating 246
9.2.7 Mutation 248
9.2.8 The Next Generation 1149
9.2.9 Performance 250
9.2.10 Enhancements 251
9.3 Continuous Genetic Algorithm 252
9.3.1 Genetic Representation 253
9.3.2 Mating 254
9.3.3 Mutation 256
9.3.4 Performance 256
9.3.5 Enhancements 257
9.4 Evolutionary Programming 259
9.4.1 Evolutionary Strategies 259
y'
237
242
T
CONTENTS
9.5
9.4.2 Evolutionary Programming
Summary 263
260
.
10 Intelligent Strategy Generation in Complex Manufacturing
Environments
10.1
10.2
10.3
10.4
10.5
xiii
y
264
Introduction 264
Model Description 265
Evolutionary Algorithm 267
Process Simulation 269
Fuzzy Logic Evaluation 271
11 Product Demand Forecasting Using Genetic Programming
11.1 Introduction 274
11.2 Algorithm Description 276
11.2.1 Chromosome Structure 276
11.2.2 Fitness Evaluation 276
11.2.3 Reproduction and Generation Evolution
11.3 Experiments and Results 278
11.4 Conclusion 281
References 282
Index 289
277
274