Download Slide - Centre for Modeling and Simulation

Computational Intelligence Based
Methodologies for Modeling and
Optimization
Somnath Nandi
Asst. Professor
Dept. of Petroleum and Petrochemical Engineering
MIT - Pune
Contents
•
•
•
•
•
•
•
Modeling
Artificial Neural Networks
Optimization
Genetic Algorithms
Differential Evolution
Case Study
Conclusion
Modeling
• Engineers and scientists required to analyze the
complex processes and develop mathematical
models which simulate their steady-state and / or
dynamic behavior.
• The objective is to construct, from theoretical and
empirical knowledge of the process, a mathematical
description.
• A mathematical model provides information on the
process behavior, over important ranges of
operating variables, in terms of equations, which
reflects at least the major features of the
underlying mechanisms.
Modeling (contd… )
• Phenomenological Approach:
- Process behavior described in terms of the
appropriate mass, momentum and energy
balance equations together with the pertinent
chemical engineering principles.
- Mathematical formulation describing the
physico-chemical phenomena underlying in the
process is formulated followed by model
fitting.
- Regression techniques based on the least
squares minimization
Modeling (contd… )
• Advantages :
- It provides a valuable insight into the process behavior
- It possesses extrapolation ability
• Disadvantages:
- Owing to the complex nature of many processes, the
underlying physico-chemical phenomenon is seldom fully
understood
- Collection of the requisite phenomenological information
is costly, time-consuming and tedious
- Nonlinear behavior common for many processes leads to
complex nonlinear models, which in most cases are not
amenable to analytical solutions; thus, computationally
intensive numerical methods must be utilized for obtaining
solutions
Modeling (contd… )
• Empirical Approach:
- Process behavior is modeled using appropriately
chosen empirical equations, for instance, polynomial
expressions.
- Model can be constructed solely from the process
input-output data without explicitly invoking the
process phenomenology.
- An appropriate functional form that possibly fits
the process data is selected in advance following
which the unknown model parameters are estimated
using a suitable function fitting procedure.
Modeling (cond… )
• Artificial Intelligence based Approach:
-
AI is science and engineering of making “intelligent”
systems, especially intelligent computer programs
- Related to task of using computers to understand
the “human intelligence”
- Intelligence can be broadly defined as
computational part of our ability to efficiently
achieve goals in the world.
AI based Modeling Approaches
• Artificial Neural Networks (ANN)
• Support Vector Regression (SVR)
• Genetic Programming (GP)
• Fuzzy Logic (FL)
Artificial Neural Networks
• Efforts to develop computer models of the
information processing of human nervous
system (Rumelhart et. al., 1986).
• Simplified mathematical models describing
the biological nervous system and
functioning.
• A highly interconnected system of simple
processing elements can learn complex
Artificial Neural Networks
Outputs
y1
y2
y3
yNO
1
2
3
NO
1
Hidden layer
NH
2
1
2
3
NI
x1
x2
x3
xNI
Inputs
Output layer
Input layer
Artificial Neural Networks
y k  f k  x, w k , k 1, 2, ..., K
Input
x1
x2
y1
yK
xN
RMSE 
  yˆ
P
K
i 1 k 1
i, k
y

p 2
i, k
Output
Artificial Neural Networks
• The distinct advantages of the ANN formalism
are:
– Can be developed solely from process input-output
–
–
–
–
–
data.
MIMO relationships can be approximated
Possesses good generalization ability
Can tolerate noisy data or incomplete information
Can be developed even using qualitative data.
Use a generic nonlinear function for function
approximation and thus there is no need to specify
system-specific data fitting function as done in
traditional regression.
Artificial Neural Networks
• Principal applications of ANNs are:
(i) nonlinear function approximation (i.e.,
process modeling),
(ii) pattern recognition and
classification,
(iii) data reduction and compression,
(iv) signal processing,
(v) noise reduction.
What is Optimization ?
• Optimization is use of specific methods to
determine the most cost-effective and efficient
solution to a problem or design for a process
• A wide variety of problems in the design,
construction, operation, and analysis of industrial
processes can be resolved by optimization
• The field of statistics treats various principles
termed "maximum likelihood," "minimum loss," and
"least squares," and business makes use of
"maximum profit," "minimum cost," "maximum use
of resources," "minimum effort," in its efforts to
increase profits
What is Optimization ?
• A typical engineering problem can be posed as follows:
a process can be represented by some equations or
perhaps solely by experimental data. You have a single
performance criterion in mind such as minimum cost
• The goal of optimization is to find the values of the
variables in the process that yield the best value of
the performance criterion
• A trade-off usually exists between capital and
operating costs. The described factors-process or
model and the performance criterion-constitute the
optimization "problem."
What is Optimization ?
• Optimization is minimization or
maximization of an objective function
(also called a performance index or goal
function) that may be subject to certain
constraints
min f (x)
Goal function
subject to,
g (x) = 0
Equality constraints
h (x) < 0
Inequality constraints
Need for Optimization
• Typical problems in engineering process design or
plant operation have many (possibly an infinite
number) solutions
• Optimization is concerned with selecting the best
among the entire set by efficient quantitative
methods
• Computers and associated software make the
necessary computations feasible and cost effective
• To obtain useful information using computers,
however, requires
(1) critical analysis of the process or design,
(2) insight about what appropriate performance
objectives are (what is to be accomplished),
(3) use of past experience, sometimes called
engineering judgment.
Applications of Optimization
• Determining the best sites for plant location
• Routing tankers for the distribution of crude and
refined products
• Sizing and layout of a pipeline
• Designing equipment and an entire plant
• Scheduling maintenance and equipment replacement
• Operating equipment, such as tubular reactors,
columns, and absorbers
• Evaluating plant data to construct a model of a
process
• Minimizing inventory charges
• Allocating resources or services among several
processes
• Planning and scheduling construction
1-Dimensional Search
2-Dimensional Search
2-Dimensional Search
2-Dimensional Search
Unimodal Optimization
Multi-modal Optimization
A function exhibiting different types of stationary points.
a-inflection point (scalar equivalent to a saddle point);
b-global maximum;
c-local minimum;
d-local maximum
Global Methods of Optimization
Performance of Classical Techniques
Multiobjective Optimization
• A MOO problem will have two or more objectives
involving many decision variables and constraints
• Consider an MOO problem with two objectives: f1(x)
and f2(x), and several decision variables (x)
Minimize f1(x)
Minimize f2(x)
With respect to x
Subject to xL ≤ x ≤ xU
h (x) = 0
g (x) ≤ 0
(1)
(2)
(3)
(4)
(5)
Multiobjective Optimization
Different Evolutionary Techniques
•
•
•
•
•
•
•
•
Genetic Algorithms (GA)
Simulated Annealing (SA)
Ant Colony Optimization (ACO)
Tabu Search (TS)
Particle Swarm Optimization (PSO)
Differential Evolution (DE)
Memetic Algorithm (MA)
Simultaneous Perturbation Stochastic
Approximation (SPSA)
What is GA ?
• GAs are computer based search and
optimization algorithms based on mechanics
of natural genetics and natural selection
• A population of initial solution is generated
within feasible region
• The main idea is
- Survival of the fittest
- Evolution of species with time
• Only best solution will survive till end
What is GA ?
• Genetic Algorithms (GAs) were invented by John
Holland and developed by him and his students and
colleagues. This lead to Holland's book "Adaptation in
Natural and Artificial Systems" published in 1975.
• All living organisms consist of cells. In each cell there
is the same set of chromosomes
• A chromosome consists of genes, blocks of DNA.
Each gene encodes a particular protein
• Complete set of genetic material (all chromosomes) is
called genome.
• Particular set of genes in genome is called genotype.
Working Principle
Let us consider the maximization problem:
Maximize
f x ,
xiL  xi  xiU ,
i  1, 2, ... , n
• Coding:
- Variable xi are first coded into binary
strings
- Length of string is determined based on
desired accuracy of solution
 x1L , x2L  T   0 0 0 0 0 0 0 0 T
x
U
1
, x2U
   1111 1111
T
T
Working Principle
• Fitness function
- GA are based on survival-of-the-fittest
- Naturally suitable for solving maximization
problems
- Minimization are transformed to suitable
maximization ones
- Fitness function is a measure of goodness of
the string
- Our target is to keep on increasing the overall
fitness functions of all the strings
- Genetic operators perform duty to manipulate
binary strings so that fitness function is keep
on increasing on successive iterations
Working Principle
• GA Operators
- Reproduction / Selection
 Selects good strings of a population
 Forms a mating pool
 Above – average stings are picked from current
population
 Multiple copies of selected strings are placed
in mating pool in a probabilistic manner
 No new strings are formed in this phase
 Roulette – Wheel or Stochastic Remainder
Selection methods
Working Principle
• Crossover
-
New strings are created
- It exchanges information among
strings of mating pool
- 2 strings are picked at random
- Point of crossover is probabilistically chosen
0 0 0 1 1 1
0 0 0 0 1 0
1 1 1 0 1 0
1 1 1 1 1 1
Parent Strings
Children Strings
Working Principle
• Mutation
- It changes 1 to 0 and vice versa
- Small probability pm generally < 0.1
- Need is to create a point in the neighborhood of
the current point
- Performs local search around current solution
- It maintains diversity of population
0 0 0 1 1 1
1 1 1 0 1 0
0 0 0 0 1 1
Mutation
1 1 0 0 1 0
Diversification
• Generate initial population covering entire range
• Visit new places
• Extract characteristics of each region
• Cover as much as possible
• Performing Global Search
• All are done by Crossover operator
Intensification
• Should be started once search space is well scanned
• Visit zones adjacent / nearby to already visited
• Check the performance
• Perform local search
• This is done by Mutation operator
Algorithm
• Step 1: Do coding, choose selection operator, crossover and
mutation probability (pc and pm). Choose population size (n),
string length (l), max. no. of iterations (Nmax)
• Step 2: Evaluate each string of population
• Step 3: Perform Reproduction on population
• Step 4: Perform crossover on random pairs of strings
• Step 5: Perform mutation on each string
• Step 6: Evaluate strings of new population
• Step 7: Set N = N + 1 and go to step 3
• Terminate if N > Nmax or no further improvement on string
performance
Advanced GA
•
Multi Point Crossover :
1 1 0 0 0 1 1 0
1 1 0 1 1 0 1 0
1 0 0 1 1 0 0 1
1 0 0 0 0 1 0 1
•
Real Coded GA :
- Real variables are directly used
- Optimal point of any desired accuracy obtained
•
Non – dominated Sorting :
- To keep versatility of population
- Give more chance to a poor performer to enhance its skills
•
Pareto GA :
- Population in a GA simulation is adaptively divided into separate
subpopulation, corresponding to each optimum point by use of sharing
functions
- Can get all the solutions of Pareto Optimal front in one shot
GA - Applications
• Reactor Design – Ammonia Synthesis
• Process Optimization
– Cumene Synthesis
- Phenol Production
• Scheduling – Refinery Operations
• Multiphase – Trickle Bed Reactor
• Polymerization Processes
– MMA Synthesis
- Polyethylene Plant
- Nylon Manufacture
• Water Distribution
Differential Evolution
• Introduced by Storn and Price in 1996
• Algorithm works with a population of size N
• Algorithm iterates as follows:
- Generate new vector by adding
weighted difference of two vectors to third
- Mix new vector with target vector to yield
trial vector
- Replace target vector with trial vector if
latter is strictly superior
Differential Evolution
Differential Evolution
• F and CR are DE control parameters
• F is a real-valued factor in the range
(0.0,1.0+]
• Upper limit on F has been empirically
determined.
• CR is a real-valued crossover factor in range
[0.0,1.0]
• CR controls the probability that a trial vector
parameter will come from the randomly
chosen noise vector
Importance of Parameters
• Optimal values are dependent both on
objective function characteristics and
on the population size, NP
• Practical advice on how to select control
parameters NP, F and CR can be found
in the literature
Crossover in DE
DE - Applications
•
•
•
•
•
•
•
•
•
•
Multiprocessor synthesis
Power minimisation
Neural network learning.
Crystallographic characterization
Design of Shell-and-Tube Heat Exchangers
Heat transfer parameter estimation in a trickle bed
reactor
Gas Transmission Network
Water Pumping and Distribution Systems
Optimization of Ammonia Synthesis Reactor
Design and Operation of Thermal Cracker
DE - Advantages
•
•
•
•
•
•
Powerful algorithm- multidimensional functions
Easy applicable to various problems.
Widely used
Literature and other materials available
Generally good accuracy for real world problems
Easy to implement as same parameter settings work
fine for a wide range of problems
• Drawback :
Somewhat slow during initial iterations
Cumene Synthesis
• Main reaction :
Benzene + Isopropyl Alcohol  Cumene + Water
(benzene alkylation)
• Secondary reactions :
Cumene + Isopropyl Alcohol  p-Di-isopropyl Benzene + Water
(cumene alkylation)
p-Di-isopropyl Benzene  m-Di-isopropyl Benzene
(isomerization)
2 Isopropyl alcohol  Di-isopropyl ether + Water
(alcohol dehydration)
Cumene Synthesis
Catalyst
• Beta is a crystalline alumino-silicate catalyst with
high silica content
• Important characteristic is that it is the only large
pore zeolite with chiral pore intersections
• It consists of 12-membered rings interconnected by
cages formed by intersecting channels
• The linear channels have pore opening dimensions of
5.7  7.5 Å
• the tortuous channels with intersections of two linear
channels have approximate dimensions of 5.6  6.5 Å
• The catalyst has pore volume of  0.2 cm3/g.
• Beta catalyst (1.5 mm extrudates with 20 % binder) in
its active protonated form with Si to Al ratio of 15
was obtained from M/s UCIL, India
Reactor
• Vapor phase isopropylation of benzene was carried
out in a pilot plant scale stainless steel reactor
• A preheater in its upstream and a condenser in the
down-stream
• Material of construction: SS 316,
• Internal diameter (ID): 25 mm
• Wall thickness: 6 mm
• Reactor length: 33 cm
• Catalyst bed height: 10-15 cm
• Heating coils are wound around the reactor to provide
proper heating and maintain temperature
• Reactor is also jacketed with insulation to minimize
the heat loss
The Operation
• The liquid mixture of benzene and
isopropyl alcohol was fed to the reactor
by a positive displacement pump
• Hydrogen was used as the carrier gas
• The condensed products collected were
analyzed with a Flame Ionization
Detector (FID) using a “Xylene Master”
capillary column fitted to a Shimadzu
15A Gas Chromatograph (GC)
Process Parametrs
• Important Operating Variables
–
–
–
–
reaction temperature (x1)
pressure (x2)
benzene to isopropyl alcohol mole ratio (x3)
weight hourly space velocity (WHSV) (x4)
• Outputs are: Cumene yield and selectivity y1, y2
y1 
100  weight of cumene formed per unit time
weight of isopropyl alcohol fed per unit time
100  weight of cumene formed per unit time
y2 
weight of total aromatics produced per unit time
Expt.
No.
Temperature
( 0C)
Pressure (atm.)
Benz/IPA
(mole ratio)
WHSV
(hr-1)
Yield
( wt %)
Selectivity
( wt %)
1
110
1
8
3.3
0.07
77.03
2a
145
1
8
3.3
11.6
58.75
3
180
1
8
3.3
15.78
79.93
4
210
1
8
3.3
17.365
90.72
5
215
1
8
3.3
16.09
91.95
6
150
4
8
3.3
12.2
65.74
7
135
4
8
3.3
12.99
74.58
8a
110
4
8
3.3
0.71
80.82
9
100
4
8
3.3
0.19
75.02
10
110
1
10
3.3
0.55
67.74
11
110
1
8
3.3
0.24
54.85
12a
110
1
6
3.3
0.37
53.63
13
110
1
3
3.3
0.2
32.13
14
110
1
1
3.3
0.14
21.62
15
110
1
8
6.8
0.24
54.85
16
110
1
8
8
0.15
44.64
17
110
1
8
9.5
0.13
37.38
18
110
1
8
10.5
0.08
39.3
19a
110
1
8
12
0.09
39.13
20
110
1
8
13
0.07
39.1
21
105
1
8
6.8
0.3
70.38
22
110
1
8
6.8
0.24
54.85
23
115
1
8
6.8
0.35
48.25
18
110
1
8
10.5
0.08
39.3
19a
110
1
8
12
0.09
39.13
20
110
1
8
13
0.07
39.1
21
105
1
8
6.8
0.3
70.38
22
110
1
8
6.8
0.24
54.85
23
115
1
8
6.8
0.35
48.25
24
130
1
8
6.8
4.61
76.68
25
185
1
8
6.8
9.2
59.23
26
210
1
6.5
3.3
20.04
91.8
27
155
1
6.5
3.3
16.93
77.4
28a
180
1
6.5
3.3
20.27
90.9
29
210
1
6.5
3.3
19.86
91.9
30
225
1
6.5
3.3
19.1
89.3
31
250
1
6.5
3.3
17.89
85.2
32
275
1
6.5
3.3
17.29
83.1
33
230
1
6.5
2.5
20.33
91.1
34
215
1
7
5
19.86
91.9
35a
215
10
7
5
19.54
92
36
215
18
7
5
18.68
89.1
37
215
25
7
5
17.74
86.8
38
195
25
6
5
18.92
85.6
39
210
25
6
5
22.1
93.7
40
230
25
6
5
22.02
93.8
41
250
25
6
5
21.35
90.7
42
280
25
6
5
20.48
86.2
Modeling of Output vs. Input
25
Yield (wt %)
20
15
10
5
Predicted
Experimental
0
0
5
10
15
20
25
30
35
40
45
Expt. No.
Selectiviyty (wt %)
100
80
60
40
Predicted
20
Experimental
0
0
5
10
15
20
25
Expt. No.
30
35
40
45
Optimization
• Best values of following GA-specific parameters were
chosen heuristically:
- population size (Npop) = 25
- crossover probability (pcross) = 0.82
- mutation probability (pmut) = 0.05
- maximum number of generations (Ngen) = 100
• In order to obtain the best set of operating conditions,
GA runs were replicated several i.e. 50 times, using
different random number generator seeds.
• The fitness function:
ˆ  wˆ 1 y1  wˆ 2 y 2  y1  y 2 ; wˆ  wˆ  0.5 ; j 1, 2, ..., N
j
1
2
pop
100
200
Optimized Results
ANN-GA
Soln.
No.
Optimized Inputs
Maximized Outputs
Temp.
(0 C)
(x1*)
Press.
(atm.)
(x2*)
Benz/IPA
(mol ratio)
(x3*)
WHSV (hr-1)
(x4*)
Yield
(wt %)
(y1*)
Selectivity
(wt %)
(y2*)
1
271.5
3.38
3.69
12.83
24.88
99.04
2
267.2
1.567
4.05
12.83
24.84
98.90
3
270.08
3.6
4.05
11.76
24.82
98.74
Experimental Verification
Experimental Conditions
Exp.
No.
Yield (output 1)
Selectivity (output 2)
Temp.
(0C)
Pressure
(atm)
Benz/IPA
(mole
ratio)
WHSV
(hr-1)
GAmaximized
value
(wt %)
Exptl.
value
(wt %)
Error
(%)
GAmaximized
value
(wt %)
Exptl.
value
(wt %)
Error
(%)
1
271.5
3.4
3.7
12.8
24.88
24.69
0.77
99.04
98.98
0.06
2
267.2
1.6
4.0
12.8
24.84
23.79
4.41
98.90
98.70
0.20
3
270.0
3.6
4.0
11.8
24.82
24.58
0.98
98.74
98.65
0.09
Published in Chemical Engineering Journal, Vol. 97, No. 2 – 3, pg: 115 – 129 (2004)
Benefit of the Study
• The work extended from pilot plant level to
commercial scale
• Implemented successfully by HPCL
• Overall profit increased by almost 18 %
• Some more research work with HPCL and
others regarding their multiphase operations
• Leads to optimization of Polypropylene
Production unit of Reliance at their Hazira
plant
Overall Conclusion
• Modeling and various approaches
discussed
• ANN-based modeling introduced
• Optimization and its necessity
• Multi-objective optimization
• Genetic Algorithm methodology
• Differential Evolution – a novel method
• Cumene synthesis – case study
References
•
•
•
•
•
•
•
•
•
Rumelhart, D., Hinton, G., and Williams, R. “Learning Representations by Backpropagating
Errors”, Nature, 323, 533 - 536 (1986).
Deb, K. “Optimization for Engineering Design: Algorithms and Examples”, Prentice Hall
of India, New Delhi (2006)
Deb, K., “Multiobjective Optimization Using Evolutionary Algorithms”, Wiley, Chichester,
UK (2001)
Nandi, S. Ghosh, S. Tambe S and Kulkarni, B. D. “Artificial Neural Network Assisted
Stochastic Process Optimization Strategies”, AIChE J, Vol. 47, pp. 126-141 (2001).
Nandi, S. Mukherjee, P, Tambe, S. S. , Kumar, R. and Kulkarni, B. D. “Reaction Modeling
and Optimization Using Neural Networks and Genetic Algorithms: Case Study Involving
TS-1 Catalyzed Hydroxylation of Benzene”, Ind. Engg. Chem. Res., Vol. 41, pp. 21592169 (2002).
Nandi, S., Badhe, Y. , Lonari, J. B., Sridevi, U., Rao, B. S. Tambe, S. S., Kulkarni, B. D.
“Hybrid Process Modeling and Optimization Strategies Integrating Neural
Networks/Support Vector Regression and Genetic algorithms: Study of Benzene
Isopropylation on HBeta Catalyst”, Chem. Engg. Jour. Vol. 97, pp. 115-129 (2004).
Price, K.V. (1999). An Introduction to Differential Evolution. In: Corne, D., Dorigo,M. and
Glover, F. (eds.) (1999). New Ideas in Optimization, pp. 79–108. McGraw-Hill, London.
ISBN 007-709506-5.
Storn, R. and Price, K.V. (1995). Differential evolution - a Simple and Efficient Adaptive
Scheme for Global Optimization Over Continuous paces. Technical Report TR-95-012,
ICSI, March 1995. Available via Internet:
ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-012.ps.Z .
Storn, R. and Price, K.V. (1997). Differential Evolution – a Simple and Efficient Heuristic
for Global Optimization over Continuous Spaces. Journal of Global Optimization, 11(4):
341–359, December 1997. Kluwer Academic Publishers.