Download Artificial Neural Network An artificial neural network (ANN)

Artificial Neural Network
An artificial neural network (ANN), usually called neural network (NN), is a mathematical
model or computational model that is inspired by the structure and/or functional aspects of
biological neural networks. A neural network consists of an interconnected group of artificial
neurons, and it processes information using a connectionist approach to computation. They are
powerful tools for modelling, especially when the underlying data relationship is unknown.
ANNs can identify and learn correlated patterns between input data sets and corresponding target
values. After training, ANNs can be used to predict the outcome of new independent input data.
ANNs have been applied to many geotechnical engineering problems such as in pile capacity
prediction, modelling soil behaviour, site characterisation, earth retaining structures, settlement
of structures, slope stability, design of tunnels and underground openings, liquefaction, soil
permeability and hydraulic conductivity, soil compaction, soil swelling and classification of
soils.
Figure 1: Three layer neural network
1
Figure1 show three layer neural network consist first layer has input neurons, second layer of
hidden neurons, third layer of output neurons. Supervised neural networks are trained in order to
produce desired outputs in response to training set of inputs. It is trained by providing it with
input and matching output patterns.
It used in the modelling and controlling of dynamic systems, classifying noisy data, and
predicting future events. Unsupervised neural networks, on the other hand, are trained by letting
the network continually adjusting itself to new input. It is or Self-organisation in which an
(output) unit is trained to respond to clusters of pattern within the inputs. Reinforcement
Learning is be considered as an intermediate form of the above two types of learning. Here the
learning machine does some action on the environment and gets a feedback response from the
environment.
For an artificial neuron, the weight is a number, and represents the synapse. A negative weight
reflects an inhibitory connection, while positive values designate excitatory connections. All
inputs are summed altogether and modified by the weights and refers as a linear combination.
Finally, an activation function controls the amplitude of the output. For example, an acceptable
range of output is usually between 0 and 1, or it could be -1 and 1.
A neuron is a real function of the input vector (x 0 , x 2 , … x k ). The out put is obtained as f(y j )
Where, f is a function, typically the sigmoid (logistic or tangent hyperbolic) function. A
graphical presentation of neuron is given in figure 2. Mathematically a Multi-Layer Perceptron
network is a function consisting of compositions of weighted sums of the functions
corresponding to the neurons.
2
Figure 2: Graphical presentation of neuron in ANN
There are several types of architecture of NNs. However, the two most widely used NNs Feed
forward networks and Recurrent networks. In a feed forward network, information flows in
one direction along connecting pathways, from the input layer via the hidden layers to the final
output layer. There is no feedback (loops) i.e., the output of any layer does not affect that same
or preceding layer. Feed-forward neural networks, where the data ow from input to output
units is strictly feedforward. The data processing can extend over multiple (layers of) units, but
no feedback connections are present.
3
Figure 3: A multi-layer feed forward neural network
These networks differ from feed forward network architectures in the sense that there is at least
one feedback loop. Thus, in these networks, for example, there could exist one layer with
feedback connections as shown in figure below. There could also be neurons with self feed back
links, i.e. the output of a neuron is fed back into itself as input.
Recurrent neural networks that do contain feedback connections. In some cases, the activation
values of the units undergo a relaxation process such that the neural network will evolve to a
stable state in which these activations do not change anymore. In other applications, the change
of the activation values of the output neurons are significant, such that the dynamical behaviour
constitutes the output of the neural network (Pearlmutter, 1990).
4
Figure 4: A recurrent neural network
Black propagation network
The back-propagation algorithm is a non-linear extension of the least mean squares (LMS)
algorithm for multi-layer perceptrons. It is the most widely used of the neural network paradigms
and has been successfully applied in many fields of model-free function estimation. The back
propagation network (BPN) is expensive computationally, especially during the training process.
Properly trained BPN tends to produce reasonable results when presented with new data set
inputs.
A BPN is usually layered, with each layer fully interconnected to the layers below and above it.
The first layer is the input layer, the only layer in the network that can receive external input. The
second layer is the hidden layer, in which the processing units are interconnected to the layers
below and above it. The third layer is the output layer. Each unit of the hidden layer is
interconnected with the units of the output layer. Units are not interconnected to other units
within the same layer. Each interconnection is assigned an associative connection strength,
expressed as weight (Figure 1). Weights are adjusted during the training of the network. In BPN,
the training is supervised, in which case the network is presented with target values for each
input pattern. The input space of the network is considered to be linearly separable.
5
The various steps in developing a neural network model are: summarized below & the example
is shown by metlab software.
Variable selection
The input variables important for modeling variable(s) under study are selected by suitable
variable selection procedures.
Formation of training, testing and validation sets
The data set is divided into three distinct sets called training, testing and validation sets. The
training set is the largest set and is used by neural network to learn patterns present in the data.
The testing set is used to evaluate the generalization ability of a supposedly trained network. A
final check on the performance of the trained network is made using validation set.
Neural network architecture
Neural network architecture defines its structure including number of hidden layers, number of
hidden nodes and number of output nodes etc. Number of hidden layers: The hidden layer(s)
provide the network with its ability to generalize. In theory, a neural network with one hidden
layer with a sufficient number of hidden neurons is capable of approximating any continuous
function. In practice, neural network with one and occasionally two hidden layers are widely
used and have to perform very well.
• Number of hidden nodes: There is no magic formula for selecting the optimum number of
hidden neurons. However, some thumb rules are available for calculating number of hidden
neurons. A rough approximation can be obtained by the geometric pyramid rule proposed by
Masters (1993). For a three layer network with n input and m output neurons, the hidden layer
would have sqrt(n*m) neurons.
• Activation function: Activation functions are mathematical formula that determine the output
of a processing node. Each unit takes its net input and applies an activation function to it. Non
linear functions have been used as activation functions such as logistic, tanh etc. Transfer
functions such as sigmoid are commonly used because they are nonlinear and continuously
differentiable which are desirable for network learning.
6
Evaluation criteria
The most common error function minimized in neural networks is the sum of squared errors.
ther error functions offered by different software include least absolute deviations, least fourth
powers, asymmetric least squares and percentage differences.
Neural network training
Training a neural network to learn patterns in the data involves iteratively presenting it with
examples of the correct known answers. The objective of training is to find the set of weights
between the neurons that determine the global minimum of error function. This involves decision
regarding the number of iteration i.e., when to stop training a neural network and the selection of
learning rate.
Figure 5:
The various reserche have used ANN to predict to the slope slabiling or slope failure or factor of
safety. The Back propagation neural network is used to calculate the factor of safety. Nine input
parameters and one output parameter are used in the analysis. The output parameter is the factor
of the safety of the slopes, the input parameters are the height of slope, the inclination of slope,
7
the height of water level, the depth of firm base, the cohesion of soil, the friction angle of soil,
the unit weight of soil, but the important input parameters are horizontal and vertical seismic
coefficients.
Slope failures are complex natural phenomena that constitute a serious natural hazard in many
countries. To prevent or mitigate the landslide damage, slope-stability analyses and stabilization
require an understanding and evaluation of the processes that govern the behavior of the slopes.
The factor of safety based on an appropriate geotechnical model as an index of stability, is
required in order to evaluate slope stability. Many variables are involved in slope stability
evaluation and the calculation of the factor of safety requires geometrical data, physical data on
the geologic materials and their shear-strength parameters (cohesion and angle of internal
friction), information on pore-water pressures, etc.
The determination of the non-linear behaviour of multivariate dynamic systems often presents a
challenging and demanding problem. The impact of these parameters on the stability of slopes is
investigats through the use of computational tools called neural networks. the input data for
slope stability estimation consist of values of geotechnical and geometrical input parameters. the
network estimates the factor of safety (FS) that can be modelled as a function approximation
problem, or the stability status (S) that can be modelled either as a function approximation
problem or as a classification model. The performance of the network is measured and the results
are compared to those obtained by means of standard analytical methods.
A series of ANNs were created in order to predict the safety factor and estimate stability against
the circular failure mechanism and the wedge failure mechanism.
Ann and fuzzy set could primarily be used in two ways in slope stability. One is prediction of
various strength and physico mechanical properties by previously used properties.
Other is direct prediction of factor of safety or stability based on simulation of huge data set or
incorporating the case studies.
The rock slopes have important role for the design and excavation in various open pit mine and
also civil engineering projects all around the world. Initial the condision and friction angle can be
trained by neural network. input parameter of compressive strength and later on cohesion and
friction angle were calculated by compressive strength and these properties were used as input
for finite difference code to analysing slope stability and determine the factor of safety.
8
Rock Properties
Compressive strength as input
Cohesion & friction angle as output
Trained Data by ANN
Calculate the Cohesion &
Friction by input
Compressive Strength
Input the data in Software
Analysis of Slope Stability
Figure 6: Flow Chart to determine the properties and analysis the Slope stability by ANN
9
Fuzzy Inference System
Fuzzy logic is a form of many-valued logic and it deals with reasoning that is approximate rather
than fixed and exact. The nature of uncertainty in a slope design is a very important that should
considered. Fuzzy set theory was developed specially to deal with uncertainties that are
nonrandom in nature.
The fuzzy set was first introduced in 1965 by Lofti Zadeh as a mathematical way to represent
linguistic vagueness . It can be considered as a generalization of classical set theory. In a classical
set, an element belongs to or does not belong to a set. That is, the membership of an element is
crisp (0, 1), and an ‘‘A’’ crisp set of real objects are described by a unique membership function
such as X A in fig.3.1 (a).
Fig. 3.1 (a) Crisp set and (b) Fuzzy set
Contrary, a fuzzy set is a generalization of an ordinary set which assign the degree of
membership for each element to range over the unit interval between 0 and 1 as shown in fig.
3.1(b). In addition, fuzzy set theory can be used for developing rule-based models which
combine physical insights, expert knowledge and numerical data in a transparent way that
closely resembles the real world. An element of the variable can be a member of the fuzzy set
through a membership function that can take values in the range from 0 to 1. Membership
functions (MF) can either be chosen by the user arbitrarily based on the user’s experience or can
also be designed using machine learning methods (e.g., artificial neural networks, genetic
algorithms, etc.). There are different shapes of membership functions; triangular, trapezoidal,
10
piecewise-linear, Gaussian, bell shaped, etc. The fuzzy rules provide a system for describing
complex (uncertain, vague) systems by relating input and output parameters using linguistic
variables. A fuzzy if–then rule assumes the form ‘‘if x is A then y is B,’’ where A and B are
linguistic values defined by fuzzy sets on universes of discourse X and Y, respectively.
Fuzzy inference is the process of formulating an input fuzzy set map to an output fuzzy set using
fuzzy logic. In fact, the core section of a fuzzy system is the FIS part, which combines the facts
obtained from the fuzzification with the rule base and conducts the fuzzy reasoning process.
Generally, the basic structure of a FIS consists of three conceptual components, rule base,
database, and reasoning mechanism. A rule base contains a selection of fuzzy rules and a
database defines the membership functions used in the fuzzy rules. A reasoning mechanism
performs the fuzzy reasoning based on the rules and given facts to derive a reasonable output or
conclusion. There are several FISs that have been employed in various applications. The most
commonly used include:
•
Mamdani Fuzzy Model;
•
Takagi-Sugeno-Kang fuzzy (TSK) model;
•
Tsukamoto fuzzy model;
•
Singleton fuzzy model.
The differences between these FISs lie in the consequents of their fuzzy rules, and thus their
aggregation and defuzzification procedures differ accordingly. Defuzzification is a process of
reducing an aggregated (or clipped) fuzzy set into a crisp number, presumably the most
representative value of that fuzzy set interval. There are two methods which are generally used
for defuzzification i.e. Centre of area (Centroid) method and Ranking index method.
The Mamdani Fuzzy model is often used in geotechnical problems because of its simplicity and
effectiveness to handle linguistic variables. Basically, rule base, database and reasoning
mechanism are three conceptual elements of a FIS. The fuzzy rules constitute the rule base and
the database determines the membership functions associated with the inputs parameters to be
used in the rule base while the reasoning mechanism provides the platform to derive an adequate
11
conclusion (output) by using fuzzy logic. At this stage the extraction of a crisp set from a fuzzy
set, called defuzzification is performed.
Fuzzy logic provides an inference structure that enables the human reasoning capabilities to be
applied to artificial knowledge-based systems. Fuzzy logic provides a means for converting
linguistic strategy into control actions and thus offers a high-level computation.Fuzzy logic
provides mathematical strength to the emulation of certain perceptual and linguistic attributes
associated with human cognition, whereas the science of neural networks provides a new
computing tool with learning and adaptation capabilities. The theory of fuzzy logic provides an
inference mechanism under cognitive uncertainty, computational neural networks offer exciting
advantages such as learning, adaptation, fault tolerance, parallelism, and generalization.
Table 1: Distribution of the main references according to the fuzzy inference techniques
employed.
Type of fuzzy set/logic
Number of Topics
papers
Basic fuzzy set/logic
13
Slope
stability,
foundation,
rock
mass
classification, geotechnical project scheduling
and cost planning determination geotechnical
parameters,
prediction
of
soil
uniaxial
compressive strength, fizzyfication of Chen
plastic model of concrete and rock sawability.
Mandani systems
18
Rock mass blastability, penetrability, diggability,
rippability, excavability; rock mass classification
systems, prediction of flyrock in mining surface,
burden, rock fragmentation, backbreak in openpit blasting and TBMs thrust and torque
requirement
Seguno-Tagaki systems
5
Prediction of maximum charge par delay in
surface mining, impact hammer performance,
12
constitutive modeling, swelling potential of
compacted soils, rock engineering classification
system and rock slope stability assessment
Hybrid system using Neural 12
Constitutive modeling of undrained response of
network (ANFIS and other)
sand mixtures, angle of shearing resistance of
soils,
tunnel
modeling,
boring
machine
liquefaction
performance
prediction,
footing
response modeling, modulus of deformation of
jointed rock masses, slake durability of shaly
rock and landside susceptibility mapping
Hybrid system using Genetic 5
Optimum design of dynamic compaction of soil,
Algorithms
slope
stability
and
decision
making
in
geotechnical engineering.
Table 1: A general classification of fuzzy set technique in use in geotechnical engineering
Fuzzy techniques
Basic
fuzzy
Example of applications
set/fuzzy
Rock
logic and inference
mass
classification
(Nguyen
&
Ashworth,1985), slope stability (Kacewicz,
1987:
Juan
et
al,
1998),
sawability
classification of building stones (Tutmez et al.,
2007) and risk assessment for rock stability
(Wang et al., 2011).
Advanced fuzzy inference Mandani
systems
systems
type A new Mandani-based model to predict burden
from rock geomechanical propertics (Monjezi
& Rezaei, 2011) and Mandani fuzzy inference
model prediction of the blastability designation
of rock (Azimi et al., 2010)
Sugeno
type Rock
engineering
classification
system
(Jalalifar et al., 2011), rock slope stability
13
systems
Systems
assessment (Chen et al., 2011)
using Constitutive modeling of undrained response
Neural Network
of sand mixtures (Calabar et al., 2010) and
prediction of maximum charge per delay in
surface mining (Alipour & Ashtiani,2011).
Systems
Genetic
using Slope stability (Zhang & Lin 2006; Xue et al.
2007)
Algorithm
Hybrid
Soft computing techniques based model of the
formulation
angle
of
shearing
resistance
of
soils
(Dodagoudar
&
(Kayadelen et al., 2009)
Fuzzy probability theory
Slope
reliability
Venkatachalam,2000)
Fuzzy plasticity theory
Cyclic constitutive modeling (klisinski,1988),
rock
fragmentation
(Mishnaevsky
&
Schmauder, 1996) and soil-water hysteresis
model for unsaturated sands (Min & Phan,
2010)
Neuro-fuzzy inference systems have been used in many areas in civil engineering applications. A
stability assessment model for epimetamorphic rock slopes has been developed by using
Adaptive Neuro-Fuzzy Inference System (ANFIS) for its capacity of dynamic nonlinear
analyses. the inference system is employed to predict the stability of the slope by choosing bulk
density γ, the height H, the inclination β, the shear strength parameters c and ϕ, of the slope as
inputs, while the stability state as output.
In order to forecast the factor of safety (FS) or the status of stability (S) in the case of rock or soil
slopes, the factors that influence FS and S have to be determined. The input layer data consists of
six input parameters that perfect stability in the case of failure. The output layer is composed of
a single output parameter, either the factor of safety FS, or the status of stability.
14
This fuzzy methodology enables the engineer to investigate the effect of parameter uncertainty
on the computed stability of a slope in systematic way. The resulting factor of safety fuzzy set
provides more information than does a single, fixed factor of safety value as obtained from
conventional methods. With this approach the variation and possible range of factor of safety
values which reflect uncertainty in the input parameters can be determined.
Shangguan et.al.(2010) has simulated Probabilistic Neural Networks Forecasting the stability of
Slope. Unit weight, Cohesion, Internal friction angle, Slope angle, Slope height and Pore
pressure used as input parameter and factor of safety is simulated as output parameter. Then he
also compare between the estimated and practical states of slope safety with different methods.
ANN Among all data, we 80% used for the training and remaining for validating the prediction
capability The dataset must covers a wide spectrum of soil and seismic parameters. When
preparing input data for a particular site, of primary importance is the recognition of the
conditions which caused the slope to become unstable and the processes which triggered that
movement. From the results presented here, it can be observed that the neural network results are
considerably close to value calculated by Bishop’s classical method. In all cases, it is over 92%
and in most cases, it is over 95%.
15
ANN Several important parameters, including total stress, effective stress, angle of slope,
coefficient of cohesion, internal friction angle, and horizontal coefficient of earthquake, were
used as the input parameters, while the slope stability was the output parameter. The results are
compared with the classical methods of limit equilibrium to check the ANN model’s validity.
The application of fuzzy set theory is used to SMR classification by incorporating fuzzy sets and
assesses the stability of rock slope. The Mamdani fuzzy algorithm may be used to construct the
if–then rules for evaluating rock slope stability.
Thus, the FSMR method provides a better assessment of slope stability than other slope stability
classification systems and can also predict of rock slope failure. It is also noted that engineering
judgment is required in the stability analysis, and fuzzy set theory has also been found to be a
useful tool for rock engineers and engineering geologists who study rock slope stability.
Fuzzy Slope engineering is an important branch in geotechnical engineering. Slope engineering
is a complicated systematic engineering. A lot of Engineering is related to slope stability, such as
16
mining, road and bridge, water conservancy and structure engineering, etc. Its stabilization
directly concerns the safety of engineering. With economical development and large-scale
construction cause, it takes up more and more important place.Based on reviews of evaluation
methods of slope stability, the main research work conducted in this paper is as follows:
(1)Considering the uncertain problems of stability analysis which have the characteristics of
random and fuzziness, the author uses the maximum membership degree principle to analyze and
evaluate the slope stability. Ridge distribution in effect factor of quantity and trapezium
distribution in the effect factor of ration are applied here to construct membership function. The
gradation analysis method is used here to determine the proportion of importance of each effect
factor. The method of two class synthesis assessment is adopted to analyze the stability of slope.
(2)There are eleven effect factors chosen to analyze fuzzily the slope stability. We selected angle
of cut slope, state of underwater, angle between surface of cut slope and major structure plane,
efflorescence, etc. as major factors effect slop stability. The slop stability is assessed by each
factor.
(3)Based on one concrete engineering case, the method of fuzzy analysis is examined, and this
result demonstrates that eleven membership functions, constructed by the author, are reasonable.
So the proportion of importance is reasonable. The membership functions and the distribution of
the proportion of importance can also be applied to analyze the stability of similar slopes.
(4)Put the judgment of fuzzy comprehensive evaluation as the input of neural network by
MATLAB. We transport out the final judgment through the neural network that possess learning
ability.
17
18