Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Zero-configuration networking wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Computer network wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Network tap wikipedia , lookup
Airborne Networking wikipedia , lookup
List of wireless community networks by region wikipedia , lookup
Application of Recurrent Neural Network in Dynamic Modeling of Sensors SHI MENG,TIAN SHEPING, JIANG PINGPING, YAN GUOZHENG Department of Information Measurement Technology and Instruments Shanghai Jiaotong University Room 2201,Building 5,1915 Zhen Guang Rd.Shanghai,200333 CHINA Abstract: - Dynamic modeling of sensors is an important aspect in the field of instrument technique. The recurrent neural network is proposed for nonlinear dynamic modeling of sensors,as its architecture is determined only by the number of nodes in the input, hidden and output layers. With the feedback behavior, the recurrent neural network can catch up with the dynamic response of the system. A recursive prediction error algorithm, which converges fast, is applied to training the recurrent neural network. Experimental results show that the the performance of the recurrent neural network model conforms to the sensor to be modeled, proving the method is not only effective but of high precision. Key-Words: - Recurrent neural network; Sensor; Dynamic modeling; Recursive prediction error algorithm; Dynamic response; Mechanical sensor 1 Introduction It is an important aspect to construct the model of a sensor with the dynamic calibrating data in the field of dynamic measurement[1]. Several methods have been applied to dynamic modeling of sensors. A common approach is to find the difference equation of a sensor using the discrete domain calibrating data. The transfer function of the sensor can be gotten from corresponding difference equation through bilinear transform. Above method is suitable for linear dynamic modeling of sensors and is useless when the sensors display nonlinear characteristics. Several artificial neural network paradigms and neural learning schemes have been used in many dynamic system identification problems, and many promising results are reported. Most people made use of the feedforward neural network, combined with tapped delays, and the back propagation training algorithm to solve the dynamical problems[2]; however, the feedforward network is a static mapping and without the aid of tapped delays it does not represent a dynamic system mapping. On the other hand, the recurrent neural networks have important capabilities that are not found in feedforward networks, such as the ability to store information for later use and higher predicting precision. Thus the recurrent neural network is a dynamic mapping and is better suited for dynamical systems than the feedforward network. This paper discusses the application of the recurrent neural networks in nonlinear dynamic modeling of sensors. With the feedback behavior, the recurrent neural network can capture the dynamic response of the system. A recursive prediction error algorithm, which converges fast, is applied to training the recurrent neural network. Experimental results show that the dynamic modeling method is effective. 2 Dynamic modeling of sensors based on recurrent neural network model 2.1 Recurrent neural network model The fully connected recurrent neural network, however, where all neurons are coupled to one another, is difficult to train and to make it converge in a short time. With the requirement of fewer weights and a shorter training time for the neural network model, a simplified recurrent neural network is proposed. The architecture of the simplified version is a modified model of the fully connected recurrent neural network. It normally has an input layer, an output layer, and one hidden layer. The hidden layer is comprised of self-recurrent neurons, each feeding its output only into itself and not to other neurons in the hidden layer. On the other hand, the neurons in the output layer are linear type neurons, and do not have feedback weights. It was shown that with a proper choice of input and output weights, the simplified recurrent neural network paradigm can be made equivalent to a fully connected recurrent neural network. As shown in Fig.1, the recurrent neural network’s characteristics are determined by the connected weights of neighboring layers. W ijI W jD W jO I ij and nodes in hidden layer respectively. For a recurrent network with single output, it has (p+2) q weights. g(.) is the activation function of neurons which is usually chosen as the sigmoid function: g ( z) I i (t )W W O j (t ) (3) If the input of the recurrent network is denoted as {u (t-1), y’ (t-1)}, output is y’ (t), the input –output mapping of the recurrent neural network can be represented by: y' (t ) f [u (l ), y' (l ), l t 1] (4) Where f (.) is a nonlinear function. So f (.) is a nonlinear dynamic mapping. Fig. 2 shows the framework of dynamic modeling of sensors based on recurrent neural network model, in which y’ (t) is the output of the recurrent neural network model. The determination of weights can be achieved by applying certain training algorithm. u(t) O j 1 1 ez y(t) sensor + O (t ) - … … I i (t ) y’(t) -1 z Fig. 1 A recurrent neural network model Each neuron in the hidden layer is a recurrent one Fig. 2 Dynamic modeling of sensors based on recurrent neural network model with a nonlinear activation function. I i (t ) is the ith input; Sj (t), Xj (t) are the sum of input and output of the jth neuron respectively; O (t) represents the output of the whole network. The weighting vectors of the input layer, hidden layer and output layer compose the vector W={wI, wD, wO}. So the relation between the input and output can be described as: q O(t ) W jO X j (t ) X j (t ) g[ S j (t )] j 1 (1) p S j (t ) W jD X j (t 1) WijI I i (t ) (2) i 1 Where p and q refer to the numbers of network inputs 2.2 Training algorithm of the recurrent neural network Several methods such as back propagation algorithm can be used to estimate the weights of a recurrent network model. In this paper we utilize the recursive prediction error algorithm (RPE) to train the recurrent network. Prediction error method achieves the parameters’ estimates by minimizing the prediction error criterion. First, let’s define the prediction error as: t,W yt y' t,W (5) After N data have been recorded, a criterion function can be expressed by the following sum of squared prediction errors: J (W ) N 1 2N T (t , w)(t , w) the Gauss-Newton search direction of J (W ) to make J→min . The basic equation is: W t W t 1 st W t 1 (7) Where s (t) is the step size, µ (w) represents the Gauss-Newton search direction. Where J W 1 N t , W t , W (9) W N t 1 dy ' t , W t , W dW pt N t ,W t ,W T (11) t 1 (12a) 1 1 pt 1 pt 1t t I T t pt 1t T t pt 1 t W t W t 1 pt t t y' (t ) W jO Pj (t ) W jD (14b) y ' (t ) W jO Qij (t ) I Wij (14c) Pj (t ) X j (t ) W j D , Q ij (t ) X j (t ) Wij I and meet the following conditions: D Pj (t ) X j (t 1) W j Pj (t 1), Pj (0) 0 D Qij (t ) I i (t 1) W j Qij (t 1), Qij (0) 0 (15a) (15b) (10) The RPE algorithm based on above rules is described by following equations[3,4]: t yt y' t (14a) T H (w) can be calculated from the expression as follows: 1 H W N y' (t ) X j (t ) W jO In which Pj (t), Qij (t) are defined as: (8) Here the gradient of J(w) towards w is denoted as ▽ J(w), H(w) is the second order derivative of J(w), namely the Hessian matrix of J(w). It can be easily derived out that: J W (13) ψ(t) can be obtained by equation : The unknown weighing vectors are updated along 1 t 0 t 1 1 0 (6) i 1 W H W J W the above requirements: (12b) (12c) p (t) is called the middle matrix, representing the covariance matrix of parameters when t→∞, whose initial value p (0) is usually chosen from the range of 104I to 105I, where I is the identity matrix.λ(t)is called the forgetting factor. It’s desirable to setλ(t)<1 at the initial stage so that rapid adaptation takes place and then to letλ(t)→1 as t→∞. Following equation can meet 3 Experimental results From above analysis, the process of applying recursive network to dynamic modeling of a sensor can be summarized as follows: 1. According to the performance of the sensor, determine the node number of the hidden layer and relevant initial values of them. 2. Acquire experimental data, apply RPE algorithm to training recurrent network until its convergence. For example: to model the system whose input-output function is: y(t ) 0.8 1 exp 0.5 y(t 1) 0.6u (t 1) 0.9 Apparently, above system is highly nonlinear. In the simulation, normally distributed random noise with mean value of 0 and standard variance of 0.1 is added to the experimental data. The input data for training the recurrent neural network model are pseudo random binary sequences whose amplitude is ±1 and length is 64. The number of nodes in hidden layer is taken as q=5. The weighting vectors converge in about 800 iterations. Fig. 3 shows the simulation testing results when the trained model is imposed by the input signal which is taken as u (t ) sin( t / 50) e(t ) where e(t) is uncorrelated random noise with mean value of 0 and standard variance of 0.05. In Fig.3, curve one represents the output of the system and curve two represents the output of the recurrent neural network model. The results indicate that the two outputs coincide very well. Fig. 3 The simulation test results To verify the validity of above method, it is carried out to construct the model of a mechanical sensor. Fig.4 gives the modeling results of a mechanical sensor. The training data for the recurrent neural network are taken as the data of the impulse response of the sensor. The number of nodes in hidden layer is taken as q=8. In Fig.4, curve one represents the step response of the mechanical sensor while curve two represents the output of the recurrent neural network model. Fig.4 Dynamic modeling of mechanical sensor 4 Conclusion According to above analysis, the recurrent neural network possesses the ability of nonlinear dynamic mapping. Its architecture is determined only by the number of nodes in the input, hidden and output layers, so applying the recurrent neural network to dynamic modeling of sensors is a valid approach and especially significant for modeling highly non-linear sensors. From the experimental results, we learn that the performance of the recurrent neural network model conforms to the sensor to be modeled, proving the method is not only effective but of high precision. But to ensure the correctness of the dynamic modeling, the training data chosen must be typical. Increasing the number of nodes in the hidden layer is also helpful to improve the precision of dynamic modeling, but will be at the cost of training time. References: [1] Xu kejun, Chen rongbao, Zhang chongwei, Common Techniques in Automatic Measurement and Instrument, Tsinghua University Press, 2000 [2] K. S. Narendra and K. Parthasarathy, Identificationand Control of Dynamical Systems using Neural Networks, IEEE Trans. on Neural Networks, Vol.1,No.1,1990,pp.4~27 [3] Ku C C and Lee K Y, Diagonal Recurrent Neural Network for Dynamic Systems Control, IEEE Trans on NN,Vol.6,No.1,1995,pp.144-155 [4] L.ljiung and T.Soderstrom, Theory and Practice of Recursive Identification ,The MIT Press, 1983 [5] Draye,J.,Pavisc,D.,Libert,G., Dynamic Recurrent Neural Networks: a Dynamical Analysis,IEEE Trans. On Systems Man and Cybernetics, Vol.26, No.5, 1996, pp.692-706 [6] Parlos,A.,Chong,K.,Atiya,A., Application of the Recurrent Multilayer Perceptron in Modelling Complex Process Dynamics,IEEE Trans.On Neural Networks,VOL.5, No.2 ,1994, pp.255-2 [7] Kosmatopoulos,E., Polycarpou,M., Iannou,A., High-order neural network structures for identification of dynamical systems, IEEE Trans. On Neural Networks,vol6,no.2,1995,pp.422-431