Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Learning Markov-chain embedded recurrence relations for chaotic time series analysis Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Outline Introduction High-order Markov processes for stochastic modeling Nonlinear recurrence relations for deterministic modeling Recurrence relation approximation by supervised learning of radial or projective basis functions Markov-chain embedded recurrence relations Numerical Simulations Conclusions High-order Markov assumption Let Z[t] denote time series, where t is positive integers High-order Markov assumption Given chaotic time series are oriented from a generative source well characterized by a high-order Markov process. An order- Markov process obeys memory-less property Pr( Z[t ] | Z[t 1],..., Z[t ],..., Z[1]) Pr( Z[t ] | Z[t 1],..., Z[t ]) Current event only depends on instances of most recently events instead of all historic events Recurrence relation Conditional expectation of an upcoming event to most recently events is expressed by a recurrence relation y[t ] Z[t] | z[t - 1], , z[t - ] G ( z[t 1],..., z[t ]) equivalently y[t ] G (x[t ]), where x[t ] ( z[t 1],..., z[t ])T denotes previous events y[t ] z[t ] denotes current event Recurrence relation for time series modeling 5 z[t ] G( z[t 1],..., z[t ]) sin( a1 z[t 1] a2 z[t 2] a5 z[t ]), t , ,1000 z[t ] y[t ] x[t ] z[t 1], z[t 2], , z[t ] T predictor target y[t ] G (x[t ]) sin( a T x[t ]) a (a1 , a2 ,, a ) T Mackey-Glass 30 chaotic time series data 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 500 1000 Chaotic time series Laser data 10000 from the SFI competition 1500 RECURRENCE RELATION APPROXIMATION • Learning neural networks for approximating underlying recurrence relation y[t ] G(x[t ]) F (x[t ] ) • F denotes a mapping realized by radial or projective basis functions • denotes adaptive network parameters Recurrence relation approximation Form paired predictor and target (x[t ], y[t ]) by assigning x[t ] ( z[t 1],..., z[t ])T and y[t ] z[t ] for all t Define the mean square error of approximating y[t ] by F (x[t ] ) 1 E ( ) N 2 ( y [ t ] F ( x [ t ] | ) t Apply Levenberg-Marquardt learning to resolve unconstrained optimization opt min E ( ) Apply the proposed pair-data generative model to formulate F Pair-data generative model (PGM) K sub-models Mixtures of paired Gaussians A stochastic model for formation emulation of given paired data Each time one of joined pairs is selected according to a set of prior probabilities Apply the selected paired Gaussians to generate paired data (x[t ], y[t ]) Exclusive Memberships Each pair is exactly generated by a sub-model Let δ[t ] e1 , e2 ,, eM denote the exclusive membership of x[t ], y[t ] where e i denotes a unitary vector with the ith bit active By exclusive membership ei 0,,1,,0 δ[t ] e k if x[t ], y[t ] is generated by the kth sub - model The conditional expectation of y to given x is defined by y x δT r r denotes local means of the target variable Overlapping memberships A Potts random variable δ is applied to encode overlapping membership The probability of being the kth state is set to Pr δ e k exp( x[t ] μ k ) 2 where modulates the overlapping degree and μ k denotes local mean of the predictor Normalized radial basis functions ( NRBF ) Since Pr δ e k exp( x[t ] μ k ) and 2 it follows vk Pr δ e k exp x[t ] μ k M 2 exp x[t ] μ h h 1 M Prδ e 1 k 1 k 2 Let v v1 , , vM T yx v r exp x[t ] μ rk exp x[t ] μ k T 2 2 k h h The conditional expectation exactly sketches a mapping realized by normalized radial basis functions Figure 4 Mackey-Glass 17 chaotic time series data source 1.5 1 0.5 0 0 500 1000 1500 approximation 1.5 1 0.5 0 0 50 100 150 -3 4 200 250 300 350 400 450 500 350 400 450 500 approximating error x 10 2 0 -2 -4 0 50 100 150 200 Figure 9 250 300 Multiple recurrence relations Multiple recurrence relations for modeling more complex chaotic time series Chaotic time series Laser data 10000 from the SFI competition Markov-chain embedded recurrence relations A Markov chain of PGMs (pair-data generative models) Transition matrix Ti , j Ti , j denotes the probability of transition from model i to model j Data generation Emulate data generation by a stochastic Markov chain of PGMs Inverse problem of Markov chain embedded PGMs Given an ordered sequence generated by Markov - chain embedded PGMs The inverse problem for model reconstruc tion can be stated as 1. Find switching points for segmentati on. 2. Estimate transitio n probabilit ies. Segmentation for phase change A time tag is regarded as a switching point if its moving average error greater than a threshold value error[t ] t 2 ( y [ t ] F ( x [ t ]; θ )) i t A simple rule for merging two PGMs The goodness of fitting the ith PGM to paired data in Sj is defined by Ei, j 1 Si 2 ( y [ t ] F ( x [ t ]; θ )) j x[ t ]S j Two PGMs are merged. Si and Sj are regarded from the same PGM if (Ei,j+Ej,i)/2 is less than a threshold value 1 If Ei , j E j ,i 2 Si and S j are regarded oriented form the same hidden state. NUMERICAL SIMULATIONS – Synthetic data K : the number of required hidden states after reduction : short time scale. : a pre - determined positive threshold N 0 : the window size of auto - regressive sampling for fixed - length segmentati on. post - nonlinear functions : sin a1T x , cosaT2 x , exp aT3 x ( K , , , N 0 ) [ 3, 5, 0.1, 200 ] Translatio n matrix Temporal sequence generated by MC-embedded PGMs Numerical results – original and reconstructed MCembedded PGMs Translatio n matrix Learning M=60,[K, , , N0 ] = [ 10, 10, 0.001, 500 ] Chaotic time series Laser data 10000 from the SFI competition Markov chain embedded recurrence relations Generated chaotic time series Conclusions This work has presented learning Markov-chain embedded recurrence relations for complex time series analysis. Levenberg-Marquardt supervised learning of neural networks has been shown potential for extracting essential recurrence relation underlying given time series Markov-chain embedded recurrence relations are shown applicable for characterizing complex chaotic time series The proposed systematic approach integrates pattern segmentation, hidden state absorption and transition probability estimation based on supervised learning of neural networks