* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Microsoft Word - UWE Research Repository
Survey
Document related concepts
Transcript
Development of a new EDRNN procedure in Control of Human Arm Trajectories Shan Liu*,Yongji Wang* and Quanmin Zhu** * Key Lab of Image Processing and Intelligent Control, Department of Control Science and Engineering, Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China [email protected] **Faculty of Computing, Engineering and Mathematical Sciences (CEMS), University of the West of England (UWE), Frenchay Campus, Coldharbour Lane, Bristol, BS16 1QY, UK Abstract: In this paper the trajectory tracking control of a human arm moving on the sagittal plane is investigated by an interdisciplinary approach with the combination of neural network mapping, evolutionary computation, and dynamic system control. The arm in the study is described by a musculoskeletal model with two degrees of freedom and six muscles, and the control signal is applied directly in the muscle space. A new control system structure is proposed to manipulate the complicated nonlinear dynamical arm motion. To design the intelligent controller, an evolutionary diagonal recurrent neural network (EDRNN) is integrated with proper performance indices, in which genetic algorithm (GA) and evolutionary program (EP) strategy are effectively integrated with the diagonal recurrent neural network (DRNN). The hybrid GA with EP strategy is applied to optimize the DRNN architecture and an adaptive dynamic back-propagation algorithm (ADBP) with momentum for the multi-input multi-output (MIMO) systems is used to obtain the network weights. The effectiveness of the control scheme is demonstrated through a simulated case study. 1 Introduction For whatever reasons, human beings have skillfully performed various movements to achieve their targeted objects. Understanding the process and mechanisms of performing such motion activities can provide a significant insight into the neural mechanisms of motor learning and adaptation, development of interactions between robots and human operators, and enormous applications in computer animation of the human motion and neuromotor rehabilitation. It may be advantageous to use humanlike control strategies for improving the performance of robot operation, in particular with regard to safety. Although the use of control theories to provide an insight into neurophysiology has been a long history, the complexity associated with the human movement arising from the highly redundant and nonlinear dynamics has not been perfectly resolved, nevertheless addressed. There have been widespread research activities on modeling and controlling of the human arm movements in the last five decades. Hierarchical model [15], neural network [12] [13] [20], optimal control [1] [10] , adaptive control [6] [11] , iterative learning [14] , and combination of these methods [17] have been applied in various motion controls of the human arm. So far most of the studies have focused on the arm movement control on the horizontal plane, with gravity compensation. However, human arm works in the gravity field. Therefore, the human arm movement on the sagittal plane, that is, in the vertical plane, is more natural and realistic. Accordingly proper research efforts should be contributed in this field to provide a new angle to understand the operational mechanism of the normal human arm movement. Some experimental findings do suggest that the gravity force, at least for the vertical arm movements in the gravity field, have an important role on the arm movement planning process [17] [18] . The gravitational torque can either initiate or brake arm movements and consequently should be represented in the motor command. And, the gravitational force can be considered as an actuator which can be used to drive the movements. The gravitational force acting to the arm even is dependent of the arm posture [19] . Thus researches on the arm movements on the sagittal plane should establish some relationships between the gravity, joint torques and movements. It has been witnessed that neural networks have been increasingly adopted in the biological motor control systems, due to their model-free approximation capability to the complex decision making processes. Compared with the static neural network, the diagonal recurrent neural network (DRNN) is more suitable for the dynamic representations to efficiently capture the dynamic behavior of a complex system. Despite the success of neural networks in solving complex problems, their design procedure is based on still trial-and-error skills. A major problem in designing a neural network is the proper choice of the network architecture, especially used as a controller for multi-input multi-output (MIMO) nonlinear systems. The architecture of a neural network includes its topological structure and the transfer function of each node in the network, which has significant role in the way to process information. Recently, there have been many attempts in this field, in which the combination of evolutionary algorithm and neural networks has attracted a great deal of attention. Design of the optimal architecture for a neural network can be formulated as a search problem in the architecture space where each point represents the architecture. Previous studies have indicated that the evolutionary approach is a better candidate for searching the space than the other heuristic algorithms [3] [4]. In the main, the contributions of the study are intended 1) To understand the movement mechanism of the human arm on the sagittal plan; 2) To propose a new control strategy to manipulate the complicated nonlinear dynamical arm motions; 3) To implement the controller by integrating the DRNN with genetic algorithm (GA) and evolutionary program (EP) strategy; 4) To implement the adaptive dynamic back-propagation (ADBP) algorithm with momentum to obtain the DRNN weight matrices in the control of MIMO systems. 5) To undertake simulation studies to demonstrate the new design procedure and compare it with other representative approaches to show the enhancement. This paper is organized as follows. In section 2, a realistic musculoskeletal model of the human arm moving on the sagittal plane and the control system structure are presented to set a foundation for the control system design. In section 3, the DRNN based controller is constructed. Subsequently the hybrid GA and EP approach for optimizing the DRNN architecture and the ADBP algorithm with momentum for determining the DRNN weights are presented. In section 4, the feasibility of the proposed neural control system is demonstrated by numerical simulation studies. Finally in section 5, the conclusions and the future works are provided. 2 Problem Statement 2.1 Human Arm Model The plant used in this study is a simulated, two degree-of-freedom, planar arm model moving in the sagittal plane. A schematic picture of the arm model is shown in Fig. 1. In this model there are two joints (i.e. shoulder and elbow) driven by three pairs of opposing muscles; two pairs of muscles individually actuate the shoulder and the elbow joints, while the third pair actuates both joints simultaneously. The movement range of the shoulder is from -80° to 90° and that of the elbow is from 0° to 150°. For simplify, it is assumed that the reference trajectory exists in the joint space, the mass effect of each muscle is not considered in the model because the mass transfer of each muscle coming from the movements of muscles is not so large, and the external force only involves the gravity. The inverse dynamics of the arm model is M ( ) C ( , ) B G ( ) m (1) Fig.1 Schematic picture of the arm model In Eq.(1) , R 2 is the joint angle vector ( [1 , 2 ]T , 1 : shoulder, 2 : elbow), M ( ) R 22 is the positive definite symmetric joint inertia matrix, C ( , ) R22 is the vector of centrifugal and Coriolis torques, B R22 is the joint friction matrix, G( ) R 2 is the vector of gravity torques, and m R 2 is the joint torque produced by six muscles. The applied muscle model is a simplified version of the model proposed by [2]. It represents the combined activation a and length l and velocity v dependent contractile forces. The tension produced by a muscle is represented as T (a, l , v) a( FL (l ) FV (l , v) FP (l )) (2) where 2 FL (l ) exp( l 2 1 ) 6 v v0 6 2lv , FV (l , v) 2 1 (3 5l 3l )v , v 0 1 v FP (l ) 0.02 exp(13.8 18.7l ) and the muscle force is (3) F (a, l , v) F0maxT (a, l , v) where F0max is the maximum isometric force. The moment arm is defined as the perpendicular distance from the muscle's line of action to the joint's center of rotation, which is equal to the transpose of the Jacobian matrix from the joint space to the muscle space. Here, the relation between muscle forces and joint torques can be expressed by using the principle of virtual work: m Mo( ) F (a, l , v) 0 (4) where Mo( ) R 26 is the moment arm. 2.2 Control System Structure The control system is illustrated in Fig. 2, where the system is composed of single DRNN controller (In Fig. 2, k is the discrete time.) and the plant. The DRNN controller plays an important role in the trajectory tracking control. The architecture of the DRNN is optimized by GA and EP; and the weights are trained by the ADBP. The inputs of the DRNN controller are the desired position ( r (k ) ) in the reference input ( yr (k ) [ r (k ), r (k )] ), the previous tracking error between the desired and the actual response of the plant ( e _ y(k 1) yr (k 1) y(k 1) , y(k ) [ (k ), (k )] ), and the previous control signal to the plant, i.e. the previous muscular activation ( u (k 1) a (k 1) ), and the integral terms of the position tracking errors. The reference input is set in advance, so no delay exists in the controller for the reference input. The DRNN makes advantage of the input and output information of the plant to generate the proper control signal. Details of the controller design are described in Section 3. Fig. 2 Block diagram of the human arm control system 3 Evolutionary Diagonal Recurrent Neural Network Controller The DRNN is a simple recurrent neural network and has a similar dynamic mapping ability to the fully connected recurrent neural network. A typical DRNN structure comprises three layers: an input layer, a hidden layer, and an output layer [5] . The hidden neurons are dynamic neurons in that the output of each hidden neuron is fed back to its input through a delay unit. These local feedback paths introduce a dynamic behavior into the network, so the DRNN has a more powerful capability to express the dynamical systems than the feed-forward neural networks. 3.1 Structure of the DRNN In the DRNN controller, the numbers of the neurons in the input layer and the output layer are fourteen and six, respectively. The input I (k ) includes the input and the output information of the plant ( I (k ) [e _ y(k 1) r (k ) a(k 1) k 1 0 e( )d ] , e(k ) r (k ) (k ) ). The last one in I (k ) is the integral of position errors to eliminate the steady-state position error [9]. The number of the hidden neurons is determined by the hybrid GA and EP approach. The mathematical model for the DRNN is shown below: al (k ) W jlO X j (k ), X j (k ) f ( S j (k )) j S j (k ) WijI I i (k ) W jD X j (k 1) (5) i where for each discrete time k, I i (k ) is the i th input to the DRNN, S j ( k ) is the sum of inputs to the j th recurrent neuron, i.e. the j th hidden neuron, X j (k ) is the output of the j th recurrent neuron, and al (k ) is the l th output of the DRNN. Here f () is the nonlinear transfer function, and W I , W D and W O are input, recurrent and output weight matrix, respectively in R n14 , Rn , and R 6n . n is the number of hidden neurons. 3.2 Optimization of the DRNN Structure For practical purpose, the simultaneous evolution of the neural network connection weights and the architecture is beneficial. However, in this paper, the DRNN controller is a fourteen inputs six outputs neural network, the amount of the connection weights is n (14 1 6) . The evolution of the architecture with connection weights will be time consuming. So, we just optimize the architecture of the network; the connection weights are trained through the ADBP algorithm. Several researches on evolving the neural network architectures for control problems have been carried out in recent years [8] [25] [26]. In [26], genetic algorithm with internal copy operator is used to organize the structure in a network, but the effectiveness depends on the gene coding and how the gene blocks are chosen and copied, which is still an open problem. Pasemann et al. [25] adopted an ENS3-algorithm (evolution of neural systems by stochastic synthesis) to generate recurrent neural networks with non-trivial internal dynamics, which is used for development of the network structure, optimizing parameters as well. In terms of the required computation time, the ENS3 can not compete with those learning algorithms like the hybrid GA with EP strategy from [8]. In this paper, the optimum DRNN architecture is determined by implementing both GA and EP methods, which evolves a pool of DRNN architectures until the control objective is achieved, because GA can explore a wider area of search in a shorter number of evolutions and EP can maintain the behavioral link of the selected offspring after the competition stage [8] . The goal of the controller is to minimize the cost function J ( Eq.(7) , i.e. the sum-squared of the tracking error (SSE)), with T the number of simulation epochs. Fig. 3 shows the process of the evolution procedure for the optimization of the DRNN architecture. At each generation, all networks in the population are trained to track the desired trajectories. The population is divided into three different groups, namely the best, strong and weak group according to their cost function. T 2 J 12 [ jr (k ) j (k )]2 (6) k 1 j 1 Binary string representation is used to represent the DRNN architecture. In the DRNN, the sigmoid transfer functions are usually used in the hidden nodes [22] . The transfer function has been shown to be an important part of the neural network architecture and have a significant impact on the network's performance. The transfer function is often assumed to be same for all the nodes in the same layer. White et al. [27] introduced a simple approach to evolve both architectures and connection weights. The evolution was used to decide the optimal mixture between two transfer functions (sigmoid transfer function and Gaussian transfer function). In [28], EP is used to evolve general neural networks with both sigmoidal and Gaussian nodes by adding or deleting a node (either sigmoidal or Gaussian); and good performances were reported for some prediction problems. However, it is still unclear about the performance of the network with different transfer functions for control problems, and the training process of such a network controller becomes complicated. We assume transfer functions are same for all nodes in the same layer. The parameters evolved are the number of the neurons and the type of the transfer functions in the hidden layer. Fig. 4 shows an example of the chromosome representation in a binary string, which means that there are 28 neurons and a log sigmoid transfer function in the hidden layer. The explanation of the major steps in Fig. 3 is as follows: 1. Generate m number of networks in one population as depicted in Fig. 3. Initialize weight matrix in each network with uniformly distributed random numbers. 2. Train each network with the ADBP algorithm and calculate the cost functions of the networks (i.e. each network in the population is used to control the human arm model, and then the control performance is monitored.). Fig. 3 Flow chart of the evolution procedure Fig. 4 Binary string representation of DRNN architecture 3. Divide these networks into three groups through the rank based selection method. If one network in the best group can make the behavior converge satisfactorily to the desired target, the procedure ends. Otherwise, continues. 4. Apply GA onto the individuals in the weak group to explore a wider area of search in a shorter number of evolutions. The multipoint crossover and simple flip over mutation are used in the computations. While these offspring networks are used to control the human arm, their cost functions are compared with those of their parents. If the offspring is better, it replaces its parent. Otherwise, the parent stays alive. 5. Apply EP onto each individual in the strong group to maintain the behavioral link of the selected offspring. Three hidden neurons are deleted from each network, then the new network is used for controlling the plant and calculating the offspring's cost function. If the offspring is better than its parent, it replaces the parent. Otherwise, three hidden neurons are added, the new network is used to control the plant again. If the offspring is better than the parent, it replaces its parent. Otherwise, the parent stays alive. 6. Combine these networks obtained from steps 5 and 6 with networks in the best group to get a new generation. 7. Repeat steps 3 to 6. 3.3 DRNN Weight Training An ADBP algorithm with momentum for the MIMO systems is used for training the DRNN weights. Back-propagation often gets trapped in a local minimum of the error function and is incapable of finding a global minimum, and the evolutionary approach can search all of solution space without the local minimum problem, but nevertheless the evolutionary approach consumes an amount of time in network learning. The ADBP with momentum for the MIMO systems can catch the dynamic information contained in the plant, and the convergence of the DRNN based control system is guaranteed through Lyapunov stability theorem [23]. The error function for a training cycle is defined as 2 E 12 [ jr (k ) j (k )]2 (7) j 1 The gradient of error in (7) with respect to an arbitrary weight matrix W R mr is represented by E (k ) a (k ) e( k ) e(k ) a (k ) (8) W W W where e(k ) r (k ) (k ) is the error between the desired and the output responses of the plant, and the factor a (k ) (k ) / a(k ) represents the sensitivity of the plant with respect to its input signal. From the mathematical model of the human arm (1) - (4), a (k ) m((kk)) am((kk)) M ( ) 1 h 2 F0max Mo( )( FL (l ) FV (l , v) FP (l )) (9) where h is the sampling time. Upon the DRNN model described in Section 3.1, the output gradients with respect to the output, the recurrent, and the input weights, respectively, are given by al (k ) X j (k ) W jlO al (k ) W jlO Pj (k ) D W j (10) al (k ) W jlO Qij (k ) I Wij where Pj (k ) X j (k ) / WjD , Qij (k ) X j (k ) / WijI and satisfy Pj (k ) f '( S j (k ))( X j (k 1) W jD Pj (k 1)), Pj (0) 0 (11) Qij (k ) f '( S j (k ))( I i (k ) W jDQij (k 1)), Qij (0) 0 We make use of the following gradient steepest descent method [24] to adapt the weights of the DRNN: E ) (W (k ) W (k 1)) (12) W I I D O where is the learning rate matrix ( D , , , represents the learning O W (k 1) W (k ) ( rate matrix corresponding to W I , W D , W O ), the maximum learning rate m in is chosen as 0 m 2 ei (k ) e j (k ) e (k ) e (k ) e (k ) e (k ) , ) ( i D , j D ) ( i O , j O ) I I W W W W W i , j 1 2 2 (13) ( W where (a, b) is the inner product between vector a and vector b , ei (k ) and e j (k ) is the i th and j th tracking error of the controller respectively. is a momentum factor affecting the change W (k ) W (k 1) of the DRNN’s weights at the k th iteration. [23] shows that (13) can be used as a guideline to find the optimal learning rate, and the convergence can be guaranteed. 4 Simulation Results In the simulation study, the EDRNN was programmed using MATLAB, the plant parameters were obtained from [14], and the SSE function was monitored to demonstrate the performance of the proposed control scheme. The reference trajectory equation was assigned as 1r 60sin t 60 60 cos t 2r (13) 1r 60 cos t 60sin t 2r In the simulation, some limitations have been imposed on the DRNN. Though there is no general solution for determining the number of hidden neurons in a neural network [20], it was assumed that the range of the hidden neuron number was from one times to four times than the number of the inputs. Therefore each network had a different hidden neuron number from 14 to 56. For investigating the effectiveness of other differentiable functions in the DRNN, tan-sigmoid, log-sigmoid, Gaussian, and the saturating linear were chosen as the candidates of the transfer functions. 4.1 Evolution Process In the evolution process, the number of the networks in one population m was set as m = 14, which is equal to the number of the network inputs; the crossover probability Pc and the mutation probability Pm were set as Pc = 0.9 and Pm = 0.03 respectively; these two parameters were determined from a trial and error procedure. The results of the evolution over thirty generations are shown in Fig. 5 to show the best cost function value of each population decreasing sharply in the first six generations, and converged after 20 generations. Table 1 indicates that the number of the hidden neurons is converged to from twenty eight to thirty. Judging from the cost function and the running time shown in Table 1, twenty eight is the optimum number of the hidden neurons. The transfer function converges as a log sigmoid. For comparison, the GANNet algorithm presented in [27] was implemented to evolve the architecture of the DRNN. Because the GA is sensitive to the length of the chromosome string, the shorter this string can be made, the better, the GANNet was modified to optimize the architecture of the DRNN; then the DBP was used to obtain the weights. Comparisons of the results through the modified GANNet algorithm and the GA and EP approach are shown in Table 2, which indicates the hybrid GA and EP approach is more effective. Fig. 5 Result of evolution Table 1. Comparison of DRNN with different number of neuron No. of neuron 26 27 28 29 30 Transfer Function tans logs logs logs logs Running time(sec) 17.92 18.13 18.86 23.64 30.02 Cost Function / T 0.04 0.034 0.02 0.02 0.02 Table 2. Comparison of the evolution results through the modified GANNet and GA+EP Type of algorithm Modified GANNet GA+EP No. of hidden neurons 30 28 Running time(sec) 621.42 380.20 Cost function / T 0.024 0.020 Type of transfer function Gauss logs Logs Probability of transfer function 0.05 0.95 / No. of generation 60 20 4.2 Control Result In running the simulation studies, each DRNN controller was trained for 700 epochs. The adaptive learning rates O , D , and I were used starting from the initial rate of 0.2, 0.2 and 0.2 respectively. After 100 training epochs, the learning rates were adjusted to 0.03, 0.05 and 0.005 respectively. The momentum factor was assigned as 0.25. In this study, randomly generated weight matrices were used as the initial weights. Therefore, the whole program had been run for several times until satisfactory results were achieved. The best simulation results are shown in Fig. 6 and Fig. 7. For comparison, the iterative LQG (ILQG) presented in [16] was also applied to control the human arm on the sagittal plane. It has been noticed that the ILQG was used to control the human arm movement on the horizontal plane, and good simulation results were reported. Therefore ILQG was selected as a critical bench test to compare the new procedure. Inspection of Fig. 6 and Fig. 7, the EDRNN controller is capable of driving the human arm to follow the reference joint trajectories with small tracking errors, and the performance of the EDRNN control is better than that generated from the ILQG. For investigating the generalization of the DRNN, the obtained DRNN controller from the trajectory tracking control simulation was further used for the posture control in the presence of external forces. In the same way, the ILQG was used as a comparable method to compare with the EDRNN controller. The external force parallels the ordinate, and is set as 9.8N . The simulation results are shown in Fig. 8 and Fig. 9. These figures indicate that the obtained DRNN controller can maintain the arm at different location within the allowable bound. Moreover, both methods can implement the posture control in the presence of the external force. Fig. 6 Trajectory of the joint angles Fig. 7 Graph of the joint angle velocities EDRNN control ILQG control reference theta1(deg) 20 0 -20 0 100 200 300 400 Epochs 500 600 700 theta2(deg) 50 EDRNN control ILQG control reference 40 30 20 10 0 100 200 300 400 Epochs 500 600 700 Fig. 8 Angle response curve under the action of the external force dtheta1(deg/s) 20 0 -20 -40 -60 -80 0 dtheta2(deg/s) EDRNN control ILQG control reference 100 200 300 400 Epochs 500 600 700 EDRNN control ILQG control reference 100 0 -100 0 100 200 300 400 Epochs 500 600 700 Fig.9 Angle velocity curve under the action of the external force Subsequently some pulse signals as disturbance were added into the inputs of the arm. Each input had the same disturbance signal. The pulse amplitude was set as 1, the period was set as 350 epochs, and the width as 15% of the period. Fig. 10 and Fig. 11 show the best simulation results. Compared with Fig. 6 and Fig. 7, it is observed that disturbance affects the performance of these controllers. But, the arm still can follow the desired joint inputs with the tracking error in tolerance. Further, the performance and robustness of the EDRNN control is better than those delivered from the ILQG. Then, the obtained DRNN from the above simulation testing is used for the posture control with disturbance. The simulation results are shown in Fig. 12 and Fig. 13. These figures illustrate the performance and robustness of the two methods for the posture control in the presence of disturbance and external forces is all satisfactory. Fig. 10 Trajectory of the joint angles with disturbance Fig. 11 Graph of the joint angle velocities with disturbance 60 EDRNN control ILQG control reference theta1(deg) 40 20 0 -20 0 100 200 300 400 Epochs 500 700 EDRNN control ILQG control reference 60 theta2(deg) 600 40 20 0 100 200 300 400 Epochs 500 600 700 dtheta1(deg/s) Fig. 12 Angle response curve with disturbance and external force 0 -100 0 dtheta2(deg/s) EDRNN control ILQG control reference -50 100 200 300 400 Epochs 500 600 700 EDRNN control ILQG control reference 100 0 -100 0 100 200 300 400 Epochs 500 600 700 Fig. 13 Angle velocity curve with disturbance and external force For better comparing the EDRNN and ILQG method, some more simulation results are included. The running time of the two methods in different simulation testing are shown in Table 3. In the trajectory tracking control simulation, the running time of the DRNN is counted during the course of optimizing the architecture of DRNN and training the DRNN. In the others simulations, the running time of the DRNN is the training time of the obtained DRNN from the tracking control simulation. Table 3 indicates that the computation burden of tuning the two controllers in the tracking simulation is almost equal, the ILQG is somewhat better; however, the obtained DRNN controller can be used for the different reference signals, and moreover, the ILQG need to repeat all the optimizing procedure for different simulation testing. Table 3 Comparison of computation burden of EDRNN and ILQG Type of controller EDRNN ILQG Trajectory tracking control 380.2 369.4 Trajectory tracking control with disturbance 25.14 409.3 Posture control with external force 16.02 261.2 Posture control with disturbance and external force 19.23 304.8 Running time (sec) Type of simulation 5 Conclusions In the first stage of concept development and system design, a number of cutting edge techniques have been properly tailed into an interdisciplinary approach for the trajectory tracking control of a human arm model on the sagittal plane. For the initial feasibility study, simulation results are provided to show that the GA and EP approach can find out the suitable architecture of the DRNN, the ADBP algorithm with momentum can guarantee the convergence of the system, and the controlled arm is well behaved to follow up those requested trajectories. It should be noticed that the proposed control procedure for the human arm movement can be further expanded for motor control, rehabilitation robot control, and various related applications. With its favorable dynamical network structure, the DRNN has played a key role in the tracking control in this study. The authors have tried to use a multilayer perceptron network in the trajectory tracking, but unfortunately the outcomes are unacceptable, which are not included in this paper. Furthermore, the performance comparisons between the DRNN and the iterative LQG show that DRNN is a better method to control the human arm motion in the sagittal plane. Compared with the modified GANNet algorithm, the hybrid GA and EP strategy has a faster converging rate and is more effective. In the second stage, further investigations will be expanded to the controller design for the trajectory tracking control in the Cartesian coordinate rather than in the joint space. The following development is also targeted for improving the computational efficiency. It has already been mentioned that evaluating a structure involves training the corresponding network and thus requires a large computational power. Obviously, speeding up training would greatly reduce the required effort. Some efficient algorithms should be developed while a fast real-time controller is the request. 6 Acknowledgements This work is supported by grants from the National Nature Science Foundation of China, No 60674105, the Ph.D. Programs Foundation of Ministry of Education of China, No 20050487013, and the Nature Science Foundation of Hubei Province of China, No 2007ABA027. The authors would like to thank Prof. Jiping He for the guidance regarding the muscular control and providing the arm model in format of Matlab/Simulink. Further the authors are grateful to the editor and the anonymous reviewers for their helpful comments and constructive suggestions with regard to the revision of the paper. References 1. Alfred C. Schouten, Erwin de Vlugt, Frans C. T. van der Helm, Guido G. Brouwn: Optimal Posture Control of a Musculo-Skeletal Arm Model. Biol. Cybern. 84 (2001) 143-152 2. Brown, I. E., Cheng, E. J., and Leob, G.: Measured and Modeled Properties of Mammalian Skeletal Muscle. II. the Effects of Stimulus Frequency on Force-Length and Force-Velocity Relationships. J. of Muscle Research and Cell Motility 20 (1999) 627-643 3. Xin Yao: Evolving Artificial Neural Network. Proceedings of the IEEE. 87 (1999) 1423-1477 4. P.A. Castillo, J.J. Merelo, M.G. Arenas, and G. Romero, Comparing Evolutionary Hybrid Systems for Design and Optimization of Multilayer Perceptron Structure along Training Parameters. Information Science 177 (2007) 2884-2905 5. Chao-Chee Ku, Kwang Y. Lee: Diagonal Recurrent Neural Networks for Dynamic System Control. IEEE Trans. on Neural Networks. 6 (1995) 144-156 6. David W. Franklin, Theodore E. Milner: Adaptive Control of Stiffness to Stabilize Hand Position with Large Loads. Exp. Brain Res. 152 (2003) 211-220 7. Ernest J. Cheng, Ian E. Brown, Gerald E. Loeb: Virtual Muscle: a Computational Approach to Understanding the Effects of Muscle Properties on Motor Control. Journal of Neuroscience Methods 101 (2000) 117-130 8. S. B. Hussenin, H. Jamaluddin, M. Mailah, A. M. S. Zalzala. A Hybrid Intelligent Active Force Controller for Robot Arms Using Evolutionary Neural Networks. In: Peter J. Angeline (eds.) Proceedings of the 2000 Congress on Evolutionary Computation, Vol. 1. IEEE Press, New Jersey (2000) 117-130 9. Kunihiko Nakazono, Masahiro Katagiri, Hiroshi Kinjo, Tetsuhiko Yamamoto: Force and Position Control of Robot Manipulator Using Neurocontroller with GA based Training. In: Yasushi Nakauchi (eds.): Proceedings of 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation, Vol. 3. IEEE Press, New Jersey (2003) 1354-1357 10. Ning Lan: Analysis of an Optimal Control Model of Multi-Joint Arm Movements. Biol. Cybern. 76 (1997) 207-117 11. Robert M. Sanner, Makiko Kosha: A Mathematical Model of the Adaptive Control of Human Arm Motions. Biol. Cybern. 80 (1999) 369-382 12. Shan liu, Yongji Wang, Jian Huang: Neural Network based Posture Control of a Human Arm Model in the Sagittal Plane. In: J. Wang et al. (eds.): International Symposium on Neural Networks 2006, Vol. 3973. Springer-Verlag, Berlin Heidelberg New York (2006) 792-798 13. Stroeve SH: Impedance Characteristics of a Neuromusculoskeletal Model of the Human Arm. I. Posture Control. Biol. Cybern. 81 (1999) 475-494 14. Weiwei Li, Emannuel Todorov: Iterative Quadratic Regulator Design for Nonliear Biological Movement system. In: Joaquim Filipe (eds.): Proceedings of the 1st International Conference on Informatica in Control, Automation and Robotics, Vol. 1. Kluwer Academic Publisher, Netherlands(2004) 222-229 15. Weiwei Li, Emanuel Todorov and Xiuchuan Pan: Hierarchical Optimal Control of Redundant Biomechanical Systems. In: D. Hudson, Z.P. Liang (eds.): Proceeding of 26th IEEE EMBS. IEEE Engineering in Medicine and Biology Society, Vol. 26. IEEE Press, New Jersey (2004) 4618-4621 16. Emanuel Todorov, Weiwei Li: A Generalized Iterative LQG Method for Locallyoptiml Feedback Control of Constrained Nonlinear Stochastic Systems. In: Suhada J., S.N. Balakrishnan (eds.): Proceeding of 2005 ACC. American Control Conference, Vol. 1. IEEE Press, New Jersey (2005) 300-306 17. Charslsmbos Papaxanthis, Thierry Pozzo, and Joe McIntyre: Arm End-point Trajectories Under Normal and Micro-gravity Environments. Acta Astronautica. 43(1998) 153-161 18. Charslsmbos Papaxanthis, Thierry Pozzo, and Marco Schieppati: Trajectories of arm pointing movements on the sagittal plane vary with both direction and speed. Exp. Brain Res. 148 (2003) 498-503 19. Hiroyuki Kambara, Kyoungsik Kim, and Duk Shin, etc.: Motor Control-Learning Model for Reaching Movements. In: 2006 International Joint Conference on Neural Networks, (2006) 555-562 20. Andrew H. Fagg, Nathan Sitkoff, james C. Houk, and Andrew G. Barto: Cerebellar Learning for Control of a Two-Link Arm in Muscle Space. In: 1997 IEEE International Conference on Robotics and Automation, 3(1997) 2638-2644 21. Hagan, M. T., H. B. Demuth, and M. H. Beale: Neural Network Design. Boston, MA: PWS Publishing, 1996 22. Yongji Wang, Jian Tu: Neural Network Control. Beijing: China Machine Press, 1998 23. Pan Wang, Youfeng Li, Shan Feng, Wei Wei: Convergence of Diagonal Recurrent Neural Networks’ Learning. In: Proceedings of the 4th World Congress on Intelligent Control and Automation. (2002) 2365-2369 24. H. Youlal, A. Kada, M. Ramzi and L. El-ismaili: Fast Diagonal Recurrent Neural Networks for Identification and Control of Nonlinear System. In: International Conference on Control 1994. 1(1994) 104-109 25. Pasemann, F., Steinmetz U., and Dieckman U. Evolving Structure and Function of Neurocontrollers. in: Proceedings of the 1999 Congress on Evolutionary Computation. 3(1999) 1973-1978 26. Toru Kumagai, Mitsuo Wada, Sadayoshi Mikami, and Ryoichi Hashimoto. Structured Learning in Recurrent Neural Network Using Genetic Algorithm with Internal Copy Operator. (1997) 651-656 27. D. White, P. Ligomenides. GANNet: A Genetic Algorithm for Optimizing Topology and Weights in Neural Network Design. In Proc. Int. Workshop Artificial Neural Network (IWANN 93), Lecture Notes in Computer Science. 686(1993) 322-327 28 Y. Liu, X. Yao. Evolutionary Design of Artificial Neural Networks with Different Nodes. In Proc. 1996 IEEE Int. Conf. Evolutionary Computation (ICEC 96). (1996) 670-675 Shan Liu is a doctoral student at Key Laboratory of Image Processing and Intelligent Control, Department of Control Science and Engineering, Huazhong University of Science and Technology, Wuhan, China. She obtained her B.S. degree in Mechanical Automation, Hubei University of Technology, Wuhan, China, in 2001 and M.S degree in system engineering, Huazhong University of Science and Technology, in 2004. Her main research interests involve application of advanced control to nonlinear multivariate system, such as rehabilitation robots, the neuromusculoskeletal model of human arm posture and movement, and industrial plant; neural network based intelligent and optimal control for neural prostheses; development of neural implant devices for epidural spinal cord stimulation for metabolic study in vivo of cat, and the modeling, control and optimality of metabolic systems. Yongji Wang was born in Ji’an, Jiangxi, P.R. China, in 1955. He received the undergraduate degree in Electrical Engineering from Shanghai Railway University, Shanghai, P.R. China, the M.S. degree and the Ph. D degree in automation from Huazhong University of Science and Technology, Wuhan, P.R. China, in 1982, 1984 and 1990, respectively. Since 1984, he has been with Huazhong University of Science and Technology, Wuhan, P.R. China, where he is currently a Professor of Electrical Engineering. His main interest is in intelligent control, and he has done research in neural network control, predictive control and adaptive control. Dr. Wang is a member of IEEE, USA, President of Hubei Automation Association, China, Standing member of council of Electric Automation Committee of Chinese Automation Society and member of council of Intelligent Robot Committee of Chinese Artificial Intelligence Society. He is an Area editor (Asia and Pacific) of Int. J. of Simulation Identification and Control. Quanmin Zhu is Professor in control systems at the Faculty of Computing, Engineering and Mathematical Sciences (CEMS) University of the West of England (UWE), Bristol, UK. He obtained his MSc in Harbin Institute of Technology, China in 1983 and PhD in Faculty of Engineering, University of Warwick, UK in 1989. His main research interest is in the area of nonlinear system modelling, identification, and control. Recently Professor Zhu started investigating electrodynamics of acupuncture points and sensory stimulation effects in human body, modelling of human meridian systems, and building up electro-acupuncture instruments. He has published over one hundred papers on these topics. Currently Professor Zhu is acting as Associate Editor of International Journal of Systems Science (http://www.tandf.co.uk/journals/titles/00207721.asp), Member of Editorial Committee of Chinese Journal of Scientific Instrument (http://www.asiatest.org/), and Editor (and Founder) of International Journal of Modelling, Identification and Control (http://www.inderscience.com/ijmic). Prof Zhu’s brief CV can be found at http://www.ias.uwe.ac.uk/people.htm