Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Engineering MECHANICS, Vol. 11, 2004, No. 2, p. 83 – 96 83 EFFICIENT Q-LEARNING MODIFICATION APPLIED ON ACTIVE MAGNETIC BEARING CONTROL Tomáš Březina*, Jiřı́ Krejsa** The paper is focused on use of Q-learning for active magnetic bearing (AMB) control. Q-learning belongs to the reinforcement learning methods which are the part of real time machine learning approaches. The essence of proposed method is in separating the Q-learning into two phases – prelearning phase, which uses mathematical model of real system and tutorage phase, which works with the real system and is used for further improvement of Q-values found during the prelearning phase. Proposed learning method, used in prelearning phase, proved to be highly efficient. Controller based on Q-learning shows better results (regarding the number of successful trials) than referential PID controller after only 1000 table passes. The control quality criterion results are comparable. The policy found by learning also shows high robustness against errors of system variables observations, even if only very simple reinforcement function in shape of simple reduced penalty is used. Key words : reinforcement learning, Q-learning, active magnetic bearing 1. Introduction Active magnetic bearing (AMB) is relatively new and highly perspective design element which eliminates direct mechanical contact by rotor levitation in controlled magnetic field. From mechanical point of view it is characterized by negligible frictional resistance and abrasion, low energy demands and noise and also high precision and stiffness. The possibility of dynamic properties change while machine is in operation is highly convenient regarding control abilities. Application under extreme or special conditions is indispensable from operational point of view. AMB itself is unstable and must be stabilized by feedback control loop. The feedback is usually made of contactless rotor position sensor and controller with power circuit. The principle consists in evaluation of rotor deviation, which controls the feeding current (voltage) of bearings magnets after processing via the controller. There are high demands on the control electronics which leads to the high cost of the bearings, however this type of bearing is commercially produced for years and used in certain large machines (turbines, turbo compressors, large electric machines) when its cost is admissible and its advantages are the most profitable. Artificial intelligence methods, which use real time machine learning, can represent the starting point for the new control methods proposition. New control methods either improve * doc. RNDr. Ing. T. Březina, CSc., Brno University of Technology, Faculty of Mechanical Engineering, Technická 2, 616 69, Brno ** Ing. J. Krejsa, PhD, Institute of Thermomechanics – Brno department, Czech Academy of Science, Technická 2, 616 69, Brno