Download efficient q-learning modification applied on active magnetic bearing

Engineering MECHANICS, Vol. 11, 2004, No. 2, p. 83 – 96
83
EFFICIENT Q-LEARNING MODIFICATION APPLIED
ON ACTIVE MAGNETIC BEARING CONTROL
Tomáš Březina*, Jiřı́ Krejsa**
The paper is focused on use of Q-learning for active magnetic bearing (AMB) control.
Q-learning belongs to the reinforcement learning methods which are the part of real
time machine learning approaches. The essence of proposed method is in separating
the Q-learning into two phases – prelearning phase, which uses mathematical model
of real system and tutorage phase, which works with the real system and is used
for further improvement of Q-values found during the prelearning phase. Proposed
learning method, used in prelearning phase, proved to be highly efficient. Controller
based on Q-learning shows better results (regarding the number of successful trials)
than referential PID controller after only 1000 table passes. The control quality criterion results are comparable. The policy found by learning also shows high robustness
against errors of system variables observations, even if only very simple reinforcement
function in shape of simple reduced penalty is used.
Key words : reinforcement learning, Q-learning, active magnetic bearing
1. Introduction
Active magnetic bearing (AMB) is relatively new and highly perspective design element
which eliminates direct mechanical contact by rotor levitation in controlled magnetic field.
From mechanical point of view it is characterized by negligible frictional resistance and
abrasion, low energy demands and noise and also high precision and stiffness. The possibility
of dynamic properties change while machine is in operation is highly convenient regarding
control abilities. Application under extreme or special conditions is indispensable from
operational point of view.
AMB itself is unstable and must be stabilized by feedback control loop. The feedback
is usually made of contactless rotor position sensor and controller with power circuit. The
principle consists in evaluation of rotor deviation, which controls the feeding current (voltage) of bearings magnets after processing via the controller. There are high demands on
the control electronics which leads to the high cost of the bearings, however this type of
bearing is commercially produced for years and used in certain large machines (turbines,
turbo compressors, large electric machines) when its cost is admissible and its advantages
are the most profitable.
Artificial intelligence methods, which use real time machine learning, can represent the
starting point for the new control methods proposition. New control methods either improve
* doc. RNDr. Ing. T. Březina, CSc., Brno University of Technology, Faculty of Mechanical Engineering,
Technická 2, 616 69, Brno
** Ing. J. Krejsa, PhD, Institute of Thermomechanics – Brno department, Czech Academy of Science, Technická 2, 616 69, Brno