Download Centrally controlled heart rate changes during mental practice in

Centrally controlled heart rate changes during mental practice in
immersive virtual environment: A case study with a tetraplegic
Pfurtscheller, G.1), Leeb, R.1), Friedman, D.2), Slater, M.2,3)
1)
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz
University of Technology, Krenngasse 37, A-8010 Graz, Austria
2)
Department of Computer Science, University College London, WC1E 6BT, UK
3)
ICREA, Universitat Politecnica de Catalunya, E-08028 Barcelona, Spain
Number of text pages (including figures):
Number of figures:
2
Number of tables:
1
12
Corresponding author:
Gert Pfurtscheller
Laboratory of Brain-Computer Interface
Graz University of Technology
Krenngasse 37,
A-8010 Graz,
Austria
phone: +43-316-873-5300
fax:
+43-316-873-5349
email: [email protected]
url:
http://www.bci.tugraz.at
1
Abstract
A tetraplegic patient was able to induce midcentral localized beta oscillations in the
electroencephalogram (EEG) after extensive mental practice of foot motor imagery. These
beta oscillations were used to simulate a wheelchair movement in virtual environments (VEs).
The analysis of electrocardiogram (ECG) data revealed that the induced beta oscillations were
accompanied by a characteristic heart rate (HR) change in form of a preparatory HR
acceleration followed by a short-lasting deceleration in the order of 10 to 20 bpm (beats-perminute). This provides evidence that mental practice of motor performance is accompanied
not only by activation of cortical structures but also by central commands into the
cardiovascular system with its nuclei in the brain stem.
Keywords: heart rate acceleration, Motor imagery, virtual environment, Brain-Computer
Interface
1. Introduction
Two types of heart rate (HR) changes can be differentiated, an event-related HR deceleration
known as orienting response and an event-related HR acceleration known as defence response
(for a recent review cf. Sokolov et al. 2002). With respect to information processing two types
of HR deceleration responses can be distinguished. An early response related to stimulus
anticipation and registration (Lacey and Laces 1980, Van der Molen etc 1989, Bohlin and
Kjellberg 1979) and a second type related to motor preparation. While Bohlin and Krellberg
(1979) favoured the view that expectancy is responsible for the HR deceleration, the work of
Damen and Brunia (1987) and Papakostopolos et al. (1990) give strong evidence that motor
preparation is the dominant factor of HR deceleration. Of interest are two recent publications
reporting HR deceleration. During world translations, a highly cognitive task, the HR
displayed a significant deceleration (Pfurtscheller et al. 2007). Imagination of hand or foot
movement during biofeedback training revealed also a HR deceleration (Pfurtscheller et al.
2006a) similar as has been reported previously regarding self-paced finger movements
(Florian et al. 1998).
Beside the deceleration also HR acceleration is a frequent response to activation of cortical
structures. HR acceleration was not only reported during cognitive processing (e.g. Danilova
2
et al. 1994), but also during mental simulation of movement (Oishi et al. 2000) and during
motor imagery sessions (Papadelis et al. 2007).
In this paper we report on HR data recorded during simulated wheel chair control in an
immersive virtual environment. A tetraplegic patient had to “move” a wheel chair in a virtual
street through thought. The movement in the street was controlled by mentally induced
electroencephalogram (EEG) bursts and the HR analysed during EEG-burst generation.
2. Methods
2.1. Subject:
The patient enrolled in the study is a 30-year old male, who sustained a traumatic spinal cord
injury (SCI) in 1998. He has complete motor and sensory paralysis at the level of the fifth
cervical spinal vertebra. During an intensive training period he has learned to induce beta
burst at 17 Hz during foot motor imagery (MI) (Pfurtscheller et al. 2000).
2.2. EEG- and ECG- recording
One single EEG channel was recorded with electrodes placed bipolarly at position Cz over the
foot representation area. The EEG was amplified and band-pass filtered between 0.5 and 30
Hz with a biosignal amplifier (g.tec, Guger Technologies, Graz, Austria). The
electrocardiogram (ECG) was recorded bipolarly from the thorax with the same amplifier and
filtered between 0.5 and 100 Hz. A 50 Hz (power line) notch filter was applied before the
biosignals were digitized with a sampling rate of 250 Hz. The real-time processing was
performed under Simulink (MathWorks, Inc., Natick, USA).
A single logarithmic band power (BP) feature was estimated from the ongoing EEG by
digitally band-pass filtering (15 – 19 Hz) the EEG signal, squaring, averaging (moving
average) the samples over the past second and computing the logarithm from this time series.
A simple threshold was used in the online experiments to distinguish between foot movement
imagination (control or intentional state) and rest (non-control or non-intentional state).
Whenever the band power exceeded the threshold a foot MI was detected and the signal was
used to control a Virtual Environment (VE) (for details see Leeb et al., 2007).
2.3. Virtual Environment and experimental paradigm
The tetraplegic patient was placed with his wheelchair in the middle of a multi-projection
based stereo and head-tracked VE system of the type commonly known as a “CAVE” (Cruz3
Neira et al. 1993). The actual system used was a Trimension ReaCTor. The VE depicted a
virtual street populated with 15 virtual characters (avatars), which were lined up along the
street.
The task of the participant was to “walk” (move the wheelchair) from avatar to avatar towards
the end of the virtual street by imagination of movement of his paralyzed feet. Only during the
time when foot MI was detected (intentional control) the subject moved forward. Every time
he approached towards, but before reaching, an avatar, he had to stop very close to it. Each
avatar was conceptually enclosed by an invisible communication sphere and the subject had to
stop spatially within this sphere. Communication was only possible within the sphere, when
the avatar would say something encouraging to the subject; if the subject stopped too early or
too close to the avatar nothing happened. After a while, at his own choice, the subject could
imagine another foot movement and start to walk again, until the end of the street was
reached. The walked distance depended only on the duration of the foot motor imagery,
longer foot MI resulted in a larger distance than short bursts of MI. The avatars were placed
on the same positions in all experiments and the participant started from the same point. In the
case of 15 correct stops in one run, the performance was 100%. Ten experiments (runs) on 2
different days were carried out in total.
2.4. Calculation of heart rate (HR) changes
The first step in ECG processing is to detect the QRS (ventricular contraction) complexes in
the ECG signal. The QRS complexes determine the distance in time from one heart
contraction to the next one (RR interval). The QRS complexes were detected automatically
based on an algorithm using a filter bank to decompose the ECG signal into various subbands.
For a detailed description of the algorithm, see (Alfonso et al., 1999). From the RR-interval
time series the instantaneous heart rate (IHR) was calculated by linear interpolation between
consecutive RR-interval samples (De Boer et al., 1985). After the selection of IHR trials with
5s prior to EEG band power maximum and 5s thereafter, averaging was performed across the
trials of each run. The results show event-related HR time courses together with the sampleby-sample inter-trial standard error (SE) of the mean.
3. Results
3.1. Performance measure
4
The results of the 2-day experiments are summarized in Table 1. Altogether the tetraplegic
subject spent about 40 minutes in the VE with self-paced EEG-based VE control, whereby in
4 runs he achieved a performance of 100 %. This means that he was able to generate at free
will a specific pattern of brain oscillations (beta bursts with frequencies of 17 Hz) by foot
motor imagery and control therewith the VE (street) and the movement of the wheelchair,
respectively. On average the performance was 90 %, the heart rate 77 bpm and the BCI time
232 seconds.
Table 1: Summarized data of the 10 runs on 2 days. The performance
(in %), the mean HR rate and the total time needed for the BCI control
are indicated.
day
run
performance [%]
mean HR [bpm]
BCI time [s]
day
run
performance [%]
mean HR [bpm]
BCI time [s]
1
1
1
2
1
3
1
4
1
5
1
6
100,0%
64,1
360,8
93,3%
72,2
327,2
100,0%
92,0
218,7
100,0%
87,4
180,8
73,3%
76,0
203,7
73,3%
70,5
255,0
2
7
2
8
2
9
2
10
grand
average
93,3%
165,7
100,0%
209,9
80,0%
217,7
86,6%
177,2
90,0%
77,0
231,7
3.2. HR changes during motor imagery in VE
The ECG was recorded only during the first day in runs 1 to 6 whereby the HR varied from
64.1 bpm (run 1) to 92 bpm (run 3) (see Table 1). While the HR was relatively high and stable
in the runs 3, 4 and 5, the HR displayed a characteristic pattern during motor imagery in the
runs 1, 2 and 6 (Fig. 1 right side). This pattern showed a slow HR acceleration, followed by a
fast deceleration and a slow return to the baseline. This HR pattern was found to correspond
to the beta burst in the EEG. In other words, foot motor imagery was accompanied not only
by a midcentral localized beta burst but also by a preparatory HR acceleration terminated by a
short-lasting HR deceleration. The period of the HR change associated with motor imagery
was about 10s.
5
Figure 1: Examples of band pass-filtered EEG trials selected by the
band power maximum at “t=0” (upper left). Averaged band power and
IHR trials (mean ± SD) for runs 1, 2, and 6 (right side) and grand
averaged band power and IHR over the runs 1, 2, and 6 (lower left).
6
Figure 2: Single trial raw EEG, instantaneous heart rate (IHR) and the
time course of the band power are shown. Remarkable is that the HR
increase is starting some seconds before the band power enhancement.
The averaged trials of this run (first run on the first day) are displayed
in the right bottom corner.
4. Discussion
In the present study we investigated HR changes during self-paced foot motor imagery in one
tetraplegic subject sitting in a wheel chair, with the goal to control the wheel chair through
thought (Leeb et al. 2007). The analysis of the data revealed a significant preparatory HR
acceleration terminated by a HR deceleration during mental practice and induction of
midcentral beta burst in the EEG.
Our results are in correspondence with the work of Decety et al. (1991) who reported that
mental imagery of the motor action of running on a treadmill provided an increase in HR.
Similarly, a HR increase was found during actual and mental practice of an electronic flight
simulator program (Papadelis et al. 2007). In both studies the HR increase correlated with the
degree of task difficulty and mental effort, respectively. In general the HR displays an
increase, when the situation becomes more demanding or during simulation with increased
7
mental effort (Beyer et al. 1990, Mulder et al. 2005). Beside mental effort also emotional
processing results in a HR increase, e.g. Critcheley et al. (2005) found that “happy” facial
expressions reduced the HR whereas “sad” or “angry” expressions increased the HR.
A similar coupling between EEG and HR changes with a HR acceleration was found in
preterm infants. In this case discontinuous slow-wave delta bursts were found in parallel with
a weak, but significant HR acceleration in the order of 2-5 bpm (Pfurtscheller 2005). In
contrast to the reported HR acceleration during foot motor imagery, this interaction between
brain and heart is non-intentional.
Special emphasis should be paid to the fact that preparation of a specific movement and
imagination of the same movement involves similar cortical networks (Porro et al. 1996,
Lotze et al 1999) and are accompanied in general by a HR deceleration (Papakostopolous et
al. 1990, Florian et al. 1998). In training sessions with a cue-based 2-class BCI (hand versus
foot motor imagery) and feedback on a desktop PC the common HR response was a
deceleration (see Fig. 3 in Pfurtscheller et al. 2006a). During EEG-based control of “walking”
in VE, the same mental strategy can induce, however, a HR acceleration (see Fig. 3 in
Pfurtscheller et al. 2006a). This provides evidence that the increased mental effort and
emotional processing (“walking” in immersive VE) are driving forces behind the HR
acceleration.
It was shown recently, that the vividness of movement imagery was significantly correlated
with the activation of the motor cortex (Alkadhi et al. 2005). The extent of activation in motor
areas was higher in paraplegic patients when compared to healthy subjects. We can suggest
therefore, that our tetraplegic patient was highly motivated and directed increased attention to
“walk” in the immersive VE.
It has to be noted, however, that an increased mental effort or the performance of a more
stressful task results not always in a HR increase. Increased word difficulty in translation from
one to another language was accompanied by an increased HR deceleration (Pfurtscheller et
al. 2007).
The novel result of this study is not that the HR response was time-locked with a significant
brain pattern, namely bursts of beta oscillations, but that the HR acceleration started several
seconds prior to the burst onset (single trial example see Fig. 2). This can be interpreted as the
intentional process preceding the generation of beta bursts was already accompanied by
8
changes in slow cortical potentials similar to the 1 – 2 s prior to self-paced movement as
reported by (Papakostopolouset al. 1990) and associated with a central command into the
cardiovascular brain stem system. The importance of such central commands into medullary
cardiovascular nuclei was discussed by Verberne and Owens (1998). Due to signal recording
with a lower cut-off frequency of 0.5 Hz, such a slow cortical potential shift was not
recordable and only the final beta burst was present in the EEG.
The reported HR changes in the order of 10 – 20 bpm during effortful mental activity
provides for the first time evidence that the performance of a brain-computer interface (BCI,
Wolpaw et al. 2002) could perhaps be improved, when beside the EEG also the HR is
analysed and classified. It could be even possible to realise a “brain switch” not only by a
brisk inspiration (Pfurtscheller et al. 2006b) but more elegantly by effortful mental activity
and HR analysis and classification only.
Acknowledgement:
This work was carried out as part of the PRESENCCIA project, an EU funded Integrated
Project under the IST programme (Project Number 27731). The authors are grateful to T.S.
for his participation in the CAVE experiments at the University College London.
9
References
Afonso, V. X., Tompkins, W. J., Nguyen, T. Q., & Luo, S. (1999). ECG beat detection using
filter banks. IEEE Transactions on Biomedical Engineering, 46, 192-202.
Beyer, L., Weiss, T., Hansen, E., Wolf, A., & Seidel, A. (1990). Dynamics of central nervous
activation during motor imagination. Int J Psychophysiol, 9(1), 75-80.
Bohlin, G., & Kjellberg, A. (1979). Orienting activity in two-stimulus paradigms as reflected
in heart rate. In: Kimmel, H., Van Olst, E., & Orlebeke, J. F. (Eds.), The orienting
reflex in humans (pp. 169-198). Hillsdale, NJ: Erlbaum.
Critchley, H.D., Rotshtein, P., Nagai, Y., O’Doherty, J., Mathias, C.J., Doland, R.J. (2005).
Activity in the human brai8n predicting differential heart rate responses to emotional
facial expressions. NeuroImage, 24:751-762.
Cruz-Neira, C., Sandin, D.J., & DeFanti, T.A. (1993). Surround-screen projection-based
virtual reality: the design and implementation of the CAVE. Proceedings of the 20th
annual conference on Computer graphics and interactive techniques, 135-142.
Damen, E.J.P., & Brunia, C.H.M. (1987). Changes in heart rate and slow brain potentials
related to motor preparation and stimulus anticipation in a time estimation task.
Psychophysiology, 24, 700-713.
Danilova, N.N., Korshunova, S.G., & Sokolov, E. N. (1994). Indexes of heart-rate during
solving arithmetical tasks in humans. Zhurnal Vysshei Nervnoi Deyatelnosti Imeni I P
Pavlova, 44, 932-943
De Boer, R.W., Karemaker, J. M., & Strackee, J. (1985). Relationships between short-term
blood-pressure fluctuations and heart-rate variability in resting subjects. I: A spectral
analysis approach. Medical and Biological Engineering and Computing, 23, 352-358.
Decety, J., Jeannerod, M., Germain, M., & Pastene, J. (1991). Vegetative response during
imagined movement is proportional to mental effort. Behav Brain Res, 42(1), 1-5.
Florian, G., Stancak, A., & Pfurtscheller, G. (1998). Cardiac response induced by voluntary
self-paced finger movement. International Journal of Psychophysiology, 28, 273-83.
Lacey, B.C. & Lacey, J.I. (1980). Cognitive modulation of time-dependent primary
bradycardia. Psychophysiology, 17, 209-222.
Leeb, R., Friedman, D., Müller-Putz, G.R., Scherer, R., Slater, M., Pfurtscheller, G.
(submitted, 2007) Self-paced (asynchronous) BCI control of a wheelchair in Virtual
Environments: A case study with a tetraplegic. Computational Intelligence and
10
Neuroscience: special issue “Brain-Computer Interfaces: Towards Practical
Implementations and Potential Applications”, submitted.
Lotze, M., Montoya, P., Erb, M., Hulsmann, E., Flor, H., Klose, U., Birbaumer, N., & Grodd,
W. (1999). Activation of cortical and cerebellar motor areas during executed and
imagined hand movements: an fMRI study. J Cogn Neurosci, 11(5), 491-501.
Mulder, T., de Vries S., Zijlstra, S. (2205) Observation, imagination and execution of an
effortful movement: more evidence for a central explanation of motor imagery. Exp.
Brain. Res. 163(3):344-51.
Oishi, K., Kasai, T., Maeshima, T. (2000). Autonomic Response specificity during Motor
Imagery. Journal of Physiological Anthropology and Applied Human Science, 19,
255-261.
Papadelis, C. Kourtidou-Papadeli, C., Bamidis, P., Albani, M. (in press, 2007). Effects of
imagery training on cognitive performance and use of physiological measures as an
assessment tool of mental effort. Brain and Cognition.
Papakostopoulos, D., Banerji, N. K., & Pocock, P. V. (1990). Performance, EMG, brain
electrical potentials and heart rate change during a self-paced skilled motor task in
Parkinson's disease. Journal of Psychophysiology, 4, 163.
Pfurtscheller K, Müller-Putz GR, Urlesberger B, Dax J, Muller W, Pfurtscheller G. (2005)
Synchronous occurrence of EEG bursts and heart rate acceleration in preterm infants.
Brain Dev. 27(8):558-63.
Pfurtscheller, G., Grabner, R.H., Brunner, C., Neuper, C. (in press, 2007). Phasic heart rate
changes during word translation of different difficulties, Psychophysiology, in press.
Pfurtscheller, G., Guger, C., Müller, G., Krausz, G., & Neuper, C. (2000). Brain oscillations
control hand orthosis in a tetraplegic. Neurosci Lett, 292(3), 211-4.
Pfurtscheller, G., Leeb, R., & Slater, M. (2006a). Cardiac responses induced during thoughtbased control of a virtual environment. International Journal of Psychophysiology, 62,
134-140.
Pfurtscheller, G., Scherer, R., Müller-Putz G.R. (2006b) Heart rate-controlled EEG-based
BCI: The Graz-Hybrid-BCI. Proc. 3rd International Brain-Computer Interface
Workshop and Training Course, Graz, Austria, Verlag der Technischen Universität
Graz, 100-101.
Porro, C.A., Francescato, M.P., Cettolo, V., Diamond, M.E., Baraldi, P., Zuiani, C.,
Bazzocchi, M., & di Prampero, P.E. (1996). Primary motor and sensory cortex
11
activation during motor performance and motor imagery: a functional magnetic
resonance imaging study. J Neurosci, 16(23), 7688-98.
Sokolov, E.N., Spinks, J.A., Näätänen, R. & Lyytinen, H. (2002). The orienting response in
information processing. Mahwah, NJ: Erlbaum.
Van der Molen, M.W., Boomsma, D.I., Jennings, J.R., & Nieuwboer, R.T. (1989). Does the
heart rate know what the eye sees? Cardicac/pupillometric analysis of motor
preparation and response execution. Psychophysiology, 26, 70-80.
Verberne, A. J. M., & Owens, N. C. (1998). Cortical modulation of the cardiovascular
system. Progress in Neurobiology, 54, 149-168.
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., & Vaughan, T.M. (2002).
Brain-computer interfaces for communication and control. Clin Neurophysiol, 113(6),
767-91.
12