Download System Architecture of ERS/ERD

Introduction to Brain Computer
Interface (BCI) Systems
Todor Mladenov
CSNL, GIST 2011
Outline
•
•
•
•
•
•
•
•
Introduction
Background
EEG signals
Noninvasive EEG methods
BCI Applications
Communication Issues
Research Issues
Conclusions
Introduction
• Machines that could be controlled by one's thoughts.
• Brain computer interface devices (BCI) detect and translate neural
activity into command sequences for computers and prostheses.
• Electrodes recording from the brain are used to send information
to computers so that mechanical functions can be performed.
• BCI devices aim to restore function in patients suffering from loss
of motor control e.g. stroke, spinal cord injury, multiple sclerosis
(MS) and amyotrophic lateral sclerosis (ALS).
• BCI will broaden repertoire of neurosurgical treatments available
to patients previously treated by non-surgical specialists
Background
• 1875 - Richard Caton discovered electrical properties of exposed cerebral
hemispheres of rabbits and monkeys.
• 1883 - Marxow discovers evoked potentials.
• 1924 - German Psychiatrist Hans Berger discovered alpha waves in humans and
invented the term “electroencephalogram”(EEG).
• 1929 - Berger records electrical activity from the skull.
• 1936 - Gray Walter finds abnormal activity with tumors.
• 1950s - Grey Walter developed “EEG topography” - mapping electrical activity
of the brain.
• 1970s - Research that developed algorithms to reconstruct movements from
motor cortex neurons, which control movement.
• 1980s - Johns Hopkins researchers found a mathematical relationship between
electrical responses of single motor-cortex neurons in rhesus macaque
monkeys and the direction that monkeys moved their arms (based on a cosine
function).
• 1990s - Several groups able to capture complex brain motor centre signals
using recordings from neurons and use these to control external devices.
EEG Pioneers
In 1929, Hans Berger
• Recorded brain activity from the closed skull
• Reportet brain activity changes according to the
functional state of the brain
– Sleep
– Hypnothesis
– Pathological states (epilepsy)
In 1957, Gray Walter
• Makes recordings with large numbers of electrodes
• Visualizes brain activity with the toposcope
• Shows that brain rhythms change according to the
mental task demanded
What is EEG
• An electroencephalogram
(EEG) is a measure of the
brain's voltage fluctuations
as detected from scalp
electrodes.
• It is an approximation of the
cumulative electrical
activity of neurons.
Physical Mechanism
• EEGs require electrodes
attached to the scalp with sticky
gel
• Require physical connection to
the machine
Electrode Placement (1)
• Standard “10-20 System”
• Spaced apart 10-20%
• Nasion – point between the
forehead and the skull
• Inion – Bump at the back of the
skull
• Letter for region
–
–
–
–
F - Frontal Lobe
T - Temporal Lobe
C - Center
O - Occipital Lobe
• Number for exact position
– Odd numbers - left
– Even numbers - right
Electrode Placement (2)
• International 10/20 system
Electrode Placement (3)
The EEG measures
• not action potentials
• not summation of
action potentials
• but summation of
graded Post Synaptic
Potentials (PSPs)
(only pyramidal cells:
dipoles between soma
and apical dendrites)
EEG Channels
Channel: Recording from a pair of electrodes (here with a common
reference: A1 – left ear)
Multichannel EEG recording: up to 40 channels recorded in parallel
Brain “Features”
• User must be able to
control the output:
– use a feature of the
continuous EEG output that
the user can reliably modify
(waves), or
– evoke an EEG response with
an external stimulus (evoked
potential)
Brain “Features”
• Generally grouped by frequency: (amplitudes are about 100µV
max)
Type
Frequency
Location
Use
Delta
<4 Hz
everywhere
occur during sleep, coma
Theta
5-7 Hz
temporal and parietal
correlated with emotional stress
(frustration & disappointment)
Alpha
8-12 Hz
occipital and parietal
reduce amplitude with sensory
stimulation or mental imagery
Beta
12-36 Hz
parietal and frontal
can increase amplitude during
intense mental activity
Mu
9-11 Hz
frontal (motor
cortex)
diminishes with movement or
intention of movement
Lambda
sharp,
jagged
occipital
correlated with visual attention
Vertex
higher incidence in patients with
epilepsy or encephalopathy
Brain Wave Transforms
• Wave-form averaging over
several trials
• Auto-adjustment with a
known signal
• Fourier transforms to detect
relative amplitude at different
frequencies
– seperates spontaneous EEG
signal to component
frequencies and amplitudes
– high frequency resolution
demands long (in the range of
seconds) analysis windows
Alpha Rhythm
Frequency:
Amplitude:
Location:
State of Mind:
Source:
8 – 12 Hz
5 – 100 microVolt
Occipital, Parietal
Alert Restfulness
Oscillating thalamic pacemaker neurons
Alpha blockade occurs when new stimulus is processed
Beta Rhythm
Frequency:
Amplitude:
Location:
State of Mind:
12 – 36 Hz
2 – 20 microVolt
Frontal
Mental Activity
Reflects specific information processing between cortex and thalamus
Delta Rhythm
Frequency:
1 – 4 Hz
Amplitude:
20 – 200 microVolt
Location:
Variable
State of Mind:
Deep sleep
Oscillations in Thalamus and deep cortical layers
Usually inibited by ARAS (Ascending Reticular Activation System)
Theta Rhythm
Frequency:
Amplitude:
Location:
State of Mind:
5 – 7 Hz
5 – 100 microVolt
Frontal, Temporal
Sleepiness
Nucleus reticularis slows oscillating thalamic neurons
Therefore diminished sensory throughput to cortex
Mu Waves
• Studied since 1930s
• Found in Motor Cortex
• Amplitude suppressed by Physical Movements, or intent to
move physically
• (Wolpaw, et al 1991) trained subjects to control the mu rhythm
by visualizing motor tasks to move a cursor up and down (1D)
• (Wolpaw and McFarland 2004) used a linear combination of Mu
and Beta waves to control a 2D cursor.
• Weights were learned from the users in real time.
• Cursor moved every 50ms (20 Hz)
• 92% “hit rate” in average 1.9 sec
Alpha and Beta Waves
•
•
•
•
•
Studied since 1920s
Found in Parietal and Frontal Cortex
Relaxed - Alpha has high amplitude
Excited - Beta has high amplitude
So, Relaxed -> Excited
means Alpha -> Beta
Importance of EEG for HCI (1)
• People with disabilities may have no other option for HCI than
BCI.
• Combining EEG with motion information to improve wearable
activity recognition systems.
• Performance improvement in both humans and artificial
systems strongly relies in the ability of recognizing erroneous
behavior or decisions. EEG activity evoked by erroneous gesture
recognition can be used as a feedback for HCI. [1]
• Hybrid Human Computer Interaction Systems (Hybrid – HCI).
• Thoughts recognition [2]
• Emotions recognition [3]
Importance of EEG for HCI (2)
• Recognizing the emotional state of a person and its thoughts are
directly applicable to the automotive industry – automatic
braking, sleep detection, tiredness level monitoring, anxiety and
tension monitoring.
• Cars controlled by thoughts - AutoNOMOS Labs at Freie
Universität Berlin.
• New possibilities for gaming experience utilizing EEG controls.
• Application of EEG systems in education. Example are the apps
based on NeuroSky’s ( light a comfortable EEG sensors with
custom fully integrated ASIC chip) equipment.
Reading the Brain (1)
• Direct Neural Contact ( Invasive )
- Most accurate method
- Highly invasive
- Not possible with current technologies
- Perhaps possible in future with e.g. nanobots
Reading the Brain (2)
• Electroencephalogy (EEG)
-
Measures electical activity in brain
-
Non-invasive
-
Susceptible to noise
-
Easy to use + low cost + portable
-
Most commonly used device in BCIs
Reading the Brain (3)
• Magnetoencephalogy (MEG)
-
Measures magnetic fields produced by electrical activity in brain
-
Non-invasive
-
Very accurate
-
High equiment requirements and maintenance costs
Reading the Brain (4)
• Functional Magnetic Resonance Imaging (fMRI)
- Measures blood flow in brain using MRI (haemodynamics)
- Blood flow correlates to neural activity
- Studies the brain‘s function
- Very accurate
- Very high costs due to MRI
Direct (noninvasive) interfaces in EEG
• The on-going electrical activity of the brain measured from
scalp electrodes is called the electroencephalogram or EEG.
• An event-related potential (ERP) is any measured brain
response that is directly the result of a thought or
perception. More formally, it is any stereotyped
electrophysiological response to an internal or external
stimulus.
• Direct Interfaces via EEG
–
–
–
–
VEP – Visual Evoked Potential
SSVEP – Steady-State Visual Evoked Potential
P300 – ERP elicited by infrequent, task-relevant stimuli.
ERS/ERD – Event related synchronization/desynchronization
Event Related Potentials (ERP)
• Averaging of trials
following a stimulus
• Noise reduction: The
noise decreases by the
squareroot of the number
of trials
• Far field potentials
require up to 1000
measurements
• Assumption: no
habituation occurs
(participants don‘t get
used to stimulation)
Visual Evoked Potential (VEP)
• Caused by Visual Stimulus
• Occurs with flashing lights (3-5 Hz)
• Have been used to monitor function during surgery for lesions
involving the pituitary gland, optic nerve and chiasma.
• Application:
Steady-State Visual Evoked Potential
(SSVEP)
• SSVEP are signals that are natural responses to visual
stimulation at specific frequencies. When the retina is
excited by a visual stimulus ranging from 3.5 Hz to 75 Hz, the
brain generates electrical activity at the same (or multiples
of) frequency of the visual stimulus.
• Excellent signal-to-noise ratio and relative immunity to artifacts.
• Applications:
– SSVEP-controlled robots (Boston University)
– User-friendly interface ( NeuroSky) [4]
P300 (1)
• P300 is thought to reflect processes involved in stimulus
evaluation or categorization.
• It is usually elicited using the oddball paradigm in which lowprobability target items are inter-mixed with highprobability non-target (or "standard") items.
• Results in a positive curve on EEG after 300ms.
• Strongest signal at parental lobe.
P300 (2)
• (Farwell and Donchin 1988)
• 95% accuracy at 1 character per 26s
ERS/ERD (1)
• Event-related desynchronization (ERD) and event-related
synchronization (ERS) is the change of signal's power
occurring in a given band, relative to a reference interval.
• People have naturally occurring brain rhythms over areas of
the brain concerned with touch and movement. When
people imagine moving, these brain rhythms first become
weaker, then stronger. These two changes are called ERD
and ERS, respectively.
• ERS
– oscillatory power increase
– associated with activity decrease?
• ERD
– oscillatory power decrease
– associated with activity increase?
ERS/ERD (2)
• The imagination of either a
left or right hand movement
results in(5):
– An amplitude attenuation
(Event-Related
Desynchronization (ERD)) of
μ (8-12Hz) and central beta
EEG-rhythms (13-30Hz) at the
contralateral sensorial motor
representation area and,
– in some cases, in an
amplitude increase (EventRelated Synchronization
(ERS)) within the γ-band (3040Hz) at the ipsi-lateral
hemisphere(6).
EEG recorded from C3 electrode.
ERS/ERD (3)
ERS/ERD (4)
Apha response
ERS
Time
Alpha
Amplitude
Theta
Amplitude
Typical Theta response
ERD
Time
Pre-Stimulus Post-Stimulus
Pre-Stimulus Post-Stimulus
Stimulus
Stimulus
Test power
ERD
Theta
Test power
Reference
power
ERS
Reference
power
ERS/ERD (5)
Alpha
ERS/ERD (6)
ERS/ERD (7)
Semantic task:
upper alpha relevant
Visuo-spatial
semantic LTM task
Visuo-spatial
learning WM task
High activity (ERD red)
is related to
good performance
Working and short term
memory task
upper alpha is less relevant
Inhibition (ERS) of
irrelevant semantic processes
leads to good performance
Communication Issues
Typical training time versus communication bitrate for the three main types
of noninvasive BCIs.
BCI Applications (1)
– For people with certain disabilities ( paraplegia, amyotrophia,.. ) BCI
might be the only way of communication.
– Human enhancements:
• Cybernetic Organisms
• Brainwave Synchronization
• Exocortex ( intelligence booster)
– Human manipulation
• Mind-control
• “Neurohacking”: unwanted reading of information from brain.
– Neuroprosthetics:
• Surgically implanted devices used as replacement for damaged neurons.
BCI Applications (2)
BCI – operated robot
BCI Applications (3)
BCI Applications (4)
BCI Applications (5)
Research Issues (1)
1. Make BCI easy, convenient and fun to use for the most
popular consumer devices and apps.
Research Issues (2)
2. Energy budget analysis – Is it feasible to spend energy
preprocessing data at the BCI headset in order to reduce the
amount of transmitted data and save energy from the
communication link.
3. Improve the signal to noise ration (SNR) for dry sensor
electrodes.
4. Deliver EEG signals with medical systems quality for
consumer applications and low price. New signal processing
methods and algorithms.
Research Issues (3)
5. Currently, research is only beginning to crack the electrical
information encoding the information in a human subject's
thoughts.
Understanding this “neural code” can have significant impact in
augmenting function for those with various forms of motor
disabilities.
Conclusions
1. BCI research has tremendous implications to the field of
medecine.
2. Lot’s of on-going active research driven by the enormous
interest.
3. Consumer Electronics companies such as Emotive and
Neurosky are coming up with user friendly headsets.
4. New areas of application outside of medecine – gaming,
control, education.
Thank you!
Q&A
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
“Adaptation of Hybrid Human-Computer Interaction Systems using EEG Error-Related Potentials”,
Ricardo Chavarriaga, Andrea Biasiucci, Killian F¨orster, Daniel Roggen, Gerhard Tr¨oster and Jos´e
del R. Mill´an
“Translating Thoughts Into Actions by Finding Patterns in Brainwaves”, Charles W. Anderson and
Jeshua A. Bratman
“Towards Emotion Recognition from Electroencephalographic Signals”, Kristina Schaaff and Tanja
Schultz
“A user-friendly SSVEP-based brain–computer interface using a time-domain classifier”, An Luo and
Thomas J Sullivan
Pfurtscheller G., et al., 1993, Brain Computer Interface a new communication device for
handicapped people. Journal of Microcomputer Applications, 16:293-299.
Neuper, C. et al., 1999. Motor imagery and ERD. Related Desyncronization, Handbook of
Electroencepalography and Clinical Neurophysiology Vol. 6. Elsevier, Amsterdam, pp. 303-525.
Grabner, R. H., Stern, E., & Neubauer, A. C. (2003). When intelligence loses is impact: neural
efficiency during reasoning in a familiar area. International Journal of Psychophysiology, 49, 89-98.
Brain-Computer Interfaces, Fabien Huske, Markus A. Kollotzek, Alexander Behm
EEG/MEG: Experimental Design & Preprocessing, Lena Kastner, Thomas Ditye
Classic EEG (ERP) / Advanced EEG, Quentin Noirhomme
The ElectroEncephaloGram, Cognitive Neuropsychology, January 16th, 2001
Brain-Computer Interface, Overview, methods and opportunities, Emtiyaz (Emt), CS, UBC
The emerging world of motor neuroprosthetics: a neurosurgical perspective, Neurosurgery. 2006 Jul;
59(1):1-14.
Workshop on direct brain/computer interface & control, Febo Cincotti, August 2006
List of Abbreviations
•
•
•
•
•
•
•
•
•
•
•
•
•
EEG – Electroencephalography
ERP – Event Related Potential
ERS – Event-Related Synchronization
ERD – Event-Related Desynchronization
HCI – Human Computer Interface
BCI – Brain Computer interface
VEP – Visual Evoked Potential
SSVEP – Steady-State Visual Evoked Potential
P300 – An ERP signal type
ECG – Electrocardiography
EMG – Electromyography
fMRI – Functional Magnetic Resonance Imaging
MEG – Magnetoencephalogy