Download Mental State Sensing and the Goal of Circuit - Synapse Synergy

Mental State Sensing and the
Goal of Circuit-Synapse Synergy
Patrick L. Craven, Ph.D.
Senior Member, Engineering Staff
Advanced Technology Laboratories
Cherry Hill, NJ
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
Goals of Artificial Intelligence (AI)
• Replicate human-like intelligence and cognition?
• Media portrays this technology in different ways
– Technology is Evil
– 2001: A Space Odyssey (1968)
– The Terminator (1984)
– Technology is Good
– The Terminator II (1991)
– Short Circuit (1986),
– “Number 5 is Alive”
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 2
Human-Machine Synergy
• Complement and enhance
human cognition
– Human cognition is
different than computer
processing
– Brain anatomy provides
us with powerful
computation
• Combining human and
machine is advantageous
– Firefox (1982)
– The Matrix (1999)
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 3
The Brain
• Large
– Neurons = 100,000,000,000!
– Neural Connections =
100,000,000,000,000+ !!!
• Electrochemical transmission
– Axons
– Dendrites
– Neurotransmitters
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 4
How do we measure brain activity?
•
•
•
•
Electrical activity: EEG
Bloodflow: fMRI, fNIR
Skin response: GSR
Heart Rate: EKG
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 5
Collecting EEG
• Measures electrical potential (µv microvolts) of cortex
• 10-20 electrode placement
– Even numbering: right
hemisphere
– Odd numbering : left
hemisphere
• Measured from nasion to inion
(occipital bone)
• Voltage between two points
(ground or another sensor site)
– Referential
– Differential
• Does not measure neuronal
current (amps)
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 6
Types of EEG waves
• Delta: up to 3 Hz. Slow wave sleep
• Theta: 4 to 7 Hz. Seen normally in young children. Seen also in drowsy adults
• Alpha: 8 to 12 Hz. Observed in adults when awake and relaxed
• Beta: 12 to ~30 Hz. Associated with active, busy, or anxious thinking
• Gamma: 26+Hz. Not easily recordable with EEG. May indicate neuronal binding
for common purpose
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 7
Uses of EEG
• Detecting abnormality
– Head/brain Trauma
– Seizure
– Sleep problems
• Applied EEG
– Use time-domain or
frequency-domain profiles
to convert signal into a
gauge of ‘mental states’
(drowsiness, cognitive
workload)
– Create closed-loop
systems where mental
activity influences software
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
PSD Bins
1
4
7
10 13
16 19 22 25
28 31 34
37 40
Hz
10/31/2007 8
Current Technology - Applied EEG
EEG
Eyetracking
EKG
• Available cognitive state gauges
– Cognitive workload
– Visual engagement
– Distraction
– Drowsiness
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
GSR
10/31/2007 9
Augmented Cognition
• Goal: Maximize operator cognitive performance
in dynamic, complex operational environments
• Approach: Physiological-data based
assessment of operator cognitive state
– Detects, predicts, avoids overload to reduce
operator error and maximize effectiveness
User
C2
System
Sensors
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
Cognitive
State
Assessor
• Benefit: Mitigate
negative effects of
cognitive overload
– Increase task
speed and
accuracy
– Improve critical
situation
understanding
10/31/2007 10
Education and Training
• Goal: Maximize effectiveness of training by customization based
upon individual cognitive capacities and current cognitive state
• Approach: Monitor trainees’ physiological data during training
sessions
– Identify and apply most efficacious training strategies
User
Training
application
CSS
Training
Monitor
Sensors
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
• Benefits:
– Helps identify potential
design problems
– distinguishes
workload-induced
and confusioninduced errors
– Provides objective,
real-time, taskindependent workload
measures
10/31/2007 11
Next Steps
• Sensors: smaller, faster, more
accurate, cheaper, easier to
apply
• Processing: quicker, cheaper,
more portable
• Algorithms: more efficient,
more accurate
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 12
Challenges Remaining
• Technical
– Gain a greater understanding of the
tremendous individual variation
• Ethical
– Should we be doing this?
• Practical
– What information can it provide that we
humans can’s share via actions or words?
Copyright © 2007 by Lockheed Martin Corporation. All rights reserved.
10/31/2007 13