Download A quick tour of the auditory system

A quick tour of the auditory
system
Nima Mesgarani, UCSF
The job of auditory system
• Task: take in lots of sound pressure waves,
process the signals and extract information
(what, who, where, how, . . .)
• The sound maybe mixed with lots of other
signals (e.g. cocktail party problem)
– looking at ocean waves, estimate number
of ships, their shapes, etc.
Auditory pathway
3
Auditory pathway
Cortex
Thalamus
Midbrain
Binaural Nuclei
Cochlear Nucleus
4
Number of cells in the auditory nuclei on the monkey
(20.000 hair cells in cochlea excluded )
Tobias 1972
Apparatus for hearing and balance
External ear
focuses and
filters the
sound
Motion of basilar membrane
• Vibration of tympanic
membrane
• stapes convey vibration
through oval window: air
pressure to fluid pressure
• Two fluid filled
compartments separated
by basilar membrane
• Motion causes the waves
7
Cochlear: mechanical frequency
analyzer
• Basilar membrane is wide
and stiff at base by oval
window, and narrow and
less stiff at apex
• Mechanical properties cause
selective amplification of
waves of high and low
frequencies at particular
places along membrane
• Preserved along auditory
pathway, provide tonotopy
mapping, a place code for
frequency
Cochlear anatomy
• Basilar membrane
below, tectorial
membrane above, hair
cells between
• 1 row Inner, 3 rows
outer
• Inner hair cells
transduce vibration
into electrical signal
• Outer hair cells
receive signals from
the brain and
transduce it to
mechanical vibrations
Mechanical stimulus into electrical signals
• Basilar membrane vibrates
up and down with sound
wave, causing shearing
motion by tectorial
membrane - hair bundles
are deflected
• Bending of hair cell back
and forth: excitation and
inhibition
10
Inner hair cells
• Vibration of hair cells modulates the ion channels and produce electrical
signals
Action potentials
• Electrical signals that convey information from
one cell to another
• Fixed amplitude and shape.
12
How action potential is generated
Mechano-electrical transduction of the
hair cells
• Displacement of the
bundle in the positive
direction increases the
tension in the gating
spring
• Promotes channel
opening and the influx of
cations: depolarizing
receptor potential
Outer hair cells
• Outer hair cells transduce
electrical signals to
mechanical vibrations
• Ear is not passive:
– Amplify the sound,
increased sensitivity
– Sharpens the frequency
resolution
16
Cochlear implant
• Stimulate the auditory nerve directly
17
Innervation of Organ of Corti
• Afferent: bottom-up
• Inner hair cells: 10:1
innervation ratio of
single hair cell
• Outer hair cells: few
• efferent innervation:
top-down
– most to outer hair
cells
Tuning curves for hair cells
• Present sound - record
from single fibers
• Present increasing sound
pressure level and count
number of spikes to each
• Produce curves of “best
frequency” for each fiber
• Tonotopic map mostly
based on mechanics of
membrane
19
Coding of the stimulus intensity
threshold of firing
sound level [dB]
Bandwidths of tuning curves increase with frequency
(frequency resolution decreases with frequency)
Frequency selec,vity decreases with amplitude
firing
rate
80 dB
60 dB 40 dB
20 dB
frequency
22
high
threshold
threshold
firing rate
intensity
Intracellular voltage changes in an inner hair cell
for different frequencies
24
Phase-locked response to stimulus
1 kHz
1.05 ms
0.97 ms
4 kHz
No synchrony above 5 kHz
Auditory pathway
Cortex
Thalamus
Midbrain
Binaural Nuclei
Cochlear Nucleus
26
Types of cells in cochlear nucleus
All these cell-types may originate from a single auditory
nerve input. This is divergence of the signal.!
27
Non-­‐monotonic rate-­‐intensity func,ons
cochlear nucleus
auditory nerve (from cochlea)
Types of cells in Primary Auditory Cortex
Auditory cortex (non-­‐monotonic rate-­‐intensity func,on)
• 90 % of rate-­‐intensity func,ons in auditory cortex are non-­‐monotonic A simple cor,cal tuning curve
• tuning curve measured
by response to pure
tones at a given
frequency
• this is a “simple” curve
(a single peak)
– more complex (multipeak) curves also exist
Central Auditory pathway
• Extends from the cohlear
nucleus to the auditory
cortex
• Several stops before the
cortex
• Sound localization
• Primary auditory cortex is
the first stop of sound in
the cortex
Spectrotemporal receptive field (STRF)
A simple way to
characterize the
function of cortical
neurons
STRF
Frequency
•
H(t,f)
STRF: S(t,f)
8
1
0
0.5
r(t)
-1
0
Time
0.2
r̂n ( t ) = # # H n ( t ! " , f ) S (" , f )
"
!
e = " ( r̂n ( t ) ! rn ( t ))
t
H n = C !1
SS CS rn
!
2
Spectrogram
Frequency
f
8
0.5
0
Time
1
Variety of STRF tunings
Neurons in Primary Auditory Cortex show a variety of
tuning properties: direction, temporal and spectral
modulations
(scale)
(rate)
(direction)
8
Frequency (KHz)
•
0.25
8
0.25
8
0.25
8
0.25
0
0.25
0
0.25
Time (s)
0
0.25
Selectivity of neural responses
•
Frequency (KHz)
Variety of tuning properties results in selective neural
responses to different phonemes in continuous speech
8
0.25
Frequency (KHz)
/s/
8
0.2
0
Neuron 2
0.2
0.25
8
/t/
0.25
Neuron 1
0.25
8
/aa/
0
Neuron 3
0.3
0
0
0.25
Time (s)
0
0.2
0
0.2
0
Time (s)
0.2
Alternative representations: Scalegram
Frequency
narrow BW
0.5
2.14
0.75
wide BW
0.37
0.18
k
l
ae
sh tclt ix n
iy
ow
pax n
s
t r
iy dx ix
n
Frequency
Scale (Cyc/Oct)
Scalegram
8
8
0.5
0
0.25
Time (s)
Alternative representations: Rategram
Fast
Frequency
0.5
Slow
k
l
ae
sh
l t ix n
iy
ow
pax n
s
t r
iy dx ix
n
Frequency
Rate (Hz)
Rategram
32
16
8
4
2
8
8
0.5
Frequency
32
16
8
4
2
k
l
ae
sh
l t ix n
iy
ow
pax n
s
t r
iy dx ix
n
Frequency
Rate (Hz)
0
Time (s)
0.25
8
0.5
8
0.5
0
Time (s) 0.25
8
Frequency
Up-­down sweeps
2
0.5
8
0.5
up-FM
0
Time (s)
0.25
down-FM
k
l
ae
sh
lt ix n
iy
ow
pax n
s
t r iy dx ix
n
Frequency
Frequency
(KHz)
Alternative representations:
Sweep direction
8
0.5
0
Time (s) 0.25
Auditory cortex
39
Higher areas of auditory
pathway
re
o
C t
l
e
B elt
ab
r
Pa
40
Thalamic projections within
auditory cortex
• Projections to and from all these areas
41
Speech representation in STG: categorical
• Representation does not linearly reflect the acoustic
parameters, but their percept
Chang et. al. 2010
Spectrogram reconstruction from neural responses
Electrodes
R(t)
0
Frequency
Superior temporal gyrus
(STG)
8
0
Time (s)
1
G(t,f)
Inverse: R(t)
Spectrogram S(t,f)
0.1
Time
1
S(t,f)
Reconstructed Ŝ(t,f)
0.1
0
Time (s)
1
(Pasley 2012, Mesgarani 2009)
Speech representation in STG: modulated
with attention
Acoustic Spectrogram: Single Speaker
SP2: ready Ringo go to Red Two
8
Frequency (KHz)
SP1: ready Tiger go toGreen Five now
8
now
0.1
0.1
Frequency (KHz)
Neural Reconstruction: Single Speaker
8
0.1
8
0
2
0.1
0
2
Frequency (KHz)
Neural Reconstruction: ATTENDED Multi-Speaker
8
0.1
8
0
Time (s)
2
0.1
0
Time (s)
2
(Mesgarani et. al., Nature 2012)
From ear to brain
From brain to ear
(about 10% of connections)
(about 90% of connections)
45