Download Auditory fMRI correlates of loudness perception for

Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
PROCEEDINGS of the 22nd International Congress on Acoustics
Psychological and Physiological Acoustics (others):
Paper ICA2016-435
Auditory fMRI correlates of loudness perception for
monaural and diotic stimulation
Stefan Uppenkamp(a), Oliver Behler(b)
Medizinische Physik, Universität Oldenburg, Germany
(a)
[email protected]
(b)
[email protected]
Abstract
Loudness as the perceptual correlate of sound intensity is formed by some neural processing
along the auditory pathway from the cochlea to the cortex. The loudness of a sound is largely
determined by its level. Still, there are several other acoustical factors like stimulus bandwidth,
duration, modulations, as well as personal factors like, e.g., the individual hearing status, that
may affect perceived loudness. Binaural loudness summation refers to the finding that a
binaural sound is perceived as louder than the same sound presented monaurally at the same
level. Some hearing impaired listeners show an increased binaural loudness summation for
broadband stimuli. The physiological background for this effect is not yet clear. We report an
auditory functional MRI study comparing results from normal hearing and hearing-impaired
listeners for monaural and diotic stimuli presented at different intensities. All listeners completed
a categorical loudness scaling procedure, to allow for an analysis of the auditory fMRI data with
respect to both, physical sound intensity as well as individual loudness perception. The results
indicate systematic differences across the different stages of the auditory pathway, when
comparing level and loudness-related brain activation. The brain activation is systematically
increasing with sound level at all stages from brainstem to cortex. Specific effects related to
individual loudness and loudness summation can be demonstrated in primary auditory cortex
and in auditory association areas in Planum temporale, while the activation in more lateral
regions on the first transverse temporal gyrus (Heschl) in cortex as well as in auditory brainstem
structures appears to be less specific for individual loudness judgements.
Keywords: Auditory fMRI, Loudness, Binaural Loudness Summation, Hearing impairment
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
Auditory fMRI correlates of loudness perception for
monaural and diotic stimulation
1 Introduction
The psychoacoustic feature of loudness plays an important role for auditory perception. The
loudness of a sound is largely determined by the stimulus intensity, typically expressed as a
sound pressure level. However, the relationship between intensity and loudness is also affected
by the bandwidth of the sound, its duration, any modulation, and possibly more physical
parameters. Moreover, there are non-acoustic factors that influence how listeners answer the
simple question “How loud is this sound?” These include the precise details of the procedure
employed to gather the loudness judgements, context effects, the individual’s hearing status,
and even non-auditory factors like, for example, personality.
Physiological investigations have assembled a comprehensive characterization of the sensory
coding of intensity in the periphery of the auditory system, including the cochlea, auditory nerve
and auditory brainstem [1]. The transformation of sensation into perception at the cortical level,
however, is much less well understood. Auditory functional MRI can help to identify the neural
correlates of loudness perception and the influence of non-acoustic factors in perception.
Binaural loudness summation refers to the finding that a binaural sound is perceived as louder
than the same sound presented monaurally at the same level. Some hearing impaired listeners
show an increased binaural loudness summation for broadband stimuli [2]. The physiological
background for this effect is not yet clear. In the first part of the current paper, results from
normal hearing listeners on auditory fMRI correlates of loudness perception for unilateral
stimulation are summarized. The second part of the current paper describes an ongoing study
in which results from a group of normal hearing listeners are compared with results from
participants with hearing impairment, to try and identify cortical correlates of increased binaural
loudness summation.
2 Results for unilateral stimulation
The interrelation of sound pressure level, ear of entry, individual loudness judgements, and
fMRI activation maps along different stages of the central auditory system and across
hemispheres has been investigated in detail for a group of 14 normal hearing listeners [3]. The
stimuli employed were bandpass filtered noise stimuli at a center frequency of 4 kHz. They were
presented monaurally to each ear with MR compatible insert earphones (Sensimetrics S14) at
levels from 37 to 97 dB SPL in steps of 15 dB, making a total of 10 unilateral sound conditions.
One diotic condition at 82 dB SPL and a silence condition were included as control conditions.
More details of the methods are provided elsewhere [3].
2
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
2.1
Loudness judgements
The participants completed a categorical loudness scaling procedure with similar stimuli prior to
the auditory fMRI measurement. Loudness scaling results were pretty much as expected, with
an approximately linear relationship between sound pressure level and perceived loudness for
levels up to about 80 dB, and a steeper growth of loudness with level at higher intensities.
Binaural loudness summation emerged as an effect of 3 dB relative to monaural stimulation,
that is, a binaural stimulus was scored as having the same loudness, when the level was
reduced by 3 dB relative to unilateral stimulation.
2.2
General effect of sound intensity
The relationship between brain activity, as inferred from blood oxygenation level dependent
(BOLD) contrasts, and both sound level and loudness estimates were analysed by means of
functional activation maps and statistical linear mixed effects models for 12 anatomically defined
regions of interest (ROI) in the ascending auditory pathway and in the cortex. The regions
included - pairwise for left and right hemisphere - inferior colliculus in the brainstem, medial
geniculate body in thalamus, and the posterior medial, central and anterolateral sections of the
first transverse temporal gyrus (Heschl’s gyrus, HG) as well as one central region in Planum
temporale. This data analysis approach allowed us to characterize the neural representation of
physical sound intensity and its perceptual correlate (i.e., sound pressure level and categorical
loudness) in great detail.
As expected, fMRI activation increased with sound intensity throughout all investigated stages
of the auditory system, i.e., brainstem, thalamic and cortical regions. In addition, a clear effect of
the ear of entry was found for nearly all investigated regions. Activation magnitudes, in general,
grew more strongly with intensity and were more closely related to intensity changes (in terms of
explained variance) for contralateral as opposed to ipsilateral stimulation.
2.3
Differences between cortical and subcortical structures
The ROI analysis also revealed distinct and systematic differences between the regions. At
cortical level, we found a similar pattern in both hemispheres with respect to the strength of the
relationship between responses and sound intensity. Specifically, we observed the steepest
growth of BOLD signal with stimulus intensity and the highest R2 values (representing the
explained variance in the data) in the posterior medial sections of the first left and right Heschl’s
gyri (HGpm), constituting a part of primary auditory cortex. In both respects, the responseintensity-relationship gradually declined along HG towards the anterolateral sections (HGal),
which were only little affected by sound intensity, especially in the left hemisphere. They were
also much less intensity-dependent than the regions in Planum temporale (PT), whose
response behavior was similar to that in the central parts of HG (HGc).
The response-intensity relationship was considerably less pronounced in the IC and the MGB
(especially for ipsilateral stimulation). This observation is in line with several previous fMRI
studies showing weaker sound induced BOLD responses and smaller increases with sound
level in these regions as compared to AC [4, 5].
3
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
2.4
Effect of ear of entry
Our data also revealed systematic differences between subcortical and cortical regions with
respect to the effect of ear of entry and its interrelation with sound intensity. These manifested
in bigger differences in the explained variance between contra- and ipsilateral model fits for the
IC and MGB as compared to cortex. In contrast, BOLD responses increased significantly as a
function of sound intensity for contra- as well as ipsilateral stimuli in all of the investigated
cortical regions. Especially in the posterior medial section of HG, changes in intensity were
highly predictive of activation magnitudes irrespective of the ear of entry.
2.5
Level vs. loudness
As sound pressure level and perceived loudness are closely related, similar patterns are
expected across regions and hemispheres when analysing the relationship between BOLD
responses and individual loudness estimates. Still, one notable distinction between the neural
representations of physical sound intensity and perceived loudness can be identified. Increasing
sound pressure levels are reflected by a nonlinear growth of activation magnitude with a
significant quadratic component, at least in cortical regions. In contrast, the relation between
activation and categorical loudness can be described as predominantly linear in all investigated
regions [3].
2.6
Summary for normal hearing listeners
Our findings for normal hearing listeners are overall in line with the notion that fMRI activation in
several regions within auditory cortex as well as in certain stages of the ascending auditory
pathway might be more a direct linear reflection of perceived loudness rather than of sound
pressure level [6, 4]. The results indicate distinct functional differences between subcortical and
cortical areas as well as between specific regions within auditory cortex, suggesting a
systematic hierarchy in terms of lateralization and the representation of level and loudness.
3 Binaural loudness summation
3.1
Motivation
The functional differences described above between subcortical and cortical areas as well as
between specific regions within auditory cortex suggest a different role of each structure in
completing loudness perception. The general loudness judgement is uniform in the sense that it
is combining acoustic input from both ears. The question then arises, at what stages in the
auditory pathway neural activation from the involved anatomical structures is integrated. One
approach to investigate this further is the comparison of activation maps for unilateral and
bilateral acoustic stimuli from normal hearing participants with those from hearing impaired
participants, who typically experience a change in their loudness perception. For example,
some hearing impaired listeners show an increased binaural loudness summation for
broadband stimuli [2]. As mentioned above, the physiological background for this effect is not
yet clear. This is the topic of this section. Pilot data from six additional participants - two of them
with hearing loss and exhibiting an increased binaural loudness summation - are presented.
4
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
3.2
Methods
All sounds were presented via an MRI compatible, opto-acoustic headphone system capable of
providing a wide frequency response. The MRI experiments were carried out using a 3-Tesla
MRI scanner Siemens Prisma. The stimulus used in this experiment was a broadband pinknoise stimulus presented either to the left ear, to the right ear, or to both ears simultaneously.
Categorical loudness scaling was employed to find the individual level that is needed to induce
the judgement of 35 categorical units for diotic stimulation, corresponding to the verbal category
“loud”. The data analysis procedure was similar to the one used before, that is, a region-ofinterest analysis for six distinct auditory regions in each hemisphere (see Fig. 2). For each ROI
in each hemisphere, the percentage change of the fMRI signal caused by the BOLD effect
(blood oxygen level dependent response) is analysed in relation to individual loudness
judgements for all three stimulus conditions, monaural to the left, monaural to the right, and
diotic stimulation.
Figure 1: Example loudness curves for one normal hearing listener (left panel, VP04) and one
participant with a hearing loss (right panel, VP07). The blue and red curves give the levelloudness functions for unilateral stimulation to the left resp. right ear. The green curves represent
the loudness curves for binaural stimulation.
3.3
Results
3.3.1 Categorical loudness scaling
Figure 1 shows results from two individuals for the relationship between sound pressure level
and perceived loudness, as determined by means of categorical loudness scaling for the
broadband pink-noise stimulus employed in this study. The left panel shows results from one
typical normal hearing listener, the right panel one example for a participant with hearing loss.
The results from the hearing-impaired listener are different in three respects: (1) the lowest
verbal category “very soft” (5 CU) is only reached at levels of about 40 dB SPL, reflecting the
increased detection threshold; (2) the slope of the loudness functions is generally steeper than
for normal hearing listeners, reflecting loudness recruitment; (3) the difference in perceived
loudness at same level between monaural and diotic sound presentation is considerably
5
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
increased for this particular listener, i.e., increased binaural loudness summation. The question
now arises whether auditory fMRI can be used to identify a neural correlate of this effect.
3.3.2 Definition of regions of interest
The two top panels in Figure 2 A show one example for a brain activation map created from
auditory fMRI data. The images represent slices through a statistical map of t-values for the
significance of a contrast between the sound condition with diotic presentation of pink noise at a
level corresponding to the individual loudness of 35 categorical units and a baseline condition
(no acoustic stimulus), superimposed on the individual anatomical MRI scan, for one normal
hearing participant from this study. The t-value is colour coded in brightness from red to white,
with a threshold corresponding to a significance level of p < 0.05. The position of these two
coronal slices are marked in the anatomical image in sagittal view in panel B, with the more
posterior slice cutting through the position of cochlear nucleus and inferior colliculus in the
auditory brainstem as well as parts of auditory cortex in the temporal lobes, and the more
anterior slice covering the position of medial geniculate bodies in thalamus and also large
portions of auditory cortex.
Figure 2. A: activation map for the contrast diotic sound presentation vs. silence, in a diotic
stimulus condition, for one normal hearing participant. Circles indicate the subcortical regions of
interest for data analysis: CN = cochlear nucleus (not analysed in the current paper), IC = inferior
colliculus, MGB = medial geniculate body. B: sagittal view of the anatomical scan visualising the
position of the coronal slices shown in A; C: circles mark the regions of interest used for a
detailed analysis of the cortical activation data.
Contrasting sound presentation with the silent baseline condition results in a distinct activation
pattern covering nearly all expected structures in the auditory pathway, both in subcortical
regions (CN, IC, MGB), as well as in all parts of auditory cortex covering the surface of both
6
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
temporal lobes, including primary auditory cortex in the posterior medial and central part of
Heschl’s gyrus, and also auditory association areas in surrounding regions. For diotic
stimulation, the activation map appears pretty symmetric across hemispheres, as expected.
Panel C in Fig. 2 shows an axial slice through the anatomical scan of the same participant, with
circles marking the regions of interest in auditory cortex that were analysed in the current study
in more detail, along with IC and MGB, as described before in section 2.
3.3.3 Relation between fMRI signal and loudness
For the combination of the brain activation obtained from the left and the right hemisphere to a
unified single loudness percept, two different, but rather simple models were put to the test. The
first option is that the overall loudness percept is dominated by the brain activation in that
hemisphere that shows the largest percentage change of the BOLD signal. The second option
is a mechanism that is somewhat integrating activation across both hemispheres. In this case
we would expect the perceived loudness to be correlated to the sum of the percentage changes
of the BOLD signal from both hemispheres. These two options have been tested using linear
mixed effects models with the pilot data from the six participants obtained so far. The idea of
this approach is to find the best (i.e. most likely) common linear trend across participants, while
accounting for differences across participants in the absolute BOLD signal strength. The results
of this analysis are depicted in Figure 3, exemplary for two of the investigated regions of
interest, inferior colliculus and the central part of Heschl’s gyrus, being part of primary auditory
cortex. A summary of the performance of the two competing models for all investigated regions
is given in Table 1, in terms of explained variance in the data.
The results in Figure 3 show, in line with previous findings, that the BOLD signal change in HG
shows overall a closer relationship with the individual perceived loudness than the signal
obtained from IC in the auditory brainstem. Also, a linear mixed effects model for the sum of the
BOLD signals from both hemispheres (panel B) outperforms a model based on the maximum of
the BOLD signals across hemispheres (panel A). This suggests that the overall loudness
percept is a correlate of the neural activation integrated across both hemispheres, rather than
just representing the dominant hemisphere, in terms of neural activation, irrespective of the
specific ear of entry. This observation is valid for all investigated regions of interest, as reflected
by the difference in statistical power (in terms of explained variance) of the two alternative
model approaches listed in Table 1. At the same time, however, the current pilot data are not
sufficient yet to detect systematic differences between normal hearing and hearing impaired
listeners, as only two hearing impaired listeners could be examined so far.
2
Table 1: Explained variance, given as R m [7], for the relationship between BOLD signal change
and individual loudness judgements, for all investigated regions of interest. Significant relationships are marked with an asterisk. (A) variance explained by the fit with the maximum in BOLD
signal change across hemispheres; (B) variance explained by the fit with the sum of the BOLD
signal changes for both hemispheres
ROI
option (A)
option (B)
IC
0.07
0.11
MGB
0.03
0.04
HGpm
0.25
0.32*
HGc
0.13
0.32*
HGal
0.28
0.33*
PT
0.01
0.24*
7
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
IC
HGc
IC
HGc
A
B
Figure 3: Linear mixed effects model for the relationship between percentage signal change of the
BOLD signal obtained with fMRI and the perceived loudness, for two regions of interest, inferior
colliculus (IC) and central part of Heschl’s gyrus (HGc). Numbers indicate the listeners in this pilot
study, colours indicate the three stimulus conditions. Listeners 6 and 7 were hearing impaired
and showed increased binaural loudness summation. Panels A and B show the results for the two
different model approaches (see text for details).
4 Conclusions
In summary, the presented data demonstrate a large potential of auditory fMRI to investigate
neural correlates of individual loudness perception. The neural activation – here represented
indirectly by the BOLD response – in the central auditory system integrated across hemispheres
forms the basis for the overall loudness judgement. Although the current pilot data are not
sufficient yet to reveal systematic differences between a group of normal hearing listeners and
hearing-impaired individuals, it can still be concluded that auditory fMRI may serve as an
important tool contributing to an individualised audiological diagnostics.
8
nd
22 International Congress on Acoustics, ICA 2016
Buenos Aires – 5 to 9 September, 2016
st
Acoustics for the 21 Century…
Acknowledgments
Oliver Behler is supported by a fellowship from the PhD programme “Signals and Cognition”,
MWK Niedersachsen.
References
[1] Pickles, J. O. An Introduction to the Physiology of Hearing, 4th edition, 2012. London: Emerald.
[2] Oetting, D.; Hohmann, V.; Appell, J.E.; Kollmeier, B.; Ewert, S.D. Spectral and binaural loudness
summation for hearing-impaired listeners. Hearing Research, 335, 2016, pp. 179-192.
[3] Behler, O.; Uppenkamp, S. The representation of level and loudness in the central auditory system
for unilateral stimulation. NeuroImage, 2016, in press.
[4] Röhl, M.; Uppenkamp, S. Neural coding of sound intensity and loudness in the human auditory
system. Journal of the Association for Research in Otolaryngology, 13, 2012, pp. 369-379.
[5] Boyen, K.; de Kleine. E.; van Dijk, P.; Langers, D.R.M. Tinnitus-related dissociation between cortical
and subcortical neural activity in humans with mild to moderate sensorineural hearing loss. Hearing
Research, 312, 2014, pp. 48-59.
[6] Langers, D.R.M.; van Dijk, P.; Schoenmaker, E.S.; Backes, W.H. fMRI activation in relation to sound
intensity and loudness. NeuroImage, 35(2), 2007, pp. 709-718.
2
[7] Nakagawa, S.; Schielzeth, H. A general and simple method for obtaining R from generalized linear
mixed‐effects models. Methods in Ecology and Evolution, 4(2), 2013, pp. 133-142.
9