Download Music Recognition, Music Listening, and Word

Music Recognition, Music Listening, and Word
Recognition by Deaf Children with Cochlear Implants
Chisato Mitani, Takayuki Nakata, Sandra E. Trehub, Yukihiko Kanda,
Hidetaka Kumagami, Kenji Takasaki, Ikue Miyamoto, and Haruo Takahashi
Objectives: To examine the ability of congenitally
deaf children to recognize music from incidental
exposure and the relations among age at implantation, music listening, and word recognition.
they would have experienced with normal hearing or
with hearing aids. Congenitally deaf children who
received an implant at a young age would not
experience an adult’s sense of loss with respect to
music, but their musical input would be similarly
impoverished.
Child implant users, unlike their adult counterparts, are involved in a variety of musical activities,
including listening, singing, dancing, and instrument
lessons (Gfeller et al., 1998; Nakata et al., 2005b;
Vongpaisal et al., 2006), which implies that their
musical experiences and skills differ from those of
adults. Vongpaisal et al. (2006) evaluated implanted
children’s and adolescents’ ability to identify popular
songs in a closed-set task. They found that children
and teens, all of whom were congenitally or prelingually deaf, could identify the songs based on excerpts
from the original and instrumental recordings. Nevertheless, these listeners failed to recognize synthesized
piano versions of the main melody that younger hearing children could recognize.
Familiarity with popular songs often arises from
deliberate listening by means of portable digital devices (e.g., MP3 players), stereo systems, or radios. For
the most part, however, musical exposure is incidental
rather than deliberate, arising from the presence of
music in a variety of contexts such as television,
movies, computer games, and public spaces. Nevertheless, incidental musical exposure often results in the
ability to recognize or reproduce the themes or background music from movies or television programs
(Nakata et al., 2005b; Schellenberg & Trehub, 2003).
Comparable incidental exposure to music may have
very different consequences for deaf implant users.
To examine the consequences of incidental music
listening, Nakata et al. (2005b) evaluated the ability
of children with implants to recognize the theme
songs from familiar TV programs. Congenitally deaf
children 4 to 9 yr of age successfully identified
excerpts from the original recordings, but they were
unable to recognize excerpts from original instrumental versions without the lyrics. In the present
study, we sought to replicate and extend those
findings. In addition to evaluating the recognition of
music from popular children’s television programs
with a larger sample of children with implants, we
sought to examine relations among other factors
Design: Seventeen child implant users who were 4
to 8 yr of age were tested on their recognition and
liking of musical excerpts from their favorite television programs. They were also assessed on openset recognition of three-syllable words. Their parents completed a questionnaire about the children’s
musical activities.
Results: Children identified the musical excerpts at
better than chance levels, but only when they heard
the original vocal/instrumental versions. Children’s
initiation of music listening at home was associated
with younger ages at implantation and higher word
recognition scores.
Conclusions: Child implant users enjoy music more
than adult implant users. Moreover, younger age at
implantation increases children’s engagement with
music, which may enhance their progress in other
auditory domains.
(Ear & Hearing 2007;28;29S–33S)
Cochlear implants enable many deaf individuals
to achieve improved speech perception and production skills relative to what is possible with hearing
aids. The situation is very different for music perception because of signal-processing strategies that
transmit insufficient information about pitch (Gfeller
et al., 2005 Wilson et al., 2005). One consequence is
that implant users, both adults (Fujita & Ito, 1999;
Gfeller et al., 2000b) and children (Stordahl, 2002;
Vongpaisal et al., 2006), cannot recognize a familiar
melody based its pitch patterning. Postlingually
deaf adults typically lose interest in music after
implantation (Gfeller et al., 2000; Leal et al., 2003),
even if they were music aficionados beforehand. Obviously, the musical input that they receive through
their implants differs radically from the music that
Graduate Course in Humanistic Studies Specialized Research
into Humanistic Studies, Department of Psychology, Nagasaki
Junshin Catholic University, Nagasaki, Japan (C.M., T.N.); Department of Psychology, University of Toronto at Mississauga,
Mississauga, Ontario, Canada (S.E.T.) and Division of Otorhinolaryngology, Department of Translational Medical Science, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan (Y.K., H.K., K.T., I.M., H.T.).
0196/0202/07/282 Supplement-0029S/0 • Ear & Hearing • Copyright © 2007 by Lippincott Williams & Wilkins • Printed in the U.S.A.
29S
30S
EAR & HEARING / APRIL 2007
such as age at implantation, at-home music listening, and word recognition. In general, younger ages
at implantation have been associated with better
language progress (Houston et al., 2003; Lin & Peng,
2003; Oh et al., 2003), but the impact of early
implantation on music perception, memory, and
enjoyment is unknown.
It is also unclear whether musical exposure and
training have transfer effects on the nonmusical
skills of child implant users, as they do for hearing
children. For example, young children who received
1 to 3 yr of piano or voice lessons as part of an
experimental treatment showed improvement in IQ
scores (Schellenberg, 2004), academic achievement
(Gardiner et al., 1996), or self-esteem (Costa-Giomi,
2004) compared to a matched control sample. Moreover, brief episodes of listening to age-appropriate
music generate short-term improvement on nonmusical tasks (Schellenberg et al., 2007). Comparable
consequences for child implant users would depend
on their ability to perceive, remember, and enjoy
music. Such abilities could create an important role
for music in aural habilitation.
METHODS
Subjects
The participants were 17 congenitally deaf children
who were 4 to 8 yr of age (mean ⫽ 6.5) at the time of
music-recognition testing and 4 to 9 yr (mean ⫽ 6.5) at
word-recognition testing. Before implantation, all children used bilateral hearing aids. Their surgery was at
2 to 7 yr of age (mean ⫽ 3.8), and their use of an
implant exceeded 1 yr (range, 1.2– 4.10). Children used
a cochlear nucleus implant with the ACE (n ⫽ 12) or
SPEAK (n ⫽ 3) coding strategy or an Advanced Bionics implant with the MPS (n ⫽ 1) or SAS (n ⫽ 1)
strategy. Twelve children used hearing aids in their
unimplanted ear. All children communicated exclusively by oral means. They attended regular classes in
community schools, except for one child who attended
an auditory/oral school for the deaf. All parents were
native speakers of Japanese.
Apparatus and Stimuli
Testing was conducted in a double-wall, soundattenuating booth. Music stimuli consisted of theme
songs from 14 popular television programs for children. There were three versions of the songs: (1)
original, which corresponded exactly to the version
that appeared on television; (2) instrumental, which
was identical to the original version except for the
absence of the sung portion; and (3) a synthesized
flute version of the main melody. The melody version preserved the timing of the original song and
TABLE 1. Words (with English translations) from the word
recognition task
Nezumi (mouse)
Sakana (fish)
Piano (piano)
Kemuri (smoke)
Megane (glasses)
Yaoya (vegetable store)
Tamago (egg)
Daruma (traditional good-luck doll)
Atama (head)
Gorira (gorilla)
Kitsune (fox)
Hiyoko (chick)
Kuruma (car)
Usagi (rabbit)
Ichigo (strawberry)
Suika (watermelon)
Maiku (microphone)
Banana (banana)
Karasu (crow)
Hanabi (fireworks)
Senaka (back)
Tsukue (desk)
Tomato (tomato)
Rakuda (camel)
Kyabetsu (cabbage)
corresponded to the portion that was sung. Musical
materials were presented by means of a laptop
computer (Macintosh iBook) and loudspeaker. Before testing began, the music was adjusted to a
comfortable listening level for each child.
The word recognition task consisted of 25 common, three-syllable words (Table 1). The materials
(CD for the Evaluation of Fitting Condition with
Hearing Aids, TY-89) are often used in Japan to
facilitate hearing-aid fitting (Yonemoto, 1995).
Procedure
Children were tested on music-recognition and
word-recognition tasks on different days. Before
music-recognition testing, the experimenter consulted with the children and their parents to determine the TV programs that each child watched
regularly. On that basis, we selected three to five
programs. Musical excerpts were presented in
blocks of trials, beginning with the original versions,
followed by the instrumental versions, and ending
with the melody versions. Each block had two trials
of the relevant version of each of the three to five
songs. Within each block of trials, excerpts were
presented in random order except that no song
occurred twice in succession.
Children were asked to pay close attention to each
song and to identify the program by clicking on one
of the three to five images on the computer monitor
that depicted characters (and titles) from each program. The experimenter presented the excerpts only
when children were attentive. Each excerpt started
at the beginning of the recording and continued until
the child responded or the excerpt ended (55– 65 sec).
Children were told to guess if they were unsure.
After their selection, they rated how much they liked
each excerpt on a 5-point unidimensional scale in
which 1 corresponded to “do not like at all” and 5
corresponded to “like very much.” Accompanying
EAR & HEARING, VOL. 28 NO. 2 SUPPLEMENT
depictions of ice cream cones clarified the meaning
of different points on the rating scale. Parents completed a questionnaire about children’s musical activities (e.g., singing, listening), affective responses
to music, and initiation of music listening.
On a separate occasion, before or after the musicrecognition task, children completed the word-recognition task as part of a routine clinical assessment. They
were tested without their hearing aid (i.e., cochlear
implant only), and the words were presented at 70 dB
SPL. Children were required to repeat each of 25
words presented in succession. Their responses were
scored as correct if they were intelligible reproductions, regardless of articulation proficiency.
RESULTS
Because children differed in the number of songs
on which they were tested (3, 4, or 5), performance
(proportion correct) was corrected for the proportion
of correct responses that would be expected by
chance (0.33, 0.25, and 0.20 in the cases of 3, 4, and
5 songs, respectively). The proportion corresponding
to chance responding was subtracted from the numerator (proportion of songs correctly identified)
and denominator (perfect performance). A score of 1
indicated perfect performance, 0 indicated performance expected by chance, and negative values
indicated performance below chance. Accuracy on
the original versions (mean ⫽ 0.346, SD ⫽ 0.44)
significantly exceeded chance levels, t(16) ⫽ 3.46,
p ⬍ 0.01, but performance on instrumental (mean ⫽
0.03, SD ⫽ 0.19) and melody (mean ⫽ ⫺0.05, SD ⫽
0.24) versions was at chance levels. Appraisals of the
excerpts were significantly more favorable than the
neutral midpoint (3) for original, instrumental, and
melody versions (mean ⫽ 3.46, 3.43, and 3.63, respectively, all p values ⬍0.05).
Performance on word recognition was highly variable, ranging from 32% correct for one child to 100%
for four others. Parent questionnaires revealed that
all children participated in musical activities. Most
sang spontaneously, but only nine initiated music
listening. The difference in age at implantation
between children who initiated music listening
(mean ⫽ 37.44 months, SD ⫽ 11.61) and those who
did not (mean ⫽ 52.63 months, SD ⫽ 19.41) approached conventional levels of significance [t(15) ⫽
1.99, p ⫽ 0.066]. In addition, music-listening initiators had significantly higher word-recognition scores
(mean ⫽ 89.33% correct, SD ⫽ 14.00) than noninitiators (mean ⫽ 70.63, SD ⫽ 19.54, t(15) ⫽ 2.29, p ⬍
0.05. This effect was not mediated by age at implantation or duration of implant use because word
recognition was uncorrelated with age at surgery or
implant experience.
31S
DISCUSSION
When congenitally deaf children with cochlear implants heard recorded music from their favorite television programs, they identified the music at better
than chance levels from a closed response set. Successful recognition occurred for renditions that incorporated all features of the original music—instruments,
timing, and vocals— but not for renditions that retained the instrumental portions without the vocals.
Children were also unable to identify synthesized flute
renditions that retained the pitch patterning and timing of the main melody (i.e., the portion that is sung)
but differed in other respects (i.e., flute rather than
voice and no accompaniment). Their difficulty recognizing songs from the melody alone is consistent with
previous reports by Stordahl (2002), Nakata et al.
(2006), and Vongpaisal et al. (2006).
The ability of child implant users to identify
instrumental renditions from deliberate listening
experience (i.e., focused attention to popular recordings), as in Vongpaisal et al. (2006), and their
inability to do so from incidental listening experience (i.e., while focusing on television programs)
highlight the differential impact of these exposure
contexts on hearing and hearing-impaired children.
For children with normal hearing, incidental listening or “overhearing” commonly results in successful
auditory learning, just as incidental observation
leads to successful visual learning. For implant
users, however, following the dialogue and narrative
thread of a television program takes a great deal of
effort, which diminishes the cognitive resources
available for learning and memory. Even “star”
adult implant users note the great concentration
that underlies their stellar performance on speech
perception tasks (Wilson et al., 2005).
Children’s favorable appraisals of the musical
excerpts are consistent with Vongpaisal et al.
(2006), but they contrast with the unfavorable appraisals reported by Stordahl (2002). Participants’
selection of TV programs for the current study and
pop songs for Vongpaisal et al. (2006) may account
for their positive appraisals. Although child implant
users in Vongpaisal et al. provided favorable ratings
for original and instrumental versions, they provided neutral ratings for the melody versions, which
implies that their recognition of the materials affected their appraisals. In the present study, however, all versions were rated equally favorably, regardless of children’s failure to recognize
instrumental and melody versions. In short, music
listening, even in the laboratory, was a positive
experience for the present participants. Perhaps
their young age (4 – 8 yr) compared to the implanted
32S
EAR & HEARING / APRIL 2007
listeners in Vongpaisal et al. (8 –18 yr of age) accounts for the differences.
Children’s positive attitudes toward music were
corroborated by parental questionnaires. Despite
the poor pitch resolution reported for implant users
(Fujita & Ito, 1999; Vandali et al., 2005; Wilson et
al., 2005), several children sang regularly and gleefully. From other research, however, it is clear that
the singing of children with implants is substantially out of tune (Nakata et al., 2006). Nine children
regularly initiated music listening. Moreover, musiclistening initiation was associated with younger ages
at implantation and with enhanced word recognition.
What can account for the association between age
at implantation and favorable inclinations toward
music? Presumably, some age-at-implant differences arise from listeners’ need to adapt to drastic
changes from the input experienced before the implant (via amplification) and the input experienced
after the implant (Fu & Shannon, 1999). As listeners
get older, their representations of sound patterns in
their environment become entrenched and resistant
to change (Johnson & Munakata, 2005). Early childhood seems to be a period of enhanced perceptual
flexibility. For example, hearing infants acquire
novel sound patterns more readily than adults do
(Hannon & Trehub, 2005; Kuhl et al., 2003). Ease of
auditory learning would promote greater interest in
music and greater familiarity with it.
What underlies the unexpected association between music listening and word recognition? Based
on the modest association between these factors in a
small sample of children, our proposals are necessarily speculative. Music listening, especially focused music listening, may involve the training of
auditory attention. In other words, learning that
occurs in the context of music listening may transfer
to other auditory but nonmusical contexts. For example, music lessons for individuals with normal
hearing are associated with enhanced verbal memory (Ho et al., 2003) and sensitivity to prosody
(Thompson et al., 2004). Regular music listening by
young cochlear implant users may provide incidental training of auditory attention, one consequence
of which is the promotion of word recognition and
verbal learning. Accordingly, the present study provides impetus for further exploration of the role of
music in aural habilitation.
ACKNOWLEDGMENTS
Funding for this study was provided by a grant from the Canadian Institutes for Health Research to Sandra E. Trehub. The
authors are grateful to the staff members of the Nagasaki Bell
Hearing Center for their continuing cooperation and to the
parents of the children who participated in this study.
Address for correspondence: Dr. Sandra E. Trehub, Department
of Psychology, University of Toronto at Mississauga, Mississauga,
Ontario, Canada, L5L 1C6. E-mail: [email protected].
Received December 17, 2005; accepted May 1, 2006.
REFERENCES
Costa-Giomi, E. (2004). Effects of three years of piano instruction
on children’s academic achievement, school performance and
self-esteem. Psychology of Music, 32, 139–152.
Fu, Q.-J., Shannon, R. V. (1999). Recognition of spectrally degraded and frequency-shifted vowels in acoustic and electric
hearing. Journal of the Acoustical Society of America, 105,
1889–1900.
Fujita, S., Ito, J. (1999). Ability of nucleus cochlear implantees to
recognize music. Annals of Otology, Rhinology & Laryngology,
108, 634–640.
Gardiner, M. F., Fox, A., Knowles, F., Jeffrey, D. (1996). Learning
improved by arts training. Nature, 381, 284.
Gfeller, K., Christ, A., Knutson, J. F., Witt, S., Murray, K. T.,
Tyler, R. S. (2000a). Musical backgrounds, listening habits,
and aesthetic enjoyment of adult cochlear implant recipients.
Journal of the American Academy of Audiology, 11, 390–406.
Gfeller, K., Olszewski, C., Rychener, M., Sena, K., Knutson, J. F.,
Witt, S., Macpherson, B. (2005). Recognition of “real-world”
musical excerpts by cochlear implant recipients and normalhearing adults. Ear & Hearing, 26, 237–250.
Gfeller, K., Witt, S. A., Spencer, L. J., Stordahl, J., Tomblin, B.
(1998). Musical involvement and enjoyment of children who
use cochlear implants. Volta Review, 100, 213–233.
Gfeller, K., Witt, S., Stordahl, J., Mehr, M., Woodworth, G. (2000).
The effects of training on melody recognition and appraisal by
adult cochlear implant recipients. Journal of the Academy of
Rehabilitative Audiology, 33, 115–138.
Hannon, E. E., Trehub, S. E. (2005). Tuning in to musical
rhythms: Infants learn more readily than adults. Proceedings
of the National Academy of Sciences, 102, 12639–12643.
Ho, Y. C., Cheung, M. C., Chan, A. S. (2003). Music training
improves verbal but not visual memory: Cross-sectional and
longitudinal explorations in children. Neuropsychology, 17,
439–450.
Houston, D. M., Ying, E. A., Pisoni, D. B., Kirk, K. I. (2003).
Development of pre-word-learning skills in infants with cochlear implants. The Volta Review, 105, 303–326.
Johnson, M. H., Munakata, Y. 2005. Processes of change in brain and
cognitive development. Trends in Cognitive Sciences, 9, 152–158.
Kuhl, P. K., Tsao, F.-M., Liu, H.-M. (2003). Foreign-language
experience in infancy: effects of short-term exposure and social
interaction on phonetic learning. Proceedings of the National
Academy of Sciences, 100, 9096–9101.
Leal, M. C., Shin, Y. J., Laborde, M.-L., Calmels, M.-N., Verges,
S., Lugardons, S., Andrieu, S., Deguine, O., Fraysse, B. (2003).
Music perception in adult cochlear implant recipients. Acta
Oto-Laryngologica, 123, 826–835.
Lin, Y. S., Peng, S.-C. (2003). Acquisition profiles of syllableinitial consonants in Mandarin-speaking children with cochlear implants. Acta Oto-Laryngologica, 123, 1046–1053.
Nakata, T., Trehub, S. E., Mitani, C., & Kanda, Y. (2006). Pitch
and timing in the songs of deaf children with cochlear implants. Music Perception, 24, 147–154.
Nakata, T., Trehub, S. E., Mitani, C., Kanda, Y., Shibasaki, A.,
Schellenberg, E. G. (2005). Music recognition by Japanese
children with cochlear implants. Journal of Physiological Anthropology and Applied Human Science, 24, 29–32.
Oh, S.-H., Kim, C. S., Kang, E. J., Lee, D. S., Lee, H. J., Chang, S.
O., Ahn, S. H., Hwang, C. H., Park, H. J., Koo, J. W. (2003).
EAR & HEARING, VOL. 28 NO. 2 SUPPLEMENT
Speech perception after cochlear implantation over a 4-year
time period. Acta Oto-Laryngologica, 123, 148–153.
Schellenberg, E. G. (2004). Music lessons enhance IQ. Psychological Science, 15, 511–514.
Schellenberg, E. G., Nakata, T., Hunter, P. G., Tamoto, S. (2007).
Exposure to music and cognitive performance: Tests of children
and adults. Psychology of Music.
Schellenberg, E. G., Trehub, S. E. (2003). Good pitch memory is
widespread. Psychological Science, 14, 262–266.
Stordahl, J. (2002). Song recognition and appraisal: a comparison
of children who use cochlear implants and normally hearing
children. Journal of Music Therapy, 39, 2–19.
Thompson, W. F., Schellenberg, E. G, Husain, G. (2004). Decoding
speech prosody: do music lessons help? Emotion, 4, 46–64.
Vandali, A. E., Sucher, C., Tsang, D. J., McKay, C. M., Chew,
33S
J. W., McDermott, H. J. (2005). Pitch ranking ability of cochlear implant recipients: a comparison of sound-processing
strategies. Journal of the Acoustical Society of America, 117,
3126–3138.
Vongpaisal, T., Trehub, S. E., Schellenberg, E. G. (2006). Song
recognition by children and adolescents with cochlear implants. Journal of Speech, Language, and Hearing Research.
Wilson, B. S., Schatzer, R., Lopez-Poveda, E. A., Sun, X., Lawson,
D. T., Wolford, R. D. (2005). Two new directions in speech
processor design for cochlear implants. Ear & Hearing, 26,
73S–81S.
Yonemoto, K. (1995). Hotyoki tekigo hyoka-yo CD-no tokutyo [CD
for the evaluation of fitting condition with hearing aids (TY-89)].
Journal of Otolaryngology, Head and Neck Surgery, 11, 1395–
1401.