Download pdf

THE NEUROSCIENCES AND MUSIC III—DISORDERS AND PLASTICITY
Music in the Lives of Deaf Children
with Cochlear Implants
Sandra E. Trehub,a Tara Vongpaisal,a and Takayuki Nakatab
a
b
University of Toronto, Mississauga, Ontario, Canada
Future University–Hakodate, Hakodate, Hokkaido, Japan
Present-day cochlear implants provide good temporal cues and coarse spectral cues.
In general, these cues are adequate for perceiving speech in quiet backgrounds and
for young children’s acquisition of spoken language. They are inadequate, however, for
conveying the rich pitch-patterning of music. As a result, many adults who become
implant users after losing their hearing find music disappointing or unacceptable. By
contrast, child implant users who were born deaf or became deaf as infants or toddlers
typically find music interesting and enjoyable. They recognize popular songs that they
hear regularly when the test materials match critical features of the original versions.
For example, they can identify familiar songs from the original recordings with words
and from versions that omit the words but preserve all other cues. They also recognize
theme songs from their favorite television programs when presented in original or
somewhat altered form. The motivation of children with implants for listening to music
or melodious speech is evident well before they understand language. Within months
after receiving their implant, they prefer singing to silence. They also prefer speech in
the maternal style to typical adult speech and the sounds of their native language-to-be
to those of a foreign language. An important task of future research is to ascertain the
relative contributions of perceptual and motivational factors to the apparent differences
between child and adult implant users.
Key words: music perception; singing; children; deafness; cochlear implants
Introduction: Challenges
of Implants
Many congenitally deaf children achieve
high levels of proficiency in speech perception
and production on the basis of electrical input
from their cochlear implants.1 Adults who become profoundly deaf escape the challenges of
language acquisition, but they still face daunting communication challenges, which are often
overcome through the use of cochlear implants.
Although typical cochlear prostheses optimize
temporal-envelope cues, which are critical for
speech perception, they provide limited cues to
temporal fine structure, which are critical for
precise pitch resolution.2
Address for correspondence: Sandra E. Trehub, Department of
Psychology, University of Toronto, 3359 Mississauga Road North, Mississauga, Ontario, Canada L5L 1C6. Voice: +1-905-828-5415; fax:
+1-905-569-4850. [email protected]
Limitations in pitch resolution have enormous implications for the perception of music.3
Adult implant users commonly require differences of 4–12 semitones before they can identify one tone as higher or lower in pitch than
another.4–6 Such resolution is obviously inadequate for the appreciation of melodies whose
successive notes are often separated by 1 or 2
semitones.7 Some sense of the challenges faced
by implant users can be gained by listening to
simulations of music via cochlear implants.8
Although poor pitch resolution is the principal impediment to music processing in implant users, there are other difficulties. Degraded cues to spectral shape also interfere with
timbre perception, specifically, the differentiation of instrumental timbres from one another,
including those from different instrument families.9,10 It is not surprising, then, that many
adult implant users find music disappointing
The Neurosciences and Music III—Disorders and Plasticity: Ann. N.Y. Acad. Sci. 1169: 534–542 (2009).
c 2009 New York Academy of Sciences.
doi: 10.1111/j.1749-6632.2009.04554.x 534
Trehub et al.: Children with Cochlear Implants
or unacceptable in the postimplant period.11,12
For a limited number of adult implant users
with extensive musical training, heroic efforts
coupled with technical and personal accommodations have enabled them to return to
careers as performing musicians. In general,
however, adult implant users fail to recognize
familiar melodies13–15 or to differentiate unfamiliar melodies from one another on the basis
of pitch cues alone.3
Child Implant Users Like Music
The situation for child implant users, especially those who are congenitally or prelingually
deaf, differs substantially from that of adult implant users. Although children must contend
with similar device limitations, they do not
evaluate music with reference to acoustic standards or the way music used to sound. For the
most part, these young implant users find music highly engaging, and they participate in varied musical activities, such as singing, dancing,
concert-going, and instrument lessons.16–18 In
principle, such favorable attitudes and greater
musical involvement could result in better music perception skills than those of adult implant users. In practice, however, children with
normal or impaired hearing typically make
poorer use of available auditory cues than
their older counterparts.19,20 It remains unclear, then, whether children with implants
achieve the potential perceptual benefits from
their musical engagement. Unfortunately, there
have been no direct comparisons of the music perception skills of child and adult implant
users, so answers to this question must await
further research. Nevertheless, studies of implanted children are shedding light on their
music recognition and production skills.
Child Implant Users Identify
Familiar Songs
In one investigation, 8- to 15-year-old implant users had difficulty recognizing simple in-
535
strumental versions of well-known songs such as
Happy Birthday, and Row, Row, Row Your Boat.21
In that respect, they seemed much like adult
implant users.13–15 For hearing listeners, who
have abstract as well as detailed mental representations of familiar music, 22–24 there are multiple routes to identification. For example, most
hearing listeners would recognize Happy Birthday in different pitch registers (e.g., high or low)
and at different tempi (e.g., fast or slow) when
performed with or without variations by solo
singers, unaccompanied choral singers, solo instrumentalists, or full orchestras. By contrast,
child implant users are likely to have different
mental representations that depend less, if at all,
on relative pitch cues. If their representations
are less abstract or less detailed than those of
hearing listeners, they could have difficulty linking instrumental renditions of familiar songs to
the sung renditions that they usually hear.
Considerations such as these led us to explore
the possibility that music recognition in young
implant users required all of the features usually heard or a specific subset of those features.
Accordingly, we focused on implanted children
and adolescents who were fans of popular music and listened regularly to recordings by their
favorite vocalists or groups.18 Our plan was to
test these implant users and a control sample of
age-matched hearing listeners on various renditions of the specific recordings most familiar
to them: the original version, which included
a single vocalist or group with instrumental
accompaniment, an instrumental version that
matched the original except for the absence of
vocals, and a synthesized piano version of the
main (sung) melody that preserved the original
pitch, tempo, and rhythm. For each version, listeners were required to identify the target song
from choices depicted on a monitor (i.e., song
titles and photos of artists or groups). Each listener selected 3–5 songs that they knew best
from a set of 14 songs that were highly popular
at that time with children and adolescents.
First, we established that 5- to 8-year-olds
and adults with normal hearing could identify the songs from the aforementioned versions
536
Figure 1. Mean accuracy scores for hearing children and adults on original, instrumental, and melody
versions of popular songs. Error bars represent standard errors. (From Vongpaisal et al .18 Reprinted with
permission.)
(Fig. 1). Hearing children and adults performed
well above chance levels (0 translating to chance
performance and 1 to perfect performance),
but the performance of both groups declined
as the difference increased between the original stimuli and the test stimuli. Children performed more poorly than adults only on the
melody version. This finding may reflect children’s inability to use retrieval cues as flexibly as
adults do. Alternatively, their mental representations of music may be less robust than those
of adults.
Subsequently, we found that young implant users 8–18 years of age (mean age of
12 years) performed more poorly than agematched hearing listeners on all versions of the
songs (Fig. 2). As with the hearing children,
young implant users’ performance was best for
test stimuli that matched all features available
at the time of encoding (i.e., during recreational
listening) and worst for those with the greatest
divergence. Specifically, implant users’ performance was well above chance levels on the original versions, less accurate but still above chance
levels on the instrumental versions, and at
chance levels on the melody versions. Although
implanted children with excellent speech perception skills have difficulty decoding the lyrics
of unfamiliar songs,25 the lyrics of the famil-
Annals of the New York Academy of Sciences
Figure 2. Mean accuracy scores of hearing (NH)
and implanted (CI) children on original, instrumental,
and melody versions of popular songs. Error bars
represent standard errors.(From Vongpaisal et al .18
Reprinted with permission.)
iar songs obviously contributed to their performance in this study.
Child and adolescent implant users’ success
on the instrumental versions confirmed that
verbal cues were not critical for song identification. This finding implies that despite implant users’ difficulty with timbre (instrument)
identification,9,10 timbre cues still contribute to
music recognition. Recall that the instrumental
renditions matched the original recordings in
all respects except for the absence of vocals, so
they provided a rich array of timbre and timing cues. By contrast, the melody renditions,
which deviated substantially from the original
music, were insufficient for song identification
even though they retained the pitch and timing
cues of the original sung melody. As can be seen
in Figure 3, individual differences in the performance of implant users were substantial.
Child Implant Users Identify TV
Theme Songs
If child implant users are able to recognize
music that they hear by choice, might they
also recognize music that they hear incidentally
while watching TV? Children’s TV programs
typically feature theme songs that become well
Trehub et al.: Children with Cochlear Implants
Figure 3. Accuracy scores of individual agematched hearing (NH) and implanted (CI) children
and adolescents on the original, instrumental, and
melody versions of popular songs.(From Vongpaisal
et al .18 Reprinted with permission.)
known to young hearing viewers, but it is unclear whether such songs would be engaging
and memorable for young implant users. We
examined the ability of Japanese child implant users to recognize such music.26,27 North
American rehabilitation practices are relatively
similar to those in Japan, with the principal
focus on speech and language and little, if any,
attention to music. The Japanese children, who
were 4–8 years of age, were tested on various
renditions of the theme music, including the
original version, an instrumental version that
preserved all cues except for the vocals, and a
synthesized flute version of the main melody.
537
After listening to a musical excerpt, children
selected the actors or cartoon characters from
the target TV show from alternatives depicted
on a monitor.
Japanese child implant users reliably identified the original versions, but they showed no
recognition of the other versions. Perhaps incidental exposure to the TV theme songs was insufficient for generating robust representations
of the music that could generalize to other versions. Alternatively, these children could have
had less overall exposure to the music than
did the Canadian pop song fans in the previous study. In Japan, the theme songs of children’s TV programs typically change at 3- to
12-month intervals, in contrast to the considerably longer availability of specific popular songs
or TV theme songs in North America.
When Canadian 4- to 11-year-olds with
cochlear implants were given a comparable
identification task with TV theme songs, their
performance was above chance levels on all
versions,28 lending credence to the potential
contribution of exposure. Although the Canadian children were older, on average, than the
Japanese children, age is unlikely to underlie
the observed differences in performance. In
fact, the two youngest children in the Canadian sample—one 4-year-old and one 5-yearold—were the best performers. More significantly, perhaps, these children received their
implant at 12 months of age, a year or more
earlier than other children in the Canadian or
Japanese samples. Implantation of congenitally
deaf children before 2 years of age is known
to facilitate speech perception and language
development.29,30 Early implantation may result in comparable processing advantages for
music.
The most unexpected finding was child
implant users’ successful identification of the
melody versions of the TV theme songs. Not
only did these Canadian children outperform the Japanese TV viewers, but they also
outperformed the implanted children and teens
who failed to identify the melody versions of
familiar pop songs.18 One potentially relevant
538
Annals of the New York Academy of Sciences
difference between the Canadian pop song fans
and TV viewers is the younger age of implantation of the latter group. Another possible difference is children’s greater cumulative exposure
to the TV theme music than to the pop songs.
No information on exposure was available, but
self-reports or parental reports of cumulative
exposure to music listening or TV viewing are
unlikely to be reliable in any case. It is important to ascertain the consequences of musical
exposure in auditory-alone or auditory-visual
contexts by means of comparable stimuli and
exposure time across contexts as well as participants of similar age and implant experience.
Finally, it is possible that timbre differences between the pop song melodies (piano) and TV
melodies (flute) contributed to the differences
in performance. According to some reports, piano tones pose particular difficulty for implant
users.3,4
Child Implant Users Sing
The favorable music evaluations of child implant users16,18,26 could be driven largely by
social conventions. Similarly, their varied musical activities could arise from parental pressure.
Such external factors, however potent, are unlikely to account for the high incidence of spontaneous singing in this population,26 which attests to their interest in music. Obviously, it is
of considerable interest to ascertain the nature
of implant users’ singing in view of their limitations in pitch processing3,4,6 and their relative
strength in rhythm processing.14
During a routine clinic visit, we asked
Japanese children to sing their favorite songs,
and they obliged without hesitation. We measured the mean fundamental frequency and duration (inter-onset intervals) for each note sung
by these children and a comparison group of
hearing children.17 For each child, we compared observed durations to expected durations, basing the latter on notated versions of
the song adjusted to the child’s tempo. We also
compared observed pitches to target pitches.
The pitch and timing characteristics of song
excerpts (first 15 beats) from a subsample of
children with cochlear implants and those with
normal hearing are depicted schematically in
Figure 4. Although the observed timing of
songs deviated somewhat from the expected or
ideal timing, those deviations were no greater
for implanted children than for hearing children. In any case, both groups provided clearly
recognizable reproductions of the original
timing.
By contrast, implanted and hearing children’s songs were dramatically different in pitch
patterning. As can be seen in Figure 4, implanted children used a very limited pitch
range, roughly a third of the range of hearing children. Whereas hearing children’s pitch
directional changes—up or down—matched
those of the target song, those of implanted children were unrelated to the target song. Despite
their substantial differences in pitch patterning,
the sung renditions of child implant users and
hearing children revealed comparable energy,
vitality, and joy.
Listening Preferences of Toddlers
with Implants
It is clear that deaf children with cochlear
implants enjoy music, remember aspects of the
music they hear regularly, and, in many cases,
sing for their own pleasure. In that sense, their
motivation for music is much like that of hearing children. For hearing children, however, interest in music is evident from the earliest days
and months of life.31 For example, preverbal
infants are tantalized by mothers’ melodious
speech, preferring it to adult-directed speech,
which is less variable in pitch and dynamics
and less affect-laden.32,33 They also prefer the
maternal style of singing, which is highly expressive, to typical informal singing, which is
much less expressive.34,35 Although maternal
speech and singing are both highly engaging to
infants, maternal singing sustains their attention for considerably longer periods.36
539
Trehub et al.: Children with Cochlear Implants
Figure 4. Sample illustrations of expected (open symbols) and observed (solid symbols)
pitch and duration (inter-onset intervals) for the first 15 beats of familiar songs sung by children
with cochlear implants (CI) and hearing children (NH). (From Nakata et al .17 Reprinted with
permission.)
Congenitally deaf children, who typically
receive their implants at 12 months of age
or later, have not had comparable exposure
to maternal speech, singing, and environmental sounds from birth. Their exposure begins
about a month after implantation, when their
implants are activated. We are currently attempting to assess the listening preferences of
implanted toddlers by means of a headturn preference procedure (Volkova et al.,
manuscript in preparation). Toddlers are pre-
sented with one auditory pattern when they
look at one display (e.g., to their left) and a contrasting auditory pattern when they look at another display (e.g., to their right). Over a series
of trials, cumulative looking times associated
with each of the auditory patterns provide an
estimate of their relative interest in those patterns. Preliminary evidence reveals that within
a few months of implant activation, toddlers
prefer an audiovisual display of a mother (not
their own) singing a play song to a soundless
540
Annals of the New York Academy of Sciences
version of the same visual display. Well before they show evidence of understanding
any words, these toddlers prefer motherese (i.e.,
speech in the maternal style) to typical adultdirected speech, presumably on the basis of its
prosody or music. They also prefer motherese in
their language-to-be to motherese in a foreign
language, which presumably reflects their familiarity with the speech sounds and prosody
in their everyday environment.
Preschool Implant Users Identify
TV Theme Songs
We have begun testing preschool implant
users, from 2.5 or 3 years of age, on their
recognition of TV theme music from programs
watched regularly (Volkova et al., manuscript
in preparation). Of the limited number of
preschoolers tested to date, most have been
successful in identifying original and instrumental versions of these songs. Some have
even succeeded in identifying the melody versions. It is as yet unclear whether their uniformly early age of implantation is responsible
for their high level of engagement with music
and their success on various music recognition
tasks. Increasingly, very young deaf children
are receiving bilateral implants rather than
the unilateral implants that were prevalent for
many years. It is too early to know whether
bilateral implantation will enhance music
perception.
Future Directions
The available data seem to suggest that music is more engaging and memorable for children with cochlear implants than it is for adult
implant users. At this point, the relative contributions of motivational and perceptual factors
to these differences remain unclear. Specifying
such contributions is an important task for future research. Although limitations of current
implant technology interfere with fine pitch res-
olution and musical pitch processing,2,3 these
limitations do not preclude rewarding musical
experiences, especially for children. We know
much more about children’s music perception skills than we did 5 years ago, but many
questions remain unanswered. What specific
features contribute to child implant users’ engagement with music? Do these children rely
primarily, even exclusively, on timing cues? For
children with implants who can identify simple
melodic renditions of TV theme songs, are they
doing so solely on the basis of relative timing
patterns rather than the relative pitch patterns
that are central for hearing listeners? Can they
capitalize on gross spectral cues? Answers to
such questions should be forthcoming over the
next few years.
Acknowledgments
Preparation of this paper was assisted by
grants from the Natural Sciences and Engineering Research Council and the Social Sciences
and Humanities Research Council of Canada,
and from the Japan Society for the Production
of Science.
Conflicts of Interest
The authors declare no conflicts of interest.
References
1. Svirsky, M.A., A.M. Robbins, K.I. Kirk, et al. 2000.
Language development in profoundly deaf children with cochlear implants. Psychol. Sci. 11: 153–
158.
2. Shannon, R.V., Q.-J. Fu, J. Galvin, et al. 2004. Speech
perception with cochlear implants. In Cochlear Implants: Auditory Prostheses and Electric Hearing, Springer
Handbook of Auditory Research. F.-G. Zeng, A.N. Popper
& R.R. Fay, Eds.: 334–376. Springer-Verlag. New
York.
3. Cooper, W.B., E. Tobey & P.C. Loizou. 2008. Music
perception by cochlear implant and normal hearing listeners as measured by the Montreal Battery for Evaluation of Amusia. Ear Hear. 29: 618–
626.
Trehub et al.: Children with Cochlear Implants
4. Fujita, S. & J. Ito. 1999. Ability of nucleus cochlear
implantees to recognize music. Ann. Otol. Rhinol.
Laryngol. 108: 634–640.
5. Green, T., A. Faulkner & S. Rosen. 2002. Spectral and temporal cues to pitch in noise-excited
vocoder simulations of continuous-interleavedsampling cochlear implants. J. Acoust. Soc. Am. 112:
2155–2164.
6. Vandali, A.E., C. Sucher, D.J. Tsang, et al. 2005.
Pitch ranking ability of cochlear implant recipients: a
comparison of sound-processing strategies. J. Acoust.
Soc. Am. 117: 3126–3138.
7. Vos, P. & J. Troost. 1989. Ascending and descending melodic intervals: statistical findings and
their perceptual relevance. Music Percep. 6: 383–
396.
8. http://www.healthaffairs.uci.edu/hesp/Simulations/
simulationsmain.htm
9. Gfeller, K., S. Witt, G. Woodworth, et al. 2002.
Effects of frequency, instrumental family, and
cochlear implant type on timbre recognition and
appraisal. Ann. Otol. Rhinol. Laryngol. 111: 349–
356.
10. McDermott, H.J. 2004. Music perception with
cochlear implants: a review. Trends. Amplif. 8: 49–
82.
11. Gfeller, K., A. Christ, J.F. Knutson, et al. 2000. Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. J. Am.
Acad. Audiol. 11: 390–406.
12. Lassaletta, L., A. Castro, M. Bastarrica, et al. 2007.
Does music perception have an impact on quality of
life following cochlear implantation? Acta. Otolaryngol.
127: 682–686.
13. Galvin, J.J., Q.-J. Fu & G. Nogaki. 2007. Melodic
contour identification by cochlear implant listeners.
Ear Hear. 28: 302–319.
14. Gfeller, K., G. Woodworth, D.A. Robin, et al. 1997.
Perception of rhythmic and sequential pitch patterns
by normally hearing adults and cochlear implant
users. Ear Hear. 18: 252–260.
15. Kong, Y.-Y., R. Cruz, J.A. Jones, et al. 2004. Music
perception with temporal cues in acoustic and electric
hearing. Ear Hear. 25: 173–185.
16. Gfeller, K., S.A. Witt, L.J. Spencer, et al. 1998.
Musical involvement and enjoyment of children
who use cochlear implants. Volta Rev. 100: 213–
233.
17. Nakata, T., S.E. Trehub, C. Mitani, et al. 2006. Pitch
and timing in the songs of deaf children with cochlear
implants. Music Percept. 24: 147–154.
18. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg.
2006. Song recognition by children and adolescents
with cochlear implants. J. Speech Lang. Hear. Res. 49:
1091–1103.
541
19. Eisenberg, L.S., R.V. Shannon, A.S. Martinez, et al.
2000. Speech recognition with reduced spectral cues
as a function of age. J. Acoust. Soc. Am. 107: 2704–
2710.
20. Stalinski, S.M., E.G. Schellenberg & S.E. Trehub.
2008. Developmental changes in the perception of
pitch contour: distinguishing up from down. J. Acoust.
Soc. Am. 124: 1759–1763.
21. Stordahl, J. 2002. Song recognition and appraisal: a
comparison of children who use cochlear implants
and normally hearing children. J. Music Ther. 39: 2–
19.
22. Dowling, W.J. 1999. The development of music perception and cognition. In The Psychology of Music, 2nd
ed. D. Deutsch, Ed.: 603–625. Academic Press. San
Diego, CA.
23. Schellenberg, E.G., P. Iverson & M.C. McKinnon.
1999. Name that tune: identifying popular recordings from brief excerpts. Psychon. Bull. Rev. 6: 641–
646.
24. Schellenberg, E.G. & S.E. Trehub. 2003. Accurate
pitch memory is widespread. Psychol. Sci. 14: 262–
266.
25. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg.
2005. Challenges of decoding the words of songs.
Presented at the 10th Symposium on Cochlear Implants in Children. Dallas, TX, March 17.
26. Nakata, T., S.E. Trehub, C. Mitani, et al. 2005. Music recognition by Japanese children with cochlear
implants. J. Physiol. Anthropol. Appl. Human Sci. 24:
29–32.
27. Mitani, C., T. Nakata, T., S.E. Trehub, et al. 2007.
Music recognition, music listening, and word recognition by deaf children with cochlear implants. Ear
Hear 28: 29S–33S.
28. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg.
2009. Identification of TV tunes by children with
cochlear implants. Music Percept. In press.
29. Nicholas, J.G. & A.E. Geers. 2006. Effects of early
auditory experience on the spoken language of deaf
children at 3 years of age. Ear Hear. 27: 286–
298.
30. Svirsky, M.A., S.-W. Teoh & H. Neuburger. 2004.
Development of language and speech perception in
congenitally, profoundly deaf children as a function
of age at cochlear implantation. Audiol. Neurotol. 9:
224–233.
31. Trehub, S.E. 2008. Music lessons from infants. In
The Oxford Handbook of Music Psychology. S. Hallam, I.
Cross & M. Thaut, Eds.: 229–234. Oxford University
Press. Oxford, UK.
32. Fernald, A. 1991. Prosody in speech to children:
prelinguistic and linguistic functions. In Annals of
Child Development, Vol. 8. R. Vasta, Ed.: 43–80. Jessica Kingsley Publishers. London, UK.
542
33. Singh, L., J.L. Morgan & C.T. Best. 2002. Infants’
listening preferences: baby talk or happy talk? Infancy
3: 365–394.
34. Trainor, L.J. 1996. Infant preferences for infantdirected versus noninfant-directed playsongs and lullabies. Infant. Behav. Dev. 19: 83–92.
Annals of the New York Academy of Sciences
35. Trainor, L.J., E.D. Clark, A. Huntley, et al. 1997.
The acoustic basis of preferences for infant-directed
singing. Infant Behav. Dev. 20: 383–396.
36. Nakata, T. & S.E. Trehub. 2004. Infants’ responsiveness to maternal speech and singing. Infant Behav. Dev.
27: 455–464.