Download The Central Origin of the Pitch of Complex Tones: Evidence from

4.9, 4.11
Received22 September1971;revised8 November1971
The Central Origin of the Pitch of Complex Tones: Evidence
'
from Musical Interval Recognition *
A. J. M. HOUTSMA
ANDJ. L. GOLI)STEINi
Centerfor Communications
Sciences,
Research
Laboratoryof Electronics,
Massachusetts
Institute o.fTechnology,
Cambridge,Massachusetts
02139
The human auditory system'sability to recognizesimplemelodiesthat correspond
to fundamentalperiods
in sequences
ofperiodicsounds
devoidoffundamental
energywasstudiedthroughmusical
intervalidentification experiments.Stimuli comprisingtwo randomlychosensuccessive
upper harmonicswere presented
both monotically(two harmonicsto oneear) and dichotically(oneharmonicto eachear). Subjectscould
recognize
melodies
equallywellwith bothmodesof stimuluspresentation.
The resultsimply that thepitchof
thesecomplextonesis mediatedby a centralprocessor
operatingon neuralsignalsderivedfrom those
effectivestimulusharmonicsthat are tonotopicallyresolved.
introducedto auditionby Ohm in 1843 (Plomp and
Smoorenburg,1970). This earlier work is discussed
An essentialbasis for practicing the art of music is
morefully in Secs.V and VI. We reportin this papera
the human auditory system'sability to perceivemelody number of new psychophysical
experimentsthat were
in a sequenceof periodic or near-periodicsounds.In designedwithin a musicalframeworkto explorehow
Western culture, a musical note scale has been develthe auditorysystemretrievesand encodes
the informaoped which characterizesmusical sounds solely by tion of the fundamentalfrequencyfrom a sequence
of
their fundamentalfrequencies,and a melodycomprises periodicsounds.
a sequenceof noteson sucha scale.Musical instruments
are designedin sucha way that when a note is played
I. METHODS
a periodic or quasiperiodicsoundis generatedwhose
fundamental frequencyis designatedby the note. On
A. General Paradigm
listening to a sequenceof suchmusicalsoundsone can
In order to simulate musical behavior as closely as
easily retrieve the original series of notes from the
the basicprocedurewasto presenta subject
auditory sensation,regardlessof the particular Fourier possible,
consisting
of a numberof
spectrum generatedby the instrument. Some instru- with a simplemusicalmessage
andto havehim respondby identifying
ments (e.g., strings) produce sounds with a rich musicalsounds,
or melodies
spectrum comprising the fundamental and many it. In caseswherethe setof musicalmessages
partials; others (e.g., flute) have little more than the was known to the subjectbeforehand,a responsebox
bare fundamental; and still others (French horn) was used; otherwisethe subjectwas askedto write
melodyusinga relativenote scale.
generatelower noteswith relatively little energy at the downthe perceived
The musicalsoundswere complextonescomprising
lower partials, including the fundamental (Saunders,
upperharmonicsof equalintensity;no
1937, 1946). Alsomelodiesplayedon any giveninstru- two successive
ment can easily be recognizedafter the soundhas been energyat the fundamentalfrequencyitselfwaspresent.
passed through a bandpassfilter with a bandwidth Despitethefact that morecomplex
stimulimighthave
much narrowerthan the spectraof the originalsounds; provided better simulation of musical-instrument
one can readily observethis when listening to music sounds,theserelativelysimplesyntheticstimuliwere
they minimizethe numberof free
from an inexpensivetransistorradio.This phenomenon preferredbecause
of melody invariance over a large class of spectral stimulusparameters,and offerbetter controlover the
transformationshas had profoundimpact on auditory locationof spectralenergyin the effectivestimulus.
because
a largebodyof
scienceever sincethe conceptof Fourier spectrumwas In addition,theywereselected
INTRODUCTION
520
Volume 51
Number 2 (Part 2)
1972
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
PITCH
OSCILLATOR
OF
COMPLEX
TONES
DUAL
DUAL
CHANNEL
CHANNEL
ELECTRONIC
AUDIO
I
PDP-4
COMPUTER
OSCILLATOR
2
SWITCH
AMPLIFIER
'I• ATTENUATOR
I
• ATTENUATOR[
ANSWER
BOX
FIG. 1. Equipmentdiagramfor all two-toneexperiments.
participatedin somemore descriptiveand qualitative
experiments.
All had at least somedegreeof musical
training and were familiar with musical notation and
dictation. A few could not perform the required
If the same harmonic numbers were chosen for each
identification or recognition tests initially and were
note, one couldeasilyrecoverthe melody describedby given sometraining (lessthan 1 h) starting with tone
harmonics,which
the (missing)fundamentalsmerely by tracking one of complexesof up to six successive
the two partials, since the frequenciesof harmonic were then graduallyreducedto the two-tonestimuli
partialsare alwaysintegralmultiplesof the fundamental usedin the test experiments.
frequency. In this situation, both partials would
IDENTIFICATION
OF
describethe same melody as the fundamental, merely II. BASIC EXPERIMENTS:
EIGHT
KNOWN
MUSICAL
INTERVALS
transposedupward in key. In order to prevent the
subject from using this trivial cue, the harmonic
A. General
Procedures
numberswere randomizedfrom note to note, and hence
the subject was required to retrieve the notes from
Five basicexperiments
werecarriedout havingmany
possiblyrelevantpsychophysical
and physiologicaldata
alreadyexistedfor thesestimuli (Goldstein,1967, 1970;
Goldstein and Kiang, 1968; Kiang, 1965; Sachs and
Kiang, 1968; Sachs,1969).
hearing a sequenceof pairs of randomly chosensucces- features in common. The task of the subject was to
siveupper harmonics.
identify on eachtrial whichout of eightknowntwo-note
melodies(or musicalintervals)was presented[-Fig.
2(a)-]. These simple melodieshad identical envelope
B. Apparatus
time structure[-Fig. 2 (b)-], and all beganwith the same
The apparatusconsistedof two programmableoscilnote.The noteswereapproximatelytunedto the natural
lators (General Radio 1161-A and KrohnHite 4031 R),
scale,that is, frequencyratiosof 16/15, 9/8, 6/5, and
an electronic switch (Grason-Stadler), a two-channel
5/4 for the minor and major second,and minor and
audio amplifier, and a set of TDH-39 headphones.
major third, respectively.The stimuli representing
the notes comprisedtwo successive
harmonics,the
the headphoneswas better than 50 dB below stimulus
number of the lower harmonicbeing chosenrandomly
level. (In one experiment, where four successive
for eachnote over a rangeof three successive
integers
harmonicswere needed,we useda third programmable
[-Fig.2(c)-]. The middleof the lowerharmonicnumber
oscillator, KrohnHite 4031 R, and a dual multiplier,
range,•, and the fundamentalfrequencyof the first
CBL 47.) Subjectswere seatedin an IAC model 1200
note,
f0, werechosenasindependent
parametersin measound-insulated chamber. Switches and oscillators were
suringidentificationperformanceexpressed
as the percontrolledby a DEC PDP-4 computerwhich generated
centageof correctresponses.
We chosethis particular
all random events, performedstimuluscomputations,
set of musical intervals, not becausewe assumethey
stored responses,and controlled feedback. An equiphaveany inherentsignificance,
but merelybecause
they
ment diagram is shownin Fig. 1.
provide, at least in the Western culture, a convenient language which musically trained people can
C. Subjects
Harmonic
and intermodulation
distortion
measured
at
understand.
Three subjectsparticipatedin the basicexperiments.
All weremale, between18 and 31 yearsof age, and had
quite extensivemusicaltraining and experience(majors
in organ,viola, and singing).All subjectscouldperform
the required tests with little special training; test
sessionswere limited to a maximum of 2 h daily. An
additional sevensubjects,four male and three female,
The subjectswerefirst testedin preliminaryexperimentsfor their ability to identify musicalintervalsby
using periodic sounds with fundamental energy; a
control run of 50 trials was carried out, the stimuli
beingsquarewaveswith fundamentalfrequencies
equal
to the notesin Fig. 2(a). Subjectsweregivena "key"
ascribinga numberto eachintervalandwereinstructed
The Journalof the AcousticalSocietyof America
521
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
HOUTSMA
AND
i
1•4•
,4
[x]J
'"
"'
GOLDSTEIN
eachtrial,whileNH t•referred
nofeedback.
A typical
I..,1"'-' I..I
•'.'
01...11•01,..I
i'-'
]'.'
11.4
]",o
II
•" o]'.'
•1
I I,.l
II
cl •"',,•]
•f(•/fo
5/4 6/5 9/8 16/15
15/168/9 5/6 4/5
run consistedof 50 trials for subjectAH and 25 trials
for the other subjects.Fundamentalfrequencieswere
incremented
in stepsof 100Hz (or sometimes
200 Hz),
and for eachfundamentalfrequencya number of runs
wastaken,eachtime incrementingthe averageharmonic
number ff by one. As many runs were taken as were
500
250
500
TIME
(ms)
(b)
necessaryto make performancedrop from perfect
(100%correct)to chance(12.5ø/o
correct,corresponding
to one out of the eight intervals guessedcorrectly);
typicallyaboutsixrunsat eachfundamentalfrequency
were required. The resulting psychometricfunctions
werethenremappedinto"equal performance
contours,"
relating performance level to both fundamental
frequency,f0, and averageharmonicnumber,if.
B. Monotic Experiments
n-!
n-I
n
n
Stimuli were presentedmonotically at a sensation
level of 20 dB. Equal performancecontoursare shown
n
n+l
n
n+l
n+l
n+2
n+l
f/fo
n+2
f/fo
concentrated
(c)
[R•GHT
"ILEFT
EAR
ß
-J 50
tn 35 -
I, AR [
I
•
,
n
20
•
I
I
!
n-2
n-I
n
in Fig. 3(a). On onesubject(AH) a controlexperiment
wasperformedwith the partials of eachnote presented
in time sequence,rather than simultaneously.We
chosevaluesfor f0 and ff which had previouslyyielded
perfectperformance.Despite the fact that f0 can easily
be computedfrom this stimulus,the subjectwas not
able to perform better than chance,even after many
n+l
f/fo
(d)
Fro. 2. Experimental paradigm for musicalinterval identifica-
tion experiments:(a) musical intervals to be identified, (b)
time-envelope
of the total stimulus,(c) the threepossibletwo-tone
stimuli for each of the two notes; for each note a random choice
wasmadeamongthe threepossiblestimuli, (d) stimulusconfigura-
tion for one note of a dichoticallypresentedmelody,with the
efforts. We concluded that randomization
of harmonic number forces the subject to use both
partials simultaneously.
The experimentalresults show clearly that the best
performance is achieved with the lowest harmonic
numbers, that is, the larger the spacing between
harmonics on a log-frequency scale, the better the
performance. This finding, together with what is
known about the physiologyof cochlearfrequency
analysis (Kiang, 1965; Goldstein, Baer, and Kiang,
1971)and the psychophysics
of resolutionof partialsin
complextones (Plomp, 1964) suggestedthe hypothesis
that both harmonicsof the soundsemployed in this
experimentare processedthrough separatechannelsof
the cochlearoutput to obtain successfulidentification.
C. Dichotic Experiments and Relation to
Monotic
Results
addition of simulated combination tones.
The separate-channelhypothesiswas tested employing probably the most extreme channelseparationone
to push a correspondingbutton on an answer box can think of, namely separateears. We thought that,
within 4 sec after each stimuluspresentation,after while negative resultswould not necessarilyinvalidate
whichfeedbackwasprovided.All threesubjectsscored the hypothesis,positive results would prove it to be
perfectly, which indicatesthat they did possess
the correct. The experimentalparadigm was the same as
requiredmusicalskills,andthat performance
limitations in the monatic experiment,except that the stimulus
so often encounteredin identificationexperimentsof
this kind and which are commonlyattributed to
imperfectmemory (Miller, 1956) were apparently
of little significancehere.
Next, in the basicexperiments,
subjectsweretested
individually.AH and SW were given feedbackafter
522
Volume51
Number2 (Part 2)
partials were presenteddichotically,one to each ear.
The resultsare shownin Fig. 3(b). Comparisonwith
the monatic resultsshowsthat eachsubject'sperformance is essentially the same under both stimulus
conditions, suggestingthat a central mechanism
integratesandprocesses
informationfrombothcochleas,
1972
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
PITCH
OF
COMPLEX
SUBJECT
SW
14-
TONES
NH
AH
_
12 -
MONOTI
C
IO-Pc;o= _ 20dBSL
-
2-IOO•
,•- ' ' ' ' '
0 ,,;,o
•;o,•oo
,;oo
•ooo
.o-"••o,,,, •o•.s•
4-
20dBSL
•,
•o•.•
80
2- I00•
o, ,
,
,
' "
o
,;oo
,;oo
;oo
f
L• I I I I
IF
DICHOTIC
I2•
50dBSPL
IOI- Pc= •
(d)
•/
•o•-,,•_•
0L
•
I
I
I
I
14FP
c=
DICHOTIC
'•1- •o,,,,.
50 •a
0t
•
i
i
i
i
200 400 600 800 I000
fo(Hz)
b
I
I
i
i
•
200 400 600 800 I000
fo(Hz)
I
J
I
I
i
i
I
1
I
200 400 600 800 I000
fo(Hz)
Fro.
3.Performance
contours
for
musical
interval
identification
experiments
with
pairs
ofrandomly
chosen
successive
upper
harmonics.
Stimulus
conditions:
(a)monotic,
20
dBSL,
(b)
dichotic,
20
dBSL,
(c)monoti'c,
50(40)
dBSPL,
(d)dichotic,
50(40)
dBSPL,
(e)
dichotic,
50
(40)dBSPL,
withtwo
simulated
combination
tones.
andthattheinputsto thismechanism
aresimilarunder andfourthexperiment
undertaken
at higherstimulus
both monoticand dichoticstimulation.
intensity
levels,
50dBSPLforsubjects
AH andSW,
Theeffectof stimulus
intensity
uponbothmonotic and40 dB SPLforsubject
NH. Thesesound-pressure
anddichotic
performance
wasinvestigated
in a third levels(SPLs)werechosento avoidthe auraldifference
TheJournal
of theAcoustical
Society
of America 523
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
HOUTSMA
AND
GOLDSTEIN
tone, f•.--f•, in the monoticcase(Plomp,1965; Goldstein, 1967), and interaural crosstalkin the dichotic
case (Zwislocki, 1953). Equal performancecontours
are shownin Figs.3(c) and 3(d).
human listenexto retrieve the notes representedby a
sequenceof periodicsounds.While these experiments
provide informationabout the conditionsunder which
subjectscan perform a certain interval identification
task, they do not show that the subject "hears"
D. Influence of Aural Combination Tones
melody.The subjectcouldhave learnedto discriminate
and identify eight stimuli merely on the basisof some
Comparing results from all four experiments,it
unspecified,
subjective criterion, rather than by
appearsthat each subject'sperformanceis essentially
recognizing
each
stimulus as a particular musical
the sameunder all four stimulusconditions,exceptthat
interval,
although
each
subjectassertedthat the latter
for higher-intensitymonotic stimulation[-Fig. 3(c)• was true.
performancecontoursare shiftedupwardsby approxTo obtainmoreobjectiveevidencethat the measured
imately 2 or 3 harmonic numbers.Such an upward
performance truly reflects the musical behavior of
shift might be expected becauseof the presenceof
aural combinationtones of the type fl--k (f•.-- fl) melody perception,a control experimentwas carried
generatedin the peripheral ear for a monotic stimulus out in which subjectswere requiredto name previously
unknown four-note melodies using standard musical
comprisingthe frequenciesfl and f•. (Zwicker, 1955;
notation. On each trial four periodic soundsat 50 dB
Plomp, 1965; Goldstein, 1967, 1970; Goldsteinand
Kiang, 1968). These combinationtones provide the SPL were presentedin regular time sequence,each
sound being a two-tone complexwith frequenciesat
ear with two or three harmonics below those contained
successiveharmonicsof some (missing) fundamental
in the stimulus, which are probably very useful since
frequency
in the range of 200-400 Hz. The lower
all the results thus far indicate that performance
harmonic
number
was chosenrandomly from '3, 4,
improves with decreasingharmonic number. These
and 5 for each sound. Since each note could be repcombination tones could make the e•ective average
resentedby three different sounds,a four-notemelody
harmonic number about 2 or 3 lower than the actual
could
be presentedin 34 (i.e., 81) different ways.
value of • in Fig. 3. In the dichotic experiments,
Four different four-note melodies were used, which
combination tones are not present; in the monotic
were unknownto the subjectsprior to the experiment.
experimentat 20 dB SL they would be near or below
In the first part of the experiment, subjectscould
threshold(Goldstein,1967).
There are two obviousways to test this combination- determine which of the four melodieswas presented,
and their task was to recognize all four by writing
tone hypothesis.One approachwould be to repeat the
them down using a relative note scale. Since each
third experiment(mediumintensity,monotic)and add
a number of pure tones to the stimulus having the melody could be representedin 81 different ways,
exact frequency, amplitude, and phase to cancel all subjectswere allowedto listen to severalpresentations
aurally generatedcombinationtones(Goldstein,1967); of each melody before answering.In the secondpart,
all four of the four-note melodiesused in the first part
identificationperformanceshouldthen approachthat
were presentedat random, and subjectswere askedto
for dichotic experiments.The technical and experimental difficultiesof providingthe correctcancellation identify eachmelody presentedby pushingthe approamplitude and phasefor eachnote causedus to eschew priate button on a responsebox. This experimentwas
this approach.Instead, an experimentwas carried out performed both with monotically and dichotically
presentedtone complexes,using 10 subjectsall of
usingdichoticstimuliat 50 dB SPL (40 dB for subject
NH) with the additionof two moretonesthat approx-
whom were familar
with musical dictation.
Since our
imately simulated the aural combinationtones which principal interest was to investigate whether the
reflecteda naturalmusicalbehavior
the ear generates under monotic conditions. The subject'sresponse
rather than someingeniouslyacquiredskill for performstimulus paradigm is shown in Fig. 2(d) and the
experimentalresultsin Fig. 3 (e). Performancecontours ing a complicatedtask, very little time was spent on
showedthe same upward shift as is seenin the third training.
The results of the monotic experiment were that
experiment,and the similaritiesof Figs. 3(c) and 3(e)
nine
out of ten subjectscharacterized
eachsequence
of
are considered
strongevidencethat suchperformance
four soundsby the missing fundamentalsin the first
differences as occur between monotic and dichotic
part and scoredperfectlyor nearperfectlyin the second
stimulus conditions can be attributed to combination
part. The samewastrue for six subjectsin the dichotic
tonesgeneratedin the peripheralauditorysystem.
experiment;three had somedifficultiesin both parts,
III. EXPERIMENTAL
TESTS
FOR
and the one subjectwho couldnot perceiveconsistent
MUSICAL
BEHAVIOR
melodiesin the first part undermonoticconditionswas
The purpose of the experimentsreported in the also unable to do so for dichotic stimuli. The three
previoussectionwasto investigatemelodyperception, subjectswho participatedin our previousbasicexperiwhich was taken operationallyas the ability of the ments had no difficulties with either the monotic or
524
Volume 51
Number 2 (Part 2)
1972
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
PITCH
OF
COMPLEX
dichotic tests. These results provide direct evidence
that the phenomenonunder study doesindeed reflect
normal musical behavior rather than a sophisticated
learned skill that is not directly relevant to melody
perception.
IV.
EVIDENCE
FOR
PERCEPTION
Since random
TWO
OF
variations
MODES
COMPLEX
in harmonic
OF
MUSICAL
TONES
number
from
one note to the next are highly unusual in normal
musicalpractice,we might wonderwhat would happen
in melody recognitionexperimentslike thosediscussed
in Sec.III if the harmonicsof eachnotevaried systematically rather than randomly. As was noted earlier, a
choice of the same harmonic number for every note
would lead to a trivial situation where retrieving the
frequencyof either the missingfundamentalor a single
upper harmonic would lead to a report of the same
melody.A more interestingstimuluswould be onewhere
TONES
soundshas beenstudiedactively and reportedin terms
of musicalpitch perceptionevoked by single sounds
with fixedharmonics(Seebeck,1841;Helmholtz, 1863;
Hermann, 1912; Fletcher, 1924; Schouten, 1940b;
Thurlow and Small, 1955; deBoer,1956; Flanaganand
Guttman, 1960; Schouten,Ritsma, and Cardozo,1962;
Ritsma, 1962; Smoorenburg,1970). Reviews of this
literature have recently been given by Plomp, 1967;
Small, 1970; Schouten,1970; and Ritsma, 1970. We
have chosento performexperimentsusingsequences
of
sounds with
random
harmonic
number
because we
found that successive
notesevokedin subjectsa sense
of musical interval and provided a context for that
feature of each sound that was being contrasted. In
addition, this choiceminimized the opportunitiesfor
behavioral responsesthat are directly correlated with
tracking of partials. Finally, it enabledus to demonstrate directly the role of fundamental retrieval in
musical
behavior.
Despite our departuresfrom earlier procedures,we
the fundamental
and harmonics describe different
believe
that our investigations involve the same
melodies.Such stimuli are given by any of the sound
phenomenon
as has been described in earlier work.
sequencesused earlier for which the harmonic numbers
were not all equal. Of the 81 possible four-sound De Boer (1956) reported pitch matches in which
sequences
usedin Sec. III to representeach four-note inharmonic complex tones comprisingfive or seven
melody, one sequencewas chosen for which no two partials with uniform frequencyspacingwere aurally
successive
noteswere representedby soundswith the matched to periodiccomplextoneswith a fundamental
same harmonic numbers. A recognition experiment that differedsystematicallyfrom the spectralspacing
of the inharmonic sound. Schouten, Ritsma, and
similar to the first part of the experimentin Sec. III
was performed. Four different four-note melodieswere Cardozo (1962) producedsimilar data for AM comused,whichwereunknownto the subjectsprior to the plexes(three partials) which are reproducedin Fig. 4,
experiment. Each melody was representedby one and Smoorenburg(1970) found essentiallythe same
four-soundsequence.Thesestimuli provideconsistent results using complexesconsistingof only two tones.
but different information dependingon whether the All these tone complexeswere presentedmonotically
subjectretrievesthe missingfundamentalor a partial. or diotically (i.e., one ear stimulated, or both ears
A number of subjectswere asked to listen to such identicallystimulated).Usinga musical-intervalmatchstimuliand to reportwhichnote sequence
they would ing paradigm, we obtained very similar results with
associatewith each of them. Some reported a note two-tone stimuli presented both monotically and
sequence which was consistent with the missing dichotically (Houtsma and Goldstein, 1971). Two
fundamentals;
othersreporteda melodycorrespondingadditionalexperimentswereperformedto examinethe
to one of the partials; one subjectwas able to report relation betweenthe findingsof pitch-matchingexperimelodies
described
by boththe upperandlowerpartial. ments and our experimentson interval identification.
Helmholtz(1863) mentionedthat complexperiodic The first was an interval identification experiment
soundscan be perceived "synthetically," i.e., the with the same paradigm as was used in the basic
complexis perceivedas one sound,havingone pitch; experiments,except that for each note the two freor "analytically,"i.e., partialsare heard individually, quencies (with successiverandom harmonic numbers)
eachonehavingits own pitch. Crossand Lane (1963) were uniformly shifted up or down in frequency by
showed that these two modes of musical behavior can
be controlledby previoustraining.Our experiments
do
indeedsupporttheseobservations
andshow,moreover,
that listeners can switch from one mode to the other
when this providesthem with relevant informationfor
a particular task.
V. RELATION
TO EARLIER
WORK
FUNDAMENTAL
RETRIEVAL
ON
For well over a century,the phenomenon
of fundamental retrievalin the perceptionof periodicmusical
some random
increment
less than
one-fourth
of the
fundamental frequency. This limit was imposed to
avoid pronouncedpitch ambiguities that are to be
expectedfor larger inharmonic frequency shifts from
the data shown in Fig. 4. Stimuli were presented
monoticallyat 50 dB SPL to one subject{AH) and at
40 dB to another (NH). We chosevaluesfor i and f0
that had yielded perfect identification performance
without the random frequencyshift. The results are
shown in Fig. 5. It is obvious that performance is
considerablyless than perfect, although the relevant
The Journal of the AcousticalSocietyof America
525
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
HOUTSMA
AND
n=7
,/
240
GOLDSTEIN
n=8
/
• //
n =9
/
o
//
=.
,•/
Fod'
/
//
n/=I0
,,/•/
o/
FIG. 4. Pitch as a function of
,..•/c),
/ • n=II
200
n=
12
18o
/•o
•
/•
•
160
/ u
,/
1200
/
/
//-
1400
/
/
//
•'•
/Z
/
1600
CENTER
/
•
/•B
/ '•
a ./
/
1800
FREQUENCY
/'
/• o
/
/o
. _..... z
o SUBJECTH
ß
f•
2000
2200
RR
the center frequency for a
three-componentcomplextone.
Stimulus intensity is 35 dB SL.
The fundamental frequency, p,
of an amplitude-modulated
soundcomprisingthe harmonic
frequencies (n- 1)p, np, and
(n+l)p is adjusted to match
aurally the musical pitch of
another amplitude-modulated
sound comprisingthe inharmanic frequenciesf-200 Hz,
f, and f4-200 Hz. The first
effect is defined by the dashed
lines of slope 1/n. (From
Schouten,Ritsma,andCardozo,
1962, with permission from
The Journal of the Acoustical
Societyof A •nerica).
2400
(Hz)
messagewas preservedin the differencefrequenciesof
the complex tones, indicating that inharmonic frequency shift of a complex tone does indeed alter its
pitch. Performancewas better than chance (12.5%
correct) and improved with increasing harmonic
number •. This was predictable from the particular
choiceof musicalintervals to be identifiedand the pitch
uncertainty introduced by the uniformly distributed
random frequency shift by using the simple decision
model outlined in the legendof Fig. 5.
In the secondexperiment we examinedthe musical
relevanceof the empirical relation between pitch of a
complex tone and inharmonic frequency shift of its
partials by applying this relation to our musicalinterval identification paradigm. The experiment
differed from the previous one in that the inharmonic
soundswere chosento preserve the original musical
intervals. For the first note a random frequencyshift,
positive or negative, was chosenwith a maximum of
one-fourth of the fundamental frequency; then the
empirical"first effect" relation (Schouten,Ritsma, and
Cardozo,1962) shownin Fig. 4 was usedto computethe
inharmonicfrequencyshift for the secondnote required
to give the originalmusicalinterval. Figure 5 showsthe
results; identification performanceis almost perfect.
Occasionalmistakes are easily accountedfor by the
additionaldifficultyfor the subjectof havingto identify
eight intervals in a roving key, sinceintervalsno longer
(1) Despite differencesin experimental procedure,
there is clear evidence that our studies on fundamental
retrieval behavior using interval identification reflect
the samebasicphenomenon
as the work of many other
investigatorson "periodicitypitch" or "residuepitch."
(2) Inharmonic tone complexesdo have a definite
musicalvalue, providedthat the deviation from the
harmonic situation is not so large that it will cause
pitch ambiguities.This value is, at leastfor the pitch
resolutionrequiredin our experiments,well described
by what is knownin the literatureas the "first effect"
of inharmonicfrequencyshift.
(3) The largedeparturesfrom the first effectwhich
havebeenreportedto occurundercertainmonaticand
diotic stimulusconditions(de Boer, 1956; Schouten,
Ritsma, and Cardozo,1962; Ritsma, 1970; Smoorenburg,1970;van denBrink, 1970)reflectonlya stimulus
modification
attributable
to aural combination
tones
(Goldsteinand Kiang, 1968; Ritsma, 1970; Smoorenburg, 1970), rather than essentialfeatures of the
central processorof musicalpitch.
VI.
DISCUSSION
The two currentlypopulartheoriesof musicalpitch
perception
are the frequencydetectiontheory(Helm-
holtz, 1863; Fletcher,1924) and the periodicity-detection theory(Schouten,
1940a;Licklider,1962).According to the formertheory,the cochlearspectrumanalyzer
started with the same note because of the random
maps the energyof the fundamentalfrequencytonoinharmonicfrequencyshift introducedin the first note. topically,and pitch is associated
with spatialposition
Having establishedthe relation betweenearlier work of the fundamental; if the stimulushas no energy at
on musicalpitch and our basicexperiments,we make the fundamental frequency, nonlinearitiesin the
the followingconclusions.
peripheralear will introduceit. Accordingto the
526
Volume 51
Number 2 (Part 2)
1972
.
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
PITCH
OF
COMPLEX
TONES
ioo
80--
Fro.
5.
Effects
of
inharmonic
fre-
quency shift on musical interval identification performanceof two subjects.
The solid line indicates performancefor
the random frequency shift paradigm
predicted by a simplifieddecisionmodel
which assumesthat (1) the musicalvalue
for each note is randomly distributed
with a rectangularpdf of width fo/2•,
(2) the means of the nine notes in Fig.
2(a) are uniformly separatedby f0/16,
and (3) the first note of each interval is
ignored,while the secondnote is detected
for maximum expected correct answers.
60--
4O
fo = 300 Hz
2O
OPEN
SYMBOLS'
CLOSED
0
2
latter theory, somemechanismis presumedto measure
temporal periodicitiesat the output of the cochlear
spectrum analyzer and then associate pitch with
temporal interval.
All the dichotic experimentsdiscussedin this paper
prove directly that neither energy at the fundamental
frequency nor fundamental periods in the cochlear
output are necessaryconditionsfor a subjectto retrieve
fundamental frequencies in a sequenceof periodic
sounds. This definite inadequacy of the periodicitydetection theory is a new finding. Earlier attempts to
reconcile the establishedsensationof a missing fundamental (Seebeck,1841)with Helmholtz'sfrequency-detection theory by meansof aurally generateddistortion
tones (Fletcher, 1924) had already proved inadequate. Musical pitch perception could be established
for which difference
SPL
A H(•)
40dB
SPL
N H (o)
CONTROLLED
SYMBOLS'
RANDOM
SHIFT
SHIFT
•
0
with stimuli
50dB
tone distortion
had to
be ruled out becauseof low stimulusintensity (Schouten, 1938; Plomp, 1967; Goldstein, 1967) or external
maskingat the differencefrequency (Licklider, 1954).
Furthermore, difference frequency did not designate
the musical pitch of inharmonic tone complexes
(Hermann, 1912; Schouten, 1940b; de Boer, 1956;
Schouten,Ritsma, and Cardozo,1962).
Considering these theories from a converse viewpoint, the results of the experimentsreported in this
paper show, moreover, that a sufficient condition for
fundamental tracking is indeed given by energy at
the fundamental, which can very easily be demonstrated, but probably not by fundamental periodsin
the cochlear output. The data suggest that such
fundamental periods are probably irrelevant for the
followingreasons:
4
6
8
(2) With monotic stimuli the possibilitiesfor fundamental periodsin the cochlearoutput are enhancedas
the harmonic number n is increased. Nevertheless,
interval identification performancealways deteriorates
with increasingharmonic number.
(3) All monotic stimuli for which identification
performancewasbetter than chanceeither consistedof
behaviorally resolvabletones, or generatedsuch tones
as combinationtones(Plomp, 1964; Goldstein,1967).
The similarity of the monoticand dichoticbehavior
and the close correlation
between the limits on funda-
mental retrieval and behavioral frequency resolution
strongly suggest that fundamentals of complex-tone
stimuli are retrieved by meansof a central mechanism
which operateson thosestimulustonesor combination
tones that can be resolved in the cochlea. This conclu-
sion is a radical departure from the theory of the
"residue,"which is definedas' "the joint perceptionof
thosehigher Fourier componentswhich the ear fails
to resolve" (Schouten,Ritsma, and Cardozo,1962).
The experimentsreported here imply that the unresolvedremainderof the harmonicseries,or residue,is
not responsible
for fundamentalretrieval. The finding
that performancefor dichotic stimuli is boundedby
harmonicnumbersimilarlyas undermonoticstimulus
conditionscannot be accountedfor by cochlearfre-
quencyresolution
alone;thecause
mustbemorecentral.
The finding that the only effectiveconveyorsof
musicalpitch are thosepartialswhichcanbe resolved
behaviorallyis actuallynot new. Ritsma (1962, 1963,
1967a,1967b)found that the existenceregionof the
"tonal residue" is bounded by an upper harmonic
number whosemagnitudedependson the complexity
(1) Monotic stimuli, which can provide cochlear of the stimulusand the experimentalparadigm; in
fundamental periods,give no better performancethan addition,he foundthat the mosteffectiveor dominant
dichotic stimuli: performance differences between pitch conveyorsare the harmonics3, 4, and 5. Our
monotic and dichotic conditions at higher intensities findingthat there is no noticeabledrop in performance
are well accounted
for bv aural combination
tones.
for harmonic numbers below 4 is not inconsistent with
The Journalof the AcousticalSocietyof America
527
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
HOUTSMA
AND
GOLDSTEIN
Ritsma's findings,sincethe effect is probably too small fibersand, at leastfor lowerfrequencies,
in the temporal
to make performancein our experimentsdeteriorate firing patterns of individual fibers (Kiang, 1965;
below perfect (Bilsen, 1971). Smoorenburg(1970) Whitfield, 1970). From our work it appearsthat all
concluded from his pitch matching data for two-tone
stimuli
that
the
effective
harmonic
numbers
for
fundamentalsof 200 Hz have an upper boundof about
9; this conclusionis supported and extended to other
fundamentalfrequenciesby our results.
The effectivenessof the lower partials alone in
conveyingmusical pitch is also consistentwith some
readily observedmusical phenomena.First, there are
somemusicalinstruments,e.g., piano strings (Young,
1952), which produce soundswhose higher partials
are inharmonic.
From our results we conclude that this
is not necessarilydisturbing, since only the lower
partials are effective in communicating the note.
Second,Mach's (1881) observation--thatdissonance
is
equally pronouncedwhen the soundsfrom two mistuned
simple tone sourcesare presented to one ear or to
separate ears, and can therefore not be attributed to
beating effectsin the cochlea--is also quite consistent
with our experimental findings. Simple tones whose
frequenciesare related by ratios of small integersare
effective conveyersof musical pitch independentlyof
whether they are presentedmonotically or dichotically.
Presumably, stimulus conditionsthat evoke no definitive musicalpitch contributeto a musician'sjudgment
of dissonance
(cf. Plomp and Levelt, 1965).
Little specificcan be said at this point about the
neural mechanism that mediates fundamental
retrieval.
Its closeconnectionwith behavioralfrequencyresolution suggeststhat neural signals that allow one to
recognizesimple tones and retrieve the missingfundamental of harmonic complex tones are common in
early stages of processing.Whatever kind of neural
mechanismis postulated, for complex-tonestimuli it
will operateon neural signalsderivedfrom peripherally
resolved tones. It is known that information
about the
frequency of such tones is-preservedafter the neural
transformationin the form of the placeof activenerve
the constraintscouldconceivablybe met by mechanisms
basedon either time or placeinformation.An important
simplification for further work on musical pitch is
offered by the empirical finding that fundamental
retrievals for dichotically and monotically presented
two-tone stimuli are essentially identical. Thus, the
basicpropertiesof the centralprocessor
of musicalpitch
can be investigatedwith the two-tone dichoticstimulus
that avoids confounding and irrelevant phenomena
caused By peripheral interactions among stimulus
partials. Earlier efforts to find cochlea-generatedplace
(yon B•k•sy, 1961) and time (Schouten,1940a)cuesin
responseto the periodicity or missingfundamental of
complex input stimuli now appear to be irrelevant;
mechanisms
more
central
than
the cochlea must
be
investigated.
ACKNOWLEDGMENTS
The authors are deeply indebted to W. F. Kelley,
D. J. Callahan, and L. D. Braida for their valuable
assistance in developing the instrumentation and
computer software that made these experimentspossible. Informative discussions
werecontributedby H. S.
Colburn, N. !. Durlach, D. M. Epstein, and W. M.
Siebert. This work was supported by the National
Institutes of Health (Grant 5 P01 GM14940-05 and
Grater5 T01 GM01555-05).
* This report is basedon a PhD thesissubmittedto the Dept.
of Electrical Engineering, Mass. Inst. of Technol., June 1971,
by A. J. M. Houtsma, and supervisedby J. L. Goldstein.The
basicresultswerepresentedat the 79th Meeting of the Acoustical
Society of America in Atlantic City, April 1970, [-A. J. M.
Houtsmaand T. L. Goldstein,J. Acoust.Soc.Amer. 48, 88(A)
(1970)].
t Also at the Eaton PeabodyLaboratory of Auditory Physiology, MassachusettsEye and Ear Infirmary, Boston, Massachusetts 02114.
REFERENCES
FLANAC,
AN, J. L., and GUTTMAN,N. (1960). "Pitch of Periodic
Pulseswithout Fundamental Components,"J. Acoust.Soc.Amer.
Observedon the CochlearModel with Nerve Supply," J. Acoust. 32, 1319-1328.
Soc. Amer. 33, 888-896.
F•.v.TCaV.
R, H. (1924). "The PhysicalCriterion for Determining
B•.smq, F. A. (1971). "Pitch Perceptibilityfor Multi-Component the Pitch of MusicalTone," Phys. Rev. 23, 427-437.
Signals,"J. Acoust.Soc.Amer. 51, 113(A).
Go•.•)sTv.
xN, J. L. (1967). "Auditory Nonlinearity," J. Acoust.
•)v. BOy.
R, E. (1956). "On the Residuein Hearing," PhD thesis, Soc.Amer. 41, 676-689.
Univ. of Amsterdam.
Go•.•)sxmN,J. L. (1970). "Aural CombinationTones," in FreDE BOER, E. (1956). "Pitch of Inharmonic Signals," Nature quencyAnalysisand PeriodicityDetectionin Hearing, R. Plomp
andG. F. Smoorenburg,
Eds. (Sythoff,Leiden,The Netherlands).
178, 535-536.
vo•qB•c•s•r, G. (1961). "Concerningthe FundamentalComponent
of
Periodic
Pulse
Patterns
and
Modulated
Vibrations
Goxa)sxv.
xN, J. L., BAY.
R, T., and KXANG,N.Y. S. (1971). "A
Theoretical Treatment of Latency, Group-delay and Tuning
Characteristicsfor Auditory Nerve Responsesto Clicks and
Tones," in Physiologyof theAuditorySystem,M. B. Sachs,Ed.
CROSS,D., and LANE, H. (1963). "Attention to SingleStimulus (National EducationalConsultants,Baltimore,Md.)
J. L., and KIANG,N.Y. S. (1968)."Neural Correlates
Properties in the Identification of Complex Tones," in Experi- GOLDSTEIN,
mentalAnalysisof theControlofSpeechProductionandPerception," of the Aural CombinationTone 2f,-f2," Proc. IEEE 56,
981-992.
Univ. of Michigan ORA Rep. No. 05613-1-P.
VAN•)v.NBRXN•, G. (1970). "Two Experimentson Pitch Perception: Diplacusisof Harmonic AM Signalsand Pitch of Inharmonic
AM Signals," J. Acoust. Soc. Amer. 48, 1355-1365.
528
Volume 51
Number 2 (Part 2)
1972
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40
PITCH
OF
COMPLEX
vo• H•.•mto•z, H. L. F. (1863).Die LehreyondenTonempfindungertals physiologische
Grundlage
fiir die Theorieder Musik
(F. Vieweg& Sohn,Braunschweig).
H•.a•t^•, L. (1912)."NeueVersuchezur Frageder Unterbrechungstone,"Arch. ges.Physiol.146, 249-294.
Hou•s•t^,A. J. M., and Go•t>s•.x•,J. L. (1971)."Perception
of
TONES
R•xs•^, R. J. (1970). "Periodicity Detection," in Frequency
Analysisand PeriodicityDetectionin Hearing,R. Plomp and G. F.
Smoorenburg,Eds. (A. W. Sijthoff, Leiden, The Netherlands).
Sacas, M. B. (1969). "StimulusResponseRelation for Auditory
Nerve Fibres: Two-Tone Stimuli," J. Acoust. Soc. Amer. 45,
1025-1036.
SACaS,M. B., and Kx^•o, N.Y. S. (1968). "Two-Tone Inhibition
of Complex
Tones,"Res.Lab. of Electron.,MIT Tech.Rep.484. in Auditory-Nerve Fibres," J. Acoust.Soc.Amer. 43, 1120-1128.
S^um>•.as,F. A. (1937). "The Mechanical Action of Violins,"
KI•O, N.Y. S. (1965).Disc_harge
Patternsof SingleFibersin the J.
Acoust. Soc. Amer. 9, 81-98.
Cat'sAuditoryNerve(MIT Press,Cambridge,Mass.).
SAum)•.as,F. A. (1946). "Analysisof the Tones of a Few Wind
Lm•c•I•)•.a,J. C. R. (1954). "PeriodicityPitch and 'Place' Instruments," J. Acoust. Soc. Amer. 18, 395-401.
Pitch," J. Acoust.Soc.Amer.26, 137(A).
Sc}touxE•, J. F. (1938). "The Perceptionof SubjectiveTones,"
Lic•c•n)•.a, J. C. R. (1962). "PeriodicityPitch and Related Proc. Kon. Nederl. Akad. Wetenschap41, 1086-1093.
Auditory ProcessModels," Int. Audiol. 1, 11-35.
S½}toux•.•,J. F. (1940a). "The Residue and the Mechanism of
M•c}t, E. (1881). "Zur Analyseder Tonempfindungen,"
Sit- Hearing," Proc. Kon. Nederl. Akad. Wetenschap43, 991-999.
zungsber. Kaisersl. Akad. Wissensch.,Mathemat.-Naturwis- Sc}toux•.•, J. F. (1940b). "The Perception of Pitch," Philips
sensch.,Klasse 92, II Abt.
Tech. Rev. 5, 286-294.
M•.a,
G. A. (1956). "The Magical Number Seven Plus or SCHOUTEN,
J. F., RITSMA,R. J., and CARDOZO,
B. L. (1962).
Minus Two," Psych. Rev. 63, 81-97.
"Pitch of the Residue," J. Acoust. Soc. Amer. 34, 1418-1424.
O}m, G. S. (1843). "Ueber die DefinitiondesTones,nebstdaran SCHOU•.N, J. F. (1970). "The ResidueRevisited," in Frequency
gekniipfterTheorie der Sirene und iihnlicher ton bildender Analysis and Periodicity Detectionin Hearing, R. Plomp, and
Vorrichtungen,"Ann. Phys. Chem.59, 513-565.
G. F. Smoorenburg,Eds., (A. W. Sijthoff, Leiden, The NetherP•o•t•, R. (1964). "The Ear as a FrequencyAnalyzer,"J. Acoust. lands).
SE•.•.CK, A. (1841). "Beobachtungenfiber einige Bedingungen
Soc. Amer. 36, 1628-1636.
P•o•t•, R. (1965). "Detectability Threshold for Combination der Entstehungvon T6nen," Ann. Phys. Chem. 53, 417-436.
S•t•LL, A.M., Ja. (1970). "Periodicity Pitch" in Foundationsof
Tones," J. Acoust.Soc.Amer. 37, 1110-1123.
Modern Auditory Theory, J. R. Tobias, Ed. (Academic, New
PLO•t•, R. (1967). "Pitch of ComplexTones," J. Acoust. Soc. York), Vol. I.
Amer. 41, 1526-1533.
S•tOO•EN]•Ua•,G. F. (1970). "Pitch Perceptionof Two-Frequency
PLOMP,R., and LEVELT,W. J. M. (1965). "Tonal Consonance Stimuli," J. Acoust. Soc. Amer. 48, 924-942.
and Critical Bandwidth," J. Acoust.Soc.Amer. 38, 548-560.
THUaLOW,W. R., and S•t•nL, A.M., Ja. (1955). "Pitch PercepPzo•tv, R., and S•tooaE•auag, G. F., Eds. (1970). Frequency tion for Certain Periodic Auditory Stimuli," J. Acoust. Soc.
Analysis and PeriodicityDetectionin Hearing (A. W. Sijthoff, Amer. 27, 132-137.
Leiden, The Netherlands).
WmTmEL•), I. C. (1970). "Central Nervous Processingin Relation
RtTS•tA,R. J. (1962). "ExistenceRegionof the Tonal Residue. to Spatio-Temporal Discrimination of Auditory Patterns," in
I," J. Acoust.Soc.Amer. 34, 1224-1229.
FrequencyAnalysis and PeriodicityDetectionin Hearing, R. Plomp,
R•TS•tA,R. J. (1963). "ExistenceRegionof the Tonal Residue. and G. F. Smoorenburg,Eds. (A. W. Sijthoff, Leiden, The
Netherlands).
II," J. Acoust.Soc.Amer. 35, 1241-1245.
YOUN%
R. W. (1955). "Inharmonicity of Plain Wire Strings,"
R•TS•tA,R. J. (1967a). "FrequenciesDominantin the Perception
of the Pitch of Complex Sounds,"J. Acoust. Soc. Amer. 42, J. Acoust.Soc. Amer. 24, 267-273.
191-198.
ZWmKE•, E. (1965). "Der Ungew6hnlicheAmplitudengangder
R•TS•tA,R. J. (1967b). "FrequenciesDominant in Pitch Percep- nichtlinearenVerzerrungendes Ohres," Acustica5, 67-74.
ZW•sLOCKI,
J. (1953). "AcousticAttenuationbetweenthe Ears,"
tion of Periodic Pulses of Alternating Polarity," IPO Ann. Progr.
J. Acoust.Soc. Amer. 25, 752-759.
Rep. 2, Eindhoven, pp. 14-24.
Musical Intervals: Evidencefor the Central Origin of the Pitch
The Journalof the AcousticalSocietyof America
520
Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 128.97.27.21 On: Tue, 17 Feb 2015 22:42:40