Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Alma Mater Studiorum University of Bologna, August 22-26 2006 Studying musical imagery: Context and intentionality Freya Bailes Sonic Communications Research Group, University of Canberra Canberra, ACT 2601, Australia [email protected] the stimuli, and melodic distinctiveness. Results point to the complementary roles of perceptual exposure and memory when participants intentionally generate a mental image of music. Although musical imagery is an intangible phenomenon, this paper argues that by examining converging evidence it is possible to discern commonalities worthy of further study. This overview also underlines the changing nature of imagery experience dependent on the contextual factors of the intention to image music and musical task. ABSTRACT The phenomenon of a conscious ‘inner hearing’ of music, when this music is not actually present, is known as musical imagery. Of interest to music cognition are when, where, why, what and how particular music is imaged. Given the methodological difficulties of imagery research, these questions have only just begun to be addressed. The research to date has begun to suggest fruitful areas to explore and useful methods to achieve this. This paper reviews converging evidence of musical imagery experience from a sampling study, interviews, and laboratory experiments. Relevant findings from a study that sampled the everyday occurrence of having a ‘tune on the brain’ are presented (Bailes, in press) as an introduction to some of the probable factors linked to the occurrence of an involuntary musical image. Analyses highlight the influence of recent exposure to particular music on what is subsequently imaged. Following this, an experimental study is described which measured the point of recognition (POR) of 120 melodies by 32 participants, with the goal of predicting POR as a function of different subjective measures of familiarity with Keywords Musical Imagery; Familiarity; Melodic distinctiveness; Recognition INTRODUCTION Musical imagery is the experience of having a conscious mental image of the sound of music that is not actually present. Musical imagery might be experienced during periods of silence, or might be superimposed as a slightly different mental representation to the actual music during listening. Moreover, it can occur spontaneously and unintentionally during everyday life as in the experience of a tune in the ‘mind’s ear’. When this is persistent, it is known as ‘earworms’. Auditory imagery can also be an involuntary corollary of musical activity, such as anticipating the next track on an album while listening to music, or working towards an ideal musical sound in performance based on internally ‘hearing’ how it should sound. Finally, a mental image of music might be intentionally brought to mind when reflecting on a particular piece of music, in the ‘silent’ analysis of musical score, or in composition. The potential breadth of musical imagery experience, from the involuntary everyday ‘tune on the brain’ to a deliberate and functional role in music, renders it both useful but also challenging as a subject of psychological research. Of in- In: M. Baroni, A. R. Addessi, R. Caterina, M. Costa (2006) Proceedings of the 9th International Conference on Music Perception & Cognition (ICMPC9), Bologna/Italy, August 22-26 2006.©2006 The Society for Music Perception & Cognition (SMPC) and European Society for the Cognitive Sciences of Music (ESCOM). Copyright of the content of an individual paper is held by the primary (first-named) author of that paper. All rights reserved. No paper from this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval systems, without permission in writing from the paper's primary author. No other part of this proceedings may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval system, without permission in writing from SMPC and ESCOM. ISBN 88-7395-155-4 © 2006 ICMPC 805 ICMPC9 Proceedings erate generation of an auditory image in a melody recognition task. terest to music cognition are when, where, why, what and how particular music is imaged. The significant methodological obstacles associated with researching an essentially private, subjective phenomenon have slowed progress. Researchers have variously asked individuals to report their imagery experience (Mainwaring, 1932; Retra, 1999), conducted experiments to infer patterns of behavior associated with imaging sound (Halpern, 1988a, 1988b; Weber & Brown, 1986; Intons-Peterson, 1992; Repp, 2001), and applied neuroscientific technologies to examine brain activity during imaging (Zatorre & Halpern, 1993; Zatorre, Halpern, Perry, Meyer & Evans, 1996; Janata, 2001; Neuhaus, 2001). Methodological challenges for research in musical imagery are seemingly greater than for visual imagery, which has a long tradition of inquiry. This is partly because music is more obviously dynamic in its progression through time, and in experimental tasks it requires sustained mental effort. Given the difficulties inherent in musical imagery research, there is a paucity of knowledge concerning imagery for sound, and in particular, music. However, by beginning to assemble findings and reports obtained using a variety of different research methodologies, we might begin to find converging evidence in response to some of these questions. This is the case for imagery for timbre, where evidence from laboratory experiments (Crowder, 1989; Pitt & Crowder, 1992; Auhagen & Schoner, 2001; Bailes, 2002; Halpern, Zatorre, Bouffard & Johnson, 2004), sampling studies (Bailes, in press a) and interviews (Bailes, 2002) collectively point to an auditory dimension that is possible to image, but one which is more prone to be imaged when it coincides with clear sound source identification, high familiarity, and when it is subjectively experienced as a meaningful component of music. While it is informative to examine a particular problem using a variety of methodologies, it is also important to acknowledge different manifestations of musical imagery in a variety of contexts, and for a variety of roles. Experimental techniques call for the deliberate generation of a mental image, while sampling techniques are designed to observe the spontaneous, everyday occurrence of the phenomenon. Research by the author sought to elucidate the relationship between musical perception and imagery ranging from the everyday ‘tune on the brain’ to the functional role in specialist musical activity. Sampling methods, case study, focus group discussion, observation of pedagogical activity, and laboratory experiments were used to this end (Bailes, 2002). Converging evidence was found contributing to the aforementioned description of timbral imagery, and in support of the notion that musical imagery is derived from perception, functioning in similar and complementary ways. This paper sets out two contrasting studies as an example of how converging evidence of musical imagery experience might be assembled. The focus is on the role of familiarity and melodic distinctiveness in both spontaneous, everyday ‘tune on the brain’ imagery, and in the delib- ISBN 88-7395-155-4 © 2006 ICMPC INVOLUNTARY MUSICAL IMAGERY Method A sampling study by the author sought to explore the often involuntary, spontaneous musical imagery of everyday life (Bailes, in press a, in press b). Eleven music students volunteered to participate in the study, in which they were signaled on their mobile phones at random times throughout the day, for a period of seven days. At each signal, participants filled out a short experience sampling form which asked for information about their current activity, location and mood, and for details of any real or imaged music they were currently experiencing. Many questions were addressed by the study, which aimed in part to question why and under what circumstances people (in this instance music students) spontaneously image music. Results A total of 417 forms were completed, of which 146 describe imagery episodes. The findings are suggestive: respondents made strong links between the music they had recently heard, rehearsed or performed, and its consequent mental re-presentation. 43% of respondents had actually heard the music they were imaging during the period since the last signal. An open-ended question asked participants to reflect on the possible reasons for imaging the music they experienced. 54 episodes or 51% of responses to this question can be categorized as ‘having heard the music previously’ or ‘having analyzed it’. Response categories and the number of episodes fitting each category can be seen in Table 1. Table 1. Reasons for imagining the music Reasons given Previously heard or analyzed the music Music to be performed Frequency 21 episodes Talked about piece 6 episodes ‘No idea’ 6 episodes Person association 4 episodes ‘Sticky’ music/automatic 4 episodes Other 12 episodes Total 107 episodes 54 episodes Respondents were asked to name the music they heard or imaged when possible. It is interesting to note that they were significantly better at naming the music they imaged than the music they were physically exposed to during the course of the day (χ2 = 41.19; d.f. = 1; p< 0.0005). This suggests they were highly familiar with the subject of their imagery. Even the composition students who participated 806 ICMPC9 Proceedings in the study tended to image music other than their own compositions, with only 8 out of 123 episodes involving imagery in composition. duration of successive pairs of notes, e.g. longer-same) were calculated from statistical analyses of 16,069 Western themes and melodies, using Humdrum (Huron, 1994). These probabilities formed the basis for the measure of each stimulus melody’s distinctiveness. Discussion For the music students in this study, a strong influence of exposure to music on subsequently imaging that music was revealed. The question can be taken further to ask whether mere exposure to music is a sufficient criterion to subsequently hearing it ‘on the brain’. A series of interviews with professional musicians (Bailes, 2002) reveals that choruses, hooks and jingles are more likely to be imaged spontaneously than avant-garde or improvised music recordings. This could be because of the typically greater exposure and repetition of the former, or it might relate to the structure of the music itself. The following section addresses the combined role of familiarity and distinctiveness in the deliberate re-presentation of music in a melodic recognition task. Procedure The task was to identify each melody as soon as possible, by naming it, singing it, or providing enough information about it to convince the experimenter of successful recognition. The method adapted the gating paradigm (Dalla Bella, Peretz & Aronoff, 2003), and involved playing the first note, followed by the first two notes, followed by the first three notes, etc. (with the rhythm and articulation of the original melody). Participants were asked to indicate when (and if) they recognized the melody. They were given no advanced knowledge of the pool of possible stimuli. Commonly, participants reported imaging a continuation of the opening cue in the case of successful identification. The note at which, in the opinion of the experimenter, they successfully identified the melody was deemed to be the POR. Using this procedure, recognition data for 960 cases were collected. At the end of the melody recognition procedure, participants were asked to provide subjective familiarity estimates for each of the stimulus melodies, as well as an exposure estimate (how many times they estimated having heard the melody during the last 12 months), and a recentness estimate (when was the last time they heard the melody). As described above (‘Stimuli’), distinctiveness values were calculated for each note of each stimulus melody. Both ‘momentary distinctiveness’ (for a given note), and ‘accumulated distinctiveness’ (accumulated for all notes up to and including a given note) measures were used. Separate distinctiveness values were used for interval, scale degree, scale-degree dyad, combined pitch information, rhythmic stress, relative duration-dyad, combined rhythmic information, and combined pitch and rhythmic information. Using the predictor variables of familiarity and distinctiveness, a dynamic model to predict POR was created. Half the POR data were used to test the model’s ability to predict 1) that a given listener will not recognize a melody and 2) the POR plus or minus 1 or 2 notes. As a baseline model, the existing POR data were scrambled and paired with their unscrambled version, using one case to predict another. INTENTIONAL MUSICAL IMAGERY An experimental study was conducted to predict the point of recognition (POR) of melodies as a function of different subjective measures of familiarity and melodic distinctiveness. Listeners were exposed to melodic fragments and asked to indicate when they recognized a melody. While the primary concern was not an exploration of musical imagery, the task reportedly evoked an auditory image of the melodies in question. It was predicted that POR would be earlier for more familiar melodies, and that it would correlate with melodically ‘distinct’ moments. Method Participants 32 volunteers from the Ohio State University community with a mean age of 27 years participated. Half were music students, while half had no formal musical training. Stimuli Participants were presented with a sub-sample of 120 melodies, played by the experimenter on a synthesizer. The stimulus melodies comprised traditional folk melodies and classical music themes. They were selected to ensure that the familiarity data and the distinctiveness measures of the melodies would also exhibit a wide variance. Intuitively, a melody is likely to be recognized sooner when it comprises a unique or distinctive musical pattern. For example, the opening notes of Maria from Westside Story are more distinctive than the opening notes of On Top of Old Smokey. This concept of distinctiveness can be formulated in terms of the probability of occurrence of particular musical events within a particular musical tradition. The probabilities of occurrence for melodic interval, scale degree, scale-degree dyad (e.g. scale degree 1 followed by 3), rhythmic stress, and relative duration-dyad (relative ISBN 88-7395-155-4 © 2006 ICMPC Results Of those melodies that were recognized, the mean POR was 5.4 notes with a mode of 4 notes. POR occurred earlier for more familiar melodies (r = -0.77, p < 0.001). Models combining subjective familiarity and distinctiveness had the best predictive accuracy, but tended to perform equally with respect to either momentary or accumulated distinctiveness. Similarly, there was little difference between models that used pitch- or rhythm-based distinctiveness meas- 807 ICMPC9 Proceedings ures to predict POR.1 Results point to the complementary roles of subjective familiarity and the information structure of the melody when participants intentionally generate a mental image of music. is an essentially intangible phenomenon, by triangulating methods and examining converging evidence it is possible to discern commonalities worthy of further study, such as the influence of perceptual exposure and distinctive melodic structure on image formation. However, assumptions that imagery is equivalent in different contexts should be challenged. In the first study, respondents reported imagery that was freely experienced during the course of everyday life (though see Bailes, in press a, for discussion). No constraints were placed on the type of music to be imaged, nor on the quality of that experience. Consequently, respondents reported not only aspects of melody, but also song lyrics, expressive nuance, harmony, timbre and kinesthetic imagery. It is less likely that the participants in the melody recognition task mentally represented musical dimensions beyond the minimal melodic content cued by the isolated notes on the synthesizer (and probably a mental image of any associated lyrics). Importantly, imagery experience depends upon the contextual factors of the intention to image music and musical task. A comprehensive account of musical imagery should combine ecologically valid methods of exploring the circumstances and nature of spontaneous, everyday imagery, while combining experimental techniques to elucidate the cognitive constraints of imagery for sound and music. COMMONALITIES The two studies reported in this paper concern very different kinds of mental imagery for music. The first concerns the mostly involuntary, everyday experience of having a ‘tune on the brain’. The second calls for the intentional mental representation of melody in response to an auditory cue. It is not obvious that such different forms of musical imagery should share the same experiential qualities. Both the physical and mental contexts of the imagery experience along with the intention to image music differ greatly. However, in trying to understand why we spontaneously image certain music it is informative to investigate what musical dimensions we are most able to image. For instance, the sampling study of everyday imagery experience demonstrated that, for music students, a strong association exists between the music imaged and their familiarity with that music through exposure to it. In a study that deliberately probed participants’ ability to recognize melody, a strong relationship was again demonstrated between the subjective measurement of familiarity, and the speed with which participants were able to mentally generate an image for that melody. While it is tautological in a recognition task to state that recognition is dependent on familiarity, it is less obvious that music students, including composition students (who might be expected more than others to generate their own original imagery), unintentionally image music to which they have been the most exposed and are the most familiar. In this paper, familiarity and distinctiveness have been described separately, yet they may be viewed as complementary dimensions of musical experience. In the melodic recognition study, a distinct event has been defined as rarely occurring in relation to typical structural patterns found in Western music. Accordingly, a distinct melody is an unusual and schematically less familiar melodic form. When reflecting on the experience of imaging music, it might seem that not only highly familiar melodies stick in the ‘mind’s ear’, but also highly distinct events, such as an unusual timbre or rhythm. A combination of subjective (personal) familiarity measures with distinctiveness measures (based on an assumed schematic representation of Western melodies) predicted POR for familiar melodies with reasonable success. ACKNOWLEDGMENTS The sampling study reported in this paper was conducted as part of my doctoral research at the University of Sheffield, UK, sponsored by the Arts and Humanities Research Board. The melody recognition project was jointly undertaken with Professor David Huron during the author’s postdoctoral fellowship at the Ohio State University (2003-4). I would like to thank Andrea Halpern and Roger Dean for their helpful comments on a previous version of this paper. REFERENCES Auhagen, W., & Schoner, V. (2001). Control of Timbre by Musicians – A Preliminary Report. In R. I. Godøy & H. Jørgensen (Eds.), Musical Imagery (pp. 201-217). Lisse: Swets & Zeitlinger. Bailes, F. (in press, a). The Use of Experience-Sampling Methods to Monitor Musical Imagery in Everyday Life. Musicae Scientiae. Bailes, F. (in press, b). The Prevalence and Nature of Imagined Music in the Everyday Lives of Music Students. Psychology of Music. CONCLUSION Juxtaposing the two studies reported in this paper demonstrates how aspects of imagery experience can be examined using different methodologies. Although musical imagery 1 Bailes, F. (2002). Musical imagery: Hearing and imagining music. University of Sheffield: PhD thesis. Crowder, R. G. (1989). Imagery for musical timbre. Journal of Experimental Psychology: Human Perception and Performance, 15 (3), 472-478. Please note that the models and their predictive success are not presented in detail in this publication. ISBN 88-7395-155-4 © 2006 ICMPC 808 ICMPC9 Proceedings Dalla Bella, S., Peretz, I., & Aronoff, N. (2003). Time course of melody recognition: A gating paradigm study. Perception & Psychophysics, 65 (7), 1019-1028. Mountain, R. (2001). Composers and imagery: Myths and realities. In R. I. Godøy & H. Jørgensen (Eds.), Musical Imagery (pp. 271-288). Lisse: Swets & Zeitlinger. Halpern, A. R. (1988a). Perceived and Imagined Tempos of Familiar Songs. Music Perception, 6 (2), 193-202. Neuhaus, C. (2001). Mental Images of Musical Scales. In R. I. Godøy & H. Jørgensen (Eds.), Musical Imagery (pp. 77-93). Lisse: Swets & Zeitlinger. Halpern, A. R. (1988b). Mental Scanning in Auditory Imagery for Songs. Journal of Experimental Psychology, 14 (3), 343-443. Pitt, M. A., & Crowder, R. G. (1992). The role of spectral and dynamic cues in imagery for musical timbre. Journal of Experimental Psychology: Human Perception and Performance, 18, 728-738. Halpern, A. R., Zatorre, R. J., Bouffard, M., & Johnson, J. A. (2004). Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia, 42, 12811292. Repp, B. H. (2001). Expressive Timing in the Mind’s Ear. In R. I. Godøy & H. Jørgensen (Eds.), Musical Imagery (pp. 185-200). Lisse: Swets & Zeitlinger. Huron, D. (1994). The Humdrum Toolkit; Reference Manual. Menlo Park, CA: Center for Computer Assisted Research in the Humanities. Retra, J. (1999). An Investigation into the Musical Imagery of Contemporary Composers. University of Sheffield: MA dissertation. Intons-Peterson, M. J. (1992). Components of Auditory Imagery. In D. Reisberg (Ed.), Auditory Imagery (pp. 4571). Hillsdale, NJ.: Lawrence Erlbaum Associates. Weber, W. J., & Brown, S. (1986). Musical Imagery. Music Perception, 3 (4), 411-426. Zatorre, R.J., & Halpern, A.R. (1993). Effect of unilateral temporal-lobe excision on perception and image of songs. Neuropsychologia, 31 (3), 221-232. Janata, P. (2001). Neurophysiological Mechanisms Underlying Auditory Image Formation in Music. In R. I. Godøy & H. Jørgensen (Eds.), Musical Imagery (pp. 27-47). Lisse: Swets & Zeitlinger. Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E., & Evans, A. C. (1996). Hearing in the Mind’s Ear: A PET Investigation of Musical Imagery and Perception. Journal of Cognitive Neuroscience, 8 (1), 29-46. Mainwaring, J. (1932). Kinaesthetic Factors in the Recall of Musical Experience. British Journal of Psychology, 23, 284- 307. ISBN 88-7395-155-4 © 2006 ICMPC 809