Download The Screen as Performer: Perspectives on composing with moving

The Screen as Performer:
Perspectives on composing with moving image for the concert environment.
Adam Melvin
Throughout history, music has played an important role as a collaborative force in
combination with a variety of artistic media, forging a number of long standing hybrid
formats. In the modern age, it is music and moving image perhaps more than any
other pairing that has evolved into the most prominent, widespread and arguably
successful collaborative partnership.
As technology has advanced, so too has the willingness to explore the possibilities of
combining the two media in a variety of artistic environments, beyond the long
established realms of cinema, e.g. art installation. At the same time, musicians and
composers have increasingly absorbed moving image into their own creative and
performance environments as a means of increasing artistic expression. Yet while
music and the moving image continue to blossom together in the gallery and cinema,
the same relationship when transferred to the live concert environment can still
constitute a somewhat problematic combination for composers, film makers, video
artists and performers alike.
Incorporating Moving Image in Live Musical Performance – Recent Trends
If we look at the past few decades, it is pop more than any other musical genre, that
has led the way in exploiting the use of moving image as a means of enhancing its
2
own musical content, both as a commercial and creative tool. Music video is, and has
been for some time now, as much a part of the very fabric of popular music as the
music itself. Lately, developments in interactive technology along with an increased
interest in searching for new ways in which to present the pop concert as a more
lavish affair, have reconfigured a further role for music video within the popular
music sphere: that of a performative tool within the live concert environment.
Whereas once video was used primarily to capture a live performance, it now often
features as an integral part of the performance itself.
U2’s Zoo TV tour of the early 90s is one particularly extravagant, if not entirely
groundbreaking examplei of a band giving greater prominence to moving image’s role
in their live performances. Incorporating an extensive use of video material on-stage,
complete with ‘channel hopping’ screens and an interactive audience video booth,
Zoo TV transformed U2’s performances from stadium rock concerts into visually
striking, almost circus-like shows. This not only enhanced their impact as a live act
but also served as a means of heralding and embodying their rebirth as a band at the
time, to the extent that the visual and interactive aspect of their concerts became as
important an artistic statement (some would argue more important) than the actual
music.
More recently, Damon Albarn and Jamie Hewlett’s Gorillaz have sought to turn the
pop concert on its head entirely by replacing live musicians with a ‘virtual band’ in
the form of four animated performers who appear either on screen or as projected 3dimensional holograms on stageii, creating a visually stunning live performance that
3
seems to capture and personify the urban, almost playful style of the band’s music as
well as facilitate the collaborative nature of their work.
In recent years, jazz and classical musicians, particular those specialising in
contemporary music, have sought to incorporate moving image in their own
performances in similar, albeit often less ornate, ways. Nowadays, the use of moving
image either to provide a more interesting performance environment or
accompaniment to the live performance of a musical work, or perhaps as part of a
more integrated collaborative piece is commonplace. Indeed, the growing interest in
staging musical performances in venues other than the traditional concert hall, i.e.
spaces somewhat free of the long-established rituals and conventions of concert
performance, would seem to have increased such practices. Elsewhere, ensembles and
orchestras have sought to recreate the cinema experience within the concert hall itself,
albeit with a heightened sense of a film’s musical content, by presenting existing
films with live concert performances of their soundtracks seemingly in an attempt to
build on the commercial popularity of film music but also perhaps to readdress the
imbalance between moving image and score which exists in the cinema by
heightening the role of the musical component through the unique dramatic quality of
live musical performance.
Problems
The main issue here is that, in each case, there can be an air of redundancy about the
whole thing in that either the visual element or the musical performance feels
somewhat disposable. A musical composition is a complex work of art in its own
4
right. Since often the inclusion of moving image is something of an afterthought,
largely unrelated to the original creative process that produced the music in the first
place (certainly a large number of performances incorporating moving image would
seem to fall into this category) no matter how striking and dynamic the visuals might
be, they often serve little artistic purpose other than one of enhanced performance
setting. At best they might articulate or draw attention to the music’s structural or
compositional make-up – at worst, they constitute little more than moving wallpaper.
And, as wonderful as it is to hear a film score ‘in the flesh’ so to speak it fails to offer
us a great deal more, in the artistic or compositional sense, than experiencing the film
in the cinema.
When the work is the result of a more integrated collaborative partnership between
composer and video artist, the result is arguably more interesting but it too can veer
into the territory of the two examples mentioned above - either one or both elements
feel too disposable or the work feels like it would be just as well served by having a
recorded rather than live musical part, perhaps existing as a cinema or installation
event. That is not to say that such works are somehow lacking artistically, they just do
not seem suited to the live concert environment.
Hybrid art forms
Naturally, these arguments have certain repercussions for the composer concerning
his/her approach to working with moving image. It is perhaps worth considering at
this point exactly what composers (myself included) and video artists might be trying
to achieve by working in collaboration with one another. Presumably, the tendency is
5
to try and create a hybrid work with a greater sense of integration or more valid use of
the two media together. Or, to put it another way, to search for a more balanced, less
subservient role for both elements in conjunction with one another that might
ordinarily exist in, for example, film music practice, perhaps even achieve a true
equilibrium between the two, while attempting to maintain our role as concert
composers and video makers alike.
‘Multimedia’ is the somewhat unhelpful, all-encompassing, umbrella term that gets
used a great deal when referring to hybrid works that fall outside the boundaries of
traditional definitions. A more useful insight in to what takes place in a collaborative
piece is offered by Jerrold Levinson in his essay, Hybrid Art Forms and, more
recently, Simon Shaw-Miller, in his book, Visible Deeds of Music. Both writers refer
to three distinct categories of collaborative art form:
The first example is juxtaposition or multidisciplinaryiii works: Those which
incorporate products of two or more different media and present them alongside one
another as one larger whole yet the individual component parts used maintain their
original identities. Examples of this might include the collaborative works of John
Cage and Merce Cunningham: While both elements may share conceptual and
temporal properties, their inner workings remain largely independent and their
identities distinct.
The second is synthesis or interdisciplinary. This is essentially the opposite of
juxtaposition and is perhaps best exemplified by Wagner’s notion of the
Gesamtkunstwerk. Here, two or more artistic elements merge into a third ‘animal’ so
6
to speak. Rather than coexisting, they fuse so that, supposedly, if one element is taken
away, the work ceases to exist.
Finally, and more unusually, there is transformation or crossdisciplinary. In many
ways this is similar to synthesis but, in this instance, there is unequal parity between
the artistic elements - one element is modified in the direction of another, taking on
its characteristics. The example both writers use is kinetic sculpture i.e. sculpture that
preserves its identity as sculpture but which adopts the movement and temporality of
dance.
As useful as these categorisations are, pigeonholing hybrid works so definitively can
prove somewhat difficult. The kind of work to which composers aspire by
incorporating moving image is often more elusive. Indeed, I would actually argue that
works can, and often do involve elements of all three. That said, these definitions do
offer a useful insight into the interplay between music and image in such hybrids as
well as provide possible solutions to combining the two. In any case, both these
writers, amongst others, refer to the importance of intersecting points of similarity and
contrast between two media in a hybrid work to create, what Nicholas Cook refers to
as emergent meanings (1998): those which are the unique result of two media
working in tandem with one another, rather than the material of individual parts and
are thus paramount in contributing to the potential success of a piece’s dynamic.
The Question of Live Musicians
7
Despite the depth of discussion that has taken place concerning artistic hybrids,
virtually all writings on the subject fail to mention one crucial issue for the composer
writing for live performance with moving image, one which potentially holds the
most significant problems as well as perhaps the best solutions for tackling the
format: the question of live performers.
Composers differ from most other creative practitioners in that they require a ‘middle
man’, in the form of musicians, to realise their work. Without getting into a lengthy
discussion about the composer-performer relationship or indeed the live music versus
recorded music debate, one has to acknowledge that live musical performance plays a
huge part in the creative process of a musical work.
When we consider this in relation to moving image, it raises an immediate practical
issue concerning temporality. Film, by and large, is a fixed entity. No matter how
many times you ‘play’ a film, the duration will be exactly the same, down to the very
last frameiv. Live music is not. A performance by even the most temporally acute
performers will always be slightly longer or shorter each time. This of course raises
questions of synchronicity. The use of click tracks, timers or cueing performers from
events on screen, all of which can be unnatural for musicians performing ‘live’, may
be needed. As a result, the composer and video artist may well have to consider only a
sporadic use of synchronism in their work, perhaps even exploring (using the terms
above) a more juxtapositional relationship between the two elements employed.
8
One way round this is to involve a fixed entity in the musical part through electronics
that can synchronize to the temporal frame of the film. Another possibility is to
involve live manipulation of the moving image although this is something of a
specialism among video artists in that it often involves having to ‘perform’ the
moving image in some way; the crossdisciplinarity of such an approach might
perhaps be too far removed from the working practice of many. Indeed, this approach
seems to lend itself to a more improvised format where neither element is temporally
or structurally fixed.
The performances of British saxophonist, Evan Parker and his Electro-Acoustic
Ensemble involving live video manipulation by artist Kjell Bjørgeengen have yielded
intriguing and dynamic results in this sense. Essentially an improvised music
ensemble comprising saxophone, piano, double bass, viola, electro-acoustic, laptop
musicians and video screens, Parker’s group extends the existing performative
relationship between instrumentalist(s) and sound processor(s) to include the video
artist by adopting similar compositional processes across all its component elements.
Thus, just as the ensemble’s electronic music part uses a combination of pre-recorded
sound and live laptop manipulation of the instrumentalists’ improvisations as the basis
of its material, the video component mixes pre-recorded footage with flickering,
abstract moving images which, thanks to audio-visual technology, are generated and
automated in response what takes place sonically during the performancev. The result
is an improvised performance where the video material occupies a textural and
dialogic role that is equal to that of the electro-acoustic musicians and instrumental
performersvi.
9
Live Performance – A Visual Event
Perhaps the most important issue regarding live performers however, is the fact that
they constitute a visual element in themselves. During any live musical performance
we, as members of the audience, have our gaze constantly drawn to the various
sources of the music that is being created, in order to perhaps gain a greater sense of
connection with what is being heard. Indeed, the ‘presence’ of the performer or
performers – how they appear to play the music – is hugely important in the
communicative experience of a performance. Even if we close our eyes during a
performance, we are consciously trying to shut out visual events to focus purely on
the music. It is no accident that, in the more traditional hybrid formats where music
appears, e.g. ballet and opera, the musicians are deliberately hidden from view in the
orchestral ‘pit’. Perhaps all of this is why electro-acoustic concerts can sometimes feel
slightly awkward to us as an audience and why, so many of us close our eyes during
them. In an environment where the musical language is so sonically and spatially rich,
staring at an empty stage or a couple of speakers constitutes a rather bland and plain
odd frontal view. The temporal advantages already mentioned aside, is it any wonder
then that electro-acoustic composers have been so keen to embrace moving image for
their work?
Once we acknowledge that music in its performed state is a visual as well as aural
experience, then we can argue that it is actually a hybrid in its own right in the form
of a crossdisciplinary work – it reaches over into the realms of theatre. There are of
course, numerous examples where composers and performers have exploited this
phenomenon without the use of any moving image. One such example is Harrison
10
Birtwistle’s Ritual Fragment. Scored for sinfonietta line up, the piece involves
individual players walking to the centre of the stage to perform solo passages over a
sustained musical backdrop. While the composition is successful in its own right, the
theatrical gesture of the movement of the players serves to heighten its effect, giving
the performance a ceremonial quality which both isolates the solo passages as grand
gestures as well as helping to enhance the steady, measured progress of the piece's
structure. Another, Berio’s Formazioni, for orchestra, fragments and reconfigures the
conventional orchestral stage plan – arguably the concert format most heavily steeped
in visual traditions - by placing groups of musicians in positions they would not
ordinarily appear. The overall effect is a piece which challenges our preconceptions
concerning the ritual of orchestral performance from the outset in a piece that is often
visually confusing yet, as a result, incredibly stimulating.
By considering and manipulating the visual impact of performers in conjunction with
moving image we can heighten this existing hybrid/visual quality in live performance
further by creating a platform for the image to enter as a performer in its own right
while maintaining a non-subservient role for the music, thus transforming a musical
piece into a hybrid work. After all, if we return to our opening examples, is this not
similar to what pop has done, utilizing the substantially visual aspects of one of its
own constituent parts (the rock concert) to absorb another (video) as a means of
enhancing its essence as a live performance entity? Indeed, the key to the success of
this is the combination of performer and video. It is perhaps worth pointing out that
the majority of live performances by our second earlier example, Gorillaz, have
featured guest musicians onstage (De la Soul, Ike Turner, the Manchester Community
Gospel Choir amongst othersvii). It would seem that, in this case, onscreen animation
11
is not enough; the performance needs live musicians or it becomes merely a cinematic
event.
For the composer/video artist collaborative, such an approach potentially offers
further considerations concerning compositional hybridity. Since moving image will
perhaps have to utilise its more crossdisciplinary elements that sympathise with
musical materials - e.g. texture, temporal rhythm and so on - to achieve a sense of
synthesis with a composition, the visual aspect of the performers can allow it to
perhaps retain more of its own visual qualities while potentially allowing a greater
and more integrated sense of visual, aural and spatial interaction between the two
media. In turn, this can allow for a more defined or indeed, redefined, sense of
performance environment; the hybridity existent in the piece’s construction can
facilitate a level of crossdisciplinary hybridity to the performance space in which it is
realised.
Examples
Perhaps the reader will forgive me if I now turn to two examples from my own work
that incorporate and, I hope, shed some light on some of the issues touched upon thus
far.
Speak (2004)
The first piece, Speak, is the result of a collaboration with video artist, Mark
Melvinviii. It is scored forix soprano saxophone, female voice, electronics and video
12
screen which takes the form of a third performer whose face we never see in its
entirety - in a nod to Beckett’s play, Not I, the character of ‘mouth’ (Fig. 1).
Insert Fig. 1
1: Sarah Dacey and Adam Melvin performing Speak at the Sassoon Gallery,
Peckham, London, 2008
One of the immediate issues of combining live performers with video is the
coexistence onstage of both a real, physical visual element
- the performers
themselves – as well as an artificial or synthetic one in the form of the video screen.
Speak deliberately attempts to exploit this apposition as a way of achieving a greater
sense of integration to the piece firstly, by considering the screen as part of the
ensemble and secondly, by incorporating both live and electronically generated
musical material.
The piece’s musical material is based around streams of text that consist of syllables
from the International Phonetic Alphabet and fragments of words from various
dialects fused together to create a kind of hybrid language of sorts x. It thus focuses on
the rhythmic patterns and musical nature of speech, rather than word meaning to
propel itself. Both sung and spoken material in the vocal line is developed and
expanded in a pre-recorded electronic part, which both revolves itself around the live
singer and provides a sonic 'backdrop' to the piece. The saxophone, meanwhile, works
in dialogue with the other two parts by intertwining with and punctuating the vocal
line and featuring, at times, in the electronics; its role can be seen as providing an
elusive, more traditionally 'musical' element to the piece. Finally, the video part takes
13
on the role of a second singer trapped inside of a screen. Very much a performer in its
own right, the visual part flirts between the tape and live vocal parts to articulate the
piece’s musical material.
If we return to Levinson’s categories, Speak includes elements of both synthesis and
juxtaposition. The video screen forms the focal point of the interaction between the
various forces employed in the piece. Sometimes it syncs with the electronic part,
sometimes with the singer, occasionally even with the saxophone, while elsewhere, it
works independently of the other elements; it constantly plays a guessing game with
the audience which, of course, the essentially nonsensical text is also doing.
Moreover, the piece takes on a further level of interaction through a strong spatial
dynamic, again, initiated by the video screen. The striking frontal visual interplay
between the rather relentless and oversized mouth and the live musicians is
complimented by a 4-channel surround speaker system which expands and shrinks the
aural landscape of the piece. Thus, on-screen movement is complimented by onstage
movement as well as the movement of the piece’s sonic material around the
auditorium in order to increase dramatic effect, The result is something that is
essentially crossdisciplinary in terms of its performance: it moves towards the
traditional performance environments of theatre or cinema while maintaining its
fundamental identity as a musical piece.
By Product (2007)
The second example explores similar ideas of performance space and the counterpoint
between the live and the automated in a slightly more ambitious way. Entitled By
14
Product and scored for bass clarinet, alto saxophone, 4-channel electronics and 5
video screens, it is concerned with the aural and visual waste products of live
performance as the basis of its material, e.g. excess breath sounds, unwanted key
noise – essentially that which is ordinarily edited out in the recording studio yet
characterises the ‘live’ quality of performance.
There is also an underlying theme of duality or mirroring in the piece that manifests
itself in each of the component parts employed. The piece’s overall trajectory is that
of gradual disintegration – all elements are synchronized at first and gradually
separate over the piece’s temporal frame. However, both the electronic music and live
instrumental parts move in opposite directions in terms of the musical material they
employ. Thus, whereas the saxophone and bass clarinet parts gradually move from
pitch material to improvised ‘noise’, the electronics gradually move from noise to
pitch as the piece progresses. This concept of duality is given further expression in the
electronic part which, from the half-way stage of the piece begins to introduce sample
sounds from the performers’ second/doubling instruments (Bb clarinet and sho)xi. It is
also reflected in the video parts which act as both performers and staging.
Insert Fig 2.1
2.1 Duo X perform By Product at CESTA, Tábor, Czech Republic, 2007
The performance environment for By Product consists of concentric spaces. The four
speakers that diffuse the music’s electronics speakers form the corners of the
performance space, which encloses the audience in the round and the live performers’
15
space within that (Fig 2.2). The two instrumentalists stand diagonally facing one
another and read from scores that are suspended from the ceiling with fishing wirexii.
Insert Fig 2.2
2.2 Aerial stage plan for By Product
Facing each performer is a corresponding monitor that plays video footage of the
respective performer’s instrument either as a disassembled construct or being
performed. In the case of the latter, the camera focuses on areas of the instrument(s)
where the performer’s hands and body are not visible, given the illusion that the
instruments are living entities. In a similar way to the live music parts, the monitors
begin very much in sync with the electronics and gradually separate from them as the
piece progresses (Fig 2.3).
Insert Fig. 2.3
2.3 Still from bass clarinet floor monitor footage
A third video is projected from an overhead cradle onto the players, acting as an
enhanced stage light of sortsxiii. Its material is actually footage of the exterior of the
performance space focusing on walls and doors that are partitioned into two
contrasting colours or textures, again reinforcing the theme of duality. Since the
performers are instructed to wear white, they become ‘screens’ themselves, mirroring
the concept of the screens becoming ‘performers’ (See Fig. 2.1)
The remaining video elements consist of two laptops which are situated on top of the
two monitors. These play sped-up footage of the performers’ eye and forehead
16
movements recorded during a practice session (Fig 2.4). The laptops remain closed
for the first half of the piece but, in one final twist, are opened one at a time by each
player, serving as a score for the improvised section of the piece; the performers are
asked to improvise noise material based on the rhythmic movement of the foreheads
on screenxiv.
Insert Fig. 2.4
Fig 2.4 Mock up showing two stills from saxophone and bass clarinet monitor
footage sandwiched between still images from both laptop sequences xv.
There are two points to note here in relation to this discussion, both of which concern
the use of multiple screens. Whereas the screen forms the performative nucleus for all
that takes place aurally and visually in Speak, here its role is much more multifaceted. Firstly, the relationship between musician and moving image as well as
between the piece’s live and automated elements is much more complex. The screens
outnumber the musicians, yet the source of their material is made up entirely of
footage of them. Furthermore, it is the laptop footage of the two performers’ faces
that appears much more ‘automated’ (sped up and glitching) than that of the
seemingly ‘living’ footage of the mechanical instruments. Also, the musical element
in By Product, particularly the live instrumental component feels somewhat
incomplete. The waste product-like, fragmented character of much of its notated
material including its predominantly improvised second half (which, while having
instructions on the score to indicate musical content, is temporally manipulated by the
laptop screens) results in a musical component that appears unable to stand on its
own. Unlike Speak, its identity as a piece of music is lost at the expense of a
heightened level of interdisciplinarity between it and the piece’s video component.
17
The other issue is that the use of screens makes for a performance environment the
crossdisciplinary nature of which is considerably far removed from the traditions of
live musical performance. Rather like some theatrical settings, the audience for By
Product witnesses the piece ‘in the round’. However, the positioning of
instrumentalists and screens is such that the piece cannot be viewed in its entirety in
one sitting. The audience must experience more than one performance and will have
to change their seating position either between or during performances to view all its
visual componentsxvi, the musical context of which will change each time due to the
amount of instrumental improvisation employed. Thus, it not only undergoes a
transformation away from the concert recital environment but sits somewhere
between theatre and installation.
Summary
While obviously very different in construction, both pieces incorporate moving image
as an integral part of their composition and performance, focusing on elements of
juxtaposition and synthesis as well as moving the works into the realms of theatre,
cinema and installation while, to one degree or another, maintaining their essential
characteristics as concert pieces. In doing so, I believe they go at least some way to
tackling the live performance and moving image format, forming what I hope are
unique and stimulating hybrid works that both capture and reveal the very essence of
the creative process. As Simon Shaw Miller argues:
18
Rather than thinking of purity at all, it might be better to see hybridity as the more
natural state for art, because purity is a historical contingency, whereas hybridity is
part of the flux of the creative process: the putting of things together (2002: 27)
Notes
i
While Zoo TV could certainly be viewed as landmark in terms of the extent of technology it
employed, it is perhaps less groundbreaking conceptually. Eamon Dunphy states that the
original idea behind the Zoo TV may have emerged from lighting designer Pete Williams’s
work on David Bowie’s Sound and Vision tour. See Eamon Dunphy, Unforgettable Fire: The
Story of U2 (Penguin, London 1993) p389.
ii
The actual performing musicians who make up the band are usually silhouetted behind a
screen.
iii
In each case, the first term referred to is that of Levinson while the second is the
supplementary heading Shaw-Miller assigns to each example for the purposes of analysis. See
Simon Shaw-Miller, Visible Deeds of Music (Yale, 2002) pp11-29
iv
This, of course, is not entirely correct as temporal discrepancies can and do exist in moving
image, particularly concerning different models and manufacturers of devices used to play
film. For example the same film footage played on two different DVD players, commencing
at precisely the same time may well end a split second apart (or perhaps more depending on
the running time of the footage). However, in comparison to live musical performance the
margin of error is significantly less (one could argue almost negligible).
v
See Biography at www.kjellbjorgeengen.com
vi
The information given here is in particular reference to a performance given at the
Lawrence Batley Theatre in November 2002 as part of the Huddersfield Contemporary Music
Festival.
19
vii
These are examples of acts that appear on the DVD release, Demon Days: Live at the
Manchester Opera House (EMI, 2006). That said, Gorillaz have performed with these acts
and others on numerous other occasions.
viii
…who is, incidentally, my brother! Mark Melvin also produced the video parts for By
Product
ix
While the piece is notated conventionally, it is scored in 30-second systems with specific
cues given in minutes and seconds on the stave. Both performers use a stopwatch to
synchronize with each other and with the electronics and video.
x
My wife is a speech pathologist and had to study phonetics as part of her training. I became
very interested in the different sounds of the International Phonetic Alphabet that she studied
during her University years, which greatly influenced Speak.
xi
By Product was originally composed for the group, Duo X whose members are a
clarinettist/bass clarinettist and a saxophonist who also plays the Japanese sho.
xii
The piece employs a similar use of stopwatches to Speak. The performers are also
responsible for synchronizing the floor monitors (via remote control).
xiii
The overhead projection is triggered by remote control. This can be done by the individual
operating the audio set up and may be activated at any point during the first few seconds of
the piece.
xiv
The laptop footage has a far longer running time than the piece itself. This allows the
laptop footage to be activated some minutes before the performance actually begins.
Therefore, exactly where the piece joins the laptop parts when they are opened is left utterly
to chance.
xv
At no point in the piece does this image appear as it is shown here. This is merely a mock-
up simultaneously presenting what might occur in four of the piece’s five video components
at a particular moment in the piece.
xvi
The audience is free to move around the performance space, excluding the ‘stage’ area
within the floor monitors and overhead projection, at any point during the performance.