Download 1 - Carleton University

Interviewed by Maryse de la Giroday, science and technology writer,
Link to interview: (http://www.frogheart.ca)
1. I was quite intrigued the title of your talk (A Multimodal Approach to
Discourse Studies: A paradigm with new research questions, agendas and
directions for the digital age) at the 2009 Congress for the Humanities and
Social Sciences held in Ottawa, Canada this May. Could you briefly describe
a multimodal approach for people who aren't necessarily in the field of
education?
Traditionally, language has been studied in isolation, largely due to an emphasis on
the study of printed linguistic texts and existing technologies such as print media,
telephone and radio where language was the primary resource which was used.
However, various forms of images, animations and videos form the basis for sharing
information in the digital age, and thus it has become necessary to move beyond the
study of language to understand contemporary communicative practices. In a sense,
the study of language alone was never really sufficient because analysing what
people wrote or said missed significant choices such as typography, layout and the
images which appeared in the written texts, and the intonation, actions and gestures
which accompanied spoken language. In addition, disciplinary knowledge (e.g.
mathematics, science and social science disciplines) involves mathematical
symbolism and various kinds of images, in addition to language. Therefore,
researchers in language studies and education are moving beyond the study of
language to multimodal approaches in order to investigate how linguistic choices
combine with choices from other meaning-making resources.
Basically multimodal research explores the various roles which language, visual
images, movement, gesture, sound, music and other resources play, and the ways
those resources integrate across modalities (visual, auditory, tactile, olfactory etc) to
create meaning in artefacts and events which form and transform culture. For
example, the focus may be written texts, day-to-day interactions, internet sites,
videos and films and 3-D objects and sites. In fact, one can think of knowledge and
culture as specific choices from meaning-making resources which combine and
unfold in patterns which are familiar to members of groups and communities.
Moreover, there is now explicit acknowledgement in educational research that
disciplinary knowledge is multimodal and that literacy extends beyond language.
The shift to multimodal research has taken place as a result of digital media which
not only serves as the object of study, but also because digital media technologies
offer new research tools to study multimodal texts. Such technologies have become
available and affordable, and increasingly they are being utilised by multimodal
researchers in order to make complex multimodal analysis possible. Lastly, scientists
and engineers are increasingly looking to social scientists to solve important
problems involving multimodal phenomena, for example, data analysis, search and
retrieval and human computer interface design. Computer scientists and social
sciences face similar problems in today’s world of digital media, and interdisciplinary
collaboration is the promise of the future in what has become the age of information.
2. Could you describe the research questions, agendas and directions that are most
compelling to you at this time?
Multimodal research involves new questions and problems such as:
- What are the functionalities of the resources (e.g. language versus image)?
- How do choices combine to make meaning in artefacts and events?
- What types of reconstruals take place within and across semiotic artefacts and events and what
type of metaphors consequently arise?
- How is digital meaning expanding our meaning-making potential?
The most compelling agendas and directions in multimodal research include developing new
approaches to annotating, analysing, modeling, and interpreting semiotic patterns using digital
media technologies, particularly in dynamic contexts (e.g. videos, film, website browsing, online
learning materials). The development of new practices for multimodal analysis (e.g. multimodal
corpus approaches) means we can investigate social cultural patterns and trends and the nature
of knowledge and contemporary life in the age of digital media, together with its limitations.
Surely new media offers us the potential for new research paradigms and making new types of
meanings which will lead us to new ways of thinking about the world. Also, multimodal
approaches offer the promise of new paradigms for educational research where classroom and
pedagogical practices and disciplinary knowledge can be investigated in their entirety.
Multimodal research opens up a new exciting world, one which is being eagerly embraced by
academic researchers and postgraduate students as the way forward (in my experience at least).
3. I notice that you have a project examining PowerPoint in the classroom
and in corporate settings which you are conducting for the Australian
Research Council. Could you explain a little bit about the project?
The project ‘Towards a Social Theory of Semiotic Technology: Exploring PowerPoint's
Design and its Use Higher Education and Corporate Settings awarded by the
Australian Research Council (ARC) (Discovery Grant No. DP09889939) is a
collaborative project between Chief Investigator Professor Theo van Leeuwen (Dean
for Faculty of Arts and Social Sciences, University of Technology, Sydney), Dr Emilia
Djonov (Post-doctoral Fellow, University of Technology, Sydney) and myself. The
following description of the project is drawn from our research proposal.
PowerPoint has become the dominant technology for designing and delivering
presentations, particularly in education and business settings where success often
depends on skills in the use of the application. Powerpoint is the subject of much
debate and it creates strong reactions, both positive and negative. It’s either praised
for increasing presenters’ confidence and eloquence (e.g. Gold 2002) or it’s
condemned for limiting users’ ability to present complex ideas through an oversimplification of information presented in bullet points, linear slide-by-slide formats
and illegible graphics (e.g. Tufte 2003).
From the multimodal perspective, Powerpoint is a semiotic technology which has a
range of options (i.e. grammar) from which presenters make selections with regards
to the linguistic text, images, animations and sounds. There are default themes
which the presenter may choose as well. These choices integrate in multimodal
presentations which are recontextualised by the speaker during the presentation.
Most studies of Powerpoint adopt a different approach, however, by either exploring
lecturers’ and students’ perceptions of PowerPoint to support learning, or
alternatively they are experimental studies which investigate the effects of
PowerPoint versus transparency-supported lectures on learning.
Our project adopts a multimodal approach to (a) conceptualise the grammar of
Powerpoint through the study of its systems of meaning; (b) analyse and compare
the choices which are made in higher education and corporate settings; and (c)
investigate how these choices are contextualised in presentations. In this way, we
will explore how the design of PowerPoint supports or hinders the achievement of the
various goals of the presenters. At the moment, there are no studies which
investigate differences in the use of Powerpoint across educational and corporate
settings, and furthermore, there is no evidence for arguments that PowerPoint
cannot support the representation of knowledge in technical disciplines such as
engineering (Tufte, 2003) or the rich narrative and interpretative skills required for
social science disciplines (Adams, 2006), nor is there evidence that PowerPoint has
introduced corporate rhetoric into educational practices (Turkle, 2004). In addition,
the study will provide guidelines for evaluating and improving the design and use of
PowerPoint and other similar presentation software.
Bibliography
Adams, C. (2006). PowerPoint, habits of mind, and classroom culture. Journal of
Curriculum Studies, 38(4), 389 - 411.
Gold, R. (2002). Reading PowerPoint. In N. J. Allen (Ed.), Working with words and
images: New steps in an old dance. (pp. 256-270). Westport, Connecticut: Ablex.
Tufte, E. R. (2003). The cognitive style of PowerPoint (2nd edition). Cheshire,
Connecticut: Graphics Press.
Turkle, S. (2004). The fellowship of the microchip: global technologies as evocative
objects. In M. Suárez-Orozco & D.B. Qin-Hilliard (Eds.), Globalization: Culture and
Education in the New Millennium (pp. 97-113). Berkeley, CA: University of California
Press.