Analyzing Multimodal Interaction: within a Classroom Setting

Visible Language, 2006 by Moura, Heloisa

ABSTRACT

Human interactions are multimodal in nature. From simple to complex forms of transferal of information, human beings draw on a multiplicity of communicative modes, such as intonation and gaze, to make sense of everyday experiences. Likewise, the learning process, either within traditional classrooms or Virtual Learning Environments, is shaped by learners' perceptions of what is being communicated multimodally to them intentionally or not, and by the perceptible pedagogical affordances of the environment.

This paper examines the specific place of action and multimodal interaction within the learning process. It starts by defining learning and multimodal interaction. Next, it expands on an existing methodological framework for analyzing multimodal interaction in order to include affordances for learning and to visually map the central role of action to learning. Finally, it makes use of the reviewed methodological framework to analyze a video ethnographic study of interactions that take place within a graduate Design classroom.

INTRODUCTION

Any view of learning reflects its underlying theories. In (he present study, it is assumed that learning is situated in particular socioculturel contexts, and it is the result of mediated experiences that are afforded (Gibson, 1986) or constrained by interactions with the situation (King et al., 2001). In this way, the possibilities and limitations for action in particular situations affect learning. Furthermore, learning takes place whenever and wherever the individual is receptive. It can have different purposes or intentions, which, according to King, Young, Drivere-Richmond and Schrader (2001), can be classified into: a) objective-driven learning, such as in instruction; b) non-objective driven learning, such as in exploration; and c) unintended learning.

With regard to the relation between learning and multimodal interaction, it is possible to affirm that learning is woven with multimodal interaction. Discourse analysis studies in educational settings (Cazden, 2001; Adger, 2001; Mehan, 1979; Gumperz and Herasimrhuk, 1975) have been trying to uncover the way in which talk in school is unique, helping to explicate the actions in which learning is realized. The emphasis on the linguistic aspect of classroom interaction, however, fails to account for the multiple fused semiotic modalities that together, rather than separately, help extend the understanding of the learning that takes place.

Jürgen Ruesch and A. Rodney Prestwood were pioneers in bringing embodiment as communication into the applied arena of the human sciences (Lanigan, 1995). In early multimodal studies, "Anxiety: Hs initiation, communication and interpersonal management" (Ruesch and Prestwood, 1949) and Communication and bodily disease: A study of vasopastic conditions (Ruesch and Prestwood, 1950), the authors affirmed that the whole body can be looked upon as an instrument of communication. In 1951, Ruesch and Bateson, Communication: The social matrix of psychiatry, examined the asymmetrical communication interactions between psychotherapy supervisors and supervisees, taking note of the embodiment applications to communication and the diagnosis of stressed embodiment. For instance, within a group of the twelve women and nine men who bad undergone major operations (Ruesch and Prestwood, 1950), the majority had significant problems with human interaction and social process. The negative embodiment was manifest in a number of communication factors in the patients' comportment, such as inadequate gestures, poor system of codification and inability to consider the double meaning of communication actions.

Within educational settings, multimodal studies are more recent. Kress, Jewitt, Ogborn, and Tsatsarelis (2001a) conducted a multimodal study of school-based leaching in order to challenge the assumption that learning and teaching are primarily linguistic accomplishments, and not visual and actional. The authors show that classroom texts are realized through the interaction of different modes of oommunication or organized means of representation. For example, the construction of the entity 'cell' in a year 7 Science classroom involved speech, action - in the form of experimentation and image. The process of construction also involved the transformation of information across modes, e.g., verbal analogy to visual analogy, and experimentation into written report. Here, communication is extended to refer to all meaningmaking systems.

Bourne and Jewitt (2005), for example, took a multimodal approach to understand the ways in which the interpretation of literary texts is constructed through social interaction. The authors look at a year 10 English classroom, showing that higher-order literacy skills are realized and constructed through the configuration of talk, writing, gesture, gaze, movement and posture. An example is the use of a diagram by the teacher to talk about the abstract notion of gender and link the behavior of male students to the characters in the story and men in general. The authors show that the understanding of teaching and learning is facilitated and extended through the multimodal analysis of social interactions.