Setting:
The method we have used in many investigations was developed some two decades ago in collaboration with Elena Levy (see McNeill and Levy 1982). Our method is to show a film or animated cartoon to subjects and have the subject retell the story to a listener immediately, from memory. The performance is thus storytelling. The subjects do not know that gestures are of interest (the experiment is described as about 'communication' or 'storytelling'). Nonetheless, the spoken narration is typically accompanied by spontaneous gestures, with no instructions to make gestures or any mention of gestures as a topic of interest. The gestures that occur, together with the narration itself, comprise the raw data of our investigation, and are recorded on videotape. We picked a cartoon stimulus partly because we wanted to use the method with children but it has proven useful with many other kinds of subjects - university students, speakers of other languages, speakers with neurological abnormalities, even deaf and blind subjects. The first results with this technique were presented in the aforementioned paper (McNeill & Levy, 1982).

Reasons for the storytelling method:
The cartoon/film narration method addresses a critical problem in the study of gesture, which has not always been noticed. If we use the content of speech to determine the meaning of a gesture, we are unable to see co-expressive speech and gesture; only redundant echoing is discernable. Having a known stimulus for the story provides a basis to interpret the gestures without relying on the speech content itself to interpret them (speech is only used to identify the event being described). This is important since it gives a non-circular basis for comparing the information content of speech and gesture. The circularity problem in part motivated our choice of the cartoon stimulus from the start.

Ultimately, the goal is to interpret gesture and speech without the crutch of a known source, and this may now be possible in many cases, since a sufficient knowledge of gesture and speech and how they work together has been built up. Nonetheless, even though we could venture forth and attempt to explain conversation, argumentation, instruction, lectures, gossip, dinner conversation, and a host of other genres, most of the examples in this work are from narrative discourse. Still, selected cases of instruction, lecturing, and conversation will also be described.

A second advantage of having a known stimulus is that it makes possible event-by-event comparisons across speakers. It is possible to look at the same event and see how it appears in speech and gesture across languages (we have collected narrations in English, Spanish, Mandarin, Japanese), ages (toddler to adult), and neurological condition (normals, aphasics, right hemisphere injured, split-brains, and the important IW case). The ability to have targeted comparisons is a powerful source of insights and, in itself, gives narrative data a special place.

The stimulus:
We chose a 7 minute long animated color cartoon of the Tweety and Sylvester series ("Canary Row", ca. 1950). The choice was based on several factors that we thought made this specific cartoon suitable for storytellers of different ages, neurological conditions, and language groups - it uses little speech, the plot line is linear and repetitive yet varies widely on the surface from episode to episode, it contains a high concentration of motion, and it is comparatively brief. Our intuitions have been validated over the years, in that we have obtained excellent retellings from non-native participants, including some who were very new to the culture at the time of taping (within weeks of arrival), as well as from children and various kinds of neurological patients. We have also used a full-length film as a narrative stimulus, an early Hitchcock (Blackmail), which is suitable obviously only with adults. In contrast to cartoon retellings, the film evokes a high proportion of metaphoric gestures (see the descriptions in Hand and Mind).

The procedure:
We show the cartoon stimulus either as a whole (unimpaired adults) or in 2 or 3 parts, and ask the subject to immediately recount the story to a listener who had not seen the cartoon, from memory. (Blackmail is shown straight through.) We employ true listeners, not confederates. The instructions frame the experiment as dealing with storytelling; there is no mention of gesture. The retelling performance, which in the case of the cartoon may last anywhere from a minute to ten minutes and in the case of Blackmail may last up to an hour, is recorded on videotape. This tape recording is the raw data for all our investigations. The tape must be transcribed to bring out the gesture and speech data. For details, download the annotation notes, which summarizes the process (and is more up-to-date that the appendix in Hand and Mind.

Gesture phases:
A gesture as it is usually defined passes through up to five phases: preparation, prestroke hold, the stroke itself, poststroke hold, and retraction; all are optional except for the stroke. The stroke caries the imagistic content of the gesture and is the phase whose synchrony with speech is maintained by the speaker. The following transcription illustrates many of these features (transcription by S. Duncan; the illustrations show three stages of the gesture):
  / tryi[ng to swing across by a rope #]
            prep   hold   stroke   hold   retract

Transcription: Iconic; 2 similar hands; A-shape; palms toward body; fingers turned down; starts at right and arcs to other side with slight wrist pivot. Hands =S's hands, character vpt = S; arc = trajectory, observer vpt. S swings on rope

Preparation

Mid-Stroke

Retraction

Gesture types:

A misconception has arisen about the nature of the gesture categories described in Hand and Mind, to wit, that they are mutually exclusive bins into which gestures should be dumped. In fact, pretty much any gesture is going to involve more than one category. Take a classic upward path gesture of the sort that many speakers produce when they describe the event of the cat climbing up the pipe in our cartoon stimulus. This gesture involves an iconic path-for-path mapping, but is also deictic, in that the gesture is made with respect to an origo --that is, it is situated within a deictic field. Even "simple" beats are often made in a particular location which the speaker has given further structure (e.g. by setting up an entity there and repeatedly referring to it in that spatial location). Metaphoric gestures are de facto iconic gestures, given that metaphor entails iconicity. The notion of a type, therefore, should be considered as a continuum --with a given gesture having more or less iconicity, metaphoricity, etc.