My research is concerned with the relationship of language to thought -- an old problem examined in a new way through the prism of the gestures that accompany utterances in discourse. A major focus is a theory in which the language-thought relationship is explained as a dynamic combination of imagery and language starting from a "growth point." A growth point can be inferred from gesture-speech synchrony, and it is a snapshot of an utterance at its beginning stage psychologically. Via gestures one can ask how a growth point self-organizes into a surface utterance that embodies core content and extends the discourse at each moment of speaking.

Data in this research consist of videotaped spoken narratives and other discourse forms together with their co-occurring spontaneous gestures (subjects view a film or cartoon, and tell the story to a listener). We have a sizable library of the same stimulus stories being retold by speakers of different languages (English, Japanese, Chinese, Korean, Georgian, Swahili, and most of the European languages), by non-native speakers at different stages of learning English, by children at various ages, by adolescent deaf children not exposed to language models, and by speakers with neurological impairments (aphasic, right hemisphere damaged, and split-brain patients).

We have also conducted experiments on gesture performance: the pickup of gesture information by listeners (detected by planting mismatching gestures in a videotaped story the subject retells), change in gestures when they are required to carry the full communicative burden (a joint experiment with Susan Goldin-Meadow and Jenny Singleton), a gesture-speech cross-overs (changes of gesture when speech is interfered with, changes of speech when gesture is interfered with).