What role does the body play with language? My research investigates this question from different perspectives using different methods, but it is built on a single theoretical framework. Working from the position that language is an embodied ability that evolved from bodily communication systems (gestures, facial expressions, eye gaze, etc.) in our evolutionary past, my work investigates how language interacts with the body in present-day communication.
I focus primarily on hand gestures that spontaneously accompany speech. These gestures are interesting because they occur simultaneously with speech but reflect meaning in a distinct way from words.
For example, the form and movement of different drinking gestures—a gentle movement with a small precision grasp depicts sipping from a sake cup whereas a more abrupt movement with a closed fist depicts drinking from a large beer mug—capture the meaning of these two actions in a direct and non-arbitrary way. In contrast, spoken words reflect meaning only indirectly and arbitrarily through the particular conventions of a language. For example, the words, “nomu” in Japanese, and, “drink” in English, are utterly unrelated to the actual act of imbibing. This difference is exactly why co-speech gesture is so interesting—it offers a direct visual complement to the conventional symbols of a language, and when combined with those symbols, provides a more complete “picture” of what a speaker means.
My research on the relationship between speech and gesture spans social, psychological, and neural levels. On the social and psychological levels, I use behavioral methods to demonstrate that gestures influence how children and adults comprehend language in different social contexts, and how they think during the explanation of difficult concepts. On the neural level, I use event-related potentials (ERPs) to show that gestures influence speech at multiple stages of language comprehension. See below for PDFs on these different topics.
A video of my research was featured on ScienCentral News (link no longer available) and the original story that first appeared in Scientific American Mind. And here is a recent Seattle television story on the role that hand gestures play in second-language learning. For more on second-language learning and instruction, visit Colgate's Center for Language and Brain.
If you are interested in more research on gesture, brain, and language, click here for a link to a special issue (June, 2007) on the topic in the journal, Brain and Language. Or visit the webpage for the International Society for Gesture Studies (ISGS) to learn more about the field of gesture research more broadly.
BA, Washington University, St. Louis, 1991; MA (1997), PhD (1999), University of Chicago
Complete curriculum vitae [PDF]
Co-speech hand gestures, cognitive neuroscience, verbal and nonverbal communication, psycholinguistics, developmental psychology, second language learning
(Hyperlinks lead to PDF files or free download sites)
* denotes Colgate undergraduate
- Church, R. B., Kelly, S., & Holcombe, D. (2013).Temporal synchrony between speech, action and gesture during language production. Language and Cognitive Processes, 1-10.
- *Hinzman, L., & Kelly, S. D. (2013). Effects of emotional body language on rapid out-group judgments. Journal of Experimental Social Psychology, 49, 152–155.
- Kelly S. D., Hansen B. C., & *Clark D. T. (2012). “Slight” of hand: The processing of visually degraded gestures with speech. PLoS ONE 7(8): e42620.
- Kelly, S. D., & *Lee, A. (2012). When actions speak too much louder than words: Gesture disrupts word learning when phonetic demands are high. Language and Cognitive Processes, 27, 793-807.
- Kelly, S. D., *Byrne, K., & Holler, J. (2011). Raising the ante of communication: Evidence for enhanced gesture use in high stakes situations. Information, 2, 579-593.
- Kelly, S. D., Ozyurek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260-267.
- Kelly, S. D., *Creigh, P., & *Bartolotti, J. (2010). Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22, 683-694.
- Hirata, Y., & Kelly, S. (2010). The effects of lips and hands on auditory learning of second language speech sounds. Journal of Speech, Language and Hearing Research, 53, 298-310.
- Kelly, S. D., *McDevitt, T., & *Esch, M. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24, 313-334.
- *Pratt, N., & Kelly, S. D. (2008). Emotional states influence the neural processing of affective language, Social Neuroscience, 3, 434-442.
- Kelly, S. D., Manning, S., & *Rodak, S. (2008). Gesture gives a hand to language and learning: Perspectives from cognitive neuroscience, developmental psychology and education. Language and Linguistics Compass, 2, 569-588.
- Ozyurek, A. & Kelly, S. D. (2007). Gesture, brain and language. Brain and Language, 101, 181-184.
- Kelly, S. D., *Ward, S., *Creigh, P., & *Bartolotti, J. (2007). An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222-233.
- Molfese, D. L., Fonaryova Key, A. Kelly, S., Cunningham, N., Terrell, S., Fergusson, M., Molfese, V. & Bonebright, T. (2006). Below-average, average, and above-average readers engage different and similar brain regions while reading. Journal of Learning Disabilities, 39, 352-363.
- Kelly, S. D., & *Goldsmith, L. (2004). Gesture and right hemisphere involvement in evaluating lecture material. Gesture, 4, 25-42.
- Kelly, S. D., *Kravitz, C., & *Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253-260.
- Molfese, D. L., Molfese, V. J., Key, S., Mogdlin, A. & Kelly, S., & Terrell, S. (2002). Reading and cognitive abilities: Longitudinal studies of brain and behavior changes in young children. Annals of Dyslexia, 52, 121-140.
- Kelly, S. D., Iverson, J., *Terranova, J., *Niego, J., *Hopkins, M., & *Goldsmith, L. (2002). Putting language back in the body: Speech and gesture on three timeframes. Developmental Neuropsychology, 22, 323-349.
- Kelly, S. D., Singer, M., Hicks, J., & Goldin-Meadow, S. (2002). A helping hand in assessing children's knowledge: Instructing adults to attend to gesture. Cognition and Instruction, 20, 1-26.
- Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: Gesture lightens the load. Psychological Science, 12, 516-522.
- Molfese, D., Molfese, V., & Kelly, S. D. (2001). The use of brain electrophysiology techniques to study language and reading: A basic guide for the beginning consumer of electrophysiology information. Learning Disabilities Quarterly, 24 (3), 177-188.
- Kelly, S. D. (2001). Broadening the units of analysis in communication: Speech and nonverbal behaviours in pragmatic comprehension. Journal of Child Language, 28, 325-349.
- Church, R. B., Kelly, S. D., & Lynch, K. (2000). Multi-modal processing over development: The case of speech and gesture detection. Journal of Nonverbal Behavior, 24 (2), 151-174.
- Kelly, S. D., Barr, D., Church, R. B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 40, 577-592.
- Kelly, S. D., & Church, R. B. (1998). A comparison between children's and adults' ability to detect children's representational gestures. Child Development, 69, 85-93.
- Kelly, S. D., & Church, R. B. (1997). Children's ability to detect nonverbal behaviors from other children. Cognition and Instruction, 15(1), 107-134.
Visiting scientist at the Max Planck Institute of Psycholinguistics in Nijmegen, The Netherlands (spring 2005, 2009); Allen Edwards Lecturer at the University of Washington in Seattle (fall 2008)
- Picker Research Fellowship, Colgate University (2005-2006, 2009)
- Picker Interdisciplinary Research Grant, Colgate University (2007-2009)
- Gastwissenschaftler (Visiting Scientist) at the MPI CBS, Leipzig, Germany (Summer, 2007)
- Post-Doctoral Research Fellowship, University of Louisville (1999-2000)
- Pre-Doctoral National Research Service Award, University of Chicago, declined (1999)
- William Rainey Harper Dissertation Fellowship, University of Chicago (1998-1999)
- John Dewey Lectureship Prize, University of Chicago (1998)
- Division of Social Sciences Scholarship, University of Chicago (1994-1998)