What role does the body play with language? My research investigates this question from different perspectives using different methods, but it is built on a single theoretical framework. Working from the position that language is an embodied ability that evolved from bodily communication systems (gestures, facial expressions, eye gaze, etc.) in our evolutionary past, my work investigates how language interacts with the body in present-day communication.
I focus primarily on hand gestures that spontaneously accompany speech. These gestures are interesting because they occur simultaneously with speech but reflect meaning in a distinct way from words.
For example, the form and movement of different drinking gestures—a gentle movement with a small precision grasp depicts sipping from a sake cup whereas a more abrupt movement with a closed fist depicts drinking from a large beer mug—capture the meaning of these two actions in a direct and non-arbitrary way. In contrast, spoken words reflect meaning only indirectly and arbitrarily through the particular conventions of a language. For example, the words, “nomu” in Japanese, and, “drink” in English, are utterly unrelated to the actual act of imbibing. This difference is exactly why co-speech gesture is so interesting—it offers a direct visual complement to the conventional symbols of a language, and when combined with those symbols, provides a more complete “picture” of what a speaker means.
My research on the relationship between speech and gesture spans social, psychological, and neural levels. On the social and psychological levels, I use behavioral methods to demonstrate that gestures influence how children and adults comprehend language in different social contexts, and how they think during the explanation of difficult concepts. On the neural level, I use event-related potentials (ERPs) to show that gestures influence speech at multiple stages of language comprehension. See below for PDFs on these different topics.
My research and a paper I co-authored with a Colgate student were cited in a BBC article that focused on body movements and how they can help you learn. For a general background on gesture, see this article in Scientific American Mind. For more on second-language learning and instruction, visit Colgate's Center for Language and Brain.
If you are interested in more research on gesture, brain, and language, click here for a link to a special issue (June, 2007) on the topic in the journal, Brain and Language. Or visit the webpage for the International Society for Gesture Studies (ISGS) to learn more about the field of gesture research more broadly. For more on my work and the work of my collaborators, please visit my page on Research Gate.
You can find me on Twitter at @sdk_lab.
(Hyperlinks lead to PDF files or download sites)
"Why Gesture?," edited by Breckie Church, Martha Alibali and Spencer Kelly, asks why we move our hands when we speak and why that is such an important interdisciplinary question.
Church, R. B., Alibali, M., & Kelly, S. D. (2017). Why gesture? How the hands function in speaking, thinking and communicating. John Benjamins Publishing: Amsterdam. pp. 1-433.
- Kelly, S. D., Alibali, M. W., & Church, R. B. (2017). Understanding gesture: Description, mechanism and function. In R. B. Church, M. W. Alibali & S. D. Kelly (Eds.), Why gesture? How the hands function in speaking, thinking and communicating. John Benjamins Publishing: Amsterdam. pp. 3-10.
- Kelly, S. D. (2017). Exploring the boundaries of gesture-speech integration during language comprehension. In R. B. Church, M. W. Alibali & S. D. Kelly (Eds.), Why gesture? How the hands function in speaking, thinking and communicating. John Benjamins Publishing: Amsterdam. pp. 243-265.
- *Billot-Vasquez, K., *Lian, Z., Hirata, Y., & Kelly, S. D. (2020). Emblem gestures improve perception and evaluation of non-native speech. Frontiers in Psychology, 11, 1-16.
- Sekine, K., Schoechl, C., Mulder, K., Holler, J., Kelly, S. D., Furman, R., & Özyürek, A. (2020). Evidence for children’s online integration of simultaneous information from speech and iconic gestures: an ERP study. Language, Cognition and Neuroscience, 1-11.
- *Zheng, A., Hirata, Y., & Kelly, S. D. (2018). Exploring the effects of imitating hand gestures and head nods on L1 and L2 Mandarin tone production. Journal of Speech, Language, and Hearing Research, 61(9), 2179-2195.
- Kelly, S. D., *Bailey, A. H., & Hirata, Y. (2017). Metaphoric gestures facilitate perception of intonation more than length in auditory judgments of non-native phonemic contrasts. Collabra, 3(1), 7.
- *Bailey, A. H., & Kelly, S. D. (2016). Body posture and gender impact neural processing of power-related words. The Journal of Social Psychology, 1-11.
- *Siciliano, R., Hirata, Y., & Kelly, S. D. (2016). Electrical stimulation over left inferior frontal gyrus disrupts hand gesture’s role in foreign vocabulary learning. Educational Neuroscience, 1, 1-12.
- *Bailey, A. H., & Kelly, S. D. (2015). Picture power: Gender versus body language in perceived status. Journal of Nonverbal Behavior, 1-21.
- Obermeier, C., Kelly, S. D., & Gunter, T. C. (2015). A speaker's gesture style can affect language comprehension: ERP evidence from gesture-speech integration. Social Cognitive and Affective Neuroscience, nsv011.
- Hirata, Y., Kelly, S. D., *Huang, J., & *Manansala, M. (2014). Effects of hand gestures on auditory learning of second-language vowel length contrasts. Journal of Speech, Language, and Hearing Research, 57(6), 2090-2101.
- Holler, J., Schubotz, L., Kelly, S., Hagoort, P., Schuetze, M. & Özyürek, A. (2014). Social eye gaze modulates processing of speech and co-speech gesture. Cognition, 132, 696-697.
- Kelly, S., *Healey, M., Özyürek, A., & Holler, J. (2014). The processing of speech, gesture, and action during language comprehension. Psychonomic Bulletin & Review, 1-7.
- Kelly, S., Hirata, Y., *Manansala, M., & *Huang, J. (2014). Exploring the role of hand gestures in learning novel phoneme contrasts and vocabulary in a second language. Frontiers in Psychology, 5, 673.
- Holler, J., Kokal, I., Toni, I., Hagoort, P., Kelly, S., & Özyürek, A. (2014). Eye'm talking to you: speakers' gaze direction modulates co-speech gesture processing in the right MTG. Social Cognitive & Affective Neuroscience.
- Church, R. B., Kelly, S., & Holcombe, D. (2013). Temporal synchrony between speech, action and gesture during language production. Language and Cognitive Processes, 1-10.
- *Hinzman, L., & Kelly, S. D. (2013). Effects of emotional body language on rapid out-group judgments. Journal of Experimental Social Psychology, 49, 152–155.
- Kelly S. D., Hansen B. C., & *Clark D. T. (2012). “Slight” of hand: The processing of visually degraded gestures with speech. PLoS ONE 7(8): e42620.
- Kelly, S. D., & *Lee, A. (2012). When actions speak too much louder than words: Gesture disrupts word learning when phonetic demands are high. Language and Cognitive Processes, 27, 793-807.
- Kelly, S. D., *Byrne, K., & Holler, J. (2011). Raising the ante of communication: Evidence for enhanced gesture use in high stakes situations. Information, 2, 579-593.
- Kelly, S. D., Ozyurek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260-267.
- Kelly, S. D., *Creigh, P., & *Bartolotti, J. (2010). Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22, 683-694.
- Hirata, Y., & Kelly, S. (2010). The effects of lips and hands on auditory learning of second language speech sounds. Journal of Speech, Language and Hearing Research, 53, 298-310.
- Kelly, S. D., *McDevitt, T., & *Esch, M. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24, 313-334.
- *Pratt, N., & Kelly, S. D. (2008). Emotional states influence the neural processing of affective language, Social Neuroscience, 3, 434-442.
- Kelly, S. D., Manning, S., & *Rodak, S. (2008). Gesture gives a hand to language and learning: Perspectives from cognitive neuroscience, developmental psychology and education. Language and Linguistics Compass, 2, 569-588.
- Ozyurek, A. & Kelly, S. D. (2007). Gesture, brain and language. Brain and Language, 101, 181-184.
- Kelly, S. D., *Ward, S., *Creigh, P., & *Bartolotti, J. (2007). An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222-233.
- Molfese, D. L., Fonaryova Key, A. Kelly, S., Cunningham, N., Terrell, S., Fergusson, M., Molfese, V. & Bonebright, T. (2006). Below-average, average, and above-average readers engage different and similar brain regions while reading. Journal of Learning Disabilities, 39, 352-363.
- Kelly, S. D., & *Goldsmith, L. (2004). Gesture and right hemisphere involvement in evaluating lecture material. Gesture, 4, 25-42.
- Kelly, S. D., *Kravitz, C., & *Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253-260.
- Molfese, D. L., Molfese, V. J., Key, S., Mogdlin, A. & Kelly, S., & Terrell, S. (2002). Reading and cognitive abilities: Longitudinal studies of brain and behavior changes in young children. Annals of Dyslexia, 52, 121-140.
- Kelly, S. D., Iverson, J., *Terranova, J., *Niego, J., *Hopkins, M., & *Goldsmith, L. (2002). Putting language back in the body: Speech and gesture on three timeframes. Developmental Neuropsychology, 22, 323-349.
- Kelly, S. D., Singer, M., Hicks, J., & Goldin-Meadow, S. (2002). A helping hand in assessing children's knowledge: Instructing adults to attend to gesture. Cognition and Instruction, 20, 1-26.
- Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: Gesture lightens the load. Psychological Science, 12, 516-522.
- Molfese, D., Molfese, V., & Kelly, S. D. (2001). The use of brain electrophysiology techniques to study language and reading: A basic guide for the beginning consumer of electrophysiology information. Learning Disabilities Quarterly, 24 (3), 177-188.
- Kelly, S. D. (2001). Broadening the units of analysis in communication: Speech and nonverbal behaviours in pragmatic comprehension. Journal of Child Language, 28, 325-349.
- Church, R. B., Kelly, S. D., & Lynch, K. (2000). Multi-modal processing over development: The case of speech and gesture detection. Journal of Nonverbal Behavior, 24 (2), 151-174.
- Kelly, S. D., Barr, D., Church, R. B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 40, 577-592.
- Kelly, S. D., & Church, R. B. (1998). A comparison between children's and adults' ability to detect children's representational gestures. Child Development, 69, 85-93.
- Kelly, S. D., & Church, R. B. (1997). Children's ability to detect nonverbal behaviors from other children. Cognition and Instruction, 15(1), 107-134.
* denotes Colgate undergraduate
BA (History), Washington University in St. Louis, 1991
*Student-at-large (Experimental Psychology), Northeastern Illinois University, 1992-1994
MA & PhD (Developmental Psychology), University of Chicago, 1997 & 1999
*Special note: I am very grateful for the transformative experience of taking courses and doing research in the Psychology department at NEIU after graduating from college. It was there where I became fascinated by the scientific study of why we gesture.
Download complete CV
Co-speech hand gestures, cognitive neuroscience, verbal and nonverbal communication, psycholinguistics, developmental psychology, second language learning
Event-related potentials (ERPs) are one of the cognitive neuroscience tools that I use in my lab. Check out this short video to learn how they work and what questions they can answer.
Dr. Kelly is the co-director of Colgate's Center for Language and Brain (CLB) which focuses on, among other things, issues involved in foreign language instruction and learning. Check out the video below to learn how the CLB promotes the psychological and social benefits of learning a foreign language at Colgate.
In collaboration with Professors Yukari Hirata and Douglas Johnson, Dr. Kelly developed an easy-to-navigate online resource offering 13 tips to help students transition from high school to Colgate. 'GATE WAY
Visiting scientist at the Max Planck Institute of Psycholinguistics in Nijmegen, The Netherlands (spring 2005, 2009); Allen Edwards Lecturer at the University of Washington in Seattle (fall 2008)
- Included in the Princeton Review's The Best 300 Professors, 2012
- Picker Research Fellowship, Colgate University (2005-2006, 2009)
- Picker Interdisciplinary Research Grant, Colgate University (2007-2009)
- Gastwissenschaftler (Visiting Scientist) at the MPI CBS, Leipzig, Germany (Summer, 2007)
- Post-Doctoral Research Fellowship, University of Louisville (1999-2000)
- Pre-Doctoral National Research Service Award, University of Chicago, declined (1999)
- William Rainey Harper Dissertation Fellowship, University of Chicago (1998-1999)
- John Dewey Lectureship Prize, University of Chicago (1998)
- Division of Social Sciences Scholarship, University of Chicago (1994-1998)
"The limits of my language mean the limits of my world."
- Ludwig Wittgenstein, Tractatus Logico-Philosophicus
Colgate's Center for Language Brain explores questions regarding foreign language instruction and learning.