Jump to ContentJump to Main Navigation
Show Summary Details
More options …


Journal of the International Association for Semiotic Studies / Revue de l'Association Internationale de Sémiotique

Editor-in-Chief: Danesi, Marcel

IMPACT FACTOR 2018: 0.509

CiteScore 2018: 0.23

SCImago Journal Rank (SJR) 2018: 0.232
Source Normalized Impact per Paper (SNIP) 2018: 0.478

Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca: Classe A

See all formats and pricing
More options …
Volume 2013, Issue 197


Beyond speech balloons and thought bubbles: The integration of text and image

Neil Cohn
Published Online: 2013-10-23 | DOI: https://doi.org/10.1515/sem-2013-0079


Speech balloons and thought bubbles are among the most recognizable visual signs of the visual language used in comics. These enclosed graphic containers provide a way in which text and image can interface with each other. However, their stereotypical meanings as representing speech or thought betray much deeper semantic richness. This paper uses these graphic signs as a platform for examining the multimodal interfaces between text and image, and details four types of interfaces that characterize the connections between modalities: Inherent, Emergent, Adjoined, and Independent relationships. Each interface facilitates different levels of multimodal integration, tempered by principles of Gestalt grouping and underlying semantic features. This process allows the possibility of creating singular cohesive units of text and image that is on par with other multimodal interfaces, such as between speech and gesture.

Keywords: visual language; comics; multimodality; graphic communication; thought bubbles

About the article

Neil Cohn

Neil Cohn (b. 1980) is a postdoctoral fellow at University of California at San Diego 〈 〉. His research interests include narrative structure, comics, and visual language. His publications include “A different kind of cultural frame: An analysis of panels in American comics and Japanese manga” (2011); “Comics, linguistics, and visual language: The past and future of a field” (2012); “(Pea)nuts and bolts of visual narratives: Structure and meaning in sequential image comprehension” (with M. Paczynski, R. Jackendoff, P. Holcomb & G. Kuperberg, 2012); and The visual language of comics: Structure, psychology, culture (2013).

Published Online: 2013-10-23

Published in Print: 2013-10-25

Citation Information: Semiotica, Volume 2013, Issue 197, Pages 35–63, ISSN (Online) 1613-3692, ISSN (Print) 0037-1998, DOI: https://doi.org/10.1515/sem-2013-0079.

Export Citation

©[2013] by Walter de Gruyter Berlin Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Language and Cognition, 2019, Volume 11, Number 1, Page 66
William J. White
Technical Communication Quarterly, 2017, Volume 26, Number 2, Page 101
Neil Cohn and Sean Ehly
Journal of Pragmatics, 2016, Volume 92, Page 17

Comments (0)

Please log in or register to comment.
Log in