Pow, Punch, Pika, and Chu: The Structure of Sound Effects in Genres of American Comics and Japanese Manga
Nimish K. Pratha
, Natalie Avunjian
, and Neil Cohn
1 Center for Research in Language, University of California at San Diego, San Diego, CA, USA
2 Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, The Netherlands
Nimish K. Pratha
Nimish K. Pratha will receive a Bachelors of Science in Physiology and Neuroscience and a Bachelors of Arts in Linguistics at the University of California at San Diego. He is particularly interested in the Japanese language and manga, and as a fluent speaker of Japanese, has worked translating them into English for several years. His other interests include neurolinguistics, dance, and game design.
Natalie Avunjian holds a Bachelors of Science in Cognitive Science from the University of California at San Diego with a minor in Computer Science. She currently works as an engineer in the field of information technology, and holds interests in the visual and fine arts, web design, and music.
Neil Cohn is an assistant professor at the Tilburg center for Cognition and Communication at Tilburg University. He is internationally recognized for his research on the overlap of the structure and cognition of sequential images and language. His book, The Visual Language of Comics (Bloomsbury, 2013), introduces a broad framework for studying visual narratives in the linguistic and cognitive sciences. His edited volume, The Visual Narrative Reader (Bloomsbury, 2016), integrates interdisciplinary research on visual narratives into a unified field within the cognitive sciences. His work is online at www.visuallanguagelab.com.
As multimodal works, comics are characterized as much by their use of language as by the style of their images. Sound effects in particular are exemplary of comics’ language-use, and we explored this facet of comics by analyzing a corpus of books from genres in the United States (mainstream and independent) and Japan (shonen/boys’ and shojo/girls’). We found variation between genres and between cultures across several properties of the content and presentation of sound effects. Foremost, significant differences arose between the lexical categories of sound effects (ex. onomatopoetic: Pow! vs. descriptive: Punch!) between genres within both culture’s works. Additionally, genres in Japanese manga vary in the scripts used to write sound effects in Japanese (hiragana vs. katakana). We argue that, in English, a similar function is communicated through the presence or absence of textual font stylization. Altogether, these aspects of variation mark sound effects as important carriers of multimodal information, and provide distinctions by which genres and cultures of comics can be distinguished.
Cohn, N., Taylor-Weiner, A., and Grossman, S. (2012). Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Psychology – Cultural Psychology, 3:1–12.
Coulmas, F. (1989). The Writing Systems of the World. Malden, MA: Blackwell Publishers Inc.
Drummond-Mathews, A. (2010). What boys will be: A study of Shōnen manga. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum Books.
Duncan, R., and Smith, M. J. (2009). The Power of Comics. New York: Continuum Books.
Flyxe, M. (2002). Translation of Japanese onomatopoeia into Swedish (with focus on lexicalization). Africa & Asia, 2:54–73.
Forceville, C. (2013). Creative visual duality in comics balloons. In: The Agile Mind, T. Veale, K. Feyaerts, and C. Forceville (Eds.). Berlin: Mouton de Gruyter.
Forceville, C., Veale, T., and Feyaerts, K. (2010). Balloonics: The visuals of balloons in comics. In: The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form, J. Goggin, and D. Hassler-Forest (Eds.). Jefferson: McFarland & Company Inc.
Forster, I., Borgwaldt, S. R., and Neef, M. (2012). Form follows function: Interjections and onomatopoetica in comics. Writing Systems Research, 4:122–139.
Iwahara, A., Hatta, T., and Maehara, A. (2003). The effects of a sense of compatibility between type of script and word in written Japanese. Reading and Writing, 16:377–397.10.1023/A:1023668804595)| false
Kovanen, S., and Aizawa, K. (2015). A layered method for determining manga text bubble reading order. Image Processing (ICIP), 2015 IEEE International Conference on, 27–30 September 2015, 4283–4287.10.1109/ICIP.2015.7351614)| false
Smith (Shibamoto), J. S. (1996). Japanese writing. In: The World’s Writing Systems, P.T. Daniels, and W. Bright (Eds.). Oxford, UK: Oxford University Press.
Smith, J. S., and Schmidt, D. L. (1996). Variability in written Japanese: Towards a sociolinguistics of script choice. Visible Language, 30:46–71.
Takashima, M. (2002). The development of young children’s understanding of representation using thought-bubbles. The Japanese Journal of Developmental Psychology, 13:136–146.
Tsuchimochi, G. H. (1991). Education reform under the American occupation (3): Report of the United States education mission to japan and the reform of the Japanese language. Hirosaki University Repository for Academic Resources.
Walker, M. (1980). The Lexicon of Comicana. Port Chester, NY: Comicana, Inc.
Wellman, H. M., Hollander, M., and Schult, C. A. (1996). Young children’s understanding of thought bubbles and thoughts. Child Development, 67:768–788.
Witek, J. (2012). Comics modes: caricature and illustration in the crumb family’s dirty laundry. In: Critical Approaches to Comics: Theories and Methods, M.J. Smith, and R. Duncan (Eds.). New York, NY: Routledge.
Yannicopoulou, A. (2004). Visual aspects of written texts: preschoolers view comics. L1 – Educational Studies in Language and Literature, 4:169–181.
Multimodal Communication is a venue for researchers in diverse fields of study, who explore varied & multimodal ways to conduct research & illustrate findings. It is open to papers from anthropology to art, math, psychology, science, & beyond. Articles share a commitment to developing multimodality and may use visuals/writing/sound to explore a theme; be highly theoretical or may be based on an empirical study integrating a number of modes.