Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Multimodal Communication

Ed. by Norris, Sigrid

2 Issues per year

Online
ISSN
2230-6587
See all formats and pricing
More options …

Pow, Punch, Pika, and Chu: The Structure of Sound Effects in Genres of American Comics and Japanese Manga

Nimish K. Pratha / Natalie Avunjian / Neil Cohn
  • Corresponding author
  • Center for Research in Language, University of California at San Diego, San Diego, CA, USA
  • Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, The Netherlands
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2016-10-04 | DOI: https://doi.org/10.1515/mc-2016-0017

Abstract

As multimodal works, comics are characterized as much by their use of language as by the style of their images. Sound effects in particular are exemplary of comics’ language-use, and we explored this facet of comics by analyzing a corpus of books from genres in the United States (mainstream and independent) and Japan (shonen/boys’ and shojo/girls’). We found variation between genres and between cultures across several properties of the content and presentation of sound effects. Foremost, significant differences arose between the lexical categories of sound effects (ex. onomatopoetic: Pow! vs. descriptive: Punch!) between genres within both culture’s works. Additionally, genres in Japanese manga vary in the scripts used to write sound effects in Japanese (hiragana vs. katakana). We argue that, in English, a similar function is communicated through the presence or absence of textual font stylization. Altogether, these aspects of variation mark sound effects as important carriers of multimodal information, and provide distinctions by which genres and cultures of comics can be distinguished.

Keywords: visual language; multimodality; onomatopoeia; comics; manga; Japanese

References

  • Akita, K. (2009). A grammar of sound-symbolic words in Japanese: theoretical approaches to iconic and lexical properties of mimetics. Doctoral Dissertation, Kobe University.

  • Allen, K., and Ingulsrud, J. E. (2005). Reading manga: Patterns of personal literacies among adolescents. Language and Education, 19:265–280.Google Scholar

  • Baldwin, R. S., and Cody, J. M. (1978). Psycholinguistic approaches to a theory of punctuation. Journal of Literacy Research, 10(4):363–375.Google Scholar

  • Blackwell, N. L., Perlman, M., and Tree, J. E. F. (2015). Quotation as a multimodal construction. Journal of Pragmatics, 81:1–7.Google Scholar

  • Bredin, H. (1996). Onomatopoeia as a figure and a linguistic principle. New Literary History, 27:555–569.Google Scholar

  • Brenner, R. E. (2007). Understanding Manga and Anime. Westport, CT: Libraries Unlimited.Google Scholar

  • Cao, Y., Lau, R. W. H., and Chan, A. B. (2014). Look over here: Attention-directing composition of manga elements. ACM Transactions on Graphics, 33:1–11.Google Scholar

  • Catricalà, M., and Guidi, A. (2015). Onomatopoeias: A new perspective around space, image schemas and phoneme clusters. Cognitive Processing, 16:175–178.Google Scholar

  • Cohn, N. (2013a). Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica, 2013:35–63.Google Scholar

  • Cohn, N. (2013b). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. London, UK: Bloomsbury.Google Scholar

  • Cohn, N. (2016). A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition, 146:304–323.Google Scholar

  • Cohn, N., and Ehly, S. (2016). The vocabulary of manga: Visual morphology in dialects of Japanese visual language. Journal of Pragmatics, 92:17–29.Google Scholar

  • Cohn, N., Taylor-Weiner, A., and Grossman, S. (2012). Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Psychology – Cultural Psychology, 3:1–12.Google Scholar

  • Coulmas, F. (1989). The Writing Systems of the World. Malden, MA: Blackwell Publishers Inc.Google Scholar

  • Drummond-Mathews, A. (2010). What boys will be: A study of Shōnen manga. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum Books.Google Scholar

  • Duncan, R., and Smith, M. J. (2009). The Power of Comics. New York: Continuum Books.Google Scholar

  • Flyxe, M. (2002). Translation of Japanese onomatopoeia into Swedish (with focus on lexicalization). Africa & Asia, 2:54–73.Google Scholar

  • Forceville, C. (2013). Creative visual duality in comics balloons. In: The Agile Mind, T. Veale, K. Feyaerts, and C. Forceville (Eds.). Berlin: Mouton de Gruyter.Google Scholar

  • Forceville, C., Veale, T., and Feyaerts, K. (2010). Balloonics: The visuals of balloons in comics. In: The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form, J. Goggin, and D. Hassler-Forest (Eds.). Jefferson: McFarland & Company Inc.Google Scholar

  • Forster, I., Borgwaldt, S. R., and Neef, M. (2012). Form follows function: Interjections and onomatopoetica in comics. Writing Systems Research, 4:122–139.Google Scholar

  • Gravett, P. (2004). Manga: Sixty Years of Japanese Comics. New York, NY: HarperCollins.Google Scholar

  • Griffin, J. K. (1998). A brief glossary of comic book terminology. Serials Review, 24:71–76.Google Scholar

  • Groensteen, T. (2007). The System of Comics. Jackson: University of Mississippi Press.Google Scholar

  • Guynes, S. A. (2014). Four-color sound: A Peircean semiotics of comic book onomatopoeia. The Public Journal of Semiotics, 6:58–72.Google Scholar

  • Hamano, S. (1998). The Sound-Symbolic System of Japanese. Stanford: CSLI Publications.Google Scholar

  • Hatfield, C. (2005). Alternative Comics: An Emerging Literature. Jackson, MS: Univ. Press of Mississippi.Google Scholar

  • Hill, G. E. (1943a). The vocabulary of comic strips. Journal of Educational Psychology, 34:77.Google Scholar

  • Hill, G. E. (1943b). Word distortions in comic strips. The Elementary School Journal, 43:520–525.Google Scholar

  • Inose, H. (2007). Translating Japanese onomatopoeia and mimetic words. Translation Research Projects, Anthony, Pym. Alexander, Perekrestenko (eds.), 1:97–116.Google Scholar

  • Inoue, M. (2002). Gender, language, and modernity: Toward an effective history of Japanese women’s language. American Ethnologist, 29:392–422.Google Scholar

  • Ito, K. (2005). A history of manga in the context of Japanese culture and society. Journal of Popular Culture, 38:456–575.Google Scholar

  • Iwahara, A., Hatta, T., and Maehara, A. (2003). The effects of a sense of compatibility between type of script and word in written Japanese. Reading and Writing, 16:377–397.Google Scholar

  • Jüngst, H. E. (2004). Japanese comics in Germany. Perspectives: Studies in Translatology, 12:83–105.Google Scholar

  • Kadooka, K.-I. (2005). On the degree of lexicalization in English onomatopoeia from a historical perspective. The Ryukoku Journal of Humanities and Sciences, 27:1–13.Google Scholar

  • Klock, G. (2002). How to Read Superhero Comics and Why. New York, NY: A&C Black.Google Scholar

  • Kovanen, S., and Aizawa, K. (2015). A layered method for determining manga text bubble reading order. Image Processing (ICIP), 2015 IEEE International Conference on, 27–30 September 2015, 4283–4287.

  • Kurlander, D., Skelly, T., and Salesin, D. (1996). Comic chat. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, 225–236.

  • Manfredi, M., Cohn, N., and Kutas, M. (submitted). When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative.

  • Mazur, D., and Danner, A. (2014). Comics: A Global History, 1968 to the Present. London: Thames & Hudson.Google Scholar

  • Nakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In: The Visual Narrative Reader, N. Cohn (Ed.). London: Bloomsbury.Google Scholar

  • Natsume, F. (2003). Japanese Manga: Its Expression and Popularity. ABD, 34(1):3–5.Google Scholar

  • Nunberg, G. (1990). The Linguistics of Punctuation (No. 18). Stanford: Center for the Study of Language (CSLI).Google Scholar

  • Petersen, R. S. (2009). The acoustics of manga. A Comics Studies Reader, 163–171.Google Scholar

  • Piekos, N. (2009). Comics Grammar and Traditions. Blambot: Comic Fonts and Lettering (http://www.blambot.com) [Online]. Available: http://www.blambot.com/grammar.shtml (Accessed 3/23/09).

  • Prough, J. (2010). Shojo manga in Japan and abroad. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum.Google Scholar

  • Robertson, W. (2015). Orthography, foreigners, and fluency: Indexicality and script selection in Japanese manga. Japanese Studies, 35:205–222.Google Scholar

  • Schodt, F. L. (1983). Manga! Manga! The World of Japanese Comics. New York: Kodansha America Inc.Google Scholar

  • Sell, C. (2011). Manga translation and interculture. Mechademia, 6:93–108.Google Scholar

  • Smith (Shibamoto), J. S. (1996). Japanese writing. In: The World’s Writing Systems, P.T. Daniels, and W. Bright (Eds.). Oxford, UK: Oxford University Press.Google Scholar

  • Smith, J. S., and Schmidt, D. L. (1996). Variability in written Japanese: Towards a sociolinguistics of script choice. Visible Language, 30:46–71.Google Scholar

  • Takashima, M. (2002). The development of young children’s understanding of representation using thought-bubbles. The Japanese Journal of Developmental Psychology, 13:136–146.Google Scholar

  • Tsuchimochi, G. H. (1991). Education reform under the American occupation (3): Report of the United States education mission to japan and the reform of the Japanese language. Hirosaki University Repository for Academic Resources.

  • Walker, M. (1980). The Lexicon of Comicana. Port Chester, NY: Comicana, Inc.Google Scholar

  • Wellman, H. M., Hollander, M., and Schult, C. A. (1996). Young children’s understanding of thought bubbles and thoughts. Child Development, 67:768–788.Google Scholar

  • Witek, J. (2012). Comics modes: caricature and illustration in the crumb family’s dirty laundry. In: Critical Approaches to Comics: Theories and Methods, M.J. Smith, and R. Duncan (Eds.). New York, NY: Routledge.Google Scholar

  • Yannicopoulou, A. (2004). Visual aspects of written texts: preschoolers view comics. L1 – Educational Studies in Language and Literature, 4:169–181.Google Scholar

  • Zanettin, F. (2008). Comics in Translation. Manchester: St. Jerome Pub.Google Scholar

About the article

Nimish K. Pratha

Nimish K. Pratha will receive a Bachelors of Science in Physiology and Neuroscience and a Bachelors of Arts in Linguistics at the University of California at San Diego. He is particularly interested in the Japanese language and manga, and as a fluent speaker of Japanese, has worked translating them into English for several years. His other interests include neurolinguistics, dance, and game design.

Natalie Avunjian

Natalie Avunjian holds a Bachelors of Science in Cognitive Science from the University of California at San Diego with a minor in Computer Science. She currently works as an engineer in the field of information technology, and holds interests in the visual and fine arts, web design, and music.

Neil Cohn

Neil Cohn is an assistant professor at the Tilburg center for Cognition and Communication at Tilburg University. He is internationally recognized for his research on the overlap of the structure and cognition of sequential images and language. His book, The Visual Language of Comics (Bloomsbury, 2013), introduces a broad framework for studying visual narratives in the linguistic and cognitive sciences. His edited volume, The Visual Narrative Reader (Bloomsbury, 2016), integrates interdisciplinary research on visual narratives into a unified field within the cognitive sciences. His work is online at www.visuallanguagelab.com.


Published Online: 2016-10-04

Published in Print: 2016-12-01


Citation Information: Multimodal Communication, ISSN (Online) 2230-6587, ISSN (Print) 2230-6579, DOI: https://doi.org/10.1515/mc-2016-0017.

Export Citation

©2016 by De Gruyter Mouton. Copyright Clearance Center

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Mirella Manfredi, Neil Cohn, and Marta Kutas
Brain and Language, 2017, Volume 169, Page 28

Comments (0)

Please log in or register to comment.
Log in