Jump to ContentJump to Main Navigation
Show Summary Details

Multimodal Communication

Ed. by Norris, Sigrid

2 Issues per year

Online
ISSN
2230-6587
See all formats and pricing

Pow, Punch, Pika, and Chu: The Structure of Sound Effects in Genres of American Comics and Japanese Manga

Nimish K. Pratha
  • Center for Research in Language, University of California at San Diego, San Diego, CA, USA
/ Natalie Avunjian
  • Center for Research in Language, University of California at San Diego, San Diego, CA, USA
/ Neil Cohn
  • Corresponding author
  • Center for Research in Language, University of California at San Diego, San Diego, CA, USA
  • Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, The Netherlands
  • Email:
Published Online: 2016-10-04 | DOI: https://doi.org/10.1515/mc-2016-0017

Abstract

As multimodal works, comics are characterized as much by their use of language as by the style of their images. Sound effects in particular are exemplary of comics’ language-use, and we explored this facet of comics by analyzing a corpus of books from genres in the United States (mainstream and independent) and Japan (shonen/boys’ and shojo/girls’). We found variation between genres and between cultures across several properties of the content and presentation of sound effects. Foremost, significant differences arose between the lexical categories of sound effects (ex. onomatopoetic: Pow! vs. descriptive: Punch!) between genres within both culture’s works. Additionally, genres in Japanese manga vary in the scripts used to write sound effects in Japanese (hiragana vs. katakana). We argue that, in English, a similar function is communicated through the presence or absence of textual font stylization. Altogether, these aspects of variation mark sound effects as important carriers of multimodal information, and provide distinctions by which genres and cultures of comics can be distinguished.

Keywords: visual language; multimodality; onomatopoeia; comics; manga; Japanese

References

  • Akita, K. (2009). A grammar of sound-symbolic words in Japanese: theoretical approaches to iconic and lexical properties of mimetics. Doctoral Dissertation, Kobe University.

  • Allen, K., and Ingulsrud, J. E. (2005). Reading manga: Patterns of personal literacies among adolescents. Language and Education, 19:265–280.

  • Baldwin, R. S., and Cody, J. M. (1978). Psycholinguistic approaches to a theory of punctuation. Journal of Literacy Research, 10(4):363–375.

  • Blackwell, N. L., Perlman, M., and Tree, J. E. F. (2015). Quotation as a multimodal construction. Journal of Pragmatics, 81:1–7.

  • Bredin, H. (1996). Onomatopoeia as a figure and a linguistic principle. New Literary History, 27:555–569.

  • Brenner, R. E. (2007). Understanding Manga and Anime. Westport, CT: Libraries Unlimited.

  • Cao, Y., Lau, R. W. H., and Chan, A. B. (2014). Look over here: Attention-directing composition of manga elements. ACM Transactions on Graphics, 33:1–11.

  • Catricalà, M., and Guidi, A. (2015). Onomatopoeias: A new perspective around space, image schemas and phoneme clusters. Cognitive Processing, 16:175–178.

  • Cohn, N. (2013a). Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica, 2013:35–63.

  • Cohn, N. (2013b). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. London, UK: Bloomsbury.

  • Cohn, N. (2016). A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition, 146:304–323.

  • Cohn, N., and Ehly, S. (2016). The vocabulary of manga: Visual morphology in dialects of Japanese visual language. Journal of Pragmatics, 92:17–29.

  • Cohn, N., Taylor-Weiner, A., and Grossman, S. (2012). Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Psychology – Cultural Psychology, 3:1–12.

  • Coulmas, F. (1989). The Writing Systems of the World. Malden, MA: Blackwell Publishers Inc.

  • Drummond-Mathews, A. (2010). What boys will be: A study of Shōnen manga. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum Books.

  • Duncan, R., and Smith, M. J. (2009). The Power of Comics. New York: Continuum Books.

  • Flyxe, M. (2002). Translation of Japanese onomatopoeia into Swedish (with focus on lexicalization). Africa & Asia, 2:54–73.

  • Forceville, C. (2013). Creative visual duality in comics balloons. In: The Agile Mind, T. Veale, K. Feyaerts, and C. Forceville (Eds.). Berlin: Mouton de Gruyter.

  • Forceville, C., Veale, T., and Feyaerts, K. (2010). Balloonics: The visuals of balloons in comics. In: The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form, J. Goggin, and D. Hassler-Forest (Eds.). Jefferson: McFarland & Company Inc.

  • Forster, I., Borgwaldt, S. R., and Neef, M. (2012). Form follows function: Interjections and onomatopoetica in comics. Writing Systems Research, 4:122–139.

  • Gravett, P. (2004). Manga: Sixty Years of Japanese Comics. New York, NY: HarperCollins.

  • Griffin, J. K. (1998). A brief glossary of comic book terminology. Serials Review, 24:71–76.

  • Groensteen, T. (2007). The System of Comics. Jackson: University of Mississippi Press.

  • Guynes, S. A. (2014). Four-color sound: A Peircean semiotics of comic book onomatopoeia. The Public Journal of Semiotics, 6:58–72.

  • Hamano, S. (1998). The Sound-Symbolic System of Japanese. Stanford: CSLI Publications.

  • Hatfield, C. (2005). Alternative Comics: An Emerging Literature. Jackson, MS: Univ. Press of Mississippi.

  • Hill, G. E. (1943a). The vocabulary of comic strips. Journal of Educational Psychology, 34:77.

  • Hill, G. E. (1943b). Word distortions in comic strips. The Elementary School Journal, 43:520–525.

  • Inose, H. (2007). Translating Japanese onomatopoeia and mimetic words. Translation Research Projects, Anthony, Pym. Alexander, Perekrestenko (eds.), 1:97–116.

  • Inoue, M. (2002). Gender, language, and modernity: Toward an effective history of Japanese women’s language. American Ethnologist, 29:392–422.

  • Ito, K. (2005). A history of manga in the context of Japanese culture and society. Journal of Popular Culture, 38:456–575.

  • Iwahara, A., Hatta, T., and Maehara, A. (2003). The effects of a sense of compatibility between type of script and word in written Japanese. Reading and Writing, 16:377–397.

  • Jüngst, H. E. (2004). Japanese comics in Germany. Perspectives: Studies in Translatology, 12:83–105.

  • Kadooka, K.-I. (2005). On the degree of lexicalization in English onomatopoeia from a historical perspective. The Ryukoku Journal of Humanities and Sciences, 27:1–13.

  • Klock, G. (2002). How to Read Superhero Comics and Why. New York, NY: A&C Black.

  • Kovanen, S., and Aizawa, K. (2015). A layered method for determining manga text bubble reading order. Image Processing (ICIP), 2015 IEEE International Conference on, 27–30 September 2015, 4283–4287.

  • Kurlander, D., Skelly, T., and Salesin, D. (1996). Comic chat. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, 225–236.

  • Manfredi, M., Cohn, N., and Kutas, M. (submitted). When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative.

  • Mazur, D., and Danner, A. (2014). Comics: A Global History, 1968 to the Present. London: Thames & Hudson.

  • Nakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In: The Visual Narrative Reader, N. Cohn (Ed.). London: Bloomsbury.

  • Natsume, F. (2003). Japanese Manga: Its Expression and Popularity. ABD, 34(1):3–5.

  • Nunberg, G. (1990). The Linguistics of Punctuation (No. 18). Stanford: Center for the Study of Language (CSLI).

  • Petersen, R. S. (2009). The acoustics of manga. A Comics Studies Reader, 163–171.

  • Piekos, N. (2009). Comics Grammar and Traditions. Blambot: Comic Fonts and Lettering (http://www.blambot.com) [Online]. Available: http://www.blambot.com/grammar.shtml (Accessed 3/23/09).

  • Prough, J. (2010). Shojo manga in Japan and abroad. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum.

  • Robertson, W. (2015). Orthography, foreigners, and fluency: Indexicality and script selection in Japanese manga. Japanese Studies, 35:205–222.

  • Schodt, F. L. (1983). Manga! Manga! The World of Japanese Comics. New York: Kodansha America Inc.

  • Sell, C. (2011). Manga translation and interculture. Mechademia, 6:93–108.

  • Smith (Shibamoto), J. S. (1996). Japanese writing. In: The World’s Writing Systems, P.T. Daniels, and W. Bright (Eds.). Oxford, UK: Oxford University Press.

  • Smith, J. S., and Schmidt, D. L. (1996). Variability in written Japanese: Towards a sociolinguistics of script choice. Visible Language, 30:46–71.

  • Takashima, M. (2002). The development of young children’s understanding of representation using thought-bubbles. The Japanese Journal of Developmental Psychology, 13:136–146.

  • Tsuchimochi, G. H. (1991). Education reform under the American occupation (3): Report of the United States education mission to japan and the reform of the Japanese language. Hirosaki University Repository for Academic Resources.

  • Walker, M. (1980). The Lexicon of Comicana. Port Chester, NY: Comicana, Inc.

  • Wellman, H. M., Hollander, M., and Schult, C. A. (1996). Young children’s understanding of thought bubbles and thoughts. Child Development, 67:768–788.

  • Witek, J. (2012). Comics modes: caricature and illustration in the crumb family’s dirty laundry. In: Critical Approaches to Comics: Theories and Methods, M.J. Smith, and R. Duncan (Eds.). New York, NY: Routledge.

  • Yannicopoulou, A. (2004). Visual aspects of written texts: preschoolers view comics. L1 – Educational Studies in Language and Literature, 4:169–181.

  • Zanettin, F. (2008). Comics in Translation. Manchester: St. Jerome Pub.

About the article

Nimish K. Pratha

Nimish K. Pratha will receive a Bachelors of Science in Physiology and Neuroscience and a Bachelors of Arts in Linguistics at the University of California at San Diego. He is particularly interested in the Japanese language and manga, and as a fluent speaker of Japanese, has worked translating them into English for several years. His other interests include neurolinguistics, dance, and game design.

Natalie Avunjian

Natalie Avunjian holds a Bachelors of Science in Cognitive Science from the University of California at San Diego with a minor in Computer Science. She currently works as an engineer in the field of information technology, and holds interests in the visual and fine arts, web design, and music.

Neil Cohn

Neil Cohn is an assistant professor at the Tilburg center for Cognition and Communication at Tilburg University. He is internationally recognized for his research on the overlap of the structure and cognition of sequential images and language. His book, The Visual Language of Comics (Bloomsbury, 2013), introduces a broad framework for studying visual narratives in the linguistic and cognitive sciences. His edited volume, The Visual Narrative Reader (Bloomsbury, 2016), integrates interdisciplinary research on visual narratives into a unified field within the cognitive sciences. His work is online at www.visuallanguagelab.com.


Published Online: 2016-10-04

Published in Print: 2016-12-01


Citation Information: Multimodal Communication, ISSN (Online) 2230-6587, ISSN (Print) 2230-6579, DOI: https://doi.org/10.1515/mc-2016-0017. Export Citation

Comments (0)

Please log in or register to comment.
Log in