Pow, Punch, Pika, and Chu: The Structure of Sound Effects in Genres of American Comics and Japanese Manga

Nimish K. Pratha 1 , Natalie Avunjian 1 ,  and Neil Cohn 1 , 2
  • 1 Center for Research in Language, University of California at San Diego, San Diego, CA, USA
  • 2 Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, The Netherlands
Nimish K. Pratha
  • Further information
  • Nimish K. Pratha will receive a Bachelors of Science in Physiology and Neuroscience and a Bachelors of Arts in Linguistics at the University of California at San Diego. He is particularly interested in the Japanese language and manga, and as a fluent speaker of Japanese, has worked translating them into English for several years. His other interests include neurolinguistics, dance, and game design.
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Natalie Avunjian
  • Further information
  • Natalie Avunjian holds a Bachelors of Science in Cognitive Science from the University of California at San Diego with a minor in Computer Science. She currently works as an engineer in the field of information technology, and holds interests in the visual and fine arts, web design, and music.
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Neil Cohn
  • Corresponding author
  • Netherlands
  • Email
  • Further information
  • Neil Cohn is an assistant professor at the Tilburg center for Cognition and Communication at Tilburg University. He is internationally recognized for his research on the overlap of the structure and cognition of sequential images and language. His book, The Visual Language of Comics (Bloomsbury, 2013), introduces a broad framework for studying visual narratives in the linguistic and cognitive sciences. His edited volume, The Visual Narrative Reader (Bloomsbury, 2016), integrates interdisciplinary research on visual narratives into a unified field within the cognitive sciences. His work is online at www.visuallanguagelab.com.
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

As multimodal works, comics are characterized as much by their use of language as by the style of their images. Sound effects in particular are exemplary of comics’ language-use, and we explored this facet of comics by analyzing a corpus of books from genres in the United States (mainstream and independent) and Japan (shonen/boys’ and shojo/girls’). We found variation between genres and between cultures across several properties of the content and presentation of sound effects. Foremost, significant differences arose between the lexical categories of sound effects (ex. onomatopoetic: Pow! vs. descriptive: Punch!) between genres within both culture’s works. Additionally, genres in Japanese manga vary in the scripts used to write sound effects in Japanese (hiragana vs. katakana). We argue that, in English, a similar function is communicated through the presence or absence of textual font stylization. Altogether, these aspects of variation mark sound effects as important carriers of multimodal information, and provide distinctions by which genres and cultures of comics can be distinguished.

  • Akita, K. (2009). A grammar of sound-symbolic words in Japanese: theoretical approaches to iconic and lexical properties of mimetics. Doctoral Dissertation, Kobe University.

  • Allen, K., and Ingulsrud, J. E. (2005). Reading manga: Patterns of personal literacies among adolescents. Language and Education, 19:265–280.

    • Crossref
    • Export Citation
  • Baldwin, R. S., and Cody, J. M. (1978). Psycholinguistic approaches to a theory of punctuation. Journal of Literacy Research, 10(4):363–375.

  • Blackwell, N. L., Perlman, M., and Tree, J. E. F. (2015). Quotation as a multimodal construction. Journal of Pragmatics, 81:1–7.

    • Crossref
    • Export Citation
  • Bredin, H. (1996). Onomatopoeia as a figure and a linguistic principle. New Literary History, 27:555–569.

    • Crossref
    • Export Citation
  • Brenner, R. E. (2007). Understanding Manga and Anime. Westport, CT: Libraries Unlimited.

  • Cao, Y., Lau, R. W. H., and Chan, A. B. (2014). Look over here: Attention-directing composition of manga elements. ACM Transactions on Graphics, 33:1–11.

  • Catricalà, M., and Guidi, A. (2015). Onomatopoeias: A new perspective around space, image schemas and phoneme clusters. Cognitive Processing, 16:175–178.

    • Crossref
    • PubMed
    • Export Citation
  • Cohn, N. (2013a). Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica, 2013:35–63.

  • Cohn, N. (2013b). The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. London, UK: Bloomsbury.

  • Cohn, N. (2016). A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition, 146:304–323.

    • Crossref
    • PubMed
    • Export Citation
  • Cohn, N., and Ehly, S. (2016). The vocabulary of manga: Visual morphology in dialects of Japanese visual language. Journal of Pragmatics, 92:17–29.

    • Crossref
    • Export Citation
  • Cohn, N., Taylor-Weiner, A., and Grossman, S. (2012). Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Psychology – Cultural Psychology, 3:1–12.

  • Coulmas, F. (1989). The Writing Systems of the World. Malden, MA: Blackwell Publishers Inc.

  • Drummond-Mathews, A. (2010). What boys will be: A study of Shōnen manga. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum Books.

  • Duncan, R., and Smith, M. J. (2009). The Power of Comics. New York: Continuum Books.

  • Flyxe, M. (2002). Translation of Japanese onomatopoeia into Swedish (with focus on lexicalization). Africa & Asia, 2:54–73.

  • Forceville, C. (2013). Creative visual duality in comics balloons. In: The Agile Mind, T. Veale, K. Feyaerts, and C. Forceville (Eds.). Berlin: Mouton de Gruyter.

  • Forceville, C., Veale, T., and Feyaerts, K. (2010). Balloonics: The visuals of balloons in comics. In: The Rise and Reason of Comics and Graphic Literature: Critical Essays on the Form, J. Goggin, and D. Hassler-Forest (Eds.). Jefferson: McFarland & Company Inc.

  • Forster, I., Borgwaldt, S. R., and Neef, M. (2012). Form follows function: Interjections and onomatopoetica in comics. Writing Systems Research, 4:122–139.

    • Crossref
    • Export Citation
  • Gravett, P. (2004). Manga: Sixty Years of Japanese Comics. New York, NY: HarperCollins.

  • Griffin, J. K. (1998). A brief glossary of comic book terminology. Serials Review, 24:71–76.

    • Crossref
    • Export Citation
  • Groensteen, T. (2007). The System of Comics. Jackson: University of Mississippi Press.

  • Guynes, S. A. (2014). Four-color sound: A Peircean semiotics of comic book onomatopoeia. The Public Journal of Semiotics, 6:58–72.

    • Crossref
    • Export Citation
  • Hamano, S. (1998). The Sound-Symbolic System of Japanese. Stanford: CSLI Publications.

  • Hatfield, C. (2005). Alternative Comics: An Emerging Literature. Jackson, MS: Univ. Press of Mississippi.

  • Hill, G. E. (1943a). The vocabulary of comic strips. Journal of Educational Psychology, 34:77.

    • Crossref
    • Export Citation
  • Hill, G. E. (1943b). Word distortions in comic strips. The Elementary School Journal, 43:520–525.

    • Crossref
    • Export Citation
  • Inose, H. (2007). Translating Japanese onomatopoeia and mimetic words. Translation Research Projects, Anthony, Pym. Alexander, Perekrestenko (eds.), 1:97–116.

  • Inoue, M. (2002). Gender, language, and modernity: Toward an effective history of Japanese women’s language. American Ethnologist, 29:392–422.

    • Crossref
    • Export Citation
  • Ito, K. (2005). A history of manga in the context of Japanese culture and society. Journal of Popular Culture, 38:456–575.

    • Crossref
    • Export Citation
  • Iwahara, A., Hatta, T., and Maehara, A. (2003). The effects of a sense of compatibility between type of script and word in written Japanese. Reading and Writing, 16:377–397.

    • Crossref
    • Export Citation
  • Jüngst, H. E. (2004). Japanese comics in Germany. Perspectives: Studies in Translatology, 12:83–105.

    • Crossref
    • Export Citation
  • Kadooka, K.-I. (2005). On the degree of lexicalization in English onomatopoeia from a historical perspective. The Ryukoku Journal of Humanities and Sciences, 27:1–13.

  • Klock, G. (2002). How to Read Superhero Comics and Why. New York, NY: A&C Black.

  • Kovanen, S., and Aizawa, K. (2015). A layered method for determining manga text bubble reading order. Image Processing (ICIP), 2015 IEEE International Conference on, 27–30 September 2015, 4283–4287.

    • Crossref
    • Export Citation
  • Kurlander, D., Skelly, T., and Salesin, D. (1996). Comic chat. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, 225–236.

    • Crossref
    • Export Citation
  • Manfredi, M., Cohn, N., and Kutas, M. (submitted). When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative.

  • Mazur, D., and Danner, A. (2014). Comics: A Global History, 1968 to the Present. London: Thames & Hudson.

  • Nakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In: The Visual Narrative Reader, N. Cohn (Ed.). London: Bloomsbury.

  • Natsume, F. (2003). Japanese Manga: Its Expression and Popularity. ABD, 34(1):3–5.

  • Nunberg, G. (1990). The Linguistics of Punctuation (No. 18). Stanford: Center for the Study of Language (CSLI).

  • Petersen, R. S. (2009). The acoustics of manga. A Comics Studies Reader, 163–171.

  • Piekos, N. (2009). Comics Grammar and Traditions. Blambot: Comic Fonts and Lettering (http://www.blambot.com) [Online]. Available: http://www.blambot.com/grammar.shtml (Accessed 3/23/09).

  • Prough, J. (2010). Shojo manga in Japan and abroad. In: Manga: An Anthology of Global and Cultural Perspectives, T. Johnson-Woods (Ed.). New York: Continuum.

  • Robertson, W. (2015). Orthography, foreigners, and fluency: Indexicality and script selection in Japanese manga. Japanese Studies, 35:205–222.

    • Crossref
    • Export Citation
  • Schodt, F. L. (1983). Manga! Manga! The World of Japanese Comics. New York: Kodansha America Inc.

  • Sell, C. (2011). Manga translation and interculture. Mechademia, 6:93–108.

    • Crossref
    • Export Citation
  • Smith (Shibamoto), J. S. (1996). Japanese writing. In: The World’s Writing Systems, P.T. Daniels, and W. Bright (Eds.). Oxford, UK: Oxford University Press.

  • Smith, J. S., and Schmidt, D. L. (1996). Variability in written Japanese: Towards a sociolinguistics of script choice. Visible Language, 30:46–71.

  • Takashima, M. (2002). The development of young children’s understanding of representation using thought-bubbles. The Japanese Journal of Developmental Psychology, 13:136–146.

  • Tsuchimochi, G. H. (1991). Education reform under the American occupation (3): Report of the United States education mission to japan and the reform of the Japanese language. Hirosaki University Repository for Academic Resources.

  • Walker, M. (1980). The Lexicon of Comicana. Port Chester, NY: Comicana, Inc.

  • Wellman, H. M., Hollander, M., and Schult, C. A. (1996). Young children’s understanding of thought bubbles and thoughts. Child Development, 67:768–788.

    • Crossref
    • PubMed
    • Export Citation
  • Witek, J. (2012). Comics modes: caricature and illustration in the crumb family’s dirty laundry. In: Critical Approaches to Comics: Theories and Methods, M.J. Smith, and R. Duncan (Eds.). New York, NY: Routledge.

  • Yannicopoulou, A. (2004). Visual aspects of written texts: preschoolers view comics. L1 – Educational Studies in Language and Literature, 4:169–181.

    • Crossref
    • Export Citation
  • Zanettin, F. (2008). Comics in Translation. Manchester: St. Jerome Pub.

Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Multimodal Communication is a venue for researchers in diverse fields of study, who explore varied & multimodal ways to conduct research & illustrate findings. It is open to papers from anthropology to art, math, psychology, science, & beyond. Articles share a commitment to developing multimodality and may use visuals/writing/sound to explore a theme; be highly theoretical or may be based on an empirical study integrating a number of modes.

Search