Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Semiotica

Journal of the International Association for Semiotic Studies / Revue de l'Association Internationale de Sémiotique

Editor-in-Chief: Danesi, Marcel


IMPACT FACTOR 2018: 0.509

CiteScore 2018: 0.23

SCImago Journal Rank (SJR) 2018: 0.232
Source Normalized Impact per Paper (SNIP) 2018: 0.478

Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca: Classe A

Online
ISSN
1613-3692
See all formats and pricing
More options …
Volume 2019, Issue 230

Issues

A data-driven computational semiotics: The semantic vector space of Magritte’s artworks

Jean-François Chartier
  • Corresponding author
  • Centre interuniversitaire de recherche sur la science et la technologie, Université du Québec à Montréal, C.P. 8888, succ. Centre-ville, Montreal, Quebec H3C 3P8, Canada
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Davide Pulizzotto / Louis Chartrand
  • Laboratoire D’ANalyse Cognitive de l’Information, Informatique, Universite du Quebec a Montreal, 201, avenue du Président-Kennedy, Local PK-4150, Montreal, Quebec H2X 3Y7, Canada
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Jean-Guy Meunier
  • Laboratoire D’ANalyse Cognitive de l’Information, Philosophie, Université du Québec à Montréal Faculté des sciences humaines, cp8888, Montreal, Quebec H3C 3P8, Canada
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2019-09-05 | DOI: https://doi.org/10.1515/sem-2018-0120

Abstract

The rise of big digital data is changing the framework within which linguists, sociologists, anthropologists, and other researchers are working. Semiotics is not spared by this paradigm shift. A data-driven computational semiotics is the study with an intensive use of computational methods of patterns in human-created contents related to semiotic phenomena. One of the most promising frameworks in this research program is the Semantic Vector Space (SVS) models and their methods. The objective of this article is to contribute to the exploration of the SVS for a computational semiotics by showing what types of semiotic analysis can be accomplished within this framework. The study is applied to a unique body of digitized artworks. We conducted three short experiments in which we explore three types of semiotic analysis: paradigmatic analysis, componential analysis, and topic modelling analysis. The results reported show that the SVS constitutes a powerful framework within which various types of semiotic analysis can be carried out.

Keywords: computational semiotics; semantic vector space; data-driven; paradigmatic analysis; component analysis; topic analysis; artwork mining; René Magritte

References

  • Arora, Ravneet Singh. 2012. Towards automated classification of fine-art painting style: A comparative study. Rutgers University PhD dissertation.Google Scholar

  • Arthur, D. & S. Vassilvitskii. 2007. k-means++: The advantages of careful seeding. Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms, 1027–1035. Philadelphia: Society for Industrial and Applied Mathematics.Google Scholar

  • Bar, Yaniv, Noga Levy & Lior Wolf. 2014. Classification of artistic styles using binarized features derived from a deep neural network. Workshop at the European conference on computer vision, 71–84. Berlin: Springer.Google Scholar

  • Baroni, Marco & Alessandro Lenci. 2010. Distributional memory: A general framework for corpus-based semantics. Computational Linguistics 36(4). 673–721.CrossrefGoogle Scholar

  • Blei, David M. & John D. Lafferty. 2009. Topic models. Text Mining 10(71). 34.Google Scholar

  • Bordag, Stefan & Gerhard Heyer. 2007. A structuralist framework for quantitative linguistics. In Alexander Mehler & Reinhard Köhler (eds.), Aspects of automatic text analysis, 171–189. Berlin: Springer.Google Scholar

  • Bouma, Gerlof. 2009. Normalized (pointwise) mutual information in collocation extraction. In C. Chiarcos, R. Eckart de Castilho & M. Stede (eds.), Proceedings of Biennial GSCL Conference, 31–40. Tübingen: Gunter Narr.Google Scholar

  • Burgess, Curt. 2000. Theory and operational definitions in computational memory models: A response to Glenberg and Robertson. Journal of Memory and Language 43(3). 402–408.CrossrefGoogle Scholar

  • Burgess, Curt, Kay Livesay & Kevin Lund. 1998. Explorations in context space: Words, sentences, discourse. Discourse Processes 25(2–3). 211–257.CrossrefGoogle Scholar

  • Carneiro, Gustavo, Nuno Pinho Da Silva, Alessio Del Bue & João Paulo Costeira. 2012. Artistic image classification: An analysis on the Printart database. In A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato & C. Schmid (eds.), Computer vision – ECCV 2012 (Lecture Notes in Computer Science 7575), 143–157. Berlin: Springer.Google Scholar

  • Crowley, Elliot J. & Andrew Zisserman. 2014. The state of the art: Object retrieval in paintings using discriminative regions. In M. Valstar, A. French & T. Pridmore (eds.), Proceedings of the British machine vision conference. Birmingham: BMVA Press.Google Scholar

  • De Souza, Clarisse Sieckenius. 2005. The semiotic engineering of human-computer interaction. Cambridge, MA: MIT Press.Google Scholar

  • Dunning, Ted. 1993. Accurate methods for the statistics of surprise and coincidence. Journal Computational Linguistics 19(1). 61–74.Google Scholar

  • Erk, Katrin. 2009. Supporting inferences in semantic space: Representing words as regions. Proceedings of the eighth international conference on computational semantics, 104–115. Morristown: Association for Computational Linguistics.Google Scholar

  • Evans, James A. & Pedro Aceves. 2016. Machine translation: Mining text for social theory. Annual Review of Sociology 42. 21–50.CrossrefGoogle Scholar

  • Firth, J. R. 1957. A synopsis of linguistic theory, 1930–1955. Special issue, Studies in Linguistic Analysis, 1–32.Google Scholar

  • Floridi, Luciano. 1999. Philosophy and computing: An introduction. London: Psychology Press.Google Scholar

  • Gärdenfors, P. 2000. Conceptual spaces: The geometry of thought. Cambridge, MA: MIT Press.Google Scholar

  • Gärdenfors, P. 2014. The geometry of meaning: Semantics based on conceptual spaces. Cambridge, MA: MIT Press.Google Scholar

  • Graham, Daniel J., Jay D. Friedenberg, Daniel N. Rockmore & David J. Field. 2010. Mapping the similarity space of paintings: Image statistics and visual perception. Visual Cognition 18(4). 559–573.CrossrefGoogle Scholar

  • Greimas, Algirdas Julien, Frank Collins & Paul Perron. 1989. Figurative semiotics and the semiotics of the plastic arts. New Literary History 20(3). 627–649.CrossrefGoogle Scholar

  • Griffiths, Thomas L., Mark Steyvers & Joshua B. Tenenbaum. 2007. Topics in semantic representation. Psychological Review 114(2). 211–244.CrossrefGoogle Scholar

  • Groupe Mu. 1992. Traité du signe visuel: Pour une rhétorique de l’image. Paris: Seuil.Google Scholar

  • Hamilton, William L., Jure Leskovec & Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In A. van Den Bosch, K. Erk & N. A. Smith (eds.), Proceedings of the 54th annual meeting of the Association for Computational Linguistics, vol. 1, 1489–1501. Stroudsbourg, PA: Association for Computational Linguistics.Google Scholar

  • Hare, Jonathon S., Paul H. Lewis, Peter G. B. Enser, Christine J. Sandom, et al. 2006. Mind the gap: Another look at the problem of the semantic gap in image retrieval. In Ei Y. Chang (ed.), Proceedings of multimedia content analysis, management, and retrieval 2006, 75–86. San Jose, CA: SPIE.Google Scholar

  • Harris, Zellig S. 1951. Methods in structural linguistics. Chicago & London: University of Chicago Press.Google Scholar

  • Harris, Zellig S. 1954. Distributional structure. Word 10(23). 146–162.CrossrefGoogle Scholar

  • He, Kaiming, Xiangyu Zhang, Shaoqing Ren & Jian Sun. 2016. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.Google Scholar

  • Hébert, L. 2013. Magritte: Toutes les œuvres. tous les thèmes http://www.magrittedb.com/ (accessed16 March 2016).

  • Hébert, L. & Éric Trudel. 2013. Analyse des images. http://magrittedb.com (accessed 16 March 2016).

  • Johnson Jr., C Richard, Ella Hendriks, Igor J. Berezhnoy, Eugene Brevdo, Shannon M. Hughes, Ingrid Daubechies, Jia Li, Eric Postma & James Z. Wang. 2008. Image processing for artist identification. Signal Processing Magazine, IEEE 25(4). 37–48.CrossrefGoogle Scholar

  • Kell, Douglas B. & Stephen G. Oliver. 2004. Here is the evidence, now what is the hypothesis? The complementary roles of inductive and hypothesis-driven science in the post-genomic era. Bioessays 26(1). 99–105.CrossrefGoogle Scholar

  • Kelling, Steve, Wesley M. Hochachka, Daniel Fink, Mirek Riedewald, Rich Caruana, Grant Ballard & Giles Hooker. 2009. Data-intensive science: A new paradigm for biodiversity studies. BioScience 59(7). 613–620.CrossrefGoogle Scholar

  • Khan, Fahad Shahbaz, Shida Beigpour, Joost van de Weijer & Michael Felsberg. 2014. Painting-91: A large scale database for computational painting categorization. Machine Vision and Applications 25(6). 1385–1397.CrossrefGoogle Scholar

  • Kiela, Douwe & Stephen Clark. 2014. A systematic study of semantic vector space model parameters. Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC) at EACL, 21–30.Google Scholar

  • Kitchin, Rob. 2014. Big data, new epistemologies and paradigm shifts. Big Data & Society 1(1). https://journals.sagepub.com/doi/full/10.1177/2053951714528481.

  • Krizhevsky, Alex, Ilya Sutskever & Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou & K. Q. Weinberger (eds.), Advances in neural information processing systems, vol. 25, 1097–1105. Red Hook, NY: Curran.Google Scholar

  • Landauer, Thomas K., Peter W. Foltz & Darrell Laham. 1998. An introduction to latent semantic analysis. Discourse Processes 25(2–3). 259–284.CrossrefGoogle Scholar

  • Landauer, T. K. & S. T. Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review 104(2). 211–240.CrossrefGoogle Scholar

  • Larsen, Kai R. & David E. Monarchi. 2004. A mathematical approach to categorization and labeling of qualitative data: The latent categorization method. Sociological Methodology 34(1). 349–392.CrossrefGoogle Scholar

  • Lemaire, Benoît & Guy Denhière. 2006. Effects of high-order co-occurrences on word semantic similarity. Current Psychology Letters: Behaviour, Brain & Cognition 18(1). https://journals.openedition.org/cpl/471.

  • Leopold, Edda. 2005. On semantic spaces. LDV Forum 20. 63–86.Google Scholar

  • Li, Jia & James Z. Wang. 2004. Studying digital imagery of ancient paintings by mixtures of stochastic models. Image Processing, IEEE Transactions 13(3). 340–353.CrossrefGoogle Scholar

  • Liu, Ying, Dengsheng Zhang, Lu Guojun & Ma Wei-Ying. 2007. A survey of content-based image retrieval with high-level semantics. Pattern Recognition 40(1). 262–282.CrossrefGoogle Scholar

  • Lombardi, Thomas Edward. 2005. The classification of style in fine-art painting. Pace University PhD dissertation.Google Scholar

  • Lu, Qin. 2015. When similarity becomes opposition: Synonyms and antonyms discrimination in DSMs. Italian Journal on Computational Linguistics 1(1).Google Scholar

  • Lund, Kevin & Curt Burgess. 1996. Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers 28(2). 203–208.CrossrefGoogle Scholar

  • Manning, C. & H. Schütze. 1999. Foundations of statistical natural language processing. Cambridge, MA: MIT Press.Google Scholar

  • Mayaffre, Damon. 2008. De l’occurrence à l’isotopie: Les co-occurrences en lexicométrie. Syntaxe & Sémantique 9. 53–72.CrossrefGoogle Scholar

  • Mehler, Alexander. 2003. Methodological aspects of computational semiotics. SEED Journal 3(3). 71–80.Google Scholar

  • Meunier, Jean-Guy. 1989. Artificial intelligence and sign theory. Semiotica 77(1/3). 43–64.Google Scholar

  • Meunier, J. G. 2014. Humanités numériques ou computationnelles: Enjeux herméneutiques. Sens Public. http://www.sens-public.org/spip.php?article1121&lang=fr (accessed 17 June 2019).

  • Meunier, J. G. 2017. Vers une sémiotique computationnelle edited by S. Badir, I. Darrault, L. Hébert, P. Michelucci, and É. Trudel. Applied Semiotics/Sémiotique Appliquée 16.Google Scholar

  • Michel, J. B., Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, The Google Books Team, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, Steven Pinker, Martin A. Nowak & Erez Lieberman Aiden. 2011. Quantitative analysis of culture using millions of digitized books. Science 331(6014). 176–182.CrossrefGoogle Scholar

  • Mikolov, Tomas, Wen-tau Yih & Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. Proceedings of NAACL-HLT, 746–751.Google Scholar

  • Mimno, David. 2012. Computational historiography: Data mining in a century of classics journals. Journal on Computing and Cultural Heritage 5(1). 1–19.CrossrefGoogle Scholar

  • Mitchell, Jeff & Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive Science 34(8). 1388–1429.CrossrefGoogle Scholar

  • Nadin, Mihai. 2011. Information and semiotic processes the semiotics of computation. Cybernetics & Human Knowing 18(1–2). 153–175.Google Scholar

  • Neuman, Yair, Yochai Cohen & Dan Assaf. 2015. How do we understand the meaning of connotations? A cognitive computational model. Semiotica 205(1/4). 1–16.Google Scholar

  • Osgood, Charles E. 1952. The nature and measurement of meaning. Psychological Bulletin 49(3). 197–237.CrossrefGoogle Scholar

  • Osgood, Charles E. 1964. Semantic differential technique in the comparative study of cultures. American Anthropologist 66(3). 171–200.Google Scholar

  • Osgood, Charles Egerton, George John Suci & Percy H. Tannenbaum. 1957. The measurement of meaning. Urbana, IL: University of Illinois Press.Google Scholar

  • Pankratius, Victor, Li Justin, Michael Gowanlock & David M. Blair. 2016. Computer-aided discovery: Toward scientific insight generation with machine support. IEEE Intelligent Systems 31(4). 3–10.CrossrefGoogle Scholar

  • Pincemin, Bénédicte. 1999. Sémantique interprétative et analyses automatiques de textes: Que deviennent les sèmes? Sémiotiques 17. 71–120.Google Scholar

  • Rastier, F. 1996. La sémantique des textes: Concepts et applications. Hermes 9(16). 15–37.Google Scholar

  • Rastier, F. 2011. La mesure et le grain: Sémantique de corpus. Paris: Honoré Champion.Google Scholar

  • Rieger, Burghard B. 1981. Feasible fuzzy semantics: On some problems of how to handle word meaning empirically. In H. J. Eikmeyer & H. Rieser (eds.), Words, worlds, and contexts: New approaches in word semantics (Research in Text Theory 6), 193–209. Berlin: de Gruyter.Google Scholar

  • Rieger, Burghard B. 1983. Clusters in semantic space. Actes Du Congrès International Informatique et Science Humaines. 805–814.Google Scholar

  • Rieger, Burghard B. 1989. Distributed semantic representation of word meanings. Workshop on parallel processing: Logic, organization, and technology, 243–273. Berlin: Springer.Google Scholar

  • Rieger, Burghard B. 1992. Fuzzy computational semantics. Fuzzy systems: Proceedings of the Japanese-German-Center symposium, series, vol. 3, 197–217. Berlin: Publications of the JGCB.Google Scholar

  • Rieger, Burghard B. 1999. Semiotics and computational linguistics. In L. A. Zadeh (ed.), Computing with words in information/intelligent systems, vol. 1, 93–118. Berlin: Springer.Google Scholar

  • Rousseeuw, Peter J. 1987. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Computational and Applied Mathematics 20. 53–65.CrossrefGoogle Scholar

  • Sahlgren, M. 2006. The word-space model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. University of Stockholm PhD dissertation.Google Scholar

  • Sahlgren, Magnus. 2005. An introduction to random indexing. Methods and applications of semantic indexing workshop at the 7th international conference on terminology and knowledge engineering, TKE, vol. 5.Google Scholar

  • Sahlgren, Magnus. 2008. The distributional hypothesis. Italian Journal of Linguistics 20(1). 33–54.Google Scholar

  • Saleh, Babak & Ahmed Elgammal. 2016. Large-scale classification of fine-art paintings: Learning the right metric on the right feature. Digital Art History 2.Google Scholar

  • Santus, Enrico, Alessandro Lenci, Lu Qin & Sabine Schulte Im Walde. 2014. Chasing hypernyms in vector spaces with entropy. Proceedings of the 14th conference of the European chapter of the Association for Computational Linguistics, vol. 2, 38–42.Google Scholar

  • Schütze, Hinrich & Jan Pedersen. 1993. A vector model for syntagmatic and paradigmatic relatedness. Proceedings of the 9th annual conference of the UW centre for the new OED and text research, 104–113.Google Scholar

  • Shamir, Lior. 2012. Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2). 149–154.CrossrefGoogle Scholar

  • Shamir, Lior. 2015. What makes a Pollock Pollock: A machine vision approach. International Journal of Arts and Technology 8(1). 1–10.CrossrefGoogle Scholar

  • Shamir, Lior & Jane A. Tarakhovsky. 2012. Computer analysis of art. Journal on Computing and Cultural Heritage 5(2). 7.Google Scholar

  • Shen, Jialie. 2009. Stochastic modeling Western paintings for effective classification. Pattern Recognition 42(2). 293–301.CrossrefGoogle Scholar

  • Shutova, Ekaterina. 2010. Models of metaphor in NLP. Proceedings of the 48th annual meeting of the association for computational linguistics, 688–697. Morristown, NJ: Association for Computational Linguistics.Google Scholar

  • Smeulders, Arnold W. M., Marcel Worring, Simone Santini, Amarnath Gupta & Ramesh Jain. 2000. Content-based image retrieval at the end of the early years. Pattern Analysis and Machine Intelligence, IEEE Transactions 22(12). 1349–1380.CrossrefGoogle Scholar

  • Stork, David G. 2009. Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature. In Gerald Sommer, Kostas Daniilidis & Josef Pauli (eds.), Computer analysis of images and patterns, 9–24. Berlin: Springer.Google Scholar

  • Sylvestre, David (ed.). 1997. René Magritte: Catalogue raisonné. Anvers: Fonds Mercator.Google Scholar

  • Tanaka-Ishii, Kumiko. 2010. Semiotics of programming. Cambridge, MA: Cambridge University Press.Google Scholar

  • Tanaka-Ishii, Kumiko. 2015. Semiotics of computing: Filling the gap between humanity and mechanical inhumanity. In Peter Pericles Trifonas (ed.), International handbook of semiotics, 981–1002. Berlin: Springer.Google Scholar

  • Turney, P. D. & P. Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research 37(1). 141–188.CrossrefGoogle Scholar

  • Van Rijsbergen, Cornelis Joost. 2004. The geometry of information retrieval. Cambridge: Cambridge University Press.Google Scholar

  • Widdows, Dominic. 2003. Orthogonal negation in vector spaces for modelling word-meanings and document retrieval. Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, 136–143. Morristown, NJ: Association for Computational Linguistics.Google Scholar

  • Widdows, Dominic. 2004. Geometry and meaning. Stanford, CA: CSLI.Google Scholar

  • Widdows, Dominic. 2008. Semantic vector products: Some initial investigations. https://ai.google/research/pubs/pub33477 (accessed 17 June 2019).

  • Widdows, Dominic & Trevor Cohen. 2014. Reasoning with vectors: A continuous model for fast robust inference. Logic Journal of IGPL 23(2). 141–173.Google Scholar

  • Zhang, Dengsheng, Md Monirul Islam & Lu Guojun. 2012. A review on automatic image annotation techniques. Pattern Recognition 45(1). 346–362.CrossrefGoogle Scholar

  • Zujovic, Jana, Lisa Gandy, Scott Friedman, Bryan Pardo & Thrasyvoulos N. Pappas. 2009. Classifying paintings by artistic genre: An analysis of features & classifiers. Multimedia signal processing, 2009, 1–5. IEEE.Google Scholar

About the article

Published Online: 2019-09-05

Published in Print: 2019-10-25


Citation Information: Semiotica, Volume 2019, Issue 230, Pages 19–69, ISSN (Online) 1613-3692, ISSN (Print) 0037-1998, DOI: https://doi.org/10.1515/sem-2018-0120.

Export Citation

© 2019 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in