The Curious Case of Connectionism

Istvan S. N. Berkeley 1
  • 1 The University of Louisiana at Lafayette, , Lafayette, United States of America

Abstract

Connectionist research first emerged in the 1940s. The first phase of connectionism attracted a certain amount of media attention, but scant philosophical interest. The phase came to an abrupt halt, due to the efforts of Minsky and Papert (1969), when they argued for the intrinsic limitations of the approach. In the mid-1980s connectionism saw a resurgence. This marked the beginning of the second phase of connectionist research. This phase did attract considerable philosophical attention. It was of philosophical interest, as it offered a way of counteracting the conceptual ties to the philosophical traditions of atomism, rationalism, logic, nativism, rule realism and a concern with the role symbols play in human cognitive functioning, which was prevalent as a consequence of artificial intelligence research. The surge in philosophical interest waned, possibly in part due to the efforts of some traditionalists and the so-called black box problem. Most recently, what may be thought of as a third phase of connectionist research, based on so-called deep learning methods, is beginning to show some signs of again exciting philosophical interest.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Abraham, Tara. Rebel Genius: Warren S. McCulloch’s Transdisciplinary Life in Science, Cambridge, Mass.: M.I.T. Press, 2016.

  • Aizawa, Ken. “Connectionism and Artificial Intelligence: History and Philosophical Interpretations” Journal of Experimental and Theoretical Artificial Intelligence, 4 (1992), 295–313.

  • Aizawa, Ken. “Representations without Rules, Connectionism and the Syntactic Argument”. Synthese, 101:3 (1994), 465–492.

  • Aizawa, Ken. The Systematicity Arguments, Boston, Mass.: Kluwer Academic, 2003.

  • Aizawa, Ken. “Warren McCulloch’s Turn to Cybernetics: What Walter Pitts Contributed”. Interdisciplinary Science Review, 37:3 (2012), 206–217.

  • Anderson, James A. and Rosenfield, Edward (Eds.) (1998), Talking Nets: An Oral History of Neural Networks, Cambridge, Mass.: M.I.T. Press, 1998.

  • Arnold, Solvi, Suzuki, Reiji, and Arita, Takaya. “Selection for Representation in Higher-Order Adaption”, Minds and Machines, 25:1 (2015), 73–95.

  • Bechtel, William. “Contemporary Connectionism: Are the New Parallel Distributed Processing Models Cognitive or Associationist?” Behaviorism, 13 (1985), 53–61.

  • Bechtel, William and Abrahamsen, Adelle. Connectionism and the Mind: An Introduction to Parallel Processing in Networks, Cambridge, Mass.: Basil Blackwell, 1991.

  • Berkeley, Istvan, Dawson, Michael, Medler, David, Schopflocher, Donald and Hornsby, Loraine. „Density Plots of Hidden Unit Activations Reveal Interpretable Bands“, Connection Science, 7:2 (1995) 167–186.

  • Berkeley, Istvan. “What the #$*%! Is a Subsymbol?” Minds and Machines, 10:1 (2000), 1–13.

  • Berkeley, Istvan, and Gunay, Cengiz. “Conducting Banding Analysis with Trained Networks of Sigmoid Units”, Connection Science, 16:2 (2004), 119–128.

  • Berkeley, Istvan. “What the <0.70, 1.17, 0.99, 1.07> is a Symbol?” Minds and Machines, 18 (2008), 93–105.

  • Berkeley, Istvan, and Raine, Roxanne. (2011), “An Old Fashioned Connectionist Approach to a Cajun Chord Change Problem”, Connection Science, 23:3 (2011), 209–218,

  • Boden, Margaret. Mind as Machine: A History of Cognitive Science, Oxford: Oxford U.P, 2006.

  • Browne, Antony. (Ed.) Neural Network Analysis, Architectures and Applications, Philadelphia, PA: Institute of Physics, 1997.

  • Buckner, Cameron. “Empiricism Without Magic: Tranformational Abstraction in Deep Convolutional Neural Networks”, Synthese, 195:12 (2018), 5339–5372. https://doi.org/10.1007/s11229-018-01949-1.

  • Bullinaria, John. “Analyzing the Internal Representations of Trained Neural Networks”. In Neural Network Analysis, Architectures and Applications, edited by Antony Browne, (1997), 3–26.

  • Chalmers, David. “Syntactic Transformations on Distributed Representations” in Connection Science, 2:1–2 (1990), 53–62.

  • Chalmers, David. “Connectionism and Compositionality: Why Fodor and Pylyshyn were Wrong”, in Philosophical Psychology, 6:3 (1993), 305–320.

  • Charniak, Eugene. Introduction to Deep Learning, Cambridge Mass.: M.I.T. Press, 2019.

  • Cireşan Dan, Meier Ueli, Masci, Jonathan, Schmidhuber Jurgen. “Multi-column deep neural network for traffic sign classification”. Neural Networks 32 (2012), 333–338.

  • Chomsky, Noam. Aspects of a Theory of Syntax, Cambridge, Mass.: M.I.T. Press, 1965.

  • Churchland, Paul. M. “On the Nature of Theories: A Neurocomputational Perspective” in Minnesota Studies in the Philosophy of Science 14 (1989), 59–101.

  • Churchland, Patricia S. and Sejnowski, Terrence. “Neural Representation and Neural Computation”, Philosophical Perspectives, 4 (1990), 343–382.

  • Clark, Andy. Microcognition: Philosophy, Cognitive Science and Parallel Distributed Processing, Cambridge, Mass.: M.I.T. Press, 1989.

  • Clark, Andy. “A review of “Simple Minds“ by D. Lloyd, 1989, M.I.T. Press, London”, Connection Science, 1:4 (1989), 418–421.

  • Clark, Andy. “Representation, Development and Situated Connectionism”, Connection Science, 4:3–4 (1992), 171–174.

  • Clark, Andy. and Lutz, Rudi. (Eds.) Connectionism in Context (Human Centered Systems), New York, NY: Springer-Verlag, 1992.

  • Davis, Martin. “Two Notions of Implicit Rules”, in Philosophical Perspectives, 9 (1995), 153–183.

  • Davis, Stephen. (Ed.) Connectionism: Theory and Practice, New York, NY: Oxford U.P. 1992.

  • Dawson, Michael and Schopflocher, Don. “Modifying the Generalized Delta Rule to Train Networks of Non-Monotonic Processors for Pattern Classification” Connection Science, 4 (1992), 19–31.

  • Dawson, Michael. Medler, David. and Berkeley, Istvan. “PDP Networks Can Provide Models That Are Not Mere Implementations of Classical Theories” Philosophical Psychology, 10:1 (1997), 25–40.

  • Dawson, Michael. Minds and Machines: Connectionism and Psychological Modeling, Hoboken, NJ: Wiley-Blackwell, 2008.

  • Dreyfus, Herbert. and Dreyfus, Stuart. “Making a Mind verses Modeling a Brain: Artificial Intelligence Back at a Branchpoint” in S. Graubard (ed.) The Artificial Intelligence Debate: False Starts, Real Foundations, edited by Graubard, Stephen, Cambridge, Mass.: M.I.T. Press, 1988.

  • Elman, Jeff, Bates, Elizabeth, Johnson, Mark, Karmiloff-Smith, Annette, Parisi, Domenico, and Plunkett, Kim. Rethinking Innateness: A Connectionist Perspective on Development, Cambridge, Mass.: M.I.T. Press, 1996.

  • Fodor, Jerry. The Language of Thought, Cambridge, Mass.: Harvard U.P., 1975.

  • Fodor, Jerry. The Modularity of Mind, Cambridge, Mass.: M.I.T. Press, 1983.

  • Fodor, Jerry. “Discussion: Connectionism and the Problem of Systematicity (continued): Why Smolensky’s Solution Still does not Work.” Cognition, 62:1 (1997), 109–119.

  • Fodor, Jerry and Pylyshyn, Zenon. “Connectionism and Cognitive Architecture: A Critical Analysis”, Cognition, 28 (1988), 3–71.

  • Fodor, Jerry and McLaughlin, Brian. “Connectionism and the Problem of Systematicity: Why Smolensky‘s Solution Doesn‘t Work”, Cognition, 35 (1990), 183–204.

  • Haugeland, John. Artificial Intelligence: The Very Idea, Cambridge, Mass.: M.I.T. Press, 1985.

  • Heicht-Nielson, Robert. Neurocomputation, New York, NY: Addison-Wesley Pub. Co., 1990.

  • Hopfield, John. “Neural Networks and Physical Systems with Emergent Collective Computational Abilities”, Proceedings of the National Academy of the Sciences, USA, 79 (1982), 2554–2558.

  • Hopfield, John. “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons”, Proceedings of the National Academy of Sciences, USA, 81 (1984), 3088–3092.

  • Horgan, Terence and Tienson, John. “Spindel Conference 1987: Connectionism and The Philosophy of Mind”, The Southern Journal of Philosophy, Vol. XXVI, Supplement, (1987).

  • Horgan, Terence. and Tienson, John. “Representations Without Rules”, in Philosophical Topics, 17 (1989), 147–174.

  • Horgan, Terence. and Tienson, John. (Eds.) Connectionism and The Philosophy of Mind, New York, NY: Springer, 1991.

  • Horgan, Terence. and Tienson, John. “Settling into a New Paradigm”, Connectionism and The Philosophy of Mind, edited by Horgan, Terence and Tienson, John (1991), 241–260.

  • Klahr, David, Langley, Pat and Neches, Robert (eds.), Production System Models of Learning and Development, Cambridge, Mass.: MIT Press, 1987.

  • Lloyd, Dan. Simple Minds, Cambridge, Mass.: M.I.T. Press, 1989.

  • Lovecraft, Howard, P. H. P. Lovecraft, The Fiction, Complete and Unabridged, New York: Barnes and Noble, 2008.

  • Marr, David. Vision: A Computational Approach, San Francisco, CA: Freeman & Co., 1982.

  • McClelland, Jay and Rumelhart, David. Explorations in Parallel Distributed Processing, Cambridge, Mass.: M.I.T. Press, 1986.

  • McClelland, Jay, Rumelhart, David and Hinton, Geoffrey. “The Appeal of PDP”. In Parallel Distributed Processing edited by Rumelhart, McClelland and the PDP Research Group, (1986), 503–529.

  • McCloskey, Michael. „Networks and Theories: The Place of Connectionism in Cognitive Science“ in Psychological Science, 2:6 (1991), 387–395.

  • McCulloch, Warren., Embodiments of Mind, Cambridge, Mass.: MIT Press, 1965.

  • McCulloch, Warren. and Pitts, Walter. „A logical calculus of the ideas immanent in nervous activity“. In Bulletin of Mathematical Biophysics, 5 (1943), 115–133.

  • McDonald, Cynthia and McDonald, Graham. (Eds.) Connectionism: Debates in Psychological Explanation, (2 vols.), Cambridge, Mass.: Blackwell Publishers, 1995.

  • Medler, David. “A Brief History of Connectionism” in Neural Computing Surveys, 1 (1998), 61–101.

  • Miller, Alexander, and Wright, Crispin. (Eds.), Rule-Following and Meaning, Ithaca, NY: McGill-Queen’s U. P., 2002.

  • Minsky, Marvin and Papert, Seymour. Perceptrons: An Introduction to Computational Geometry, Cambridge, Mass.: M.I.T. Press, 1969.

  • Mole, Christopher. “Dead Reckoning in the Desert Ant: A Defense of Connectionist Models”, in Review of Philosophy and Psychology, 5:2 (2014), 277–290.

  • Mozer, Mike and Smolensky, Paul. „Using Relevance to Reduce Network Size Automatically“ in Connection Science, 1 (1989), 3–16.

  • Muller, Vincent. (Ed.) Fundamental Issues in Artificial Intelligence, New York, NY: Springer International, 2016.

  • Newell, Allen and Simon, Herbert. “Computer Science as Empirical Inquiry” in Communications of the ACM, 19:3 (1976), 113–126.

  • Newell Allen. “Physical Symbol Systems” Cognitive Science, 4 (1980), 135–183.

  • Pitts, Walter and McCulloch, Warren. “How We Know Universals the Perception of Auditory and Visual Forms”, Bulletin of Mathematical Biophysics, 9:3 (1947), 127–147.

  • Piccinini, Gualtiero. “The First Computational Theory of Mind and Brain: A Close Look at McCulloch and Pitts‘s‚ Logical Calculus of Ideas Immanent in Nervous Activity‘”, Synthese, 141:2 (2004), 175–215.

  • Plunkett, Kim and Elman, Jeffery. Exercises in Rethinking Innateness: A Handbook for Connectionist Simulations, Cambridge, Mass.: M.I.T. Press, 1997.

  • Pollack, Jordan. “Recursive Distributed Representations”, Artificial Intelligence, 46 (1990), 77–105.

  • Pylyshyn, Zenon. Computation and Cognition: Towards a Foundation of Cognitive Science, Cambridge, Mass.: MIT Press, 1984.

  • Ramsey, William, Stitch, Stephen, and Rumelhart, David. (Eds.) Philosophy and Connectionist Theory, Hillsdale, NJ: Lawrence Erlbaum Associated, 1991.

  • Robinson, David. (1992), “Implications of Neural Networks for How We think about Brain Function”, in Behavioural and Brain Sciences, 15 (1992), 644–655.

  • Rosenblatt, Frank “The Perceptron: A Probabilistic Model For Information Storage and Organization in the Brain”, Psychological Review, 65:60 (1958), 386–408.

  • Rosenblatt, Frank. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Washington, DC: Spartan books, 1962.

  • Rubio, Ezequiel (in press), “Computational Functionalism for the Deep Learning Era”, to appear in Minds and Machines.

  • Rumelhart, David, McClelland, Jay and the PDP Research Group, (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, M.I.T. Press, (Cambridge, MA), (2 vols).

  • Samet, Jerry, „The Historical Controversies Surrounding Innateness“, in The Stanford Encyclopedia of Philosophy Edward N. Zalta (Ed.), (2008) URL = <https://plato.stanford.edu/archives/fall2008/entries/innateness-history/>.

  • Schlatter, Mark and Aizawa, Ken. “Walter Pitts and „A Logical Calculus“”, Synthese, 162:2 (2008), 235–250.

  • Schneider, Walter. “Connectionism: Is it a Paradigm Shift for Psychology?” in Behavior Research Methods, Instruments & Computers, 19 (1987), 73–83.

  • Sejnowski, Terence. The Deep Learning Revolution, Cambridge, Mass.: M.I.T. Press, 2018.

  • Shea, N. “Representational Development Need Not Be Explicable-By-Content”, in Fundamental Issues in Artificial Intelligence, edited by Vincent Muller, (2016), 221–238.

  • Smolensky, Paul. “The Constituent Structure of Mental States: A Reply to Fodor and Pylyshyn” in Southern Journal of Philosophy, 26 (1987), 137–160.

  • Smolensky, Paul. “On the Proper Treatment of Connectionism” in Behavioural and Brain Sciences, 11 (1988), 1–74.

  • Smolensky, Paul. “Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems” Artificial Intelligence, 46 (1990), 159–216.

  • Taddeo, Mariarosaria. and Floridi, Luciano. “The Debate on the Moral Responsibilities of Online Service Providers” in Science and Engineering Ethics, 22:6 (2016), 1575–1603.

  • Thorndike, Edward. Fundamentals of Learning, New York, NY: Teachers College, Columbia University, 1932.

  • Van Gelder, Tim. “Compositionaility: A Connectionist Variant on a Classical Theme”, Cognitive Science, 14:3 (1990), 355–384.

  • Vera, Alonso. and Simon, Herbert. “Reply to Touretzky and Pomerleau: Reconstructing Physical Symbol Systems”, Cognitive Science, 18 (1994), 355–360.

  • Walker, Stephen. “A Brief History of Connectionism and Its Psychological Implications”, in Connectionism in Context (Human Centered Systems), edited by Clark and Lutz (1992), 123–144.

  • Waskan, Jonathon. “A Critique of Connectionist Semantics”, in Connection Science, 13(3) (2001), 277–292.

OPEN ACCESS

Journal + Issues

Open Philosophy is an international Open Access, peer-reviewed academic journal covering all areas of philosophy. The objective of Open Philosophy is to foster free exchange of ideas and provide an appropriate platform for presenting, discussing and disseminating new concepts, current trends, theoretical developments and research findings related to the broadest philosophical spectrum.

Search