The Evaluation of Discovery: Models, Simulation and Search through “Big Data”

Clark Glymour 1 , Joseph D. Ramsey 2 ,  and Kun Zhang 2
  • 1 Clark Glymour, Carnegie Mellon, University, United States of America
  • 2 Carnegie Mellon, University, United States of America


A central theme in western philosophy was to find formal methods that can reliably discover empirical relationships and their explanations from data assembled from experience. As a philosophical project, that ambition was abandoned in the 20th century and generally dismissed as impossible. It was replaced in philosophy by neo-Kantian efforts at reconstruction and justification, and in professional statistics by the more limited ambition to estimate a small number of parameters in pre-specified hypotheses. The influx of “big data” from climate science, neuropsychology, biology, astronomy and elsewhere implicitly called for a revival of the grander philosophical ambition. Search algorithms are meeting that call, but they pose a problem: how are their accuracies to be assessed in domains where experimentation is limited or impossible? Increasingly, the answer is through simulation of data from models of the kind of process in the domain. In some cases, these innovations require rethinking how the accuracy and informativeness of inference methods can be assessed. Focusing on causal inference, we give an example from neuroscience, but to show that the model/simulation strategy is not confined to causal inference, we also consider two classification problems from astrophysics: identifying exoplanets and identifying dark matter concentrations.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Bijterbosh, Janine, Stephen Smith and Christian Beckmann, Introduction to Resting State fMRI Functional Connectivity. Oxford: Oxford University Press, 2017.

  • Carnap, Rudolf. “The aim of inductive logic.” In Studies in Logic and the Foundations of Mathematics. Amsterdam: Elsevier, 303-318, 1966.

  • Chu, Tianjaio, Clark Glymour, Richard Scheines, and Peter Spirtes. “A statistical problem for inference to regulatory structure from associations of gene expression measurements with microarrays.” Bioinformatics, 19, 1147-1152, 2003.

  • Fisher, Ronald. A. Statistical methods for research workers. Edinburgh: Oliver and Boyd, 1925.

  • Fisher, Ronald. A. The design of experiments: New York: Macmillan, 1935.

  • Jeffrey, Richard. Carl G. Hempel: Selected Philosophical Essays. New York: Cambridge University Press, 2000.

  • Korb, Kevin B. “Introduction: Machine learning as philosophy of science.” Minds and Machines 14, 433-440, 2004.

  • Lee, B. L., H. K. C. Yee, G. Mallén-Ornelas and S. Seager. “Scientific Frontiers in Research on Extrasolar Planets,” ASP Conference Series, Vol 294, Edited by Drake Deming and Sara Seager, 413-418, 2003.

  • Peirce, Charles. S., & Joseph Jastrow. “On small differences in sensation.” Memoirs of the National Academy of Sciences, 3, 73-83, 1884.

  • Penny, William. D., Karl Friston, Joseph Ashburner, Stefan Kiebel and Thomas E. Nichols (Eds.). Statistical parametric mapping: the analysis of functional brain images. Amsterdam: Elsevier, 2006.

  • Ramsey, Joseph, Madelyn Rose Glymour, Ruben Sanchez-Romero, and Clark Glymour. “A million variables and more: the Fast Greedy Equivalence Search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images.” International journal of data science and analytics, 3, 121-129, 2017.

  • Ravanbakhsh, Siamak, Francois Lanusse, Rachel Mandelbaum, Jeff. G. Schneider and Barnabas Poczos. “Enabling Dark Energy Science with Deep Generative Models of Galaxy Images.” In AAAI 2017, 1488-1494, 2017.

  • Rawls, John. “Outline of a decision procedure for ethics.” The Philosophical Review, 60,177-197, 1951.

  • Reichenbach, Hans. Experience and prediction: An analysis of the foundations and the structure of knowledge. Berkeley: University of California Press, 1938.

  • Rubin, Vera and W. Kent Ford. “Rotation of the Andromeda Nebula from a Spectroscopic Survey of Emission Regions.” The Astrophysical Journal. 159, 379, 1970.

  • Sanchez-Romero, Ruben, Joseph D. Ramsey, Kun Zhang, Madelyn Rose Glymour, Biwei Huang, and Clark Glymour. “Estimating Feedforward and Feedback Effective Connections from FMRI Time Series: Assessments of Statistical Methods.” Network Neuroscience, (in press).

  • Spirtes, Peter, and Jiji Zhang. “A uniformly consistent estimator of causal effects under the k-triangle-faithfulness assumption.” Statistical Science, 662-678, 2014.

  • Sprites, Peter, Clark Glymour and Richard Scheines. Causation, Prediction and Search. Springer Lecture Notes in Statistics. New York: Springer, 1993.

  • Thagard, Paul. Computational Philosophy of Science. Cambridge, MA.: MIT Press, 1993.

  • Vazquez, Alberto, M. Murphy, and S. Kim. “Neuronal and physiological correlation to hemodynamic resting-state fluctuations in health and disease.” Brain connectivity 4.9; 727-740, 2014.

  • Walker, Gilbert. “Correlation in seasonal variations in climate (Introduction).” Memoirs of the India Meteorological Department 20(6).

  • Weinberg, Stephen. Dreams of a final theory. New York: Vintage, 1994.

  • Wimberly, Frank, David Danks, Clark Glymour and Tianjaio Chu. “Problems for Structure Learning Aggregation and Computational Complexity.” In Machine Learning: Concepts, Methodologies, Tools and Applications. Hershey, PA: IGI Global, 1699-1720, 2012.

  • Zwicky, Fritz. “Die Rotverschiebung von extragalaktischen Nebeln.” Helvetica Physica Acta,” 6, , 110-127, 1933.

  • Zwicky, Fritz. “On the Masses of Nebulae and of Clusters of Nebulae,” Astrophysical Journal, 86: 217, 1937.


Journal + Issues

Open Philosophy is an international Open Access, peer-reviewed academic journal covering all areas of philosophy. The objective of Open Philosophy is to foster free exchange of ideas and provide an appropriate platform for presenting, discussing and disseminating new concepts, current trends, theoretical developments and research findings related to the broadest philosophical spectrum.