Skip to content
BY-NC-ND 4.0 license Open Access Published by De Gruyter Open Access January 18, 2019

The Evaluation of Discovery: Models, Simulation and Search through “Big Data”

Clark Glymour, Joseph D. Ramsey and Kun Zhang
From the journal Open Philosophy

Abstract

A central theme in western philosophy was to find formal methods that can reliably discover empirical relationships and their explanations from data assembled from experience. As a philosophical project, that ambition was abandoned in the 20th century and generally dismissed as impossible. It was replaced in philosophy by neo-Kantian efforts at reconstruction and justification, and in professional statistics by the more limited ambition to estimate a small number of parameters in pre-specified hypotheses. The influx of “big data” from climate science, neuropsychology, biology, astronomy and elsewhere implicitly called for a revival of the grander philosophical ambition. Search algorithms are meeting that call, but they pose a problem: how are their accuracies to be assessed in domains where experimentation is limited or impossible? Increasingly, the answer is through simulation of data from models of the kind of process in the domain. In some cases, these innovations require rethinking how the accuracy and informativeness of inference methods can be assessed. Focusing on causal inference, we give an example from neuroscience, but to show that the model/simulation strategy is not confined to causal inference, we also consider two classification problems from astrophysics: identifying exoplanets and identifying dark matter concentrations.

References

Bijterbosh, Janine, Stephen Smith and Christian Beckmann, Introduction to Resting State fMRI Functional Connectivity. Oxford: Oxford University Press, 2017.Search in Google Scholar

Carnap, Rudolf. “The aim of inductive logic.” In Studies in Logic and the Foundations of Mathematics. Amsterdam: Elsevier, 303-318, 1966.10.1016/S0049-237X(09)70598-1Search in Google Scholar

Chu, Tianjaio, Clark Glymour, Richard Scheines, and Peter Spirtes. “A statistical problem for inference to regulatory structure from associations of gene expression measurements with microarrays.” Bioinformatics, 19, 1147-1152, 2003.10.1093/bioinformatics/btg011Search in Google Scholar

Fisher, Ronald. A. Statistical methods for research workers. Edinburgh: Oliver and Boyd, 1925.Search in Google Scholar

Fisher, Ronald. A. The design of experiments: New York: Macmillan, 1935.Search in Google Scholar

Jeffrey, Richard. Carl G. Hempel: Selected Philosophical Essays. New York: Cambridge University Press, 2000.Search in Google Scholar

Korb, Kevin B. “Introduction: Machine learning as philosophy of science.” Minds and Machines 14, 433-440, 2004.10.1023/B:MIND.0000045986.90956.7fSearch in Google Scholar

Lee, B. L., H. K. C. Yee, G. Mallén-Ornelas and S. Seager. “Scientific Frontiers in Research on Extrasolar Planets,” ASP Conference Series, Vol 294, Edited by Drake Deming and Sara Seager, 413-418, 2003.Search in Google Scholar

Peirce, Charles. S., & Joseph Jastrow. “On small differences in sensation.” Memoirs of the National Academy of Sciences, 3, 73-83, 1884.Search in Google Scholar

Penny, William. D., Karl Friston, Joseph Ashburner, Stefan Kiebel and Thomas E. Nichols (Eds.). Statistical parametric mapping: the analysis of functional brain images. Amsterdam: Elsevier, 2006.Search in Google Scholar

Ramsey, Joseph, Madelyn Rose Glymour, Ruben Sanchez-Romero, and Clark Glymour. “A million variables and more: the Fast Greedy Equivalence Search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images.” International journal of data science and analytics, 3, 121-129, 2017.10.1007/s41060-016-0032-zSearch in Google Scholar

Ravanbakhsh, Siamak, Francois Lanusse, Rachel Mandelbaum, Jeff. G. Schneider and Barnabas Poczos. “Enabling Dark Energy Science with Deep Generative Models of Galaxy Images.” In AAAI 2017, 1488-1494, 2017.10.1609/aaai.v31i1.10755Search in Google Scholar

Rawls, John. “Outline of a decision procedure for ethics.” The Philosophical Review, 60,177-197, 1951.10.2307/2181696Search in Google Scholar

Reichenbach, Hans. Experience and prediction: An analysis of the foundations and the structure of knowledge. Berkeley: University of California Press, 1938.Search in Google Scholar

Rubin, Vera and W. Kent Ford. “Rotation of the Andromeda Nebula from a Spectroscopic Survey of Emission Regions.” The Astrophysical Journal. 159, 379, 1970.Search in Google Scholar

Sanchez-Romero, Ruben, Joseph D. Ramsey, Kun Zhang, Madelyn Rose Glymour, Biwei Huang, and Clark Glymour. “Estimating Feedforward and Feedback Effective Connections from FMRI Time Series: Assessments of Statistical Methods.” Network Neuroscience, (in press).Search in Google Scholar

Spirtes, Peter, and Jiji Zhang. “A uniformly consistent estimator of causal effects under the k-triangle-faithfulness assumption.” Statistical Science, 662-678, 2014.10.1214/13-STS429Search in Google Scholar

Sprites, Peter, Clark Glymour and Richard Scheines. Causation, Prediction and Search. Springer Lecture Notes in Statistics. New York: Springer, 1993.10.1007/978-1-4612-2748-9Search in Google Scholar

Thagard, Paul. Computational Philosophy of Science. Cambridge, MA.: MIT Press, 1993.Search in Google Scholar

Vazquez, Alberto, M. Murphy, and S. Kim. “Neuronal and physiological correlation to hemodynamic resting-state fluctuations in health and disease.” Brain connectivity 4.9; 727-740, 2014.10.1089/brain.2014.0276Search in Google Scholar

Walker, Gilbert. “Correlation in seasonal variations in climate (Introduction).” Memoirs of the India Meteorological Department 20(6).Search in Google Scholar

Weinberg, Stephen. Dreams of a final theory. New York: Vintage, 1994.10.1119/1.17723Search in Google Scholar

Wimberly, Frank, David Danks, Clark Glymour and Tianjaio Chu. “Problems for Structure Learning Aggregation and Computational Complexity.” In Machine Learning: Concepts, Methodologies, Tools and Applications. Hershey, PA: IGI Global, 1699-1720, 2012.Search in Google Scholar

Zwicky, Fritz. “Die Rotverschiebung von extragalaktischen Nebeln.” Helvetica Physica Acta,” 6, , 110-127, 1933.Search in Google Scholar

Zwicky, Fritz. “On the Masses of Nebulae and of Clusters of Nebulae,” Astrophysical Journal, 86: 217, 1937.10.1086/143864Search in Google Scholar

Received: 2018-10-23
Accepted: 2018-11-30
Published Online: 2019-01-18

© by Clark Glymour, et al., published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Scroll Up Arrow