The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences

Published by University of California Press 2017

The Practice of Reproducible Research

Case Studies and Lessons from the Data-Intensive Sciences

Edited by: Justin Kitzes

https://doi.org/10.1525/9780520967779

Overview
Contents

About this book

The Practice of Reproducible Research presents concrete examples of how researchers in the data-intensive sciences are working to improve the reproducibility of their research projects. In each of the thirty-one case studies in this volume, the author or team describes the workflow that they used to complete a real-world research project. Authors highlight how they utilized particular tools, ideas, and practices to support reproducibility, emphasizing the very practical how, rather than the why or what, of conducting reproducible research.

Part 1 provides an accessible introduction to reproducible research, a basic reproducible research project template, and a synthesis of lessons learned from across the thirty-one case studies. Parts 2 and 3 focus on the case studies themselves. The Practice of Reproducible Research is an invaluable resource for students and researchers who wish to better understand the practice of data-intensive sciences and learn how to make their own research more reproducible.

Author information

KitzesJustin:

Justin Kitzes is Assistant Professor of Biology at the University of Pittsburgh.

Daniel Turek is Assistant Professor of Statistics at Williams College.

Fatma Deniz is Postdoctoral Scholar at the Helen Wills Neuroscience Institute and the International Computer Science Institute, and Data Science Fellow at the University of California, Berkeley.

Topics

Frontmatter Publicly Available Download PDF	i
Contents Publicly Available Download PDF	v
List of Contributors Publicly Available Download PDF	xi
Preface: Nullius in Verba Philip B. Stark Publicly Available Download PDF	xvii
Introduction Justin Kitzes Publicly Available Download PDF	xxi
PART I: PRACTICING REPRODUCIBILITY
Assessing Reproducibility Ariel Rokem, Ben Marwick and Valentina Staneva Requires Authentication Unlicensed Licensed Download PDF	1
The Basic Reproducible Workflow Template Justin Kitzes Requires Authentication Unlicensed Licensed Download PDF	19
Case Studies in Reproducible Research Daniel Turek and Fatma Deniz Requires Authentication Unlicensed Licensed Download PDF	31
Lessons Learned Kathryn Huff Requires Authentication Unlicensed Licensed Download PDF	41
Building toward a Future Where Reproducible, Open Science Is the Norm Karthik Ram and Ben Marwick Requires Authentication Unlicensed Licensed Download PDF	61
Glossary Ariel Rokem and Fernando Chirigati Requires Authentication Unlicensed Licensed Download PDF	71
PART II: HIGH-LEVEL CASE STUDIES
Case Study 1: Processing of Airborne Laser Altimetry Data Using Cloud-Based Python and Relational Database Tools Anthony Arendt, Christian Kienholz, Christopher Larsen, Justin Rich and Evan Burgess Requires Authentication Unlicensed Licensed Download PDF	95
Case Study 2: The Trade-Off between Reproducibility and Privacy in the Use of Social Media Data to Study Political Behavior Pablo Barberá Requires Authentication Unlicensed Licensed Download PDF	103
Case Study 3: A Reproducible R Notebook Using Docker Carl Boettiger Requires Authentication Unlicensed Licensed Download PDF	109
Case Study 4: Estimating the Effect of Soldier Deaths on the Military Labor Supply Garret Christensen Requires Authentication Unlicensed Licensed Download PDF	119
Case Study 5: Turning Simulations of Quantum Many- Body Systems into a Provenance-Rich Publication Jan Gukelberger and Matthias Troyer Requires Authentication Unlicensed Licensed Download PDF	125
Case Study 6: Validating Statistical Methods to Detect Data Fabrication Chris Hartgerink Requires Authentication Unlicensed Licensed Download PDF	131
Case Study 7: Feature Extraction and Data Wrangling for Predictive Models of the Brain in Python Chris Holdgraf Requires Authentication Unlicensed Licensed Download PDF	139
Case Study 8: Using Observational Data and Numerical Modeling to Make Scientific Discoveries in Climate Science David Holland and Denise Holland Requires Authentication Unlicensed Licensed Download PDF	149
Case Study 9: Analyzing Bat Distributions in a Human- Dominated Landscape with Autonomous Acoustic Detectors and Machine Learning Models Justin Kitzes Requires Authentication Unlicensed Licensed Download PDF	155
Case Study 10: An Analysis of Household Location Choice in Major US Metropolitan Areas Using R Andy Krause and Hossein Estiri Requires Authentication Unlicensed Licensed Download PDF	161
Case Study 11: Analyzing Cosponsorship Data to Detect Networking Patterns in Peruvian Legislators José Manuel Magallanes Requires Authentication Unlicensed Licensed Download PDF	169
Case Study 12: Using R and Related Tools for Reproducible Research in Archaeology Ben Marwick Requires Authentication Unlicensed Licensed Download PDF	181
Case Study 13: Achieving Full Replication of Our Own Published CFD Results, with Four Different Codes Olivier Mesnard and Lorena A. Barba Requires Authentication Unlicensed Licensed Download PDF	191
Case Study 14: Reproducible Applied Statistics: Is Tagging of Therapist-Patient Interactions Reliable? K. Jarrod Millman, Kellie Ottoboni, Naomi A. P. Stark and Philip B. Stark Requires Authentication Unlicensed Licensed Download PDF	201
Case Study 15: A Dissection of Computational Methods Used in a Biogeographic Study K. A. S. Mislan Requires Authentication Unlicensed Licensed Download PDF	215
Case Study 16: A Statistical Analysis of Salt and Mortality at the Level of Nations Kellie Ottoboni Requires Authentication Unlicensed Licensed Download PDF	221
Case Study 17: Reproducible Workflows for Understanding Large-Scale Ecological Effects of Climate Change Karthik Ram Requires Authentication Unlicensed Licensed Download PDF	227
Case Study 18: Reproducibility in Human Neuroimaging Research: A Practical Example from the Analysis of Diffusion MRI Ariel Rokem Requires Authentication Unlicensed Licensed Download PDF	233
Case Study 19: Reproducible Computational Science on High-Performance Computers: A View from Neutron Transport Rachel Slaybaugh Requires Authentication Unlicensed Licensed Download PDF	241
Case Study 20: Detection and Classification of Cervical Cells Daniela Ushizima Requires Authentication Unlicensed Licensed Download PDF	247
Case Study 21: Enabling Astronomy Image Processing with Cloud Computing Using Apache Spark Zhao Zhang Requires Authentication Unlicensed Licensed Download PDF	253
PART III: LOW-LEVEL CASE STUDIES
Case Study 22: Software for Analyzing Supernova Light Curve Data for Cosmology Kyle Barbary Requires Authentication Unlicensed Licensed Download PDF	265
Case Study 23: pyMooney: Generating a Database of Two-Tone Mooney Images Fatma Deniz Requires Authentication Unlicensed Licensed Download PDF	271
Case Study 24: Problem-Specific Analysis of Molecular Dynamics Trajectories for Biomolecules Konrad Hinsen Requires Authentication Unlicensed Licensed Download PDF	277
Case Study 25: Developing an Open, Modular Simulation Framework for Nuclear Fuel Cycle Analysis Kathryn Huff Requires Authentication Unlicensed Licensed Download PDF	285
Case Study 26: Producing a Journal Article on Probabilistic Tsunami Hazard Assessment Randall J. LeVeque Requires Authentication Unlicensed Licensed Download PDF	291
Case Study 27: A Reproducible Neuroimaging Workflow Using the Automated Build Tool “Make” Tara Madhyastha, Natalie Koh and Mary K. Askren Requires Authentication Unlicensed Licensed Download PDF	297
Case Study 28: Generation of Uniform Data Products for AmeriFlux and FLUXNET Gilberto Pastorello Requires Authentication Unlicensed Licensed Download PDF	305
Case Study 29: Developing a Reproducible Workflow for Large-Scale Phenotyping Russell Poldrack Requires Authentication Unlicensed Licensed Download PDF	311
Case Study 30: Developing and Testing Stochastic Filtering Methods for Tracking Objects in Videos Valentina Staneva Requires Authentication Unlicensed Licensed Download PDF	317
Case Study 31: Developing, Testing, and Deploying Efficient MCMC Algorithms for Hierarchical Models Using R Daniel Turek Requires Authentication Unlicensed Licensed Download PDF	323
Index Requires Authentication Unlicensed Licensed Download PDF	329