Yesterday, I started this year’s version of my course, Scientific Communication, Collaboration and Design in Quantitative Research Using R. In the course, we use R and related software to visualize data, collaborate in complex data analysis projects, manage (medium-sized) data sets, and express scientific ideas as visualizations and mathematical formulas. A lot of the course is dedicated to ways we can improve our reproducible science skills.
To get started, I presented two sad stories about scientific projects being held up by shortcuts that make it harder to reproduce results. Such shortcuts could be working with raw data in Excel sheets, not scripting analyses, or not maintaining an environment with fixed package versions. The sad part of the stories is that, to finalize the projects, a lot of effort was required to offset the costs of bad habits at the start.
Roger Peng (Peng 2011) has a nice illustration of the replication/reproducibility continuum; it suggests that as more components that enable reproducibility are added to a project, it gets closer to the minimum standard of being fully reproducible. However, my experience is that we could understand the relationship between effort and reproducibility as an S-shaped curve (Figure 1). Much effort is required in the beginning to go from zero to, let’s say, 20% on the “reproducibility scale”. Then we reach a zone where we gain a lot, with little extra effort. However, really pushing a project all the way to 100% again requires considerable effort.
I hope the takeaway from the course introduction was that we should aim to do a little better at every opportunity. The aim of the course is to make it easier to do a little better.
References
Peng, R. D. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–27. https://doi.org/10.1126/science.1213847.