Rigor Champions and Resources

Rigor Champions and Resources

Efforts to Instill the Fundamental Principles of Rigorous Research

Rigorous experimental procedures and transparent reporting of research results are vital to the continued success of the biomedical enterprise at both the preclinical and the clinical levels; therefore, NINDS convened major stakeholders in October 2018 to discuss how best to encourage rigorous biomedical research practices. The attendees discussed potential improvements to current training resources meant to instill the principles of rigorous research in current and future scientists, ideal attributes of a potential new educational resource, and cultural factors needed to ensure the success of such training. Please see the event website for more information about this workshop, including video recordings of the discussion, or the recent publication summarizing the workshop.

Rigor Champions

As described in this publication, enthusiastic individuals ("champions") who want to drive improvements in rigorous research practices, transparent reporting, and comprehensive education may come from all career stages and sectors, including undergraduate students, graduate students, postdoctoral fellows, researchers, educators, institutional leaders, journal editors, scientific societies, private industry, and funders. We encouraged champions to organize themselves into intra- and inter-institutional communities to effect change within and across scientific institutions. These communities can then share resources and best practices, propose changes to current training and research infrastructure, build new tools to support better research practices, and support rigorous research on a daily basis.

If you are interested learning more, you can join this grassroots online workspace or email us at RigorChampions@nih.gov.

Rigor Resources

In order to understand the current landscape of training in the principles of rigorous research, NINDS is gathering a list of public resources that are, or can be made, freely accessible to the scientific community and beyond. We hope that compiling these resources will help identify gaps in training and stimulate discussion about proposed improvements and the building of new resources that facilitate training in transparency and other rigorous research practices. Please peruse the resources compiled thus far below, and contact us at RigorChampions@nih.gov to let us know about other potential resources.

NINDS does not endorse any of these resources and leaves it to the scientific community to judge their quality.

Resources Table

Categories of resources listed in the table include Books and Articles, Guidelines and Protocols, Organizations and Training Programs, Software and Other Digital Resources, and Videos and Courses.

Please use the "Quick Search" bar on the right to filter results based on one of these categories or a key word of your choice.

Resource Name Website URL Description Category
10 Simple Rules for Evaluating Model Credibility https://www.imagwiki.nibib.nih.gov/content/10-simple-rules-conformance-rubric Communication tool by IMAG (Interagency Modeling and Analysis Group) for modelers to organize their model development process Guidelines and Protocols
2 Min Stats https://www.youtube.com/playlist?list=PLC1xDATp-E5FAbcVgMACJKNDDnJWdhUcr Short, animated videos about statistics Videos and Courses
A Call for Transparent Reporting to Optimize the Predictive Value of Preclinical Research https://www.nature.com/articles/nature11556 Article in Nature listing important experimental design elements to report in preclinical research papers Books and Articles
ABRF Committee on Core Rigor and Reproducibility (CCoRRe) Discussion Forum http://list.abrf.org/groups/corre/ Discussion forum by a committee of The Association of Biomolecular Resource Facilities (ABRF) that focuses on building educational resources, sharing critical best practice guidelines, and providing important tools to the core community that will impact rigor, reproducibility, and transparency across the range of science and technology Organizations and Training Programs
Advice for Choosing the Right Reproducibility Tools https://neuronline.sfn.org/scientific-research/advice-for-choosing-the-right-reproducibility-tools Webinar organized by Reproducibility for Everyone (Repro4Everyone) recorded at the Society for Neuroscience (SfN) annual meeting about tools for experimental rigor and transparency Videos and Courses
Advice to Scientists: Incorporating Authentication into Everyday Culture Practice http://iclac.org/wp-content/uploads/Advice-to-Scientists_09-Jan-2014.pdf Cell culture practice guidelines by the International Cell Line Authentication Committee (ICLAC) Guidelines and Protocols
AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 https://amstar.ca/Amstar-2.php A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions Software and Other Digital Resources
Animal Study Registry https://www.animalstudyregistry.org/asr_web/index.action Online registry for scientific studies involving animals conducted around the world Software and Other Digital Resources
Antibodypedia https://www.antibodypedia.com/ Portal of validated antibodies that are scored using the validation principles outlined by the International Working Group for Antibody Validation Software and Other Digital Resources
ARRIVE Guidelines https://www.nc3rs.org.uk/arrive-guidelines Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines are intended to improve the reporting of research using animals Guidelines and Protocols
ASA's Statement on Statistical Significance and P-Values https://amstat.tandfonline.com/doi/pdf/10.1080/00031305.2016.1154108?needAccess=true& Consensus statement by the American Statistical Association on the proper use of p-values in research Guidelines and Protocols
AsPredicted https://aspredicted.org/ Standardized pre-registration that requires what is necessary to separate exploratory from confirmatory analyses Software and Other Digital Resources
BD2K Guide to the Fundamentals of Data Science Series https://bigdatau.ini.usc.edu/data-science-seminars Virtual lecture series by the Big Data to Knowledge (BD2K) Initiative on the data science underlying modern biomedical research Videos and Courses
British Neuroscience Association https://bnacredibility.org.uk/the-bna UK-based organisation representing and promoting neuroscience and neuroscientists, with a program to promote and support credibility in neuroscience Organizations and Training Programs
Capturing the Value of Biomedical Research https://www.sciencedirect.com/science/article/pii/S0092867416302604?via%3Dihub Article in Cell describing a framework for scientific evaluation at the Sidra Medical and Research Center Books and Articles
Cell Line Authentication Training https://www.atcc.org/en/About/News_and_Events/Events/Training/2018/Cell_Line_Authentication_Training.aspx Training in cell line authentication developed by Global Biological Standards Institute (GBSI) Videos and Courses
Center for Biomedical Research Transparency (CBMRT) http://www.cbmrt.org/ Organization that facilitates transparent reporting of biomedical research Organizations and Training Programs
Center for Innovation in Science Education (R3ISE) https://www.jhsph.edu/departments/w-harry-feinstone-department-of-molecular-microbiology-and-immunology/academics-and-degree-programs/R3-PhD-program/R3-Newsletter Johns Hopkins Bloomberg School of Public Health's effort which aims to enable young practitioners to do good science and view their work through the lens of social responsibility Organizations and Training Programs
Center for Open Science (COS) https://cos.io/ Organization that works to increase openness, integrity, and reproducibility of research Organizations and Training Programs
Center for Open Science (COS) Reproducible Research Training https://osf.io/7gqsi/ Materials related to educational videos posted to the COS Youtube channel Videos and Courses
Center for the Improvement of Mentored Experiences in Research (CIMER) https://www.cimerproject.org/#/ Programs including Entering Research with undergraduate and graduate trainee curricular materials and Entering Mentoring curriculum for research mentors Organizations and Training Programs
Common Online Data Analysis Platform (CODAP) https://codap.concord.org/ Open-source software for dynamic data exploration in a pre-college educational setting Software and Other Digital Resources
Complexity Explorer https://www.complexityexplorer.org/ Online courses and educational materials about complexity science produced by the Santa Fe Institute Videos and Courses
CONSORT (CONsolidated Standards of Reporting Trials) 2010 http://www.consort-statement.org/consort-2010 Guidelines developed through collaboration and consensus to improve the reporting of parallel-group randomized controlled trials (RCTs) Guidelines and Protocols
COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) https://www.cosmin.nl/ An initiative to develop methodology and practical tools for selecting the most suitable outcome measurement instrument Software and Other Digital Resources
Cross Validated https://stats.stackexchange.com/tour A question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization Software and Other Digital Resources
DAGitty http://www.dagitty.net/ Browser-based environment for creating, editing, and analyzing causal diagrams for minimizing bias in empirical studies in epidemiology and other disciplines Software and Other Digital Resources
Data Organization by Karl Broman https://kbroman.org/dataorg/ Short primer from Karl Broman on how to organize data in spreadsheets prior to analysis Guidelines and Protocols
DataONE (Data Observation Network for Earth) Best Practices https://www.dataone.org/best-practices Database of recommendations on how to effectively work with data through all stages of the data lifecycle Guidelines and Protocols
DMPTool https://dmptool.org/ Online tool to help build data management plans (DMPs) to facilitate research Software and Other Digital Resources
Educational Resources to Make the Best Out of the Corona Break https://www.bihealth.org/en/research/quest-center/service/educational-resources-to-make-the-best-out-of-the-corona-break/ List of resources compiled by the Berlin Institute of Health to improve scientific practice and professional development Software and Other Digital Resources
Electronic Lab Notebooks at HMS https://datamanagement.hms.harvard.edu/electronic-lab-notebooks Comparison grid hosted by Harvard Medical School showing electronic lab notebook (ELN) features Software and Other Digital Resources
Enhancing Reproducibility Through Rigor and Transparency https://grants.nih.gov/policy/reproducibility/index.htm Guidance for addressing rigor in NIH grant applications and progress reports Guidelines and Protocols
Equator Network Library for Health Research Reporting http://www.equator-network.org/library/ Collection of guidelines and policy documents related to transparent health research reporting Guidelines and Protocols
Estimation for Better Inference in Neuroscience https://www.eneuro.org/content/6/4/ENEURO.0205-19.2019 Article describing the estimation approach to inference, which emphasizes reporting effect sizes with expressions of uncertainty Books and Articles
Estimation Statistics https://www.estimationstats.com/#/ Web application to plot experimental data from an estimation statistics perspective Software and Other Digital Resources
European Quality in Preclinical Data (EQIPD) Consortium https://quality-preclinical-data.eu/ Consortium with the goal of improving preclinical data quality to accelerate the transition to clinical studies Organizations and Training Programs
Evaluation of Research Careers Fully Acknowledging Open Science Practices https://ec.europa.eu/research/openscience/pdf/os_rewards_wgreport_final.pdf Recommendations by the Open Science Working Group on Rewards/Recognition for ensuring researchers in Europe are recognised and rewarded for practising open science, through the Open Science Career Assessment Matrix (OS-CAM) Guidelines and Protocols
Experimental Design Assistant (EDA) https://eda.nc3rs.org.uk/ Free resource developed by the National Centre for the Replacement Refinement & Reduction of Animals in Research (NC3Rs) to help diagram experimental plans Software and Other Digital Resources
Experimental Design: Top Four Strategies for Reproducible Mouse Research https://www.jax.org/news-and-insights/jax-blog/2016/march/experimental-design-top-four-strategies-for-reproducible-mouse-research Key considerations for mouse research according to the Jackson Laboratory Guidelines and Protocols
Fiddle (FIle Drawer Data Liberation Effort) http://s-quest.bihealth.org/fiddle/ Tool to identify alternate publishing routes for well-designed experiments that are difficult to publish in traditional journals (e.g. null or neutral results, datasets) Software and Other Digital Resources
Good Research Practice in Non-Clinical Pharmacology and Biomedicine https://rd.springer.com/book/10.1007/978-3-030-33656-1 Online textbook in the Handbook of Experimental Pharmacology series that describes rigorous and robust research practices Books and Articles
Guidelines for Scientific Record Keeping in the Intramural Research Program at the NIH https://oir.nih.gov/sites/default/files/uploads/sourcebook/documents/ethical_conduct/guidelines-scientific_recordkeeping.pdf General principles underlying record keeping that are necessary to support the conduct of good science and address needs arising from the rapid growth of alternative record keeping methods, the increasing complexity of research data formats, and the influx of scientific trainees with diverse background Guidelines and Protocols
Handbook of Biological Statistics http://www.biostathandbook.com/index.html Online textbook by John H. McDonald originally developed to supplement a biological data analysis course Books and Articles
HHMI BioInteractive: Statistics and Math https://www.hhmi.org/biointeractive/statistics-and-math Statistics and math videos produced by Howard Hughes Medical Institute (HHMI) BioInteractive Videos and Courses
Hong Kong Principles for Assessing Researchers https://osf.io/m9abx/ Principles for improving research through ensuring that researchers are explicitly recognized and rewarded for behavior that leads to trustworthy research Guidelines and Protocols
How Scientists Fool Themselves - and How They Can Stop https://www.nature.com/news/how-scientists-fool-themselves-and-how-they-can-stop-1.18517 Editorial describing cognitive fallacies in research and how to minimize bias Books and Articles
ICPSR Guidelines for Effective Data Management Plans https://www.icpsr.umich.edu/icpsrweb/content/datamanagement/dmp/ Inter-university Consortium for Political and Social Research (ICPSR) data repository at the University of Michigan that offers examples, guidance, and other resources in developing a data management plan Software and Other Digital Resources
Improving the Trustworthiness, Usefulness, and Ethics of Biomedical Research Through an Innovative and Comprehensive Institutional Initiative https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000576 Article in PLOS Biology about an effort at the Berlin Institute of Health to incentivize and support responsible practices in research, funding, and hiring Books and Articles
Improving Your Neuroscience - Society for Neuroscience (SfN) Workshop https://osf.io/5awp4 OSF project website that serves as a hub for disseminating material for the "Improving Your Science" professional development workshop series at the Annual Meeting of the Society for Neuroscience Videos and Courses
INCF TrainingSpace https://training.incf.org/ Online hub hosted by the International Neuroinformatics Coordinating Facility (INCF) to make neuroscience educational materials more accessible to the global neuroscience community Videos and Courses
International Neuroinformatics Coordinating Facility (INCF) https://www.incf.org/ Standards organization that develops, evaluates, and endorses best practices that embrace the principles of Open, FAIR, and Citable neuroscience Organizations and Training Programs
Introduction to the New Statistics https://thenewstatistics.com/itns/ Companion site for the Introduction to the New Statistics textbook Books and Articles
Jamovi https://www.jamovi.org/ Free and open statistical platform Software and Other Digital Resources
Journal of Biological Chemistry: Collecting and Presenting Data http://jbcresources.asbmb.org/collecting-and-presenting-data Guidelines for reporting data and experimental conditions as fully and transparently as possible Guidelines and Protocols
Let's Experiment: A Guide for Scientists Working at the Bench https://courses.ibiology.org/courses/course-v1:iBiology+LE+2019/about Free 6-week course to build a framework for how to design experiments in biological research Videos and Courses
Materials Design Analysis Reporting (MDAR) Checklist for Authors https://science.sciencemag.org/content/sci/suppl/2020/02/26/367.6481.1003.DC1/aaz5103_MDAR_Reproducibility_Checklist.pdf A minimum set of requirements in transparent reporting applicable to studies in the life sciences Guidelines and Protocols
Messy Data Coalition http://messydata.org/ Concord Consortium initiative to bring "messy data" into the classroom to improve pedagogy in data literacy and data fluency Organizations and Training Programs
Methods: Mind the Gap https://prevention.nih.gov/education-training/methods-mind-gap Webinars that explore research design, measurement, intervention, data analysis, and other methods of interest in clinical disease prevention science Videos and Courses
Modern Statistics for Modern Biology http://web.stanford.edu/class/bios221/book/ Interactive online statistics textbook by Susan Holmes and Wolfgang Huber Books and Articles
National Centre for the Replacement Refinement & Reduction of Animals in Research (NC3Rs) https://www.nc3rs.org.uk/ Organization that works to replace, reduce, or refine the use of animals in research and ensure that animal experiments are as robust and reproducible as possible Organizations and Training Programs
Nature.com Collection: Statistics for Biologists https://www.nature.com/collections/qghhqm Nature's collection of statistical issues of which biologists should be aware and practical advice to help improve the rigor of their work Books and Articles
NC3Rs Experimental Design Hub https://www.nc3rs.org.uk/experimental-design Resources compiled by the National Centre for the Replacement Refinement & Reduction of Animals in Research (NC3Rs) for designing and reporting animal research Organizations and Training Programs
NeuroImaging Tools & Resources Collaboratory (NITRC) https://www.nitrc.org/ Web-based resource that shares comprehensive information on neuroinformatics software and data Software and Other Digital Resources
Next Generation Science Standards (NGSS) https://www.nextgenscience.org/ Research-based content standards for the K-12 (primary and secondary school) level, including education in scientific practice Guidelines and Protocols
NIGMS Clearinghouse for Training Modules to Enhance Data Reproducibility https://www.nigms.nih.gov/training/pages/clearinghouse-for-training-modules-to-enhance-data-reproducibility.aspx Products of NIH-funded grants and other relevant training modules developed to improve data reproducibility Videos and Courses
NIH Figshare https://nih.figshare.com/ Pilot program to test the usefulness of formally adding generalist data repositories to the NIH data ecosystem Software and Other Digital Resources
NIH Research Methods Resources https://researchmethodsresources.nih.gov/ NIH website that provides investigators with important research methods resources to help them satisfy new clinical trial requirements Guidelines and Protocols
Open Neuroscience https://openeuroscience.com/ Network of collaborators curating open source projects related to neurosciences Software and Other Digital Resources
Open Science Framework (OSF) https://cos.io/our-products/osf/ A free and open source project management repository that supports researchers across their entire project lifecycle Software and Other Digital Resources
Open Science MOOC https://opensciencemooc.eu/ Online courses to help equip students and researchers with the skills they need to excel in an open research environment Videos and Courses
Patient-Centered Outcomes Research Institute (PCORI) https://www.pcori.org/ Congressionally mandated nonprofit organization that helps people make informed healthcare decisions, and improves healthcare delivery and outcomes, by producing and promoting high-integrity, evidence-based information Organizations and Training Programs
Pragmatic and Group-Randomized Trials in Public Health and Medicine https://prevention.nih.gov/education-training/pragmatic-and-group-randomized-trials-public-health-and-medicine A 7-part online course to help researchers design and analyze group-randomized trials (GRTs) Videos and Courses
PreclinicalTrials https://preclinicaltrials.eu/ International prospective registry of preclinical animal study protocols Software and Other Digital Resources
PREPARE (Planning Research and Experimental Procedures on Animals: Recommendations for Excellence) https://norecopa.no/prepare Guidelines to improve the planning of research and testing using animals Guidelines and Protocols
Principles and Guidelines for Reporting Preclinical Research https://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research NIH Guidelines for transparently reporting research practices in publications Guidelines and Protocols
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) http://prisma-statement.org/ A minimum set of items for reporting in systematic reviews and meta-analyses Guidelines and Protocols
Promoting Awareness and Knowledge to Enhance Scientific Rigor in Neuroscience http://neuronline.sfn.org/TMEDR Webinar series hosted by the Society for Neuroscience (SfN) on topics of experimental rigor and transparency Videos and Courses
PROSPERO https://www.crd.york.ac.uk/prospero/ International prospective registry of systematic reviews Software and Other Digital Resources
QUEST (Quality, Ethics, Open Science, Translation) Center for Transforming Biomedical Research https://www.bihealth.org/en/research/quest-center/mission-approaches/0 Center at the Berlin Institute of Health (BIH) in Germany that strives to increase the value of biomedical research at BIH and beyond Organizations and Training Programs
Register of Misidentified Cell Lines https://iclac.org/databases/cross-contaminations/ Registry maintained by the International Cell Line Authentication Committee (ICLAC) of cell lines that are known to be misidentified through cross-contamination or other mechanisms Software and Other Digital Resources
Registry of Research Data Repositories (re3data) https://www.re3data.org/ Global registry of research data repositories that covers research data repositories from different academic disciplines Software and Other Digital Resources
ReproducibiliTea https://reproducibilitea.org/ Grassroots initiative that helps young researchers create local journal clubs at their universities to discuss papers and ideas about improving science, reproducibility, and the open science movement Organizations and Training Programs
Reproducibility https://www.lib.umn.edu/researchsupport/reproducibility University of Minnesota library webpage on reproducibility resources Organizations and Training Programs
Reproducibility for Everyone https://www.repro4everyone.org/ Organization that compiles resources and teaching workshops to improve the reproducibility of research Organizations and Training Programs
Reproducibility in Science http://ropensci.github.io/reproducibility-guide/ Github guide to enhancing reproducibility in scientific results and writing Guidelines and Protocols
Reproducibility of Research: Get Started http://campusguides.lib.utah.edu/reproducibility/start List of courses and resources for rigor and reproducibility on the University of Utah Spencer S. Eccles Health Sciences Library website Organizations and Training Programs
Research and Data Integrity (ReaDI) Program https://research.columbia.edu/ReaDI-Program Columbia University's compilation of online research integrity and scientific best practice resources Organizations and Training Programs
Research Culture: Setting the Right Tone https://elifesciences.org/articles/55543 Article in eLife that describes efforts to improve research culture and quality at the University of Glasgow Books and Articles
Research on Research Institute (RoRI) http://researchonresearch.org/ Collaboration to investigate how research is funded, practiced and evaluated, and in how research cultures and systems can be made more efficient, open, inclusive and impactful Organizations and Training Programs
Research Topic: Reproducibility and Rigour in Computational Neuroscience https://www.frontiersin.org/research-topics/5964/reproducibility-and-rigour-in-computational-neuroscience#articles Articles by Frontiers journals about reproducibility of computational studies Books and Articles
Resources for Rigor & Transparency https://libguides.brown.edu/c.php?g=553177&p=3800376 List of resources on the Brown University Library website for improving rigor and transparency Organizations and Training Programs
Rigorous Science: a How-To Guide https://mbio.asm.org/content/7/6/e01902-16 Article describing a framework for rigor that includes redundant experimental design, sound statistical analysis, recognition of error, avoidance of logical fallacies, and intellectual honesty Books and Articles
San Francisco Declaration on Research Assessment (DORA) https://sfdora.org/read/ Recommendations to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties Guidelines and Protocols
Science in Transition https://scienceintransition.nl/en Dutch initiative to consider various issues related to research quality and reliability Organizations and Training Programs
Scientific Rigor Champions https://scientificrigor.slack.com Online workspace for champions of rigorous research to discuss pilot initiatives, successes, methods of evaluation, and best practices Software and Other Digital Resources
Shake Up Conferences https://www.nature.com/news/shake-up-conferences-1.22420 Article in Nature describing potential methods for improving rigor and transparency at conferences Books and Articles
Society for the Improvement of Psychological Science https://improvingpsych.org/ Service organization which aims to increase the quality of psychological science Organizations and Training Programs
Stanford Center for Reproducible Science http://reproducibility.stanford.edu/ Neuroimaging center that provides researchers with tools to analyze data, share data, and share workflows in ways that focus on the reproducibility of the results Organizations and Training Programs
STAR Methods Guide for Authors https://www.cell.com/pb-assets/journals/research/cell/methods/Methods%20Guide.pdf Cell Press guidelines for increased clarity and transparency in reporting of methods and analysis to foster improved rigor and reproducibility in research Guidelines and Protocols
Statistical Inference in the 21st Century: A World Beyond p < 0.05 https://www.tandfonline.com/toc/utas20/73/sup1 Special issue of The American Statistician containing dozens of journal articles by statisticians offering advice on making scientific inferences from statistical tests Books and Articles
Statistical Tests, P Values, Confidence Intervals, and Power: a Guide to Misinterpretations https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4877414/ An article in the European Journal of Epidemiology on common misconceptions with p values, confidence intervals, and power calculations Books and Articles
StatisticalHelp https://www.statsdirect.com/help/ Free statistical help provided by StatsDirect Software and Other Digital Resources
Statistics Done Wrong: the Woefully Complete Guide https://www.statisticsdonewrong.com/index.html A guide to the most popular statistical errors and slip-ups committed by scientists every day, in the lab, and in peer-reviewed journals Books and Articles
StatXFinder http://webb.deu.edu.tr/tb/statxfinder/ A decision support tool for appropriate statistical test selection Software and Other Digital Resources
Strain Background and Genetic Drift in Mice https://www.jax.org/jax-mice-and-services/customer-support/technical-support/strain-background-and-genetic-drift Jackson Laboratory FAQs and resources about potential genetic confounds in mouse studies Software and Other Digital Resources
Systematic Review Center for Laboratory Animal Experimentation (SYRCLE) https://www.radboudumc.nl/en/research/departments/health-evidence/systematic-review-center-for-laboratory-animal-experimentation Center that develops, applies, and disseminates methodology and resources for responsible and rigorous systematic reviews of animal studies Organizations and Training Programs
Ten Simple Rules for Reproducible Computational Research https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285 Article describing guidelines for making computational research more accessible Books and Articles
The Many Faces of Reproducibility https://reproducibility.umn.edu/L Project supported by the College of Liberal Arts at the University of Minnesota involving an interdisciplinary exploration of different facets of reproducibility across the social, behavioral, and natural sciences Organizations and Training Programs
The National Academies Press: Reproducibility Collection https://www.nap.edu/collection/89/reproducibility Reports by the National Academies of Sciences, Engineering, and Medicine that discuss the importance and challenges of fostering reproducibility in science Guidelines and Protocols
UCLA Institute for Digital Research and Education https://stats.idre.ucla.edu/other/mult-pkg/seminars/ Seminars and course notes by the UCLA Statistical Consulting Group Videos and Courses
UCSF Introduction to Biostatistics http://davidquigley.com/talks/2015/biostatistics Biostatistics course materials by David Quigley for designing studies, performing essential analyses, and consuming statistical data Videos and Courses
UK Reproducibility Network http://www.dcn.ed.ac.uk/camarades/ukrn/ Peer-led consortium that investigates the factors that contribute to robust research, provides training and disseminates best practice, and works with stakeholders to ensure coordination of efforts across the sector Organizations and Training Programs
Utrecht University Open Science Programme https://www.uu.nl/en/research/open-science A programme that aims to stimulate and facilitate researchers to put open science into practice Organizations and Training Programs
Whole Tale https://wholetale.org/ Computational tools for telling the whole tale of the research project Software and Other Digital Resources