Requesting Input on Educational Resources in the Principles of Rigorous Research

NINDS’s pact with the US citizens is to fund research that is careful, rigorous, and of the highest quality.  Our commitment to rigorous research is especially important as labs plan and implement re-opening during the COVID-19 pandemic and trainees return to the classrooms and benches. It’s an opportune time to jump start the research enterprise with thoughtful planning that can lead to higher value and impactful science. NINDS has announced plans to enhance the quality of the science we fund, and we ask you to give us your input by replying to a Request for Information.

Several studies over the past 15 years have shown widespread problems in the design, conduct, and analysis of experiments, and many publications do not adequately report key information about how experiments were conducted1.  Without such information, it is difficult to interpret the author’s conclusions or replicate the study, which leads to confusion in the scientific field. One contributing factor for these tendencies appears to be inadequate training in, and application of, the principles of rigorous experimental design and analysis. NINDS and a collection of NIH neuroscience institutes have started to tackle this problem by requiring T32 training programs to provide education in experimental design, statistics, quantitative tools, and scientific rigor, but building effective educational tools from scratch comes with its own unique set of challenges.

To help address these challenges, NINDS will develop a new educational platform to facilitate teaching in the principles of rigorous research for the scientific community, as described at a recent NANDS Council Meeting. To ensure that the platform addresses needs effectively, we have issued a Request for Information to collect input, and we want to hear from you!

The idea for such a platform emerged from a workshop on education in the principles of rigorous research held by NINDS in October 2018. As summarized in this eLife publication, workshop participants discussed how educational resources should be freely available, easily discoverable, engaging, modular, adaptable, and upgradeable. These resources should also be beneficial to scientists at all career stages and suitable for both stand-alone use and integration into formal coursework. Although many educational resources already exist (such as those listed on the NINDS Rigor Champions and Resources website), a comprehensive educational platform for the scientific community could facilitate course development and individual learning. We would like feedback from the public, education professionals, and the scientific community on our plans to build such a platform. Please share our request for information broadly to help our efforts.

Additionally, we recognize that no educational resource will be fully successful without a change to the current culture that places inordinate pressure on investigators and their trainees to publish frequently, preferentially in high-profile journals. This will require a coordinated effort from all sectors of the scientific enterprise, including institutional leaders, researchers, trainees, funders, and publishers. To facilitate discussion between and among these groups so that they may brainstorm ideas, share best practices, and take action to change the culture, NINDS has created a Slack collaborative workspace that anyone can join by sending a request to RigorChampions@nih.gov. We hope that this space will become a hub for rigor champions - only by the efforts of such champions can we ensure the success of newly created policies, infrastructure, and resources that aim to improve the quality of science. I invite you to join this community now to share information with colleagues and be a part of the network that can utilize and promote the educational resources that NINDS will work to share and develop.

In the meantime, we want our valuable research to have the maximum benefit by transparently reporting the design, conduct, and analysis of studies. We can label our exploratory experiments clearly so that our hypothesis-testing experiments hold more weight. We can design optimal experiments to answer our research questions, including by setting up valid controls, blinding and randomizing treatments, calculating how many samples we need before starting, selecting inclusion and exclusion criteria that ensure high quality data collection, and choosing analyses that prevent p-hacking.

Good science is rigorous science, and we can work together now during this uncertain time – and going forward – to ensure its success.

Request for Information (RFI) on Developing an Online Educational Resource for Training in the Principles of Rigorous Research: https://grants.nih.gov/grants/guide/notice-files/NOT-NS-20-062.html

1Risk of Bias in Reports of In Vivo Research: A Focus for Improvement, Update of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations, A call for transparent reporting to optimize the predictive value of preclinical research