Initiative to Improve Education in the Principles of Rigorous Research

The NINDS Office of Research Quality has released two companion Funding Opportunity Announcements (FOAs) to solicit applications for developing an educational resource to improve training in the principles of rigorous research: Creating an Educational Nexus for Training in Experimental Rigor (CENTER) (UC2 Clinical Trial Not Allowed) and Materials to Enhance Training in Experimental Rigor (METER) (UE5 Clinical Trial Not Allowed). Applications for CENTER and the first round of METER are due October 21, 2021. Please see the information below to learn more.

 

Image
PowerPoint slide with principles of rigorous biomedical research organized by category or theme. For example, blinding and randomization are principles of rigorous research related to reducing the effects of cognitive biases. Additional examples can be found at http://www.ninds.nih.gov/Funding/grant_policy.

NINDS Director's Message

NINDS Director Walter Koroshetz recently wrote a Director's Message entitled "Good Science is Rigorous Science" about this initiative. In it, Dr. Koroshetz describes the initiative and states that "[e]xperiments designed, conducted, and analyzed without sufficient rigor are wasteful of resources and the scientist’s time" and that "incorporation of a scientific rigor curriculum into a lab’s culture will help maintain its research at a high level of quality." See the Director's Message for more information.

 

Video About the Initiative

This video contains an overview of the initiative as well as information about the two funding opportunities. You may also view this video directly from the NINDS Youtube channel.

 

Informational Webinar

NINDS hosted an informational webinar about these funding opportunities on August 17, 2021. It was an opportunity to learn briefly about the initiative and ask questions of NINDS staff. You may watch the recording below or directly on the NIH Videocast website.

 

Frequently Asked Questions (FAQs)

 

General Information

What is the purpose of this initiative?

Many biomedical publications do not transparently report complete methodologies, analyses, and measures taken at various stages of the research project to ensure rigor and minimize risks of bias (see https://elifesciences.org/articles/55915 for more information). This lack of transparency in reporting, and potential lack of practice, may contribute to difficulties in building upon research findings and has been partially attributed to incomplete training in the principles of rigorous research. These two Funding Opportunity Announcements (FOAs) together aim to facilitate teaching of fundamental principles of rigorous biomedical research by developing a new online educational resource for use by a broad range of researchers in an array of learning environments. Creating an Educational Nexus for Training in Experimental Rigor (CENTER) (UC2 Clinical Trial Not Allowed) will support a Center to build the online educational resource for training in experimental rigor. Materials to Enhance Training in Experimental Rigor (METER) (UE5 Clinical Trial Not Allowed) will support subject matter experts to provide educational materials to the Center in order to integrate them into the online educational resource for training in experimental rigor.

What are “principles of rigorous biomedical research”?

Principles of rigorous biomedical research are cross-cutting concepts, processes, and practices that promote rigorous, transparent, and robust scientific experiments. NIH defines scientific rigor as “the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. NIH expects full transparency in proposing and reporting experimental details so that reviewers may assess the proposed research and others may reproduce and extend the findings.” A list of example practices that fit the goals of this initiative can be found in the image at the beginning of this webpage and in Section IV.2 Research Strategy/Research Education Program Plan of the METER FOA. Additional examples of high-priority principles of rigorous research for NINDS can be found here.

What is the audience for this new educational resource? In what environment(s) could it be used?

Biomedical researchers across various disciplines and career stages should be able to use the educational units easily and effectively, including undergraduate, graduate, postdoctoral, and independent researchers. The online resource should be adaptable to different educational settings, including utilization as part of lectures, independently performed problem-based coursework, and individual pursuit of practical advice while in the laboratory. A key feature of the resource will be its customizability by users to create individualized curricula containing a subset of educational units and to select differentiated levels of information detail and depth based on learning goals. Individual users should be able to specify the exact units, modes of learning, and individual components with which they would like to engage. Additionally, educators should be able to compile individual educational units into customized curricula, choose which subsets of content or tools they would like to use in their classrooms, and personalize scientific examples to increase relevance to their students.


Creating an Educational Nexus for Training in Experimental Rigor (CENTER) (UC2 Clinical Trial Not Allowed):

Overview

Where can I find the Funding Opportunity Announcement (FOA)?

The FOA and information about applying can be found here: Creating an Educational Nexus for Training in Experimental Rigor (CENTER) (UC2 Clinical Trial Not Allowed).

What is the purpose of this FOA? What activities are expected under an award?

The purpose of this FOA is to support a center for Creating an Educational Nexus for Training in Experimental Rigor (CENTER), which will build, evaluate, and disseminate a user-friendly, harmonized, and openly accessible educational resource to promote awareness, understanding, and utilization of the principles of rigorous biomedical research. CENTER will provide comprehensive administrative oversight, expertise required to establish and maintain the educational platform, and infrastructure to create cutting-edge digital educational components (e.g. videos, interactive activities, relevant scientific examples). These components will be adapted from content provided by subject matter experts supported by the companion Materials to Enhance Training in Experimental Rigor (METER) UE5 FOA. CENTER will assemble and manage a team of platform developers, specialists in creating and evaluating engaging digital content, and experts in pedagogy and educational techniques in order to carry out these CENTER activities. CENTER will create, modify, and maintain the educational resource’s digital platform and coordinate collaborations between the external subject matter experts providing the scholarly content, champions of rigor in the scientific community, and CENTER’s technical and educational experts ensuring high-quality educational delivery and impact.

Some of the responsibilities and activities appear to overlap between CENTER and METER. Where is the line between them and who leads each activity?

CENTER will provide overall leadership to the project. This requires project management of educational resource development, creation of the free online platform that contains multiple educational units on principles of rigorous biomedical research, coordination of appropriate expertise for each component of the resource, and collaboration with a range of stakeholders. This includes coordinating Steering Committee and External Advisory Committee meetings, leading resource evaluation and dissemination efforts, and hosting the annual meetings starting at the beginning of the second budget year. METER awardees will serve as the primary subject matter experts for one or more educational unit(s) focused on individual principles of rigorous research and will provide scholarly material to CENTER for conversion into digital educational units. CENTER is responsible for harmonization across the platform, including the educational units, and for producing multi-modal, effective, and engaging content delivery. CENTER will provide educational expertise to guide METER awardees during development of their materials to ensure both harmonization across the resource and optimal educational effectiveness. METER awardees will ensure accuracy and efficacy of the digital learning materials produced by CENTER and recommend modifications as necessary. After the educational units are refined enough to begin formal evaluation, METER awardees will participate in evaluation and dissemination efforts organized by CENTER. All of these activities, however, will be performed under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff. To use a loose film analogy, during resource development CENTER will function like a producer and director while METER awardees will function like script writers. Collaboration between each of these entities is vital for the project to succeed.

How will the variety of ideas and approaches proposed between CENTER and multiple METER awards be harmonized and integrated?

CENTER is responsible for harmonization across the platform, including the educational units, which ensures functionality, seamless navigation among related units, effective user experiences, and uniform appearance and organization. CENTER will provide overall project leadership, governance, and oversight, including through effective conflict resolution, and CENTER applicants must include a Collaboration Plan in their application describing how they will forge successful collaborations with METER awardees in order to create the educational units. METER awardees, however, are encouraged to contribute innovative ideas for consideration by CENTER and the Steering Committee. This process of harmonizing and integrating ideas will be under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff.

How will the Steering Committee and External Advisory Committee work for this initiative? Who will participate and how will members be chosen?

The Steering Committee will meet regularly to facilitate effective collaboration among CENTER and METER awardees, assist in annual meeting planning, help set the scope of the educational resource, provide guidance on CENTER governance and implementation, and ensure the production, evaluation, and dissemination of a harmonized and high-quality educational resource. The Steering Committee should include at least one individual each from CENTER, NINDS staff, and the METER awards. The External Advisory Committee will meet less often and will provide the Steering Committee guidance and feedback on activities, accomplishments, and areas for improvement. The External Advisory Committee should include individuals with relevant scientific, education, evaluation, and technical expertise who are not funded by CENTER or METER awards. Membership, frequency, and governance of these committees will be negotiated with NINDS staff after funding decisions have been made.

Application Logistics

Where and how do I apply?

See the NIH Application Guide for instructions on how to apply for funding.

When are applications due?

The due date for applications is October 21, 2021. There is only one receipt date for this FOA.

Are CENTER and METER applications due at the same time?

The single receipt date for CENTER and the first receipt date for METER are the same: October 21, 2021. METER, however, has two additional receipt dates in October 2022 and October 2023.

How many CENTER applications will be awarded?

NINDS expects to award 1 application. Making an award is contingent upon NIH appropriations and the submission of meritorious applications.

What is the timeline for CENTER and METER given METER’s multiple receipt dates?

CENTER is expected to be a 5-year award, and the three annual receipt dates for METER will solicit applications with award periods of up to 3 years each. For this reason, the start dates of METER awards will be staggered over the first couple years of CENTER, but they will all end by the completion of CENTER’s 5-year award period.

Can an applicant apply for both CENTER and METER?

No. Individuals may not serve as Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) or senior/key personnel on an application submitted to one FOA and an application submitted to the companion FOA. Institutions may submit applications to both FOAs as long as the applications are distinct and the PD(s)/PI(s) and senior/key personnel do not overlap.

Can an applicant apply for more than one award?

No. Individuals may not serve as PD(s)/PI(s) or senior/key personnel on an application submitted to this FOA and an application submitted to the companion FOA. Institutions may submit multiple applications for this FOA as long as the applications are distinct and the PD(s)/PI(s) and senior/key personnel do not overlap.

Who is eligible to apply?

See the FOA for eligibility criteria. Note that key personnel on applications for this FOA cannot also be key personnel on applications for the companion FOA.

Are foreign (non-U.S.) applications allowed?

Foreign institutions and non-domestic components of U.S. organizations are not allowed while foreign components are allowed. See the FOA for eligibility criteria.

Who can I ask about whether my application idea fits the FOA?

You may direct any questions about scientific/research/programmatic content of the initiative here:

Devon Crawford
National Institute of Neurological Disorders and Stroke (NINDS)
Telephone: 301-496-1917
Email: devon.crawford@nih.gov

Who can I ask about grant policies, such as allowable costs?

You may direct any questions about grant policy here:

Chief Grants Management Officer
National Institute of Neurological Disorders and Stroke (NINDS)
Email: ChiefGrantsManagementOfficer@ninds.nih.gov

What are the indirect cost (F&A) rates for this funding opportunity?

The F&A rates for CENTER are reimbursed at the normal negotiated rates that are applicable to other types of NIH grants.

Will approval for use of human subjects be required for resource evaluation activities?

CENTER is a Clinical Trial Not Allowed funding opportunity, so any human subjects activities should not meet the NIH definition of a clinical trial. Some education evaluation activities may be exempt from federal regulations for the protection of human subjects research, but check with your Institutional Review Board (IRB) to determine if approval is required.

If my application is not funded, can I prepare a resubmission?

No. There is only one receipt date for this FOA

How do cooperative agreements differ from typical NIH grants?

According to NIH, a cooperative agreement “is distinguished from a grant in that it provides for substantial involvement between the Federal awarding agency or pass-through entity and the non-Federal entity in carrying out the activity contemplated by the Federal award.” In other words, “under a cooperative agreement, NIH expects to be substantially involved in carrying out the project.” In addition to normal stewardship of grant awards, NIH staff will provide advice and guidance to awardees to facilitate activities and achievement of goals, disseminate information about progress, and serve as a source of information and a point of contact with the NINDS. Applicants are responsible for setting project milestones with specific timelines and criteria for evaluation, which will be approved by NINDS and evaluated annually.

Where can I find the Cooperative Agreement Terms and Conditions?

See the “Cooperative Agreement Terms and Conditions of Award” under Section VI Award Administration Information in the FOA.

Application Content

What is the maximum project period that can be proposed?

The maximum project period is 5 years.

What format should the application follow?

Follow the instructions in the SF424 R&R Application Guide as modified by the instructions in the FOA. Please note that there are additional requirements in the following sections:

  • Page Limitations
  • SF424(R&R) Other Project Information Component
  • SF424(R&R) Senior/Key Person Profile Expanded
  • R&R Budget
  • PHS 398 Research Plan (Specific Aims & Research Strategy)
  • Resource Sharing Plans

Who, if anyone, besides the Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) should be listed as “key personnel?”

Key personnel include any other individuals who contribute to the development or execution of a project in a substantive, measurable way, whether or not they receive salaries or compensation under the grant. Typically, these individuals have doctoral or other professional degrees, although individuals without such degrees may be considered senior/key personnel if they have extensive expertise in relevant disciplines and their involvement meets this definition. Consultants may be considered senior/key personnel if they meet this definition. Senior/key personnel must devote measurable effort to the project whether or not salaries or compensation are requested. "Zero percent" effort or "as needed" are not acceptable levels of involvement for those designated as senior/key personnel.  See https://grants.nih.gov/grants/policy/senior_key_personnel_faqs.htm#1658 for additional information on the use of key personnel in your application.

What areas of expertise are required in the application? Are teams allowed?

Teams are encouraged. The required and strongly advised areas of expertise for Program Directors/Principal Investigators and senior/key personnel are listed under “SF424(R&R) Senior/Key Person Profile Expanded” in the FOA. Also note that a one-page attachment must be included in the “SF424(R&R) Other Project Information Component” section that describes how personnel meet the expertise requirements.

What is the maximum budget that can be proposed?

Application budgets are not prescribed but need to reflect the actual needs of the proposed project.

What should I include in my budget?

Projects under this FOA have a maximum project period of 5 years. The requested budget should include information for anticipated costs related to accomplishing the activities proposed in the application. Budgets are not limited but need to reflect the actual needs of the proposed project. Specific components that should be included in the budget can be found under “R&R Budget” in the FOA.

Will carryover of an unobligated balance from one budget period to another be permitted?

Automatic carryover of unobligated balances does not apply for cooperative agreements like this FOA. Carryover of unobligated balances requires NINDS prior approval unless otherwise noted in the Notice of Award.

How should CENTER be organized? Should it be led by a single organization with expertise in all required areas or can it be a new collaboration between experts in different fields?

Applicants are responsible for proposing CENTER organization. As described under “CENTER Administration” in Section IV.2 of the FOA, applicants need to describe the composition of the project team, how their expertise is complementary and synergistic to meet the goals of CENTER, and how diversity across multiple dimensions will bring unique and varied perspectives to the team. Keep in mind the required and strongly advised areas of expertise for Program Directors/Principal Investigators and senior/key personnel listed under “SF424(R&R) Senior/Key Person Profile Expanded.”

What should be included in the Collaboration Plan?

As described under “CENTER Administration” in Section IV.2 in the FOA, applicants should include a Collaboration Plan that describes how they plan to forge successful collaborations with externally funded subject matter experts (METER awardees) in order to create the educational units. The Plan should describe how CENTER will foster fruitful collaborations with a diverse range of potential users of the resource and other rigor champions in the broader scientific community to ensure successful evaluation, dissemination, and adoption of the educational resource as well as advance the quality of science more broadly (see the NINDS Rigor Champions and Resources website and https://elifesciences.org/articles/55915 for more information on rigor champions). Include plans for organizing and hosting the annual meeting in the Collaboration Plan.

What should be included in plans for the annual meeting?

Beginning one year after initial award (near the beginning of the year 2 budget period), and annually thereafter, CENTER must organize and host an annual meeting of CENTER personnel, METER awardees, and other relevant stakeholders to enhance communication and collaboration among awardees and the scientific community with the ultimate goal of ensuring successful development, evaluation, and dissemination of the educational resource. These additional stakeholders may include technology developers, education professionals, subject matter experts, potential resource users, rigor champions, resource evaluators, and other experts. In addition to logistical needs, the budget for this meeting should include travel costs for up to 20 invited speakers who are not already supported by CENTER or METER. CENTER and METER awardees will include travel costs in their individual budgets to attend the annual meeting. The agenda for each meeting will depend on the anticipated needs of the initiative and will be developed in conjunction with the Steering Committee.

What is an educational unit?

For the purpose of this initiative, an educational unit is defined as an assortment of educational materials, lessons, and activities that collectively address a single principle of rigorous research. Each educational unit will be independently incorporated into the online resource by CENTER. It is estimated that one educational unit, at its maximal breadth and depth of material, will provide the equivalent of one week of instructional material if used as part of a traditional undergraduate or graduate course, although real-world users will also be able to select short, individual components of the unit with which to interact (e.g. a single 5-minute video or set of step-by-step instructions). Each unit should contain effective and engaging educational components, including measurable and attainable learning objectives, a broad and deep overview of the principle of rigorous research, guidance on why and how to apply the principle, complex and realistic neuroscience-relevant scientific examples, interactive components, effective multi-media (e.g. text, videos, graphics, animations), instructional supports for diverse learners, and additional resources.

How many educational units/principles of rigorous research will the resource be expected to host?

CENTER is expected to incorporate 30-50 total educational units into the online resource by the end of the 5-year project period.

Are there similar resources already available in the scientific community after which CENTER applicants can model the structure of the resource?

Numerous training materials are available online (including those listed on the NINDS website), but there is a lack of unified, comprehensive, freely available, and community-endorsed educational resources on the principles of rigorous biomedical research, especially on the scale described in this initiative. Prior efforts to improve awareness, understanding, and utilization of the principles of rigorous biomedical research have not been completely successful, so new approaches are encouraged. As described under “Innovation” in Section IV.2 of the FOA, applicants should describe the technical and/or conceptual innovation(s) that will be brought to the project.

Can previously developed/already existing educational materials be integrated into the resource by CENTER and METER awardees?

Due to the specific requirements for educational units under this initiative, no existing resource will likely have all the required elements or already be harmonized with other educational units for direct integration into the new resource. METER applicants, however, can propose to modify and build upon already developed materials as long as permissions are obtained or not needed for such modification and usage. Keep in mind that materials are expected to be open source once integrated into the resource, so METER applicants should explain their plans and provide assurance that such modification and usage is allowed and will be consistent with the requirements of the educational resource. Similarly, if CENTER applicants plan to use already existing platforms or digital assets to build the resource and produce the digital components, they should explain this in their application and provide assurance that such usage and/or modification is allowed and consistent with the requirements of the educational resource.

What should be included in the Resource Evaluation Plan?

As described under “Resource Evaluation Plan” in Section IV.2 in the FOA, applicants should include plans and proposed timelines for piloting and assessing the educational resource and individual educational units. Applicants should plan to partner with METER awardees and other stakeholders to test functionality in an iterative fashion, measure educational effectiveness, and update content on an ongoing basis and with input from members of the scientific communities that will potentially use the resource (e.g. graduate students, laboratory scientists, educators, rigor champions, and academic institutions). The resource should be evaluated for usage, the ability to foster user learning, inclusion of diverse and underrepresented individuals, and effectiveness in raising awareness and empowering utilization of the principles of rigorous research. The applicants must specify baseline metrics as well as measures to gauge the short- and long-term success of the resource in achieving its objectives. See the FOA for additional details.

What should be included in the Resource Dissemination Plan and the Resource Sharing Plan?

As described under “Resource Dissemination Plan” in Section IV.2 in the FOA, applicants should include plans to publicize and freely disseminate the educational resource and relevant associated materials broadly to the research and higher education communities, including diverse groups of individuals. The planned activities should aim to increase the awareness of and adoption of the resource, create buy-in from the scientific community, measure and improve relevance of the resource to the scientific communities it serves, increase understanding of and appreciation for the principles of rigorous biomedical research, and maintain transparency about progress. Also note that under “Resource Sharing Plan” in Section IV.2 of the FOA, applicants must include plans for freely sharing the educational resource, associated data, and open-source components produced from the proposed project. See the FOA for additional details.

How will collaboration with METER awardees be integrated into evaluation and dissemination activities?

CENTER will provide overall leadership to the project under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff. This ensures standardization/harmonization across the educational resource as well as centralization of logistics and planning. CENTER applicants are responsible for proposing a Collaboration Plan, a Resource Evaluation Plan, and a Resource Dissemination Plan that together include how they will forge successful collaborations with METER awardees to evaluate and disseminate the educational resource.

How should CENTER ensure utilization of the resource by the scientific community? Will NIH require training grant programs to use the resource?

In the Resource Dissemination Plan, CENTER applicants are responsible for describing activities to increase the awareness of and adoption of the resource, create buy-in from the scientific community, measure and improve relevance of the resource to the scientific communities it serves, increase understanding of and appreciation for the principles of rigorous biomedical research, and maintain transparency about progress. CENTER applicants should also include in the Collaboration Plan how they will partner with rigor champions in the scientific community who are actively seeking to change the culture to advance the quality of science more broadly. NINDS training programs already “are expected to provide formal educational activities that reinforce the principles of rigorous experimental design to ensure that trainees understand the practices required for robust hypothesis testing and the application of these principles and practices to their individual research.” The proposed educational resource should be able to serve such programs as part of its adaptability to different educational settings.

Can awardees charge the external community for use of the educational resource or for resources expended during evaluation and/or dissemination activities?

No. This initiative strives to reduce accessibility barriers as much as possible, including by making access to the resource and related materials free to the scientific community. The costs for anticipated activities described in the Resource Evaluation Plan and Resource Dissemination Plan can be included in the application budget.

What will happen to the resource at the end of the 5-year funding period? Should CENTER applicants include a plan for sustaining the resource after this time?

Next steps after the end of the 5-year funding period will depend upon the evolving organization and success of the initiative. Applicants are not asked to include a sustainability plan in their application but are also not discouraged from doing so.

What may I include in the Appendix?

Allowable items are described under “Appendix” in Section IV.2 of the FOA.

What should I put in my milestones?

Milestones are clear, quantifiable goals that can be achieved by the applicant over the course of the project period. They should be described briefly in the Research Strategy section of the application as described under “Milestones” in the FOA. Additionally, a timeline describing the major milestones and activities for each funding year of the award must be included in “Other Attachments” under Section IV.2 SF424(R&R) Other Project Information.

Should I submit letters of support?

Yes. For this FOA, an Institutional Commitment Letter of Support is required, describing institutional commitment to support the proposed work (see “Letters of Support” under Section IV.2). You may also include letters of support from collaborators and other institutions/organizations providing significant support for the proposal.

Review

Who can I ask about the review process?

You may direct any questions about the review process here:

Chief, Scientific Review Branch
National Institute of Neurological Disorders and Stroke (NINDS)
Telephone: 301-496-9223
Email: nindsreview.nih.gov@mail.nih.gov

How will applications be reviewed?

Review will be by a special emphasis panel organized by the NINDS Scientific Review Branch. Reviewers will be instructed to evaluate each application according to the review criteria listed in the FOA under Section V.

Why might my application not be reviewed?

Applications that are incomplete will not be reviewed. The following are considered non-responsive or incomplete for this FOA:

  • Applications without an Institutional Commitment Letter of Support.
  • Applications without milestones included.
  • Applications without a one-page attachment with a timeline of major milestones. Applicants need to provide project timelines that include building capacity and infrastructure within the first year of funding so that CENTER is prepared to release the basic platform within the first year of funding, compile the first set of educational units during the second year of funding, and evaluate and disseminate educational units during the third, fourth, and fifth years of funding.
  • Applications without a one-page attachment describing how the senior/key personnel meet the expertise requirements of the FOA. CENTER needs to incorporate expertise in digital platform development, education, and evaluation.
  • Applications without a Collaboration Plan.
  • Applications without a Resource Evaluation Plan.
  • Applications without a Resource Dissemination Plan.
  • Applications without a Resource Sharing Plan.

What is the estimated timeline for application review and award?

Applications will be reviewed by a scientific review group (study section) in March 2022, undergo a second level of review by the National Advisory Neurological Disorders and Stroke (NANDS) Council in May 2022, and be awarded around July 2022.

How is the National Advisory Neurological Disorders and Stroke (NANDS) Council involved in the second level of peer review of these applications?

Applications in response to this FOA will be made available to the NANDS Council in the same way as other applications. Council members will be free to comment on any application and will vote to approve applications for potential funding.

Can I appeal the review of my application?

No. This FOA is a one-time request for applications. Therefore, be sure to inform NINDS of any potential reviewer conflicts in a cover letter included with the application at the time of submission.


Materials to Enhance Training in Experimental Rigor (METER) (UE5 Clinical Trial Not Allowed):

Overview

Where can I find the Funding Opportunity Announcement (FOA)?

The FOA and information about applying can be found here: Materials to Enhance Training in Experimental Rigor (METER) (UE5 Clinical Trial Not Allowed).

What is the purpose of this FOA? What activities are expected under an award?

The purpose of this FOA is to support the compilation and refinement of Materials to Enhance Training in Experimental Rigor (METER), which will comprise the core scholarly content of educational units within the online resource. These materials will enable acquisition of knowledge and skills to recognize and conduct rigorous experimental research. NINDS METER UE5 awardees will serve as the primary subject matter experts for one or more educational unit(s) focused on individual principles of rigorous research and will provide scholarly material to CENTER, which will be supported by a separate FOA to build the final web-based platform and produce the digital elements (e.g. videos, graphics, and interactive components) of the online resource.

Some of the responsibilities and activities appear to overlap between CENTER and METER. Where is the line between them and who leads each activity?

CENTER will provide overall leadership to the project. This requires project management of educational resource development, creation of the free online platform that contains multiple educational units on principles of rigorous biomedical research, coordination of appropriate expertise for each component of the resource, and collaboration with a range of stakeholders. This includes coordinating Steering Committee and External Advisory Committee meetings, leading resource evaluation and dissemination efforts, and hosting the annual meetings starting at the beginning of the second budget year. METER awardees will serve as the primary subject matter experts for one or more educational unit(s) focused on individual principles of rigorous research and will provide scholarly material to CENTER for conversion into digital educational units. CENTER is responsible for harmonization across the platform, including the educational units, and for producing multi-modal, effective, and engaging content delivery. CENTER will provide educational expertise to guide METER awardees during development of their materials to ensure both harmonization across the resource and optimal educational effectiveness. METER awardees will ensure accuracy and efficacy of the digital learning materials produced by CENTER and recommend modifications as necessary. After the educational units are refined enough to begin formal evaluation, METER awardees will participate in evaluation and dissemination efforts organized by CENTER. All of these activities, however, will be performed under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff. To use a loose film analogy, during resource development CENTER will function like a producer and director while METER awardees will function like script writers. Collaboration between each of these entities is vital for the project to succeed.

How will the variety of ideas and approaches proposed between CENTER and multiple METER awards be harmonized and integrated?

CENTER is responsible for harmonization across the platform, including the educational units, which ensures functionality, seamless navigation among related units, effective user experiences, and uniform appearance and organization. CENTER will provide overall project leadership, governance, and oversight, including through effective conflict resolution, and CENTER applicants must include a Collaboration Plan in their application describing how they will forge successful collaborations with METER awardees in order to create the educational units. METER awardees, however, are encouraged to contribute innovative ideas for consideration by CENTER and the Steering Committee. This process of harmonizing and integrating ideas will be under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff.

How will the Steering Committee and External Advisory Committee work for this initiative? Who will participate and how will members be chosen?

The Steering Committee will meet regularly to facilitate effective collaboration among CENTER and METER awardees, assist in annual meeting planning, help set the scope of the educational resource, provide guidance on CENTER governance and implementation, and ensure the production, evaluation, and dissemination of a harmonized and high-quality educational resource. The Steering Committee should include at least one individual each from CENTER, NINDS staff, and the METER awards. The External Advisory Committee will meet less often and will provide the Steering Committee guidance and feedback on activities, accomplishments, and areas for improvement. The External Advisory Committee should include individuals with relevant scientific, education, evaluation, and technical expertise who are not funded by CENTER or METER awards. Membership, frequency, and governance of these committees will be negotiated with NINDS staff after funding decisions have been made.

Application Logistics

Where and how do I apply?

See the NIH Application Guide for instructions on how to apply for funding.

When are applications due?

There are three receipt dates for this FOA. The first due date for applications is October 21, 2021, and additional applications can be submitted (or resubmitted) for the October 11, 2022 or October 10, 2023 receipt dates.

Are CENTER and METER applications due at the same time?

The single receipt date for CENTER and the first receipt date for METER are the same: October 21, 2021. METER, however, has two additional receipt dates in October 2022 and October 2023.

How many METER applications will be awarded?

The number of awards is contingent upon NIH appropriations, the submission of a sufficient number of meritorious applications, and the balance of proposed educational units with NINDS program priorities.

What is the timeline for CENTER and METER given METER’s multiple receipt dates?

CENTER is expected to be a 5-year award, and the three annual receipt dates for METER will solicit applications with award periods of up to 3 years each. For this reason, the start dates of METER awards will be staggered over the first couple years of CENTER, but they will all end by the completion of CENTER’s 5-year award period.

Can an applicant apply for both CENTER and METER?

No. Individuals may not serve as Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) or senior/key personnel on an application submitted to one FOA and an application submitted to the companion FOA. Institutions may submit applications to both FOAs as long as the applications are distinct and the PD(s)/PI(s) and senior/key personnel do not overlap.

Can an applicant apply for more than one award?

Individuals may serve as Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) or senior/key personnel on more than one application submitted to this FOA as long as the applications do not contain overlap. Individuals may not also serve as PD(s)/PI(s) or senior/key personnel on an application submitted to the companion FOA.

Who is eligible to apply?

See the FOA for eligibility criteria. Note that key personnel on applications for this FOA cannot also be key personnel on applications for the companion FOA.

Are foreign (non-U.S.) applications allowed?

Foreign institutions, non-domestic components of U.S. organizations, and foreign components are allowed. See the FOA for eligibility criteria.

Who can I ask about whether my application idea fits the FOA?

You may direct any questions about scientific/research/programmatic content of the initiative here:

Devon Crawford
National Institute of Neurological Disorders and Stroke (NINDS)
Telephone: 301-496-1917
Email: devon.crawford@nih.gov

Who can I ask about grant policies, such as allowable costs?

You may direct any questions about grant policy here:

Chief Grants Management Officer
National Institute of Neurological Disorders and Stroke (NINDS)
Email: ChiefGrantsManagementOfficer@ninds.nih.gov

What are the indirect cost (F&A) rates for this funding opportunity?

The F&A rates for METER are reimbursed at 8% of modified total direct costs (exclusive of tuition and fees, expenditures for equipment and consortium costs in excess of $25,000) rather than on the basis of a negotiated rate agreement.

If my application is not funded, can I prepare a resubmission?

Yes. There are three receipt dates for this FOA: October 21, 2021; October 11, 2022; and October 10, 2023. Unsuccessful applications may be revised and resubmitted for the next receipt date, subject to NIH policies.

How do cooperative agreements differ from typical NIH grants?

According to NIH, a cooperative agreement “is distinguished from a grant in that it provides for substantial involvement between the Federal awarding agency or pass-through entity and the non-Federal entity in carrying out the activity contemplated by the Federal award.” In other words, “under a cooperative agreement, NIH expects to be substantially involved in carrying out the project.” In addition to normal stewardship of grant awards, NIH staff will provide advice and guidance to awardees to facilitate activities and achievement of goals, disseminate information about progress, and serve as a source of information and a point of contact with the NINDS. Applicants are responsible for setting project milestones with specific timelines and criteria for evaluation, which will be approved by NINDS and evaluated annually.

Where can I find the Cooperative Agreement Terms and Conditions?

See the “Cooperative Agreement Terms and Conditions of Award” under Section VI Award Administration Information in the FOA.

Application Content

What is the maximum project period that can be proposed?

The maximum project period is 3 years.

What format should the application follow?

Follow the instructions in the SF424 R&R Application Guide as modified by the instructions in the FOA. Please note that there are additional requirements in the following sections:

  • Page Limitations
  • SF424(R&R) Other Project Information Component
  • SF424(R&R) Senior/Key Person Profile Expanded
  • R&R Budget
  • PHS 398 Research Plan (Research Strategy/Research Education Program Plan)
  • Resource Sharing Plans

Who, if anyone, besides the Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) should be listed as “key personnel?”

Key personnel include any other individuals who contribute to the development or execution of a project in a substantive, measurable way whether or not they receive salaries or compensation under the grant. Typically, these individuals have doctoral or other professional degrees, although individuals without such degrees may be considered senior/key personnel if they have extensive expertise in relevant disciplines and their involvement meets this definition. Consultants may be considered senior/key personnel if they meet this definition. Senior/key personnel must devote measurable effort to the project whether or not salaries or compensation are requested. "Zero percent" effort or "as needed" are not acceptable levels of involvement for those designated as senior/key personnel.  See https://grants.nih.gov/grants/policy/senior_key_personnel_faqs.htm#1658 for additional information on the use of key personnel in your application.

What areas of expertise are required in the application? Are teams allowed?

Teams are encouraged. The required and strongly advised areas of expertise for Program Directors/Principal Investigators and senior/key personnel are listed under “SF424(R&R) Senior/Key Person Profile Expanded” in the FOA. Also note that a one-page attachment must be included in the “SF424(R&R) Other Project Information Component” section that describes how personnel meet the expertise requirements.

What is the maximum budget that can be proposed?

Application budgets may not exceed direct costs of $250,000 over the course of the project period.

What should I include in my budget?

Projects under this FOA have a maximum project period of 3 years, and budgets are limited to a maximum of $250,000 direct costs. The requested budget should include information for anticipated costs related to accomplishing the activities proposed in the application. Specific components that should be included in the budget can be found under “R&R Budget” in the FOA.

Should the budget depend directly on the number of educational units proposed?

Although there is no specific budget cap for each educational unit developed within the $250,000 direct cost limit, applicants should include an estimate of costs to develop educational materials that is commensurate with the number and complexity of educational units that are being proposed. The budget must reflect the actual needs of the proposed project.

Will carryover of an unobligated balance from one budget period to another be permitted?

Automatic carryover of unobligated balances does not apply for cooperative agreements like this FOA. Carryover of unobligated balances requires NINDS prior approval unless otherwise noted in the Notice of Award.

How many educational units can I propose? Do they have to be related?

A typical METER award is expected to propose content for 2-5 educational units to be developed over a 3-year award period, but there is no formal limit on the number that can be proposed. If applicants propose creating materials for more than one educational unit/principle of rigorous research, each unit may be thematically distinct (i.e. they do not need to be encompassed by a single general topic listed in the FOA). CENTER is expected to incorporate 30-50 total educational units into the online resource, so a breadth of topics is expected to be covered by the total body of funded METER applications.

How will METER applicants for the second and third receipt dates know which principles of rigorous biomedical research have already been selected for METER in prior rounds?

There will be multiple avenues to help potential METER applicants determine whether their planned educational units have already been chosen for development under the initiative. For example, CENTER is expected to maintain communication about initiative progress with the scientific community, including through the annual meeting; potential applicants can view already funded awards using the NIH RePORTER tool; and NIH can issue Notices in the NIH Guide or other communications related to the initiative.

How much material should be proposed for each educational unit?

For the purpose of this initiative, an educational unit is defined as an assortment of educational materials, lessons, and activities that collectively address a single principle of rigorous research. Each educational unit will be independently incorporated into the online resource by CENTER. It is estimated that one educational unit, at its maximal breadth and depth of material, will provide the equivalent of one week of instructional material if used as part of a traditional undergraduate or graduate course, although real-world users will also be able to select short, individual components of the unit with which to interact (e.g. a single 5-minute video or set of step-by-step instructions). Applicants should include a proposed set of learning objectives for each educational unit; complex and realistic neuroscience-relevant scientific examples that demonstrate the importance of the principle and how to implement or apply the principle; as well as creative, engaging, and effective educational and narrative ways to approach digital component delivery in each educational unit (e.g. videos, graphics, activities, resources, exercises, and interactive techniques) that could potentially be incorporated in collaboration with CENTER platform developers and education specialists. Applicants also need to address each of the educational unit requirements described under Section IV.2 of the FOA.

Why should scientific examples within each educational unit be related to neuroscience?

Although the principles of rigorous research are expected to be broadly applicable across scientific disciplines, scientific examples are expected to be within NINDS areas of focus in order to ensure relevance to NINDS-funded investigators. Also note that mechanisms will be developed by CENTER for educators to incorporate customized scientific examples into educational units to increase relevance for their students.

Can previously developed/already existing educational materials be integrated into the resource by CENTER and METER awardees?

Due to the specific requirements for educational units under this initiative, no existing resource will likely have all the required elements or already be harmonized with other educational units for direct integration into the new resource. METER applicants, however, can propose to modify and build upon already developed materials as long as permissions are obtained or not needed for such modification and usage. Keep in mind that materials are expected to be open source once integrated into the resource, so METER applicants should explain their plans and provide assurance that such modification and usage is allowed and will be consistent with the requirements of the educational resource. Similarly, if CENTER applicants plan to use already existing platforms or digital assets to build the resource and produce the digital components, they should explain this in their application and provide assurance that such usage and/or modification is allowed and consistent with the requirements of the educational resource.

Since CENTER will be responsible for organizing evaluation and dissemination activities, how will METER awardees be involved?

METER awardees will assist CENTER in the evaluation, modification, and dissemination of the educational resource. Although formal evaluation and dissemination plans are not required because CENTER will provide leadership and coordination of these activities, METER applicants should describe plans to collaborate with CENTER to evaluate the effectiveness of educational units developed under this award, improve and update the content in each unit, and disseminate the finished online educational units at no cost to individuals. Proposed approaches to evaluation and dissemination will be considered and potentially integrated by CENTER under the collaborative guidance of the Steering Committee, which consists of CENTER representation, METER awardees, and NIH staff.

What may I include in the Appendix?

Allowable items are described under “Appendix” in Section IV.2 of the FOA.

What should I put in my milestones?

Milestones are clear, quantifiable goals that can be achieved by the applicant over the course of the project period. They should be described briefly in the Research Strategy/Research Education Program Plan section of the application as described under “Milestones” in the FOA. Additionally, a timeline describing the major milestones and activities for each funding year of the award must be included in “Other Attachments” under Section IV.2 SF424(R&R) Other Project Information.

Should I submit letters of support?

Yes. For this FOA, an Institutional Commitment Letter of Support is required, describing institutional commitment to support the proposed work (see “Letters of Support” under Section IV.2). You may also include letters of support from collaborators and other institutions/organizations providing significant support for the proposal.

Review

Who can I ask about the review process?

You may direct any questions about the review process here:

Chief, Scientific Review Branch
National Institute of Neurological Disorders and Stroke (NINDS)
Telephone: 301-496-9223
Email: nindsreview.nih.gov@mail.nih.gov

How will applications be reviewed?

Review will be by a special emphasis panel organized by the NINDS Scientific Review Branch. Reviewers will be instructed to evaluate each application according to the review criteria listed in the FOA under Section V.

Why might my application not be reviewed?

Applications that are incomplete will not be reviewed. The following are considered non-responsive or incomplete for this FOA:

  • Applications without an Institutional Commitment Letter of Support.
  • Applications without milestones included.
  • Applications without a one-page attachment with a timeline of major milestones. Project timelines must include producing draft material within the first year of funding and participating in evaluation, improvement, and dissemination of educational units thereafter.
  • Applications without a one-page attachment describing how the senior/key personnel meet the expertise requirements of the FOA. Applications must incorporate expertise related to the foundations and application of one or more principle(s) of rigorous biomedical research.
  • Proposed educational materials that do not address one or more principle(s) of rigorous research.
  • Proposed educational materials that do not address required elements of an educational unit.
  • Applications that do not propose materials for adaptation into a variety of media formats.
  • Applications without a Resource Sharing Plan.

What is the estimated timeline for application review and award?

Applications will be reviewed by a scientific review group (study section) in March, undergo a second level of review by the National Advisory Neurological Disorders and Stroke (NANDS) Council in May, and be awarded around July.

How is the National Advisory Neurological Disorders and Stroke (NANDS) Council involved in the second level of peer review of these applications?

Applications in response to this FOA will be made available to the NANDS Council in the same way as other applications. Council members will be free to comment on any application and will vote to approve applications for potential funding.

Can I appeal the review of my application?

No. Therefore, be sure to inform NINDS of any potential reviewer conflicts in a cover letter included with the application at the time of submission.

Was this page helpful?
Form Approved OMB# 0925-0648 Exp. Date 06/2024