Randomized, controlled clinical intervention trials require successful management of large, complex organizations, involving a wide variety of highly specialized expertise. There are often multiple performance sites and special NIH requirements, such as the additional gender and minority requirements applied to phase III clinical trials beyond the usual scientific criteria. For these reasons there are additional review procedures and criteria for clinical trial applications. For example, a fact-finding applicant interview ("reverse site visit") sometimes precedes or takes place at the study section meeting. Procedures for the applicant interview are outlined at the end of this document.
The format of the summary statement and the criteria for the review of clinical trial applications are described below.
ORGANIZATION OF THE SUMMARY STATEMENT AND WRITING ASSIGNMENTS
Note: Applicants will receive the reviewer's verbatim critiques. It is important to revise these to reflect any changes resulting from the discussion of the application.
|Title Headings||Writing Assignee|
|Resume||Chair (with input from all reviewers)|
|Overall Evaluation||assigned reviewers|
|Recommendation||Chair (with input from all reviewers)|
INTRODUCTION: This section describes the proposed research in general terms. It should include a clear and succinct description of the primary hypothesis and outcome measure, the projected sample size, the exact intervention, and trial duration, including duration of follow-up. It should also summarize the justification or rationale for the proposed trial and discuss any background considerations important in the evaluation of the application e.g., grant history, familiarity of the research group in the proposed research area, etc.
RESUME: the chairperson completes the section with significant input from members of the Review Committee. It should reflect the reviewers' overall evaluation of the clinical trial and should address the following points:
The scientific merit, objectives, significance and potential clinical impact of the trial.
The adequacy of the clinical trial organization, including the plan for data management.
The scientific and administrative leadership.
The strengths and weaknesses of the clinical trial.
The reviewers' level of enthusiasm for the proposed trial.
The goals of NIH-supported research are to advance our understanding of biological systems, improve the control of disease, and enhance health. In your written review, you should comment on the following aspects of the application in order to judge the likelihood that the proposed research will have a substantial impact on the pursuit of one or more of these goals. Please address each of these criteria, and consider them in assigning the overall score, weighting them as you feel appropriate for each application. Note that the application does not need to be equally strong in all categories to be judged likely to have a major scientific impact and thus deserve a high priority score. For example, an investigator may propose to carry out important work, that by its nature is not innovative but is essential to move a field forward.
Significance - Does this study address an important problem? If the aims of the application are achieved, how will scientific knowledge be advanced? What will be the effect of these studies on the concepts or methods that drive this field?
In addition, will the trial have a significant impact on the delivery of health care?
Approach - Are the conceptual framework, design, methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the applicant acknowledge potential problem areas and consider alternative tactics?
In evaluating multicenter clinical trials comments regarding the following factors, if applicable, should be included:
Adequacy of preliminary data.
Appropriateness of the inclusion and exclusion criteria.
Power analysis and discussion of Type I and Type II error protection provided with the projected sample size. Patient dropout before and during the study should be considered.
Randomization and blinding.
Proposed data handling procedures and quality controls for record keeping and reporting.
Evidence of ability to recruit the projected number of patients.
Plans for detecting and reporting adverse events (See note below regarding monitoring boards.).
Training of personnel involved in administering examinations, contacting and reporting data, etc.
If there is an extended observation period, are there adequate plans for standardizing other treatments that may be required and may affect the outcome of the trial?
Are the criteria for treatment failure and its management clearly described and are they appropriate?
Is the primary outcome measure defined; does it have clinical relevance? If more than one outcome measure is identified, is it clear which is of primary importance?
Innovation - Does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies? (Note: New experimental design or statistical analysis methods should be encouraged, if they are sound and address methodological problems specific to neurological disease.)
Investigator - Are the investigators appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the principal investigator and other researchers (if any)?
Environment - Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Is there evidence of institutional support?
For multicenter clinical trials reviewers should evaluate, as applicable, the availability and adequacy of necessary major equipment, the suitability of the clinical facilities at the participating centers, and the adequacy of the data entry, biostatistical, and computer facilities.
NOTE REGARDING COMPETING CONTINUATION, CLINICAL TRIAL APPLICATIONS: If the application under review is for a competing continuation of an on-going clinical trial reviewers should take into account the special ethical and scientific issues involved in termination of a clinical trial prior to its having achieved the targeted sample size. If premature termination leads to a lack of results from the study, the human subjects that previously participated may have been put at risk without any benefit in terms of knowledge acquired.
Reviewers may sometimes feel that recent evidence strongly suggests that the applicant's original hypothesis is unlikely to be supported by the study and that the study should not continue. However, unless this evidence is very compelling reviewers should give the benefit of the doubt to the applicant because of the serious implications involved in stopping a partially completed clinical trial.
Reviewers should also consider that if the study is blinded the applicants may not be able to provide any preliminary results regarding efficacy. Thus, reviewers should evaluate primarily whether or not the applicant has demonstrated satisfactory progress in accruing and following the subject population, rather than asking for preliminary evidence of efficacy. Reviewers should bear in mind that each clinical trial is monitored by an independent board that evaluates whether continuation of the trial is justified in terms of interim data analyses and adverse event reports.
OVERALL EVALUATION: In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field, weighting the review criteria, as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major scientific impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.
HUMAN SUBJECTS: Express any comments or concerns about the appropriateness of the applicant's responses to the six required points. Discuss whether the risks to the subjects are reasonable in relation to the anticipated benefits to the subjects and/or in relation to the importance of the knowledge that may reasonably be expected to result from the research. For multicenter clinical trials, reviewers should include an evaluation of the adequacy of the proposed Informed Consent procedures.
GENDER, MINORITY AND CHILDREN SUBJECTS: If human subjects are involved examine whether the minority and gender characteristics of the sample are scientifically acceptable and consistent with the aims of the project, using the categories of "1" to "4" as follows. Also examine whether there is appropriate inclusion of children (individuals under the age of 21). Also determine whether the research is a Phase III clinical trial.
Minority & non-minority
Children & adults
No children included
Minority representation unknown
Representation of children unknown
Evaluate acceptability as "A" (acceptable ) or "U" (unacceptable). If you rate the sample as "U", consider this feature a weakness or deficiency in the design of the project and reflect it in the overall score. NOTE: To the degree that acceptability or unacceptability affects the investigator's approach to the proposed research, such comments should appear under "Approach" in the five major review criteria above, and should be factored into the score as appropriate.
In evaluating HUMAN SUBJECTS, GENDER, MINORITY AND CHILDREN SUBJECTS please refer to the enclosed information document for detailed guidance.
Note to Reviewers Regarding Monitoring Boards: If the application is funded, the NINDS will establish an independent monitoring board that will guide development of a method for interim analysis. The reviewers should not expect the applicant to appoint or manage this independent board.
BUDGET: Reviewers may recommend modifications in the budget based on what is judged to be necessary to accomplish the research. Reductions in budgets may not be recommended merely to increase the enthusiasm for the research. Give specific reasons for all budget modifications. If no budget changes are recommended, write "Appropriate and justified as requested". Note any apparent overlap with other active or pending grants to alert NINDS staff to review the total funding situation. This will usually appear as an "Administrative Note", since it does not affect the technical merit review of the application.
FOREIGN: If the applicant organization is foreign, comment on any special talents, resources, populations, or environmental conditions that are not readily available in the United States or that provide augmentation of existing U.S. resources. In addition, indicate whether similar research is being performed in the U.S. and whether there is a need for such additional research. These aspects do not apply to applications from U.S. organizations for projects containing a significant foreign component.
RECOMMENDATION AND SCORING: The review should conclude with one of the following options:
Recommendation for Support: The applications has significant and substantial merit and is considered to fall in the upper half of the distribution. If the application is recommended for support, each reviewer should record a numerical score (priority) reflecting his/her opinion of the scientific merit of the application. Applications are scored from 1.0 to the recommended median score (approximately 2.5) using 0.1 intervals. To provide a context for the scoring the Scientific Review Administrator will provide information concerning the scores that should correspond to the top 5%, 10%, 20%, 33% and 50%. The frame of reference for the scoring should be the reviewer's previous experience with all similar types of applications, not just the applications under review at a particular meeting (E.g., where would this application fall in terms of merit when compared with all such applications I have previously seen? Top 5%? Top 10%? Etc.).
Unscored - If the application is considered to fall in the lower half of the distribution (below the 50th percentile) it should be designated "unscored" (UN). Such applications may have significant merit, but fall below the median. If the reviewers disagree about whether to place an application in the lower half, they are then asked to score the application using the full range of scores from 1.0 to 5.0.
Not Recommended for Further Consideration (NR): This designation is used for applications that the reviewers agree do not have significant and substantial scientific merit. This score should only be used if all reviewers are in agreement.
Reviewers are reminded that policy issues and concerns about matters other than scientific merit are to be conveyed through scientific review administrators' notes in the summary statement, and not through the priority rating.
APPLICANT INTERVIEW PROCEDURES
Some clinical trial applications may receive an applicant interview to allow the applicants to address any questions the reviewers may have about the application. Applicant interviews are conducted as follows:
Initially the Committee, in a "closed executive session", will seek to highlight areas of concern that need to be clarified during the interview of the applicant and members of the clinical trial group.
The applicant interview will proceed with a brief 30 minute presentation by the Principal Investigator and/or selected members of his/her group. This will be followed by a questions and answer session by members of the Committee. The interview should conclude within 3 hours.
Following the interview, the Committee will deliberate in closed session to assess the scientific and technical strengths and weaknesses of the application and prepare assigned sections of the summary statement. Designated reviewers are responsible for making the final revisions in their assigned portion of the summary statement.
If the applicant interview takes place prior to the study section meeting, the reviewers who were present at the applicant interview report their findings and critiques to the study section as a whole, and, following a general discussion, the application is scored.
Last Modified November 24, 2008