What is “Context of Use” (COU) and why is it critical for establishing the study design?
The Context of Use for a Biomarker is “a concise description of the biomarker’s specified use” which includes two components: (1) the BEST biomarker category (prognostic, diagnostic, pharmacodynamic/response, predictive, safety, monitoring, or susceptibility/risk) and (2) the biomarker’s intended use in drug [or therapeutic] development (FDA Biomarker Qualification Program). Each biomarker qualification effort should identify a single COU.” For the NINDS Translational Biomarker Program, applicants may also propose a context of use for biomarkers intended to be used in updating clinical practice guidelines, such as contributing to updated diagnostic criteria.
A clearly defined Context of Use is essential because it explains how the biomarker is intended to be used; the purpose of the NINDS Biomarker Program is to enable research teams to collect the data needed to evaluate the accuracy and reliability of the biomarker for the use proposed. Therefore, the statistical analysis plan, study populations, and the acceptable variance or error when measuring the biomarker are all dependent on the intended use and associated risks and benefits within that context. Biomarkers are used for making decisions at the level of the individual participant or patient; therefore, it is important to design your study to statistically determine how the value or outcome of the biomarker can be interpreted to guide decision making. Studies that only propose to statistically evaluate group differences do not reflect analyses designed to match the context of use and will be considered a lower programmatic priority.
How does the category of biomarker and Context of Use change the study design?
Each biomarker category has different experimental design expectations. Examples illustrating some differences are listed below.
Diagnostic biomarkers and biomarker signatures:
Studies proposing to validate a diagnostic biomarker/composite biomarker should evaluate the diagnostic accuracy of the biomarker against the accepted standard such as an established clinical outcome assessment battery, postmortem pathology, or against another/other established diagnostic tool(s). If the biomarker’s proposed COU is for differential diagnosis, then the study is expected to include the relevant control groups to evaluate the differential diagnostic accuracy.
Pharmacodynamic/response biomarkers and biomarker signatures:
Studies proposing to validate a pharmacodynamic/response biomarker should be able to test the biomarker using data/samples from patients undergoing the therapeutic/intervention/treatment plan of interest. Pharmacodynamic/response biomarkers should be associated with the mechanism of action or target pathway/system of the therapeutic to demonstrate either direct or indirect target engagement of the therapeutic (e.g., PET ligand binding, changes in the quantity of enzymatic products, gene expression, down-regulation of a molecule in a signaling cascade pathway, change in phosphorylation, acetylation, methylation, electrophysiological response, etc.). If the COU is for making dosing decisions, then study designs should quantitatively evaluate the relationship between the dose and exposure of the therapeutic and the relative response and timing of the response in the biomarker.
Prognostic biomarkers and biomarker signatures:
Studies proposing to validate a prognostic biomarker/composite biomarker should demonstrate the accuracy of the biomarker’s ability to predict the likelihood of a clinical event or outcome(s) within a clinically useful defined period in individuals with the disease or medical condition of interest. When using prognostic models, statistically evaluating the added value of the biomarker(s) to improve the accuracy of the model relative to the other components is expected.
Susceptibility/risk biomarkers and biomarker signatures:
Studies proposing to validate a susceptibility/risk biomarker/composite biomarker are expected be able to obtain prodromal data/samples prior to disease/condition onset and be powered to account for the prevalence of the disease or condition at risk. For this reason, these types of biomarkers can be especially challenging to validate and may rely on large retrospective analyses of natural history studies or electronic health records. Prospective validation may be challenging if the disease is slowly progressing or difficult to accurately diagnose.
Safety biomarkers and biomarker signatures:
Studies proposing to validate safety biomarkers/biomarker signatures are expected to demonstrate the association, relative change, and decision points for evaluating the biomarker’s association with the adverse responses to or pathological manifestation of an intervention or environmental exposure. These may evaluate acute or chronic outcomes as appropriate for the context of use.
Monitoring biomarkers and biomarker signatures:
Studies proposing to validate monitoring biomarkers/biomarker signatures, by definition, must be a longitudinal analysis to establish the quantitative change in the biomarker relative to other significant indicators of disease progression or improvement that may be measured with validated clinical outcome assessments, and/or convergent validation with other established biomarkers. Statically demonstrating the utility of the monitoring biomarker(s) to reduce the number or frequency of other assessments or more complex biomarkers needed for the same purpose is an important study design consideration.
Predictive biomarkers and biomarker signatures:
Studies proposing to validate predictive biomarkers/composite biomarker should test the biomarker’s ability to identify individuals who respond (or do not respond) to a particular therapeutic intervention. They should include a design that contains exposure to the intervention(s) of interest and be sufficiently powered to establish the biomarker’s discriminative thresholds based on the desired clinical improvement and outcomes. Predictive biomarkers can help identify which patients will benefit from a new therapeutic intervention and therefore help make treatment decisions in the clinic or be used to stratify patients in clinical trials. Because validating the predictive biomarker needs to be done in parallel with an intervention, these studies may need to be done as an ancillary study to an ongoing interventional or comparative effectiveness clinical trial. Alternatively, the biomarker may be validated with an already approved therapy known to target the same disease mechanism/pathway or by large retrospective analyses of existing clinical trial data to identify markers predictive of therapeutic response in completed trials.
What is the difference between the “initial clinical validation” described in the Development PAR and the “clinical validation” described in PAR-24-097 and PAR-24-096?
The term “initial” is used to indicate a proof-of-concept clinical validation study in the Development PAR (R61/R33) which may have more restrictive inclusion and exclusion criteria in order to be powered to evaluate the biomarker utility before extending to a more clinically heterogeneous population in the Clinical Validation PARs (U01 and U44s). The Clinical Validation U01s and U44s are expected to be large multi-site studies that provide the opportunity to statistically evaluate the utility in common comorbidities, and/or using broader diagnostic criteria that may extend the validation to additional disease stages or severity of a condition.
What is the difference between Analytical and Clinical Validation?
In simple terms, Analytical Validation is the process of evaluating the technical performance and reliability of the detection method that is used to measure the biomarker, whereas Clinical Validation is the process for evaluating the performance and usefulness of the biomarker as a decision-making tool for the Context of Use proposed. Analytical validation is needed to understand and reduce sources of variability in how the biomarker is measured before trying to interpret the biomarker results during the clinical validation process.
The precise definitions are listed below:
- Analytical Validation is the process of “Establishing that the performance characteristics of a test, tool, or instrument are acceptable in terms of its sensitivity, specificity, accuracy, precision, and other relevant performance characteristics using a specified technical protocol (which may include specimen/data collection, handling, and storage procedures [or signal acquisition and processing pipelines]). This is a validation of the test, tools, or instrument’s technical performance, but is not a validation of the item’s usefulness.” (BEST resource 2016).
- Clinical Validation is the process of “Establishing that the test, tool, or instrument acceptably identifies, measures, or predicts the concept of interest.” (BEST Resource 2016). The appropriate analytical approach will depend on the proposed context of use.
Can applicants propose to discover and validate composite biomarkers and biomarker signatures as well as biomarkers?
Yes, the NINDS Biomarker Program is open to applications from all biomarker categories and modalities including composite biomarkers and algorithms for biomarker signatures. As with all biomarker applications, justification of the biomarker utility, feasibility, and approach, including the statistical plan, are critical for the success of the application.
Should I apply to NINDS Biomarker program funding opportunities if I am developing or optimizing a device that can be used for measuring biomarker?
If you propose to develop a method for measuring a biomarker without also planning to test the utility of the biomarker, then you should not apply to the NINDS Translational Biomarker Notice of Funding Opportunity (NOFO). Applications focused on establishing the safety, design, or engineering features of the device should consider applying for one of the NINDS Translational Device NOFOs or through the omnibus standard SBIR/STTR funding opportunities.
How many Biomarker applications are funded each cycle?
Funding is competitive and is dependent on the availability of funds and competing program priorities across the institute. In general, the NINDS Translational Biomarker Program funding rate matches the NINDS Payline. However, applications within the top scoring range will undergo a “due diligence” process in which investigators will be asked to respond to the critiques in the summary statement to help NINDS staff, NINDS leadership, and the NINDS Advisory Council determine if the concerns can be easily addressed, or if the application needs additional input from review.
Should I still apply if my biomarker proposal is focused on the same disease and for the same context of use as one that is already funded?
Investigators are encouraged to review the list of funded projects before applying. Projects focused on validating a biomarker for the same disease or condition and the same context of use may be considered a lower program priority unless there is a unique approach or aspect to the project that makes it distinct from what is already being funded. Applicants are always encouraged to reach out to investigators with similar project goals and look for opportunities for collaboration.
What are the data-sharing requirements?
NIH has issued the Data Management and Sharing (DMS) policy (effective January 25, 2023) to promote the sharing of scientific data. Sharing scientific data accelerates biomedical research discovery, in part, by enabling validation of research results, providing accessibility to high-value datasets, and promoting data reuse for future research studies. Additional information is available at: Data Management & Sharing Policy Overview and on the NINDS website: NINDS Data Sharing: Information for Applicants and Awardees. You may also wish to review the DMS frequently asked questions.
Does the DMS policy apply to Small Business SBIR/STTR?
The DMS Policy applies to all research that generates scientific data, including:
- Research Projects
- Some Career Development Awards (Ks)
- Small Business SBIR/STTR
- Research Centers
NIH understands that some scientific data generated with NIH funds may be proprietary. Under the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Program Policy Directive, effective May 2, 2019, SBIR and STTR awardees may withhold applicable data for 20 years after the award date, as stipulated in the specific SBIR/STTR funding agreement and consistent with achieving program goals. SBIR and STTR awardees are expected to submit a Data Management & Sharing Plan per DMS Policy requirements.
Issues related to proprietary data also can arise when co-funding is provided by the private sector (for example, the pharmaceutical or biotechnology industries). NIH recognizes that the extent of data sharing may be limited by restrictions imposed by licensing limitations attached to materials needed to conduct the research. Applicants should discuss projects with proposed collaborators early to avoid agreements that prohibit or unnecessarily restrict data sharing. NIH staff will evaluate the justifications of investigators who believe that they are unable to share data.
For questions or concerns about data sharing expectations for proprietary data, please contact the Office of Science Policy. Small businesses may wish to contact the NIH SEED Office.
How do you apply for access to biospecimens from BioSEND, or how do you contribute biospecimens to BioSEND to facilitate biomarker research?
For access to biospecimens from the NINDS biomarker repository, BioSEND , investigators must first determine sample availability and submit a webform application for NINDS Biosample access. For assistance, please contact Rebecca Price, Ph.D., Program Officer, NINDS.
Applicants planning to prospectively collect biofluid samples are encouraged to share samples with the broader scientific community through the NINDS biomarker repository, BioSEND. Applicants should contact BioSEND to incorporate sharing plans and costs in their application. Note that costs for collection are NOT included as a component of the NINDS Biomarkers Repository award.