What is “Context of Use” (COU) and why is it critical for establishing the study design?
The FDA defines Context of Use for a Biomarker as: “a concise description of the biomarker’s specified use... The COU includes two components: (1) the BEST biomarker category (prognostic, diagnostic, pharmacodynamic/response, predictive, safety, monitoring, or susceptibility/risk) and (2) the biomarker’s intended use in drug [or therapeutic] development. Each biomarker qualification effort should identify a single COU.” For the NIH Biomarker program, Context of Use (COU) may also include biomarkers where the purpose is for making treatment decisions.
A clearly defined Context of Use is essential because it explains how the biomarker is intended to be used; the purpose of the NINDS Biomarker Program is to enable research teams to collect the data needed to evaluate the accuracy and reliability of the biomarker for the use proposed. Therefore, the statistical analysis plan, study populations, and amount of acceptable variance or error when measuring the biomarker will all depend on the intended use.
How does the category of biomarker and Context of Use change the study design?
Each biomarker category has different experimental design expectations. General examples to illustrate some of the differences are listed below.
Diagnostic biomarkers and biomarker signatures:
Studies proposing to validate a diagnostic biomarker/biomarker signature must be able to differentiate the patient population(s) of interest against the gold standard outcome such as an established clinical outcome assessment battery, postmortem pathology, or against another/other established diagnostic tool(s). If the biomarker’s proposed COU is for differential diagnosis, then the study must be designed to test the accuracy for the differential diagnosis, not between the condition of interest and healthy controls.
Pharmacodynamic/response biomarkers and biomarker signatures:
Studies proposing to validate a pharmacodynamic/response biomarker must be able to test the biomarker using data/samples from patients undergoing the therapeutic(s) of interest. Pharmacodynamic/response biomarkers should be clearly associated with the mechanism of action or target pathway/system of the therapeutic to demonstrate either direct or indirect target engagement of the therapeutic (e.g., PET ligand binding, changes in quantity of enzymatic products, gene expression, down regulation of a molecule in a signaling cascade pathway, change in phosphorylation, acetylation, methylation, etc.). If the COU is for making dosing decisions, then study designs should quantitatively evaluate the relationship between the dose of the therapeutic and the relative amplitude and timing of the response in the biomarker. Initial validation in pre-clinical animal models or in vitro studies may be necessary; therefore, differences in the target pathway, receptor binding, or drug metabolism rates between the animal models and humans should be addressed.
Prognostic biomarkers and biomarker signatures:
Studies proposing to validate a prognostic biomarker/biomarker signature should demonstrate the accuracy of the biomarker’s ability to predict the likelihood of a clinical event or outcome(s) within a clinically useful defined time-period. Clinical validation should include prospective validation of the biomarker or biomarker signature and if validated, may be used to stratify patient populations for therapeutic testing or for making treatment decisions in clinical practice.
Susceptibility/risk biomarkers and biomarker signatures:
Studies proposing to validate a susceptibility/risk biomarker/biomarker signature must be able to obtain prodromal data/samples and be powered to account for the prevalence of the disease or condition. For this reason, these types of biomarkers can be especially challenging to validate and may rely on large retrospective analyses of natural history studies or electronic health records. Prospective validation may be challenging if the disease is slowly progressing or difficult to diagnose accurately.
Safety biomarkers and biomarker signatures:
Validation of safety biomarkers/biomarker signatures must demonstrate the association, relative change, and decision points for evaluating the biomarker’s association with the adverse responses to or pathological manifestation to an intervention or environmental exposure. These may include acute or longitudinal designs as needed.
Monitoring biomarkers and biomarker signatures:
Validation of monitoring biomarkers/biomarker signatures, by definition, must be a longitudinal analysis to establish the associated change in the biomarker relative to other significant indicators of disease state that may be measured with validated clinical outcome assessments, patient reported outcomes, and/or convergent validation with other established biomarkers. Monitoring biomarkers generally reflect the changes caused by the disease pathophysiology and therefore may also be useful as diagnostic, prognostic or predicative biomarkers. Likewise, some prognostic and diagnostic biomarkers that appear to reflect changes in underlying pathophysiology may be good candidate monitoring biomarkers, however the nature of the associated change with disease progression or remission must be demonstrated.
Predictive biomarkers and biomarker signatures:
Studies proposing to validate predictive biomarkers/biomarker signatures must be able to differentiate between individuals who respond (or do not respond) to a particular therapeutic intervention. Therefore, studies proposing to validate a predictive biomarker must include a design that includes exposure to the intervention(s) of interest and be sufficiently powered to establish the biomarker’s discriminative thresholds based on the desired clinical improvement and outcomes. Predictive biomarkers can help ensure that a new therapeutic intervention is targeting the correct patient population and therefore be very beneficial for making treatment decisions in the clinic or be used for patient stratification in clinical trials. However, validating the predictive biomarker for use in clinical trials can be challenging as it may need to be done in parallel with the therapeutic efficacy testing. Alternatively, the biomarker may be validated with an already approved therapy known to target the same disease mechanism/pathway or studies may use large retrospective analyses to identify markers or signatures associated with positive therapeutic response in completed trials before conducing prospective validation.
What is the difference between initial clinical validation described in the Discovery PAR-19-315 and definitive clinical validation described in PAR-21-058 and PAR-21-059?
Initial clinical validation demonstrates that the biomarker has predictive value for the clinical outcomes of interest. Definitive clinical validation comprehensively evaluates the normative ranges and establishes reference values and distributions across the full spectrum of individuals the biomarker is expected to be used with (for the proposed context of use).
What is the difference between Analytical and Clinical Validation?
Analytical validation is the process of “Establishing that the performance characteristics of a test, tool, or instrument are acceptable in terms of its sensitivity, specificity, accuracy, precision, and other relevant performance characteristics using a specified technical protocol (which may include specimen/data collection, handling and storage procedures [or signal acquisition and processing pipelines]). This is validation of the test, tools, or instrument’s technical performance, but is not validation of the item’s usefulness.” (BEST resource 2016).
Clinical validation is the process of “Establishing that the test, tool, or instrument acceptably identifies, measures, or predicts the concept of interest.” (BEST resource 2016). Metrics for Clinical Validation may include the demonstration of the sensitivity and specificity of the biomarker for the concept of interest within the Context of Use, including methods for binary and/or continuous analysis, Area Under the Curve (AUC) as determined by Receiver Operator Characteristic (ROC) Analysis, estimating the prevalence of the marker within subjects or patients for the intended clinical context, and establishing the appropriate cut-off or threshold for the biomarker for decision making within the context of use. Metrics should also include the Positive Predictive Value (PPV) and Negative Predictive Value (NPV). The appropriate analytical approach will depend on the type of biomarker and proposed context of use.
Can applicants propose to discover and validate composite biomarkers and biomarker signatures as well as biomarkers?
Yes, the NINDS Biomarker Program is open to applications from all biomarker categories and modalities including composite biomarkers and algorithms for biomarker signatures. As with all biomarker applications, justification of the biomarker utility, feasibility, and approach, including the statistical plan, are critical for the success of the application.
I am developing or optimizing a device that can be used for biomarker detection, should I apply to NINDS Biomarker program funding opportunities?
The purpose of the NINDS Biomarker program is to validate biomarkers for specific contexts of use (see FAQs above). As defined by the FDA, a biomarker is a “defined characteristic that is measured as an indicator of normal biological processes, pathogenic processes, or responses to an exposure or intervention” whereas a device is an “instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other similar or related article”. You may be optimizing advice that is an instrument to measure a biomarker, so targeting the appropriate FOA depends on the goal of your research. Applications to the biomarker program that include device optimization must focus on improving the device relative to the measurement of the biomarker for the Context of Use proposed. Applications focused on establishing the safety, design, or engineering features of the device should consider applying for one of the NINDS Translational Device FOAs or through the omnibus standard SBIR/STTR funding opportunities. If needed, applicants should contact program staff to help identify the best fit.
How many Biomarker applications are funded each cycle?
There is no payline or set number of applications funded for the biomarker program funding opportunities. Funding is competitive and is dependent on availability of funds and competing program priorities across the institute. Applications within the top scoring range may undergo a “due diligence” process in which investigators will be asked to respond to the critiques in the summary statement to help NINDS staff and the NINDS Advisory Council determine if the concerns can be easily addressed, or if the application needs additional input from review.
Should I still apply if my biomarker proposal is focused on the same disease and for the same context of use as one that is already funded?
Investigators are encouraged to review the list of funded projects before applying. Projects focused on validating a biomarker for the same disease or condition and for the same context of use may be considered a lower program priority unless there is a unique approach or aspect to the project that makes it distinct from what is already being funded. Applicants are always encouraged to reach out to investigators with similar project goals and look for opportunities for collaboration.
What types of samples or data can I use for a Biomarker Discovery project?
Biomarker discovery may include samples or data from any source that is meaningful for the purpose of the biomarker. Common examples include CSF, urine, blood, plasma, tissue samples, as well as data from imaging, electrophysiological recordings, digital technology devices, and behavioral data such as quantitative sensory testing, voice audio spectrum, movement and activity, and autonomic responses. The appropriateness of the data or sample type depends on how the biomarker is intended to be used. Important considerations for selecting samples and data may include things such as how they were collected, how relevant the sample population is relative to the proposed Context of Use, and what sources of variability need to be incorporated into the statistical analysis plan.
What existing biospecimens resources are available to leverage for my biomarker research?
NINDS supports several resources for high quality biospecimens including the NINDS Human Biomarkers Biospecimen and Data Repository (BioSEND) which banks and distributes biological samples that can be used to identify biomarkers of disease susceptibility, onset and progression for neurological and neuropsychiatric diseases, the NIH NeuroBioBank which banks and distributes human post-mortem brain tissue and related biospecimens that span neurological, neuropsychiatric, and neurodevelopmental diseases and disorders, and the NINDS Human Cell and Data Repository, which is an iPSC-based biorepository with cell sources that include fibroblasts and/or induced pluripotent stem cells for Alzheimer’s Disease, Amyotrophic Lateral Sclerosis (ALS), Ataxia-telangiectasia, Frontotemporal Lobar Degeneration (FTD), Huntington’s Disease, Parkinson’s Disease, and healthy controls. Cell sources, including isogenic cell lines for current and new diseases covered by the NINDS are continuing to be added. For more information on additional resources please see the “Resources and Tools” section of the NINDS Focus on Biomarker website.
What are the data sharing requirements?
All applications must include a resource sharing plan, regardless of budget. The resource plan should describe how, when and where the data, analytical code/scripts, metadata and protocols/standard operating procedures will be shared. If patent protection is being sought, investigators should explain how and when data will be shared after filing for patent protection. If underlying data cannot be shared, an explanation must be provided as part of the resource sharing plan. For more information please see the NIH Data Sharing Policy and Implementation Guidance.
Applicants planning to prospectively collect biofluid samples are strongly encouraged to share samples with the broader scientific community through the NINDS biomarker repository, BioSEND. Applicants should contact BioSEND to incorporate sharing plans and cost in their application. Note that costs for collection are NOT included as a component of the NINDS Biomarkers Repository award.