Paylines and Percentile Scores: Understanding the Difference
In order to understand how NINDS makes funding decisions, it is first important to understand the concept of paylines and percentile scores.
Percentile Scores
Let’s start with percentile scores. Percentiles indicate relative rank. For unsolicited applications reviewed by the Center for Scientific Review (CSR), NIH converts an overall priority score into a percentile. A percentile ranks an application relative to the other applications reviewed by a study section at its last three meetings. Percentiles range from 1 to 99 in whole numbers. In contrast to usual mathematical practice, a lower number indicates a better score.
NIH uses percentiles to counter a phenomenon called "score creep" where study sections give applications increasingly better scores. As a result, scores cluster in the exceptional range, making it impossible to discriminate among applications. Each study section can apply the NIH review criteria differently, scoring either more harshly or more favorably. Percentiling counters these trends by ranking applications relative to others scored by the same study section. Altogether, percentiling makes review fairer to applicants. Our colleagues at NIAID go into further detail, describing how percentiles are calculated using sample data.
How NINDS sets paylines
We’ve talked about what a payline is and how it’s used at NINDS. But the process of determining what the payline will be in a given year is quite complex. Below we describe some of the considerations that NINDS (and most NIH ICs) must account for when setting paylines. As we’ve discussed elsewhere, the Federal appropriations process is a complex one, and often NIH doesn’t receive a final budget until mid-way through the fiscal year. Using all the information and historic data we have at our disposal, NINDS sets a conservative payline at the beginning of the fiscal year, with the hope of being able to raise it after a budget is finalized. However, even knowing how much money is available to spend in a given year isn’t quite as simple as it seems. Beyond that, many other factors go into setting the payline, including:
-
Money tied up in “outyear commitments”. Most NIH grant awards are made for more than 1 year; however, when NIH awards a grant, it pays out one year at a time. If you receive a 5-year R01, you typically receive 1 year of funding in the first year and additional installments in each of the remaining 4 years. NIH “commits” to paying the additional four years of money; however these funds come out of each subsequent appropriation.
-
Funds for new and established programs. At any point in time, the Institute commits money to support programs that capitalize on scientific opportunities, build capacity in needed areas of the scientific workforce, and/or catalyze understudied research areas. Based on the published “set aside”, funds are held in reserve to support meritorious applications submitted in response to these Funding Opportunity Announcements. It is important to remember that the large Congressionally-supported programs (BRAIN, the AD/ADRD program, HEAL, and the SBIR program) receive funds specifically to support them, and thus do not have any bearing on the NINDS payline.
-
Trends in grant applications. It is difficult to predict how many applications NINDS will receive and how much they will cost at any given payline point, but this is one of the main factors the Institute grapples with each year. This has become increasingly complex, as we have reached an inflection point in the number and size (in dollars) of applications we receive. Grant applications are at historic highs, while the number of PIs that NIH supports is stable, relatively speaking (see Research Project Grant Investigators: Number of Competing Investigators and Funding Rates chart). Moreover, increasing grant budgets make this calculus even more challenging.