Maximizing Data Transparency with Rigor Icons

Scientific meetings are an important venue for disseminating findings, but space and time for experimental design details is often limited. Thus, to facilitate increased transparency in scientific presentations, we propose the use of rigor icons to quickly and easily convey the experimental design details as described in this paper.

Rigor icons are simple symbols that can be added to figures to easily convey important experimental design details in scientific presentations.

Different sets of rigor icons will be valuable to different communities. However, a core set of reporting standards, including measures to minimize the risk of bias, are integral experimental design elements that are broadly applicable.

A proposed set of fundamental rigor icons is presented below:

Rigor-related item                                Description                                                    Icon

Blinding/Masking

 Experimenters were blinded/masked to treatment during experimentation and analysis
An outlined face and neck wearing a white blindfold enclosed in a purple circle.

Randomization

 Samples were randomly assigned to experimental groups
A single white die enclosed in a yellow-brown circle

Sample Size Estimation

 Sample size/power was calculated in advance
A capital N (white) enclosed in a blue circle.

Data Exclusions

 Outliers or other data were excluded according to pre-defined criteria
A white thumbs up thumbs down sign separated by a white slash all enclosed in a green circle.

Exploratory Experiment

 This experiment or analysis was exploratory, observational, or hypothesis-generating
Dark grey binoculars (details outlined in white) enclosed in a red circle.

Data Accessibility

 Data/code/analysis pipelines are available in a public repository
A dark blue cloud (outlined in white) and a blue symbol for data storage enclosed in a lighter blue circle.

Adding rigor icons to figures in scientific presentations maximizes data transparency and provides important information without taking up additional space:

Left, a bar graph showing data for three groups (A, B, C) with ill-defined error bars. Right, a graph showing the data from the same groups but represented as a quantile boxplot showing the mean, sample size, and individual data points. Below are three icons depicting blinding/masking (an outlined face and neck wearing a white mask enclosed in a purple circle), randomization (yellow-brown circle with a white die), and sample size calculation (blue circle with white N).

The same data are represented as (L) bar graphs with vague error bars and (R) a quantile boxplot showing the mean (purple diamond), sample size, individual data points, and the range for data spread. The rigor icons represent (left to right) blinding/masking, randomization, and sample size estimation. The graph on the right, with the same amount of space, conveys more detailed information about the methods and analysis. This figure is modified from Silberberg, et al.

Additional Resources:

  • Flyer(pdf, 365 KB) describing the icons in more detail
  • File repository with the individual icons, the descriptive flyer, and a presentation slide
  • Presentation by NINDS Director Walter Koroshetz about using the icons