ADDIS: Adaptive discarding algorithm for online FDR control

Implements the ADDIS algorithm for online FDR control, where ADDIS stands for an ADaptive algorithm that DIScards conservative nulls, as presented by Tian and Ramdas (2019). The algorithm compensates for the power loss of SAFFRON with conservative nulls, by including both adaptivity in the fraction of null hypotheses (like SAFFRON) and the conservativeness of nulls (unlike SAFFRON).

ADDIS(
  d,
  alpha = 0.05,
  gammai,
  w0,
  lambda = 0.25,
  tau = 0.5,
  async = FALSE,
  random = TRUE,
  display_progress = FALSE,
  date.format = "%Y-%m-%d"
)

Arguments

d: Either a vector of p-values, or a dataframe with three columns: an identifier (`id'), p-value (`pval'), and decision times (`decision.times').
alpha: Overall significance level of the procedure, the default is 0.05.
gammai: Optional vector of \(\gamma_i\). A default is provided with \(\gamma_j\) proportional to \(1/j^(1.6)\).
w0: Initial `wealth' of the procedure, defaults to \(\alpha/2\).
lambda: Optional parameter that sets the threshold for `candidate' hypotheses. Must be between 0 and tau, defaults to 0.25.
tau: Optional threshold for hypotheses to be selected for testing. Must be between 0 and 1, defaults to 0.5.
async: Logical. If TRUE runs the version for an asynchronous testing process. Defaults to FALSE.
random: Logical. If TRUE (the default), then the order of the p-values in each batch (i.e. those that have exactly the same date) is randomised. Only needed if async=FALSE.
display_progress: Logical. If TRUE prints out a progress bar for the algorithm runtime.
date.format: Optional string giving the format that is used for dates.

Value

out: A dataframe with the original p-values pval, the adjusted testing levels \(\alpha_i\) and the indicator function of discoveries R. Hypothesis \(i\) is rejected if the \(i\)-th p-value is less than or equal to \(\alpha_i\), in which case R[i] = 1 (otherwise R[i] = 0).

Details

The function takes as its input either a vector of p-values, or a dataframe with three columns. The dataframe requires an identifier (`id'), date (`date') and p-value (`pval'). If the asynchronous version is specified (see below), then the column date should be replaced by the decision times.

Given an overall significance level \(\alpha\), ADDIS depends on constants \(w_0\), \(\lambda\) and \(\tau\). Here \(w_0\) represents the initial `wealth' of the procedure and satisfies \(0 \le w_0 \le \alpha\). \(\tau \in (0,1)\) represents the threshold for a hypothesis to be selected for testing: p-values greater than \(\tau\) are implicitly `discarded' by the procedure. Finally, \(\lambda \in [0,\tau)\) sets the threshold for a p-value to be a candidate for rejection: ADDIS will never reject a p-value larger than \(\lambda\). The algorithm also require a sequence of non-negative non-increasing numbers \(\gamma_i\) that sum to 1.

The ADDIS procedure provably controls the FDR for independent p-values. Tian and Ramdas (2019) also presented a version for an asynchronous testing process, consisting of tests that start and finish at (potentially) random times. The discretised finish times of the test correspond to the decision times. These decision times are given as the input decision.times. Note that this asynchronous version controls a modified version of the FDR.

Further details of the ADDIS algorithms can be found in Tian and Ramdas (2019).

References

Tian, J. and Ramdas, A. (2019). ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls. Advances in Neural Information Processing Systems, 9388-9396.

Examples