ADDIS-spending: Adaptive discarding algorithm for online FWER control

Implements the ADDIS algorithm for online FWER control, where ADDIS stands for an ADaptive algorithm that DIScards conservative nulls, as presented by Tian and Ramdas (2021). The procedure compensates for the power loss of Alpha-spending, by including both adaptivity in the fraction of null hypotheses and the conservativeness of nulls.

ADDIS_spending(
  d,
  alpha = 0.05,
  gammai,
  lambda = 0.25,
  tau = 0.5,
  dep = FALSE,
  display_progress = FALSE
)

Arguments

d: Either a vector of p-values, or a dataframe with three columns: an identifier (`id'), p-value (`pval'), and lags (`lags').
alpha: Overall significance level of the procedure, the default is 0.05.
gammai: Optional vector of \(\gamma_i\). A default is provided with \(\gamma_j\) proportional to \(1/j^(1.6)\).
lambda: Optional parameter that sets the threshold for `candidate' hypotheses. Must be between 0 and 1, defaults to 0.25.
tau: Optional threshold for hypotheses to be selected for testing. Must be between 0 and 1, defaults to 0.5.
dep: Logical. If TRUE runs the version for locally dependent p-values
display_progress: Logical. If TRUE prints out a progress bar for the algorithm runtime.

Value

out: A dataframe with the original p-values pval, the adjusted testing levels \(\alpha_i\) and the indicator function of discoveries R. Hypothesis \(i\) is rejected if the \(i\)-th p-value is less than or equal to \(\alpha_i\), in which case R[i] = 1 (otherwise R[i] = 0).

Details

The function takes as its input either a vector of p-values, or a dataframe with three columns: an identifier (`id'), p-value (`pval'), and lags, if the dependent version is specified (see below). Given an overall significance level \(\alpha\), ADDIS depends on constants \(\lambda\) and \(\tau\), where \(\lambda < \tau\). Here \(\tau \in (0,1)\) represents the threshold for a hypothesis to be selected for testing: p-values greater than \(\tau\) are implicitly `discarded' by the procedure, while \(\lambda \in (0,1)\) sets the threshold for a p-value to be a candidate for rejection: ADDIS-spending will never reject a p-value larger than \(\lambda\). The algorithms also require a sequence of non-negative non-increasing numbers \(\gamma_i\) that sum to 1.

The ADDIS-spending procedure provably controls the FWER in the strong sense for independent p-values. Note that the procedure also controls the generalised familywise error rate (k-FWER) for \(k > 1\) if \(\alpha\) is replaced by min(\(1, k\alpha\)).

Tian and Ramdas (2021) also presented a version for handling local dependence. More precisely, for any \(t>0\) we allow the p-value \(p_t\) to have arbitrary dependence on the previous \(L_t\) p-values. The fixed sequence \(L_t\) is referred to as `lags', and is given as the input lags for this version of the ADDIS-spending algorithm.

Further details of the ADDIS-spending algorithms can be found in Tian and Ramdas (2021).

References

Tian, J. and Ramdas, A. (2021). Online control of the familywise error rate. Statistical Methods for Medical Research 30(4):976–993.

Examples