Class "MTP", classes and methods for multiple testing procedure output

Description

An object of class MTP is the output of a particular multiple testing procedure, for example, generated by the MTP function. It has slots for the various data used to make multiple testing decisions, such as adjusted p-values and confidence regions.

Objects from the Class

Objects can be created by calls of the form
new('MTP',
statistic = ...., object of class numeric
estimate = ...., object of class numeric
sampsize = ...., object of class numeric
rawp = ...., object of class numeric
adjp = ...., object of class numeric
conf.reg = ...., object of class array
cutoff = ...., object of class matrix
reject = ...., object of class matrix
rawdist = ...., object of class matrix
nulldist = ...., object of class matrix
nulldist.type = ...., object of class character
marg.null = ...., object of class character
marg.par = ...., object of class matrix
label = ...., object of class numeric
index = ...., object of class matrix
call = ...., object of class call
seed = ...., object of class integer
)

Slots

statistic

Object of class numeric, observed test statistics for each hypothesis, specified by the values of the MTP arguments test, robust, standardize, and psi0.

estimate

For the test of single-parameter null hypotheses using t-statistics (i.e., not the F-tests), the numeric vector of estimated parameters corresponding to each hypothesis, e.g. means, differences in means, regression parameters.

sampsize

Object of class numeric, number of columns (i.e. observations) in the input data set.

rawp

Object of class numeric, unadjusted, marginal p-values for each hypothesis.

adjp

Object of class numeric, adjusted (for multiple testing) p-values for each hypothesis (computed only if the get.adjp argument is TRUE).

conf.reg

For the test of single-parameter null hypotheses using t-statistics (i.e., not the F-tests), the numeric array of lower and upper simultaneous confidence limits for the parameter vector, for each value of the nominal Type I error rate alpha (computed only if the get.cr argument is TRUE).

cutoff

The numeric matrix of cut-offs for the vector of test statistics for each value of the nominal Type I error rate alpha (computed only if the get.cutoff argument is TRUE).

reject

Object of class 'matrix', rejection indicators (TRUE for a rejected null hypothesis), for each value of the nominal Type I error rate alpha.

rawdist

The numeric matrix for the estimated nonparametric non-null test statistics distribution (returned only if keep.rawdist=TRUE and if nulldist is one of 'boot.ctr', 'boot.cs', or 'boot.qt'). This slot must not be empty if one wishes to call update to change choice of bootstrap-based null distribution.

nulldist

The numeric matrix for the estimated test statistics null distribution (returned only if keep.nulldist=TRUE); option not currently available for permutation null distribution, i.e., nulldist='perm'). By default (i.e., for nulldist='boot.cs'), the entries of nulldist are the null value shifted and scaled bootstrap test statistics, with one null test statistic value for each hypothesis (rows) and bootstrap iteration (columns).

nulldist.type

Character value describing which choice of null distribution was used to generate the MTP results. Takes on one of the values of the original nulldist argument in the call to MTP, i.e., 'boot.cs', 'boot.ctr', 'boot.qt', 'ic', or 'perm'.

marg.null

If nulldist='boot.qt', a character value returning which choice of marginal null distribution was used by the MTP. Can be used to check default values or to ensure manual settings were correctly applied.

marg.par

If nulldist='boot.qt', a numeric matrix returning the parameters of the marginal null distribution(s) used by the MTP. Can be used to check default values or to ensure manual settings were correctly applied.

label

If keep.label=TRUE, a vector storing the values used in the argument Y. Storing this object is particularly important when one wishes to update MTP objects with F-statistics using default marg.null and marg.par settings when nulldist='boot.qt'.

index

For tests of correlation parameters a matrix corresponding to t(combn(p,2)), where p is the number of variables in X. This matrix gives the indices of the variables considered in each pairwise correlation. For all other tests, this slot is empty, as the indices are in the same order as the rows of X.

call

Object of class call, the call to the MTP function.

seed

An integer or vector for specifying the state of the random number generator used to create the resampled datasets. The seed can be reused for reproducibility in a repeat call to MTP. This argument is currently used only for the bootstrap null distribution (i.e., for nulldist="boot.xx"). See ?set.seed for details.

Methods

signature(x = "MTP")

[

: Subsetting method for MTP class, which operates selectively on each slot of an MTP instance to retain only the data related to the specified hypotheses.

as.list

: Converts an object of class MTP to an object of class list, with an entry for each slot.

plot

: plot methods for MTP class, produces the following graphical summaries of the results of a MTP. The type of display may be specified via the which argument.

1. Scatterplot of number of rejected hypotheses vs. nominal Type I error rate.

2. Plot of ordered adjusted p-values; can be viewed as a plot of Type I error rate vs. number of rejected hypotheses.

3. Scatterplot of adjusted p-values vs. test statistics (also known as "volcano plot").

4. Plot of unordered adjusted p-values.

5. Plot of confidence regions for user-specified parameters, by default the 10 parameters corresponding to the smallest adjusted p-values (argument top).

6. Plot of test statistics and corresponding cut-offs (for each value of alpha) for user-specified hypotheses, by default the 10 hypotheses corresponding to the smallest adjusted p-values (argument top).

The argument logscale (by default equal to FALSE) allows one to use the negative decimal logarithms of the adjusted p-values in the second, third, and fourth graphical displays. The arguments caption and sub.caption allow one to change the titles and subtitles for each of the plots (default subtitle is the MTP function call). Note that some of these plots are implemented in the older function mt.plot.

print

: print method for MTP class, returns a description of an object of class MTP, including sample size, number of tested hypotheses, type of test performed (value of argument test), Type I error rate (value of argument typeone), nominal level of the test (value of argument alpha), name of the MTP (value of argument method), call to the function MTP.

In addition, this method produces a table with the class, mode, length, and dimension of each slot of the MTP instance.

summary

: summary method for MTP class, provides numerical summaries of the results of a MTP and returns a list with the following three components.

1. rejections: A data.frame with the number(s) of rejected hypotheses for the nominal Type I error rate(s) specified by the alpha argument of the function MTP. (NULL values are returned if all three arguments get.cr, get.cutoff, and get.adjp are FALSE).

2. index: A numeric vector of indices for ordering the hypotheses according to first adjp, then rawp, and finally the absolute value of statistic (not printed in the summary).

3. summaries: When applicable (i.e., when the corresponding quantities are returned by MTP), a table with six number summaries of the distributions of the adjusted p-values, unadjusted p-values, test statistics, and parameter estimates.

update

: update method for MTP class, provides a mechanism to re-run the MTP with different choices of the following arguments - nulldist, alternative, typeone, k, q, fdr.method, alpha, smooth.null, method, get.cr, get.cutoff, get.adjp, keep.nulldist, keep.rawdist, keep.margpar. When evaluate is 'TRUE', a new object of class MTP is returned. Else, the updated call is returned. The MTP object passed to the update method must have either a non-empty rawdist slot or a non-empty nulldist slot (i.e., must have been called with either 'keep.rawdist=TRUE' or 'keep.nulldist=TRUE').

To save on memory, if one knows ahead of time that one will want to compare different choices of bootstrap-based null distribution, then it is both necessary and sufficient to specify 'keep.rawdist=TRUE', as there is no other means of moving between null distributions other than through the non-transformed non-parametric bootstrap distribution. In this case, 'keep.nulldist=FALSE' may be used. Specifically, if an object of class MTP contains a non-empty rawdist slot and an empty nulldist slot, then a new null distribution will be generated according to the values of the nulldist= argument in the original call to MTP or any additional specifications in the call to update. On the other hand, if one knows that one wishes to only update an MTP object in ways which do not involve choice of null distribution, then 'keep.nulldist=TRUE' will suffice and 'keep.rawdist' can be set to FALSE (default settings). The original null distribution object will then be used for all subsequent calls to update.

N.B.: Note that keep.rawdist=TRUE is only available for the bootstrap-based resampling methods. The non-null distribution does not exist for the permutation or influence curve multivariate normal null distributions.

mtp2ebmtp

: coersion method for converting objects of class MTP to objects of class EBMTP. Slots common to both objects are taken from the object of class MTP and used to create a new object of class EBMTP. Once an object of class EBMTP is created, one may use the method EBupdate to perform resampling-based empirical Bayes multiple testing without the need for repeated resampling.

Author(s)

Katherine S. Pollard and Houston N. Gilbert with design contributions from Sandrine Dudoit and Mark J. van der Laan.

References

M.J. van der Laan, S. Dudoit, K.S. Pollard (2004), Augmentation Procedures for Control of the Generalized Family-Wise Error Rate and Tail Probabilities for the Proportion of False Positives, Statistical Applications in Genetics and Molecular Biology, 3(1). http://www.bepress.com/sagmb/vol3/iss1/art15/

M.J. van der Laan, S. Dudoit, K.S. Pollard (2004), Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error Rate, Statistical Applications in Genetics and Molecular Biology, 3(1). http://www.bepress.com/sagmb/vol3/iss1/art14/

S. Dudoit, M.J. van der Laan, K.S. Pollard (2004), Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates, Statistical Applications in Genetics and Molecular Biology, 3(1). http://www.bepress.com/sagmb/vol3/iss1/art13/

Katherine S. Pollard and Mark J. van der Laan, "Resampling-based Multiple Testing: Asymptotic Control of Type I Error and Applications to Gene Expression Data" (June 24, 2003). U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 121. http://www.bepress.com/ucbbiostat/paper121

M.J. van der Laan and A.E. Hubbard (2006), Quantile-function Based Null Distributions in Resampling Based Multiple Testing, Statistical Applications in Genetics and Molecular Biology, 5(1). http://www.bepress.com/sagmb/vol5/iss1/art14/

S. Dudoit and M.J. van der Laan. Multiple Testing Procedures and Applications to Genomics. Springer Series in Statistics. Springer, New York, 2008.

See Also

MTP, MTP-methods, EBMTP, EBMTP-methods, [-methods, as.list-methods, print-methods, plot-methods, summary-methods, mtp2ebmtp, ebmtp2mtp

Examples

## See MTP function: ? MTP