In many psychological, biological, and medical trials, more than two treatment groups are involved. In these situations, one is interested in detecting any significant difference among the treatment means , i.e. to test the global null hypothesis , and, particularly, in the detection of specific significant differences, i.e. in performing multiple comparisons according to the computation of simultaneous confidence intervals (SCI). In randomized clinical trials, the computation of SCI is consequently required by regulatory authorities: *“Estimates of treatment effects should be accompanied by confidence intervals, whenever possible…”* (ICH E9 Guideline 1998, chap. 5.5, p. 25 [23]). Hereby, the family-wise error rate should be strongly controlled.

In statistical practice, however, the usual way to detect specific significant differences among the effects of interest, and to compute SCI, consists of three steps: (1) the global null hypothesis is tested by an appropriate procedure, e.g. analysis of variance (ANOVA), (2) if the global null hypothesis is rejected, multiple comparisons are usually carried out to test individual hypotheses, e.g. the *l*th partial null hypothesis , and (3) in the final step, SCI for the treatment effects of interest are computed. Although stepwise procedures using different approaches on the same data are pretty common in practice, they may have the undesirable property that the global null hypothesis may be rejected, but none of the individual hypotheses and vice versa. This means, the global test procedure and the multiple testing procedure may be non-consonant to each other Gabriel 1969 [26] and Hsu [21]. Further the confidence intervals may include the null, i.e. the value of no treatment effect, even if the corresponding individual null hypotheses have been rejected. This means, the individual test decisions and the corresponding confidence intervals may be incompatible [1]. It is well known that the classical Bonferroni adjustment can be used to perform multiple comparisons as well as for the computation of compatible SCI. This approach, however, has a low power, particularly when the test statistics are not independent.

In recent years, multiple contrast test procedures (MCTPs) with accompanying compatible SCI for linear contrasts were derived by Mukerjee et al. [2] and Bretz et al. [1]. The procedures are based on the exact multivariate distribution of a vector of *t*-test statistics, where each test statistic corresponds to an individual null hypothesis, e.g. . It will be rejected, if the corresponding test statistic exceeds a critical value being obtained from the distribution of the vector of *t*-test statistics. The global null hypothesis will be rejected, if any individual hypothesis is rejected. Therefore, the individual and global test decisions are consonant and coherent. These MCTPs take the correlation between the test statistics into account and can be used for testing arbitrary contrasts, e.g. many-to-one, all-pairs, or even average comparisons [1]. Thus, MCTPs provide an extensive tool for powerful multiple comparisons, for the computation of compatible SCI, and for testing the global null hypothesis. The results by Bretz et al. [1] were extended to general linear models by Hothorn et al. [3], to heteroscedastic models by Hasler and Hothorn [4] and Herberich et al. [5], and for ranking procedures by Konietschke and Hothorn [6], Konietschke et al. [7], and Konietschke et al. [8]. For a comprehensive overview of existing methods, we refer to Bretz et al. [27].

Comparing MCTP and the global testing procedure ANOVA, one notices that both procedures can be used to test the global null hypothesis . From a practical point of view, MCTPs demonstrate their superiority to the ANOVA in terms of providing the information which levels cause the statistical overall significance as well as by offering SCI. In quite restricted homoscedastic normal models, both procedures are exact level tests. Arias-Castro et al. [9] studied global and multiple testing procedures under sparse alternatives and emphasize *“Because ANOVA is such a well established method, it might surprise the reader – but not the specialist – to learn that there are situations where the Max test, though apparently naive, outperforms ANOVA by a wide margin”* [9, p. 2534]. The evidence of a loss in power of the MCTP to detect global alternatives, if so, has not been investigated yet [25]. Thus, exact power comparisons remain.

It is the aim of this article to investigate the exact power of MCTP and of the ANOVA to detect global alternatives. To give a fair comparison, we restrict our analysis to those linear contrasts which are embedded in the ANOVA, i.e. contrasts which compare each mean to the overall mean . In particular, we compute the *least favorable configuration* (LFC) of the alternative, i.e. the alternative which is detected with a minimal power of both the ANOVA and the MCTP. The results indicate that the LFCs of both procedures are identical. Exact power calculations show that their powers to detect the LFCs are equal.

## Comments (0)

General note:By using the comment function on degruyter.com you agree to our Privacy Statement. A respectful treatment of one another is important to us. Therefore we would like to draw your attention to our House Rules.