**Published Online**: 2013-05-29

An alternative design-based tradition, typified by Rosenbaum [35], permits hypothesis testing, confidence interval construction, and Hodges-Lehmann point estimation via Fisher’s [48] exact test. Although links may be drawn between this Fisherian mode of inference and the Neyman paradigm [13], the present work is not directly connected to the Fisherian mode of inference.

This assumption is made without loss of generality; multiple discrete treatments (or equivalently, some units not being sampled into either treatment or control) are easily accommodated in this framework. All instances of (1 – *T*_{i}) in the text may be replaced by *C*_{i}, an indicator variable for whether or not unit *i* receives the control, with one exception to be noted in Section 6.

An alternative is to also allow to vary between the treatment and control groups, particularly if effect sizes are anticipated to be large. Many of our results will also hold under such a specification, although the conditions for unbiasedness (and conservative variance estimation) will be somewhat more restrictive.

Interestingly (and perhaps unsurprisingly), is quite similar to the double robust (DR) estimator proposed by Robins [42] (and similar estimators, e.g., Ref. 43) the key differences between the DR estimator and the difference estimator follow. (a) The DR estimator utilizes estimated, rather than known, probabilities of entering treatment, and thus is subject to bias with finite *N*. (b) Even if known probabilities of entering treatment were used, in is chosen using a regression model, which typically fails to satisfy the restrictions necessary to yield unbiasedness established in Section 5.1. Thus, the DR estimator is subject to bias with finite *N*.

More formally and without loss of generality, let , where is a matrix of pretreatment covariates (that may or may not coincide with ), is an arbitrary function (e.g., the least squares fit), and is the function implied by eq. [1]. Since only is a random variable, the random variable equals some function . implies (equivalently ) which, in turn, implies .

If there are multiple treatments, the following simplification cannot be used. Furthermore, the associated estimator in Section 6.4 must apply to eq. [29], for which the derivation is trivially different.

Although the variance estimator is nonnegatively biased, the associated standard errors may not be (due to Jensen’s inequality) and any particular draw may be above or below the true value of the variance due to sampling variability.

The variance estimators derived in this paper do not reduce to those proposed by Neyman [49], Imai [40] or Middleton and Aronow [5], due to differences in how the covariance term is approximated.

, the variance estimator as applied to , is not generally guaranteed to be conservative. Specifically, when not constant, there is no guarantee that will be conservative, though an analogy to linearized estimators suggests that it should be approximately conservative with large *N*. Importantly, however, when, for all , and and the sharp null hypothesis of no treatment effect holds, is unbiased for .

## Comments (0)