Targeted Estimation of Nuisance Parameters to Obtain Valid Statistical Inference

Mark J. van der Laan

doi:10.1515/ijb-2012-0038

degruyter.com uses cookies to store information that enables us to optimize our website and make browsing more comfortable for you. To learn more about the use of cookies, please read our Privacy Policy. OK

Source Title:: The International Journal of Biostatistics
Volume/Issue:: Volume 10: Issue 1

Targeted Estimation of Nuisance Parameters to Obtain Valid Statistical Inference

Mark J. van der Laan¹

¹ Division of Biostatistics, University of California – Berkeley, Berkeley, CA, USA

Mark J. van der Laan

Corresponding author
Email
Search for other articles:
degruyter.com Google Scholar

DOI:: https://doi.org/10.1515/ijb-2012-0038
Published online:: 11 Feb 2014

PDF

Access Metrics

	All Time	Past Year	Past 30 Days
Full Text	181	181	12
PDF	63	63	3

Abstract

In order to obtain concrete results, we focus on estimation of the treatment specific mean, controlling for all measured baseline covariates, based on observing independent and identically distributed copies of a random variable consisting of baseline covariates, a subsequently assigned binary treatment, and a final outcome. The statistical model only assumes possible restrictions on the conditional distribution of treatment, given the covariates, the so-called propensity score. Estimators of the treatment specific mean involve estimation of the propensity score and/or estimation of the conditional mean of the outcome, given the treatment and covariates. In order to make these estimators asymptotically unbiased at any data distribution in the statistical model, it is essential to use data-adaptive estimators of these nuisance parameters such as ensemble learning, and specifically super-learning. Because such estimators involve optimal trade-off of bias and variance w.r.t. the infinite dimensional nuisance parameter itself, they result in a sub-optimal bias/variance trade-off for the resulting real-valued estimator of the estimand. We demonstrate that additional targeting of the estimators of these nuisance parameters guarantees that this bias for the estimand is second order and thereby allows us to prove theorems that establish asymptotic linearity of the estimator of the treatment specific mean under regularity conditions. These insights result in novel targeted minimum loss-based estimators (TMLEs) that use ensemble learning with additional targeted bias reduction to construct estimators of the nuisance parameters. In particular, we construct collaborative TMLEs (C-TMLEs) with known influence curve allowing for statistical inference, even though these C-TMLEs involve variable selection for the propensity score based on a criterion that measures how effective the resulting fit of the propensity score is in removing bias for the estimand. As a particular special case, we also demonstrate the required targeting of the propensity score for the inverse probability of treatment weighted estimator using super-learning to fit the propensity score.

Keywords:: asymptotic linearity; cross-validation; efficient influence curve; influence curve; targeted minimum loss based estimation

1 Introduction and overview

This introduction provides an atlas for the contents of this article. It starts with formulating the role of estimation of nuisance parameters to obtain asymptotically linear estimators of a target parameter of interest. This demonstrates the need to target this estimator of the nuisance parameter in order to make the estimator of the target parameter asymptotically linear when the model for the nuisance parameter is large. The general approach to obtain such a targeted estimator of the nuisance parameter is described. Subsequently, we present our concrete example to which we will apply this general method for targeted estimation of the nuisance parameter, and for which we establish a number of formal theorems. Finally, we discuss the link to previous articles that concerned some kind of targeting of the estimator of the nuisance parameter, and we provide an organization of the remainder of the article.

1.1 The role of nuisance parameter estimation

Suppose we observe n independent and identically distributed copies of a random variable O with probability distribution $P_{0}$ . In addition, assume that it is known that $P_{0}$ is an element of a statistical model $M$ and that we want to estimate $ψ_{0} = Ψ (P_{0})$ for a given target parameter mapping $Ψ : M \mapsto I R$ . In order to guarantee that $P_{0} \in M$ one is forced to only incorporate real knowledge, and, as a consequence, such models $M$ are always very large and, in particular, are infinite dimensional. We assume that the target parameter mapping is path-wise differentiable and let $D^{*} (P)$ denote the canonical gradient of the path-wise derivative of $Ψ$ at $P \in M$ [1]. An estimator $ψ_{n} = \hat{Ψ} (P_{n})$ is a functional $\hat{Ψ}$ applied to the empirical distribution $P_{n}$ of $O_{1}, \dots, O_{n}$ and can thus be represented as a mapping $\hat{Ψ} : M_{N P} \mapsto I R$ from the non-parametric statistical model $M_{N P}$ into the real line. An estimator $\hat{Ψ}$ is efficient if and only if it is asymptotically linear with influence curve $D^{*} (P_{0})$ :

ψ_{n} - ψ_{0} = \frac{1}{n} \sum_{i = 1}^{n} D^{*} (P_{0}) (O_{i}) + o_{P} (1 / \sqrt{n}) .

The empirical mean of the influence curve $D^{*} (P_{0})$ represents the first-order linear approximation of the estimator as a functional of the empirical distribution, and the derivation of the influence curve is a by-product of the application of the so-called functional delta-method for statistical inference based on functionals (i.e. $\hat{Ψ}$ ) of the empirical distribution [2–4].

Suppose that $Ψ (P)$ only depends on P through a parameter $Q (P)$ and that the canonical gradient depends on P only through $Q (P)$ and a nuisance parameter $g (P)$ . The construction of an efficient estimator requires the construction of estimators $Q_{n}$ and $g_{n}$ of these nuisance parameters $Q_{0}$ and $g_{0}$ , respectively. Targeted minimum loss-based estimation (TMLE) represents a method for construction of (e.g. efficient) asymptotically linear substitution estimators $Ψ (Q_{n}^{*})$ , where $Q_{n}^{*}$ is a targeted update of $Q_{n}$ that relies on the estimator $g_{n}$ [5–7]. The targeting of $Q_{n}$ is achieved by specifying a parametric submodel ${Q_{n} (\in) : \in} \subset {Q (P) : P \in M}$ through the initial estimator $Q_{n}$ and a loss function $O \mapsto L (Q) (O)$ for $Q_{0} = arg {min}_{Q} P_{0} L (Q) \equiv \int L (Q) (o) d P_{0} (o)$ , so that the generalized score ${\frac{d}{d \in} L (Q_{n} (\in)) |}_{\in = 0}$ spans a desired user-supplied estimating function $D (Q_{n}, g_{n})$ . In addition, one may decide to target $g_{n}$ by specifying a parametric submodel ${g_{n} (\in_{1}) : \in_{1}} \subset {g (P) : P \in M}$ and loss function $O \mapsto L (g) (O)$ for $g_{0} = arg {min}_{g} P_{0} L (g)$ , so that the generalized score ${\frac{d}{d \in_{1}} L (g_{n} (\in_{1})) |}_{\in_{1} = 0}$ spans another desired estimating function $D_{1} (g_{n}, η_{n})$ for some estimator $η_{n}$ of nuisance parameter $η$ . The parameter $\in$ is fitted with MLE $\in_{n} = arg {min}_{\in} P_{n} L (Q_{n} (\in))$ , providing the first-step update $Q_{n}^{1} = Q_{n} (\in_{n})$ , and similarly $\in_{1, n} = arg {min}_{\in_{1}} P_{n} L (g_{n} (\in_{1}))$ . This updating process that mapped a current fit $(Q_{n}, g_{n})$ into an update $(Q_{n}^{1}, g_{n}^{1})$ is iterated till convergence at which point the TMLE $(Q_{n}^{*}, g_{n}^{*})$ solves $P_{n} D (Q_{n}^{*}, g_{n}^{*}) = 0$ , i.e. the empirical mean of the estimating function equals zero at the final TMLE $(Q_{n}^{*}, g_{n}^{*})$ . If one also targeted $g_{n}$ , then it also solves $P_{n} D_{1} (g_{n}^{*}, η_{n}) = 0$ . The submodel through $Q_{n}$ will depend on $g_{n}$ , while the submodel through $g_{n}$ will depend on another nuisance parameter $η_{n}$ . By setting $D (Q, g)$ equal to the efficient influence curve $D^{*} (Q, g)$ , the resulting TMLE solves the efficient influence curve estimating equation $P_{n} D^{*} (Q_{n}^{*}, g_{n}) = 0$ and thereby will be asymptotically efficient when $(Q_{n}, g_{n})$ is consistent for $(Q_{0}, g_{0})$ under appropriate regularity conditions, where the targeting of $g_{n}$ is not needed.

The latter is shown as follows. By the property of the canonical gradient (in fact, any gradient) we have $Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = - P_{0} D^{*} (Q_{n}^{*}, g_{n}) + R_{n} (Q_{n}^{*}, Q_{0}, g_{n}, g_{0})$ , where $R_{n}$ involves integrals of second-order products of the differences $(Q_{n}^{*} - Q_{0})$ and $(g_{n} - g_{0})$ . Combined with $P_{n} D^{*} (Q_{n}^{*}, g_{n}) = 0$ , this implies the following identity:

Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = (P_{n} - P_{0}) D^{*} (Q_{n}^{*}, g_{n}) + R_{n} (Q_{n}^{*}, Q_{0}, g_{n}, g_{0}) .

The first term is an empirical process term that, under empirical process conditions (mentioned below), equals $(P_{n} - P_{0}) D^{*} (Q, g)$ , where $(Q, g)$ denotes the limit of $(Q_{n}^{*}, g_{n})$ , plus an $o_{P} (1 / \sqrt{n})$ -term. This then yields

Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = (P_{n} - P_{0}) D^{*} (Q, g) + R_{n} (Q_{n}^{*}, Q_{0}, g_{n}, g_{0}) + o_{P} (1 / \sqrt{n}) .

To obtain the desired asymptotic linearity of $Ψ (Q_{n}^{*})$ one needs $R_{n} = o_{P} (1 / \sqrt{n})$ , which in general requires at minimal that both nuisance parameters are consistently estimated: $Q = Q_{0}$ and $g = g_{0}$ . However, in many problems of interest, $R_{n}$ only involves a cross-product of the differences $Q_{n}^{*} - Q_{0}$ and $g_{n} - g_{0}$ , so that $R_{n}$ converges to zero if either $Q_{n}^{*}$ is consistent or $g_{n}$ is consistent: i.e. $Q = Q_{0}$ or $g = g_{0}$ . In this latter case, the TMLE is so-called double robust. Either way, the consistency of the TMLE relies now on one of the nuisance parameter estimators being consistent, thereby requiring the use of non-parametric adaptive estimation such as super-learning [8–10] for at least one of the nuisance parameters. If only one of the nuisance parameter estimators is consistent, and we are in the double robust scenario, then it follows that the bias is of the same order as the bias of the consistent nuisance parameter estimator. However, if the nuisance parameter estimator is not based on a correctly specified parametric model, but instead is a data-adaptive estimator, then this bias will be converging to zero at a rate slower than $1 / \sqrt{n}$ : i.e. $\sqrt{n} R_{n}$ converges to infinity as $n \mapsto \infty$ . Thus, in that case, the estimator of the target parameter may thus be overly biased and thereby will not be asymptotically linear.

1.2 Targeting the fit of the nuisance parameter: general approach

In this article, we demonstrate that if $Q \neq Q_{0}$ , then it is essential that the consistent nuisance parameter estimator $g_{n}$ be targeted toward the estimand so that the bias for the estimand becomes second order: that is, in our new TMLEs relying on consistent estimation of $g_{0}$ presented in this article one simultaneously updates $g_{n}$ into a $g_{n}^{*}$ so that certain smooth functionals of $g_{n}^{*}$ , derived from the study of $R_{n}$ , are asymptotically linear under appropriate conditions. Even if both estimators $Q_{n}^{*}$ and $g_{n}^{*}$ are consistent, but $Q_{n}^{*}$ might be converging at a slower rate than $g_{n}^{*}$ , this targeting of the nuisance parameter estimator may still remove finite sample bias for the estimand. In addition, we also present such TMLE when only relying on one of the nuisance parameters to be consistently estimated, but not knowing which one: i.e. either $Q = Q_{0}$ or $g = g_{0}$ . The same arguments applies to other double robust estimators, such as estimating equation based estimators and inverse probability of treatment weighted (IPTW) estimators [11–16]. In fact, we demonstrate such a targeted IPTW-estimator in our next section.

The current article concerns the construction of such targeted IPTW and TMLE that are asymptotically linear under regularity conditions, even when only one of the nuisance parameters is consistent and the estimators of the nuisance parameters are highly data adaptive. In order to be concrete in this article, we will focus on a particular example. In such an example we can concretely present the second-order term $R_{n}$ mentioned above and thereby develop the concrete form of the TMLE.

The same approach for construction of such TMLE can be carried out in much greater generality, but that is beyond the scope of this article. Nonetheless, it is helpful for the reader to know that the general approach is the following (considering the case that $g = g_{0}$ , but Q can be misspecified): (1) approximate $R_{n} (Q_{n}^{*}, Q_{0}, g_{n}^{*}, g_{0}) = Φ_{0, n} (g_{n}^{*}) - Φ_{0, n} (g_{0}) + R_{1, n}$ for some mapping $Φ_{0, n}$ that depends on $P_{0}$ (e.g. through $Q_{0}$ ) and the data (e.g. $Q_{n}^{*}, g_{n}^{*}$ ), and where $R_{1, n}$ is a second-order term so that it is reasonable to assume $R_{1, n} = o_{P} (1 / \sqrt{n})$ ; (2) approximate $Φ_{0, n} (g_{n}^{*}) - Φ_{0, n} (g_{0}) = Φ_{n} (g_{n}^{*}) - Φ_{n} (g_{0}) + R_{2, n}$ , where $R_{2, n}$ is a second-order term and $Φ_{n}$ is now a known (only based on data) mapping approximating $Φ_{0}$ ; (3) construct $g_{n}^{*}$ so that it is a TMLE of the target parameter $Φ_{n} (g_{0})$ thereby allowing an expansion $Φ_{n} (g_{n}^{*}) - Φ_{n} (g_{0}) = (P_{n} - P_{0}) D_{1, n} (P_{0}) + R_{3, n}$ with $D_{1, n} (P_{0})$ being the efficient influence curve of $Φ_{n} (g_{0})$ . That is, in step 3, $g_{n}^{*}$ is iteratively updated to solve $P_{n} D_{1, n} (g_{n}^{*}, η_{n}) = 0$ with $D_{1, n} (P_{0})$ depending on $P_{0}$ through $g_{0}$ and a nuisance parameter $η_{0}$ , so that $Φ_{n} (g_{n}^{*})$ is an asymptotically linear estimator of $Φ_{n} (g_{0})$ under regularity conditions. After these three steps, we have that $R_{n} (Q_{n}^{*}, Q_{0}, g_{n}^{*}, g_{0}) = (P_{n} - P_{0}) D_{1, n} (P_{0}) + R_{1, n} + R_{2, n} + R_{3, n}$ , where $R_{1, n} + R_{2, n} + R_{3, n} = o_{P} (1 / \sqrt{n})$ , and these steps provide us with the parameter $Φ_{n} (g_{0})$ that needs to be targeted by $g_{n}^{*}$ , thereby telling us how to target $g_{n}^{*}$ in the TMLE of $ψ_{0}$ . In addition, we can then conclude that this TMLE is asymptotically linear with known influence curve $D^{*} (Q, g_{0}) + D_{1} (P_{0})$ , where $D_{1} (P_{0})$ represents the limit of the efficient influence curve $D_{1, n} (P_{0})$ of $Φ_{n} (g_{0})$ : $Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = (P_{n} - P_{0}) {D^{*} (Q, g_{0}) + D_{1} (P_{0})} + o_{P} (1 / \sqrt{n})$ .

1.3 Concrete example covered in this article

Let us now formulate our concrete example we will cover in this article. Let $O = (W, A, Y) \sim P_{0}$ , W baseline covariates, A a binary treatment, and Y a final outcome. Let $M$ be a model that makes at most some assumptions about the conditional distribution of A, given W, but leaves the marginal distribution of W and the conditional distribution of Y, given $A, W$ , unspecified. Let $Ψ : M \mapsto I R$ be defined as $Ψ (P) = E_{P} E_{P} (Y | A = 1, W)$ , the so-called treatment specific mean controlling for the baseline covariates. The canonical gradient, also called the efficient influence curve, of $Ψ$ at P is given by $D^{*} (P) (O) = A / g (1 | W) (Y - \overset{ˉ}{Q} (1, W)) + \overset{ˉ}{Q} (1, W) - Ψ (P)$ , where $g (1 | W) = P (A = 1 | W)$ is the propensity score and $\overset{ˉ}{Q} (a, W) = E_{P} (Y | A = a, W)$ is the outcome regression [13]. Let $Q = (Q_{W}, \overset{ˉ}{Q})$ , where $Q_{W}$ is the marginal distribution of W, and note that $Ψ (P)$ only depends on P through $Q = Q (P)$ . For convenience, we will denote the target parameter with $Ψ (Q)$ in order to not have to introduce additional notation. A targeted minimum loss-based estimator (TMLE) is a plug-in estimator $Ψ (Q_{n}^{*})$ , where $Q_{n}^{*}$ is an update of an initial estimator $Q_{n}$ that relies on an estimator $g_{n}$ of $g_{0}$ , and it has the property that it solves $P_{n} D^{*} (Q_{n}^{*}, g_{n}) = 0$ , where we used the notation $P f = \int f (o) d P (o)$ .

For this particular example, such TMLE are presented in Scharfstein et al. [17]; van der Laan and Rubin [7]; Bembom et al. [18–21]; Rosenblum and van der Laan [22]; Sekhon et al. [23]; van der Laan and Rose [6, 24]. Since $P_{0} D^{*} (Q, g) = ψ_{0} - Ψ (Q) + P_{0} ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) ({\overset{ˉ}{g}}_{0} - \overset{ˉ}{g}) / \overset{ˉ}{g}$ [25, 26], where we use the notation $\overset{ˉ}{g} (W) = g (1 | W)$ and $\overset{ˉ}{Q} (W) = \overset{ˉ}{Q} (1, W)$ , and $P_{n} D^{*} (Q_{n}^{*}, g_{n}) = 0$ , we obtain the identity:

(1)

Ψ (Q_{n}^{*}) - ψ_{0} = (P_{n} - P_{0}) D^{*} (Q_{n}^{*}, g_{n}) + P_{0} ({\overset{ˉ}{Q}}_{0} - {\overset{ˉ}{Q}}_{n}^{*}) ({\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}) / {\overset{ˉ}{g}}_{n} .

The first term equals $(P_{n} - P_{0}) D^{*} (Q, g) + o_{P} (1 / \sqrt{n})$ if $D^{*} (Q_{n}^{*}, g_{n})$ falls in a $P_{0}$ -Donsker class with probability tending to 1, and $P_{0} {D^{*} (Q_{n}^{*}, g_{n}) - D^{*} (Q, g)}^{2} \mapsto 0$ in probability as $n \mapsto \infty$ [4, 27]. If ${\overset{ˉ}{Q}}_{n}^{*}$ and ${\overset{ˉ}{g}}_{n}$ are consistent for the true ${\overset{ˉ}{Q}}_{0}$ and ${\overset{ˉ}{g}}_{0}$ , respectively, then the second term is a second-order term. If one now assumes that this second-order term is $o_{P} (1 / \sqrt{n})$ , it has been proven that the TMLE is asymptotically efficient. This provides the general basis for proving asymptotic efficiency of TMLE when both $Q_{0}$ and $g_{0}$ are consistently estimated.

However, if only one of these nuisance parameter estimators is consistent, then the second term is still a first-order term, and it remains to establish that it is also asymptotically linear with a second-order remainder. For sake of discussion, suppose that ${\overset{ˉ}{Q}}_{n}^{*}$ converges to a wrong $\overset{ˉ}{Q}$ while ${\overset{ˉ}{g}}_{n}$ is consistent. In that case, this remainder behaves in first order as $P_{0} ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) ({\overset{ˉ}{g}}_{n} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0}$ . To establish that such a term is asymptotically linear requires that ${\overset{ˉ}{g}}_{n}$ solves a particular estimating equation: that is, ${\overset{ˉ}{g}}_{n}$ needs to be a TMLE itself targeting the required smooth functional of $g_{0}$ . This is naturally achieved within the TMLE framework by specifying a submodel through $g_{n}$ and loss function with the appropriate generalized score, so that a TMLE update step involves both updating $Q_{n}$ and $g_{n}$ , and the iterative TMLE algorithm now results in a final TMLE $(Q_{n}^{*}, g_{n}^{*})$ , not only solving $P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) = 0$ but also these additional equations that allow us to establish asymptotic linearity of the desired smooth functional of $g_{n}^{*}$ : see general description of TMLE above.

In this article, we present TMLE that targets $g_{n}$ in a manner that allows us to prove the desired asymptotic linearity of the second term in the right-hand side of eq. (1) when either ${\overset{ˉ}{g}}_{n}$ or ${\overset{ˉ}{Q}}_{n}$ is consistent, under conditions that require specified second-order terms to be $o_{P} (1 / \sqrt{n})$ . The latter type of regularity conditions are typical for the construction of asymptotically linear estimators and are therefore considered appropriate for the sake of this article. Though it is of interest to study cases in which these second-order terms cannot be assumed to be $o_{P} (1 / \sqrt{n})$ , this is beyond the scope of this article.

1.4 Relation to current literature on targeted nuisance parameter estimators

The construction of TMLE that utilizes targeting of the nuisance parameter $g_{n}$ has been carried out in earlier papers. For example, in van der Laan and Rubin [7], we target $g_{n}$ to obtain a TMLE that, beyond being double robust locally efficient, also equals the IPTW-estimator. In Gruber and van der Laan [29] we target $g_{n}$ to guarantee that, beyond being double robust locally efficient, also outperforms a user-supplied given estimator, based on the original idea of Rotnitzky et al. [28]. In that sense, the distinction of the current article with these previous articles is that $g_{n}^{*}$ is now targeted to guarantee that the TMLE remains asymptotically linear when $Q_{n}^{*}$ is misspecified. This task of targeting $g_{n}^{*}$ appears to be one step more complicated than in these previous articles, since the smooth functionals of $g_{n}^{*}$ that need to be targeted are themselves indexed by parameters of the true data distribution $P_{0}$ , and thus unknown. As mentioned above, our strategy is to approximate these unknown smooth functionals by an estimated smooth functional and develop the targeted estimator $g_{n}^{*}$ that targets this estimated parameter of $g_{0}$ .

The TMLEs presented in this article are always iterative and thereby rely on convergence of the iterative updating algorithm. Since the empirical risk increases at each updating step, such convergence is typically guaranteed by the existence of the MLE at each updating step (e.g. an MLE of coefficient in a logistic regression). Either way, in this article, we assume this convergence to hold. Since our assumptions of our theorems require $g_{n}^{*} (1 | W)$ to be bounded away from zero, we demonstrate how this property can be achieved by using submodels for updating $g_{n}$ that guarantee this property. Detailed simulations will appear in a future article.

1.5 Organization

The organization of this paper is as follows. In Section 2, we introduce a targeted IPTW-estimator that relies on an adaptive consistent estimator of $g_{0}$ , and we establish its asymptotic linearity with known influence curve, allowing for the construction of asymptotically valid confidence intervals based on this adaptive IPTW-estimator. In the remainder of the article, we focus on construction of TMLE involving the targeting of $g_{n}$ to establish the asymptotic linearity of the resulting TMLE under appropriate conditions. In Section 3, we introduce a novel TMLE that assumes that the targeted adaptive estimator $g_{n}^{*}$ is consistent for $g_{0}$ , and we establish its asymptotic linearity. In Section 4, we introduce a novel TMLE that only assumes that either the targeted ${\overset{ˉ}{Q}}_{n}^{*}$ or the targeted ${\overset{ˉ}{g}}_{n}^{*}$ is consistent, and we establish its asymptotic linearity with known influence curve. This TMLE needs to protect the asymptotic linearity under misspecification of either $g_{n}^{*}$ or ${\overset{ˉ}{Q}}_{n}^{*}$ , and, as a consequence, relies on targeting of $g_{n}$ (in order to preserve asymptotic linearity when ${\overset{ˉ}{Q}}_{n}^{*}$ is inconsistent), but also extra targeting of ${\overset{ˉ}{Q}}_{n}$ (in order to preserve asymptotic linearity when ${\overset{ˉ}{Q}}_{n}^{*}$ is consistent, but $g_{n}$ is inconsistent). The explicit form of the influence curve of this TMLE allows us to construct asymptotic confidence intervals. Since this result allows statistical inference in the statistical model that only assumes that one of the estimators is consistent, and we refer to this as “double robust statistical inference”. Even though double robust estimators have been extensively presented in the current literature, double robust statistical inference in these large semi-parametric models has been a difficult topic: typically, one has suggested to use the non-parametric bootstrap, but there is no theory supporting that the non-parametric bootstrap is a valid method when the estimators rely on data-adaptive estimation.

In Section 5, we extend the TMLE of Section 3 (that relies on $g_{n}^{*}$ being consistent for $g_{0}$ ) to the case that $g_{n}^{*}$ converges to a possibly misspecified g but one that suffices for consistent estimation of $ψ_{0}$ in the sense that $Ψ ({\overset{ˉ}{Q}}_{n}^{*})$ will be consistent. We present a corresponding asymptotic linearity theorem for this TMLE that is able to utilize the so-called collaborative double robustness of the efficient influence curve which states that $Ψ (Q) = ψ_{0}$ if $P_{0} D^{*} (Q, g) = 0$ and $g \in G (Q, P_{0})$ for a set $G (Q, P_{0})$ (including $g_{0}$ ). In order to construct a collaborative estimator $g_{n}^{*}$ that aims to converge to an element in $G (Q_{n}^{*}, P_{0})$ in collaboration with $Q_{n}^{*}$ , we use the framework of collaborative targeted minimum loss-based estimator (C-TMLE) [20, 29–35]. Our asymptotic linearity theorem can now be applied to this C-TMLE. Again, even though C-TMLEs have been presented in the current literature, statistical inference based on the C-TMLEs has been another challenging topic, and Section 5 provides us with a C-TMLE with known influence curve. We conclude this article with a discussion. The proofs of the theorems are presented in the Appendix.

1.6 Notation

In the following sections, we will use the following notation. We have $O = (W, A, Y) \sim P_{0} \in M$ , where $M$ is a statistical model that makes only assumptions on the conditional distribution of A, given W. Let $g_{0} (a | W) = P_{0} (A = a | W)$ , and ${\overset{ˉ}{g}}_{0} (W) = P_{0} (A = 1 | W)$ . The target parameter is $Ψ : M \to I R$ defined by $Ψ (P_{0}) = E_{Q_{W, 0}} {\overset{ˉ}{Q}}_{0} (1, W)$ , where ${\overset{ˉ}{Q}}_{0} (1, W) = E_{P_{0}} (Y | A = 1, W)$ , which will also be denoted with ${\overset{ˉ}{Q}}_{0} (W)$ , and $Q_{W, 0}$ is the distribution of W under $P_{0}$ . We also use the notation $Ψ (Q)$ , where $Q = (Q_{W}, \overset{ˉ}{Q})$ . In addition, $D^{*} (Q, g)$ denotes the efficient influence curve of $Ψ$ at $(Q, g)$ . We also use the following notation:

H_{A} ({\overset{ˉ}{Q}}^{r}, \overset{ˉ}{g}) = {\overset{ˉ}{Q}}^{r} / \overset{ˉ}{g}

H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}

D_{A} ({\overset{ˉ}{Q}}^{r}, \overset{ˉ}{g}) (A, W) = H_{A} ({\overset{ˉ}{Q}}^{r}, \overset{ˉ}{g}) (W) (A - \overset{ˉ}{g} (W))

H_{Y} (\overset{ˉ}{g}) (A, W) = A / \overset{ˉ}{g} (W)

{\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{Q}, \overset{ˉ}{g}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})

{\overset{ˉ}{Q}}_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0})

{\overset{ˉ}{g}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{0} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})

{\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{g}) = E_{0} (Y | A = 1, \overset{ˉ}{g}) o n l y I P T W - S e c t i o n 2

{\overset{ˉ}{Q}}_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} ({\overset{ˉ}{g}}_{0}) o n l y I P T W - S e c t i o n 2

∥ f ∥_{0} = {(P_{0} f^{2})}^{0.5} .

2 Statistical inference for IPTW-estimator when using super-learning to fit treatment mechanism

We first describe an IPTW-estimator that uses super-learning to fit the treatment mechanism $g_{0}$ . Subsequently, we present this IPTW-estimator but now using an update of the super-learning fit of $g_{0}$ , and we present a theorem establishing the asymptotic linearity of this targeted IPTW-estimator under appropriate conditions. Finally, we discuss how this targeted IPTW-estimator compares with an IPTW-estimator that relies on a parametric model to fit the treatment mechanism.

2.1 An IPTW-estimator using super-learning to fit the treatment mechanism

We consider a simple IPTW-estimator $\hat{Ψ} (P_{n}) = P_{n} D (\hat{g} (P_{n}))$ , where $D (g) (O) = Y A / \overset{ˉ}{g} (W)$ , and $\hat{g} : M_{N P} \mapsto G$ is an adaptive estimator of $g_{0}$ based on the log-likelihood loss function $L (g) (O) \equiv - log g (A | W)$ . For a general presentation of an IPTW-estimator, we refer to Robins and Rotnitzky [11], van der Laan and Robins [13], and Hernan et al. [36]. We wish to establish conditions under which reliable statistical inference based on this estimator of $ψ_{0}$ can be obtained. One might wish to estimate $g_{0}$ with ensemble learning, and, in particular, super-learning in which cross-validation [37] is used to determine the best weighted combination of a library of candidate estimators: van der Laan and Dudoit [8]; van der Laan et al. [9, 38, 39]; van der Vaart et al. [10]; Dudoit and van der Laan [40]; Polley et al. [41]; Polley and van der Laan [42]; van der Laan and Petersen [43]. The super-learner is a general template for construction of an adaptive estimator based on a library of candidate estimators, a loss function whose expectation is minimized over the parameter space by the true parameter value, a parametric family that defines “weighted” combinations of the estimators in the library. We will start with presenting a succinct description of a particular super-learner. Consider a library of estimators ${\hat{g}}_{j} : M_{N P} \mapsto G$ , $j = 1, \dots, J$ and a family of weighted (on logistic scale) combinations of these estimators $L o g i t {\hat{g}}_{α} (1 | W) = \sum_{j = 1}^{J} α_{j} L o g i t {\hat{g}}_{j} (1 | W)$ , indexed by vectors $α$ for which $α_{j} \in [0, 1]$ and $\sum_{j} α_{j} = 1$ . Consider a random sample split $B_{n} \in {0, 1}^{n}$ into a training sample ${i : B_{n} (i) = 0}$ of size $n (1 - p)$ and validation sample ${i : B_{n} (i) = 1}$ of size np, and let $P_{n, B_{n}}^{1}$ and $P_{n, B_{n}}^{0}$ denote the empirical distribution of the validation sample and training sample, respectively. Define

α_{n} = arg min_{α} E_{B_{n}} P_{n, B_{n}}^{1} L ({\hat{g}}_{α} (P_{n, B_{n}}^{0}))

= arg min_{α} E_{B_{n}} \frac{1}{n p} \sum_{i : B_{n} (i) = 1} L ({\hat{g}}_{α} (P_{n, B_{n}}^{0})) (O_{i}),

as the choice of estimator that minimizes cross-validated risk. The super-learner of $g_{0}$ is defined as the estimator $\hat{g} (P_{n}) = {\hat{g}}_{α_{n}} (P_{n})$ .

2.2 Asymptotic linearity of a targeted data-adaptive IPTW-estimator

The next theorem presents an IPTW-estimator that uses a targeted fit $g_{n}^{*}$ of $g_{0}$ , involving the updating of an initial estimator $g_{n}$ , and conditions under which this IPTW-estimator of $ψ_{0}$ is asymptotically linear. For example, $g_{n}$ could be defined as a super-learner of the type presented above. In spite of the fact that such an IPTW-estimator uses a very data adaptive and hard to understand estimator $g_{n}$ , this theorem shows that its influence curve is known and can be well estimated.

Theorem 1We consider a targeted IPTW-estimator $\hat{Ψ} (P_{n}) = P_{n} D (g_{n}^{*})$ , where $D (g) (O) = Y A / g (A | W)$ , and $g_{n}^{*}$ is an update of an initial estimator $g_{n}$ of $g_{0} \in G$ defined below.

Definition of targeted estimator $g_{n}^{*}$ : Let ${\overset{ˉ}{Q}}_{n}^{r *}$ be obtained by non-parametric estimation of the regression function $E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{n} (W))$ treating ${\overset{ˉ}{g}}_{n}$ as a fixed covariate (i.e. function of W). This yields an estimator $H_{n}^{r *} \equiv {\overset{ˉ}{Q}}_{n}^{r *} / {\overset{ˉ}{g}}_{n}$ of $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}$ , where ${\overset{ˉ}{Q}}_{0}^{r} = E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{0})$ . Consider the submodel $L o g i t {\overset{ˉ}{g}}_{n} (\in) = L o g i t {\overset{ˉ}{g}}_{n} + \in H_{n}^{r *}$ , and fit $\in$ with the MLE

\in_{n} = arg max_{\in} P_{n} log g_{n} (\in) .

We define $g_{n}^{*} = g_{n} (\in_{n})$ as the corresponding targeted update of $g_{n}$ . This TMLE $g_{n}^{*}$ satisfies

P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = 0.

Empirical process condition: Assume that $D (g_{n}^{*}), D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ fall in a $P_{0}$ -Donsker class with probability tending to 1.

Negligibility of second-order terms: Define ${\overset{ˉ}{Q}}_{0, n}^{r} \equiv E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{0} (W), {\overset{ˉ}{g}}_{n}^{*} (W))$ . Assume ${\overset{ˉ}{g}}_{n}^{*} > δ > 0$ with probability tending to 1 and assume

∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} = o_{P} (1)

∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0}^{2} = o_{P} (1 / \sqrt{n})

∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0} = o_{P} (1 / \sqrt{n})

∥ {\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0} = o_{P} (1 / \sqrt{n}) .

Then,

\hat{Ψ} (P_{n}) - ψ_{0} = (P_{n} - P_{0}) I C (P_{0}) + o_{P} (1 / \sqrt{n}),

where

I C (P_{0}) (O) = Y A / g_{0} (A | W) - ψ_{0} - H_{0}^{r} (W) (A - {\overset{ˉ}{g}}_{0} (W)) .

So under the conditions of this theorem, we can construct an asymptotic 0.95-confidence interval $ψ_{n} \pm 1.96 σ_{n} / \sqrt{n}$ based on this targeted IPTW-estimator $ψ_{n} = \hat{Ψ} (P_{n})$ , where

σ_{n}^{2} = P_{n} I C_{n}^{2} = \frac{1}{n} \sum_{i = 1}^{n} I C_{n} (O_{i})^{2},

and $I C_{n} (O) = Y A / {\overset{ˉ}{g}}_{n}^{*} (W) - ψ_{n} - H_{n}^{r *} (W) (A - {\overset{ˉ}{g}}_{n}^{*} (W))$ is the plug-in estimator of the influence curve $I C (P_{0})$ obtained by plugging in $g_{n}$ or $g_{n}^{*}$ for $g_{0}$ and ${\overset{ˉ}{Q}}_{n}^{r *}$ for ${\overset{ˉ}{Q}}_{0}^{r}$ .

Regarding the displayed second-order term conditions, we note that these are satisfied if ${\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}$ converges to zero w.r.t. $L^{2} (P_{0})$ -norm at rate $o_{P} (n^{- 1 / 4})$ , ${\overset{ˉ}{g}}_{n}^{*} > δ > 0$ for some $δ > 0$ with probability tending to 1 as $n \mapsto \infty$ , and the product of the rates at which ${\overset{ˉ}{g}}_{n}^{*}$ converges to ${\overset{ˉ}{g}}_{0}$ and $({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{Q}}_{0, n}^{r})$ converges to ${\overset{ˉ}{Q}}_{0}^{r}$ is $o_{P} (1 / \sqrt{n})$ .

Regarding the empirical process condition, we note that an example of a Donsker class is the class of multivariate real-valued functions with uniform sectional variation norm bounded by a universal constant [44]. It is important to note that if each estimator in the library falls in such a class, then also the convex combinations fall in that same class [4]. So this Donsker condition will hold if it holds for each of the candidate estimators in the library of the super-learner.

2.3 Comparison of targeted data-adaptive IPTW and an IPTW using parametric model

Consider an IPTW-estimator using a MLE $g_{n, 1}$ according to a parametric model for $g_{0}$ , and let us contrast this IPTW-estimator with an IPTW-estimator defined in the above theorem based on an initial super-learner $g_{n}$ that includes $g_{n, 1}$ as an element of the library of estimators. Let us first consider the case that the parametric model is correctly specified. In that case $g_{n, 1}$ converges to $g_{0}$ at a parametric rate $1 / \sqrt{n}$ . From the oracle inequality for cross-validation [8, 10, 38], it follows that $g_{n}$ also converges at the rate $1 / \sqrt{n}$ to $g_{0}$ possibly up to a $\sqrt{l o g n}$ -factor in case the number of algorithms in the library is of the order $n^{q}$ for some fixed q. As a consequence, all the consistency and second-order term conditions for the IPTW-estimator using a targeted $g_{n}^{*}$ based on $g_{n}$ hold. If one uses estimators in the library of algorithms that have a uniform sectional variation norm smaller than a $M < \infty$ with probability tending to 1, then also a weighted average of these estimators will have uniform sectional variation norm smaller than $M < \infty$ with probability tending to 1. Thus, in that case we will also have that $D (g_{n}^{*}), D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ fall in a $P_{0}$ -Donsker class. Examples of estimators that control the uniform sectional variation norm are any parametric model with fewer than K main terms that themselves have a uniform sectional variation norm, but also penalized least-squares estimators (e.g. Lasso) using basis functions with bounded uniform sectional variation norm, and one could map any estimator into this space of functions with universally bounded uniform sectional variation norm through a smoothing operation. Thus, under this restriction on the library, the IPTW-estimator using the super-learner is asymptotically linear with influence curve $I C (P_{0}) (O)$ as stated in the theorem. We note that $I C (P_{0})$ is the efficient influence curve for the target parameter $E_{P_{0}} E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{0} (W))$ if the observed data were $({\overset{ˉ}{g}}_{0} (W), A, Y)$ instead of $O = (W, A, Y)$ .

The parametric IPTW-estimator is asymptotically linear with influence curve $O \mapsto Y A / g_{0} (A | W) - ψ_{0} - Π (Y A / {\overset{ˉ}{g}}_{0} (W) | T_{g})$ , where $T_{g}$ is the tangent space of the parametric model for $g_{0}$ , and $Π (f | T_{g})$ denotes the projection of f onto $T_{g}$ in the Hilbert space $L_{0}^{2} (P_{0})$ [13]. This IPTW-estimator could be less or more efficient than the IPTW-estimator using the targeted super-learner depending on the actual tangent space of the parametric model.

For example, if the parametric model happens to have a score equal to $O \mapsto {\overset{ˉ}{Q}}_{0} (W) (A / {\overset{ˉ}{g}}_{0} (W) - 1)$ , then the parametric IPTW-estimator would be asymptotically efficient. Of course, a standard parametric model is not tailored to correspond with such optimal scores, but this shows that we cannot claim superiority of one versus the other in the case that the parametric model for $g_{0}$ is correctly specified.

If, on the other hand, the parametric model is misspecified, then the IPTW-estimator using $g_{n, 1}$ is inconsistent. However, the super-learner $g_{n}$ will be consistent if the library contains a non-parametric adaptive estimator, and will perform asymptotically as well as the oracle selector among all the weighted combinations of the algorithms in the library. To conclude, the IPTW-estimator using super-learning to estimate $g_{0}$ will be as good as the IPTW-estimator using a correctly specified parametric model (included in the library of the super-learner), but will remain consistent and asymptotically linear in a much larger model than the parametric IPTW-estimator relying on the true $g_{0}$ being an element of the parametric model.

3 Statistical inference for TMLE when using super-learning to consistently fit treatment mechanism

In the next subsection, we present a TMLE that targets the fit of the treatment mechanism, analog to the targeted IPTW-estimator presented above. In addition, this subsection presents a formal asymptotic linearity theorem demonstrating that this TMLE will be asymptotically linear even when ${\overset{ˉ}{Q}}_{n}^{*}$ is inconsistent under reasonable conditions. We conclude this section with a subsection showing how the iterative updating of the treatment mechanism can be carried out in such a way that the final fit of the treatment mechanism is still bounded away from zero, as required to obtain a stable estimator.

3.1 Asymptotic linearity of a TMLE using a targeted estimator of the treatment mechanism

The following theorem presents a novel TMLE and corresponding asymptotic linearity with specified influence curve, where we rely on consistent estimation of $g_{0}$ . The TMLE still uses the same updating step for the estimator of ${\overset{ˉ}{Q}}_{0}$ as the regular TMLE [7], but uses a novel updating step for the estimator of $g_{0}$ , analogue to the updating step of the IPTW-estimator in the previous section. We remind the reader of the importance of using the logistic fluctuations as working-submodels for ${\overset{ˉ}{Q}}_{0}$ in the definition of the TMLE, guaranteeing that the TMLE update stays within the bounded parameter space (see, e.g. Gruber and van der Laan [19]).

Theorem 2

Iterative targeted MLE of $ψ_{0}$ :

Definitions: Given $\overset{ˉ}{Q}, \overset{ˉ}{g}$ , let ${\overset{ˉ}{Q}}_{n}^{r} (\overset{ˉ}{Q}, \overset{ˉ}{g})$ be a consistent estimator of the regression ${\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{Q}, \overset{ˉ}{g}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ of $(Y - \overset{ˉ}{Q})$ on $\overset{ˉ}{g} (W)$ and $A = 1$ . Let $(g_{n}, {\overset{ˉ}{Q}}_{n})$ be an initial estimator of $(g_{0}, {\overset{ˉ}{Q}}_{0})$ .

Initialization: Let $g_{n}^{0} = g_{n}, {\overset{ˉ}{Q}}_{n}^{0} = {\overset{ˉ}{Q}}_{n}$ , and ${\overset{ˉ}{Q}}_{n}^{r 0} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{Q}}_{n}^{0}, {\overset{ˉ}{g}}_{n}^{0})$ . Let $k = 0$ .

Updating step for $g_{n}^{k}$ : Consider the submodel $L o g i t {\overset{ˉ}{g}}_{n}^{k} (\in) = L o g i t {\overset{ˉ}{g}}_{n}^{k} + \in H_{A} ({\overset{ˉ}{Q}}_{n}^{r k}, {\overset{ˉ}{g}}_{n}^{k})$ , and fit $\in$ with the MLE

\in_{n} = arg max_{\in} P_{n} log g_{n}^{k} (\in) .

We define $g_{n}^{k + 1} = g_{n}^{k} (\in_{n})$ as the corresponding update of $g_{n}^{k}$ . This $g_{n}^{k + 1}$ satisfies

\frac{1}{n} \sum_{i = 1}^{n} H_{A} ({\overset{ˉ}{Q}}_{n}^{r k}, {\overset{ˉ}{g}}_{n}^{k}) (W_{i}) (A_{i} - {\overset{ˉ}{g}}_{n}^{k + 1} (W_{i})) = 0.

Updating step for ${\overset{ˉ}{Q}}_{n}^{k}$ : Let $- L (\overset{ˉ}{Q}) (O) \equiv Y log \overset{ˉ}{Q} (A, W) + (1 - Y) log (1 - \overset{ˉ}{Q} (A, W))$ be the quasi-log-likelihood loss function for ${\overset{ˉ}{Q}}_{0} = E_{0} (Y | A = 1, W)$ (allowing that Y is continuous in $[0, 1]$ ). Consider the submodel $L o g i t {\overset{ˉ}{Q}}_{n}^{k} (\in) = L o g i t {\overset{ˉ}{Q}}_{n}^{k} + \in H_{Y} (g_{n}^{k})$ , and let $\in_{n} = arg {min}_{\in} P_{n} L ({\overset{ˉ}{Q}}_{n}^{k} (\in))$ . Define ${\overset{ˉ}{Q}}_{n}^{k + 1} = {\overset{ˉ}{Q}}_{n}^{k} (\in_{n})$ as the resulting update. Define ${\overset{ˉ}{Q}}_{n}^{r k + 1} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{Q}}_{n}^{k + 1}, {\overset{ˉ}{g}}_{n}^{k + 1})$ .

Iterating till convergence: Now, set $k \leftarrow k + 1$ , and iterate this updating process mapping a $(g_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{r k})$ into $(g_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{r k + 1})$ till convergence or till large enough K so that the estimating equations (2) below are solved up till an $o_{P} (1 / \sqrt{n})$ -term. Denote the limit of this iterative procedure with $(g_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{r *})$ .

Plug-in estimator: Let $Q_{n}^{*} = (Q_{W, n}, {\overset{ˉ}{Q}}_{n}^{*})$ , where $Q_{W, n}$ is the empirical distribution estimator of $Q_{W, 0}$ . The TMLE of $ψ_{0}$ is defined as $Ψ (Q_{n}^{*})$ .

Estimating equations solved by TMLE: This TMLE $(Q_{n}^{*}, g_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{r *})$ solves

P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) = 0

(2)

P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = 0.

Empirical process condition: Assume that $D^{*} (Q_{n}^{*}, g_{n}^{*})$ , $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ falls in a $P_{0}$ -Donsker class with probability tending to 1 as $n \mapsto \infty$ .

Negligibility of second-order terms: Define

\begin{matrix} {\overset{ˉ}{Q}}_{0, n}^{r} (W) \equiv E_{P_{0}} (Y - \overset{ˉ}{Q} (1, W) | A = 1, {\overset{ˉ}{g}}_{n}^{*} (W), {\overset{ˉ}{g}}_{0} (W)) \\ {\overset{ˉ}{Q}}_{0}^{r} (W) \equiv E_{P_{0}} (Y - \overset{ˉ}{Q} (1, W) | A = 1, {\overset{ˉ}{g}}_{0} (W)) \\ H_{0, n}^{r} = {\overset{ˉ}{Q}}_{0, n}^{r} / {\overset{ˉ}{g}}_{n}^{*} \\ H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}, \end{matrix}

where ${\overset{ˉ}{g}}_{n}^{*} (W)$ is treated as a fixed covariate (i.e. function of W) in the conditional expectation ${\overset{ˉ}{Q}}_{0, n}^{r}$ . Assume that there exists a $δ > 0$ , so that ${\overset{ˉ}{g}}_{n}^{*} > δ > 0$ with probability tending to 1, and

∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1)

∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} = o_{P} (1)

∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n})

∥ {\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0} = o_{P} (1 / \sqrt{n})

∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0}^{2} = o_{P} (1 / \sqrt{n})

∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0} ∥_{0} = o_{P} (1 / \sqrt{n}) .

Then,

Ψ (Q_{n}^{*}) - ψ_{0} = (P_{n} - P_{0}) I C (P_{0}) + o_{P} (1 / \sqrt{n}),

where $I C (P_{0}) = D^{*} (Q, g_{0}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0})$ .

Thus, under the assumptions of this theorem, an asymptotic 0.95-confidence interval is given by $ψ_{n}^{*} \pm 1.96 σ_{n} / \sqrt{n}$ , where $σ_{n}^{2} = P_{n} I C_{n}^{2}$ , and $I C_{n} = D^{*} (Q_{n}^{*}, g_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ .

3.2 Using a $δ$ -specific submodel for targeting g that guarantees the positivity condition

The following is an application of the constrained logistic regression approach of the type presented in Gruber and van der Lann [19] for the purpose of estimation of ${\overset{ˉ}{g}}_{0}$ respecting the constraint that ${\overset{ˉ}{g}}_{0} > δ > 0$ for a known $δ > 0$ . Recall that $A \in {0, 1}$ . Suppose that it is known that ${\overset{ˉ}{g}}_{0} (W) \in (δ, 1]$ for some $δ > 0$ , a condition the asymptotic linearity of our proposed estimators relies upon. Define $A_{δ} \equiv \frac{A - δ}{1 - δ}$ . We have ${\overset{ˉ}{g}}_{0} (W) = δ + (1 - δ) {\overset{ˉ}{g}}_{0, δ}$ , where ${\overset{ˉ}{g}}_{0, δ} = E_{0} (A_{δ} | W)$ is a regression that is known to be between $[0, 1]$ . Let $g_{δ, n}^{0}$ be an initial estimator of the true conditional distribution $g_{δ, 0}$ of $A_{δ}$ , given W, which implies an estimator ${\overset{ˉ}{g}}_{n}^{0} = δ + (1 - δ) {\overset{ˉ}{g}}_{δ, n}^{0}$ of ${\overset{ˉ}{g}}_{0}$ . Let $k = 0$ . Consider the following submodel for the conditional distribution of $A_{δ}$ , given W, through a given estimator $g_{δ, n}^{k}$ :

L o g i t {\overset{ˉ}{g}}_{δ, n}^{k} (\in) = L o g i t {\overset{ˉ}{g}}_{δ, n}^{k} + \in H_{A} ({\overset{ˉ}{Q}}_{n}^{r k}, {\overset{ˉ}{g}}_{δ, n}^{k}) .

The MLE is simply obtained with logistic regression of $A_{δ}$ on W (see, e.g. Gruber and van der Lann [19]) based on the quasi-log-likelihood loss function:

\in_{n} = arg min_{\in} P_{n} L ({\overset{ˉ}{g}}_{δ, n}^{k} (\in)),

where

- L ({\overset{ˉ}{g}}_{δ}) (O) = A_{δ} log {\overset{ˉ}{g}}_{δ} (W) + (1 - A_{δ} log (1 - {\overset{ˉ}{g}}_{δ} (W))

is the quasi-log-likelihood loss. The update ${\overset{ˉ}{g}}_{δ, n}^{k + 1} = {\overset{ˉ}{g}}_{δ, n}^{k} (\in_{n})$ implies an update ${\overset{ˉ}{g}}_{n}^{k + 1} = δ + (1 - δ) {\overset{ˉ}{g}}_{δ, n}^{k + 1}$ of ${\overset{ˉ}{g}}_{n}^{k} = δ + (1 - δ) {\overset{ˉ}{g}}_{δ, n}^{k}$ , and, by construction ${\overset{ˉ}{g}}_{n}^{k + 1} > δ > 0$ . The above submodel ${\overset{ˉ}{g}}_{n}^{k} (\in) = δ + (1 - δ) {\overset{ˉ}{g}}_{δ, n}^{k} (\in)$ and corresponding loss function $L (\overset{ˉ}{g}) = L ({\overset{ˉ}{g}}_{δ})$ generates the same score equation as the submodel and loss function used in Theorem 2. Therefore, the TMLE algorithm presented in Theorem 2 but now using this $δ$ -specific logistic regression model solves the same estimating equations, so that the same Theorem 2 immediately applies. However, using this submodel we have now guaranteed that ${\overset{ˉ}{g}}_{n}^{k} > δ > 0$ for all k in the iterative TMLE algorithm, and thereby that ${\overset{ˉ}{g}}_{n}^{*} > δ > 0$ .

4 Double robust statistical inference for TMLE when using super-learning to fit outcome regression and treatment mechanism

In this section, our aim is to present a TMLE that is asymptotically linear with known influence curve if either $g_{0}$ or $Q_{0}$ is consistently estimated, but we do not need to know which one. Again, this requires a novel way of targeting the estimators $g_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*}$ in order to arrange that the relevant smooth functionals of these nuisance parameter estimators are indeed asymptotically linear under appropriate second-order term conditions. In this case, we also need to augment the submodel for the estimator of ${\overset{ˉ}{Q}}_{0}$ with another clever covariate: that is, our estimator of ${\overset{ˉ}{Q}}_{0}$ needs to be double targeted, once for solving the efficient influence curve equation, but also for achieving asymptotic linearity in the case that the estimator of $g_{0}$ is misspecified.

Theorem 3

Definitions: For any given $\overset{ˉ}{g}, \overset{ˉ}{Q}$ , let ${\overset{ˉ}{g}}_{n}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q})$ and ${\overset{ˉ}{Q}}_{n}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q})$ be consistent estimators of ${\overset{ˉ}{g}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{P_{0}} (A | \overset{ˉ}{Q}, \overset{ˉ}{g})$ and ${\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ , respectively (e.g. using a super-learner or other non-parametric adaptive regression algorithm). Let ${\overset{ˉ}{Q}}_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ and ${\overset{ˉ}{g}}_{n}^{r} = {\overset{ˉ}{g}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ denote these estimators applied to the TMLEs $({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ defined below.

Iterative targeted MLE of $ψ_{0}$ :

Initialization: Let $(g_{n}$ , ${\overset{ˉ}{Q}}_{n})$ be an initial estimator of $(g_{0}, {\overset{ˉ}{Q}}_{0})$ . Let $g_{n}^{0} = g_{n}$ , ${\overset{ˉ}{Q}}_{n}^{0} = {\overset{ˉ}{Q}}_{n}$ and let $k = 0$ . Let ${\overset{ˉ}{g}}_{n}^{r, k} = {\overset{ˉ}{g}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{k})$ be obtained by non-parametrically regressing A on ${\overset{ˉ}{Q}}_{n}^{k}, {\overset{ˉ}{g}}_{n}^{k}$ . Let ${\overset{ˉ}{Q}}_{n}^{r, k} = {\overset{ˉ}{Q}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{k})$ be obtained by non-parametrically regressing $Y - {\overset{ˉ}{Q}}_{n}^{k}$ on $A = 1, {\overset{ˉ}{g}}_{n}^{k}$ .

Updating step: Consider the submodel $L o g i t {\overset{ˉ}{g}}_{n}^{k} (\in) = L o g i t {\overset{ˉ}{g}}_{n}^{k} + \in H_{A} ({\overset{ˉ}{Q}}_{n}^{r, k}, {\overset{ˉ}{g}}_{n}^{k})$ , and fit $\in$ with the MLE

\in_{A, n} = arg max_{\in} P_{n} log g_{n}^{k} (\in) .

Define the submodel $L o g i t {\overset{ˉ}{Q}}_{n}^{k} (\in) = L o g i t {\overset{ˉ}{Q}}_{n}^{k} + \in_{1} H_{Y} ({\overset{ˉ}{g}}_{n}^{k}) + \in_{2} H_{Y}^{1} ({\overset{ˉ}{g}}_{n}^{r, k}, {\overset{ˉ}{g}}_{n}^{k})$ , where $H_{Y}^{1} ({\overset{ˉ}{g}}^{r}, \overset{ˉ}{g}) \equiv \frac{A}{{\overset{ˉ}{g}}^{r}} \frac{{\overset{ˉ}{g}}^{r} - \overset{ˉ}{g}}{\overset{ˉ}{g}}$

Let $\in_{Y, n} = arg {min}_{\in} P_{n} L ({\overset{ˉ}{Q}}_{n}^{k} (\in))$ be the MLE, where $L (\overset{ˉ}{Q})$ is the quasi-log-likelihood loss.

We define $g_{n}^{k + 1} = g_{n}^{k} (\in_{A, n})$ as the corresponding targeted update of $g_{n}^{k}$ , and ${\overset{ˉ}{Q}}_{n}^{k + 1} = {\overset{ˉ}{Q}}_{n}^{k} (\in_{Y, n})$ as the corresponding update of ${\overset{ˉ}{Q}}_{n}^{k}$ . Let ${\overset{ˉ}{g}}_{n}^{r, k + 1} = {\overset{ˉ}{g}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1})$ and ${\overset{ˉ}{Q}}_{n}^{r, k + 1} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1})$ .

Iterate till convergence: Now, set $k \leftarrow k + 1$ , and iterate this updating process mapping a $(g_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{k}, {\overset{ˉ}{g}}_{n}^{r k}, {\overset{ˉ}{Q}}_{n}^{r k})$ into $(g_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1}, {\overset{ˉ}{g}}_{n}^{r k + 1}, {\overset{ˉ}{Q}}_{n}^{r k + 1})$ till convergence or till large enough K so that the following three estimating equations are solved up till an $o_{P} (1 / \sqrt{n})$ -term:

\begin{matrix} P_{n} D^{*} (Q_{n}^{K}, g_{n}^{K}) = o_{P} (1 / \sqrt{n}) \\ P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r, K}, {\overset{ˉ}{g}}_{n}^{K}) = o_{P} (1 / \sqrt{n}) \\ P_{n} D_{Y} ({\overset{ˉ}{Q}}_{n}^{K}, {\overset{ˉ}{g}}_{n}^{r, K}, {\overset{ˉ}{g}}_{n}^{K}) = o_{P} (1 / \sqrt{n}), \end{matrix}

where

\begin{matrix} D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) = H_{Y}^{1} ({\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) (Y - \overset{ˉ}{Q}) . \end{matrix}

Final substitution estimator: Denote the limits of this iterative procedure with ${\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{r *}, g_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*}$ . Let $Q_{n}^{*} = (Q_{W, n}, {\overset{ˉ}{Q}}_{n}^{*})$ , where $Q_{W, n}$ is the empirical distribution estimator of $Q_{W, 0}$ . The TMLE of $ψ_{0}$ is defined as $Ψ (Q_{n}^{*})$ .

Equations solved by TMLE:

\begin{matrix} o_{P} (1 / \sqrt{n}) = P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) \\ o_{P} (1 / \sqrt{n}) = P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) \\ o_{P} (1 / \sqrt{n}) = P_{n} D_{Y} ({\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) . \end{matrix}

Empirical process condition: Assume that $D^{*} (Q_{n}^{*}, g_{n}^{*})$ , $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ , $D_{Y} ({\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ fall in a $P_{0}$ -Donsker class with probability tending to 1 as $n \mapsto \infty$ .

Negligibility of second-order terms: Define ${\overset{ˉ}{Q}}_{0, n}^{r} = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n})$ and ${\overset{ˉ}{g}}_{0, n}^{r} = E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{n}^{*})$ . Assume that there exists a $δ > 0$ so that ${\overset{ˉ}{g}}_{n} > δ > 0$ with probability tending to 1, that ${\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*}$ are consistent for $\overset{ˉ}{g}, \overset{ˉ}{Q}$ w.r.t. $∥ \cdot ∥_{0}$ -norm, where either $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ or $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ , and assume that the following second-order terms are $o_{P} (1 / \sqrt{n})$ :

\begin{matrix} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1) \\ ∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} = o_{P} (1) \\ ∥ {\overset{ˉ}{g}}_{n}^{r *} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} = o_{P} (1) \\ ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0}^{2} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{n}^{r *} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n}) . \end{matrix}

Then,

Ψ (Q_{n}^{*}) - ψ_{0} = (P_{n} - P_{0}) I C (P_{0}) + o_{P} (1 / \sqrt{n}),

where

I C (P_{0}) = D^{*} (Q, g) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) - D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) .

Note that consistent estimation of the influence curve $I C (P_{0})$ relies on consistency of ${\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{Q}}_{n}^{r *}$ as estimators of ${\overset{ˉ}{g}}_{0}^{r}, {\overset{ˉ}{Q}}_{0}^{r}$ , and estimators ${\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{*}$ converging to a $\overset{ˉ}{Q}, \overset{ˉ}{g}$ for which either $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ or $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ . These estimators imply an estimated influence curve $I C_{n}$ . An asymptotic 0.95-confidence interval is given by $ψ_{n}^{*} \pm 1.96 σ_{n} / \sqrt{n}$ , where $σ_{n}^{2} = P_{n} I C_{n}^{2}$ .

If $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ , then $E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}) = \overset{ˉ}{g}$ , and therefore $D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) = 0$ for all $\overset{ˉ}{Q}$ . If $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ , then it follows that ${\overset{ˉ}{Q}}_{0}^{r} = 0$ , and thus that $D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) = 0$ for all $\overset{ˉ}{g}$ . In particular, if both $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ and $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ , then $I C (P_{0}) = D^{*} (Q_{0}, g_{0})$ . We also note that if $\overset{ˉ}{g} \neq {\overset{ˉ}{g}}_{0}$ , but $\overset{ˉ}{g}$ is a true conditional distribution of A, given some function $W^{r}$ of W for which $\overset{ˉ}{Q} (W)$ is only a function of $W^{r}$ , then it follows that $E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}) = \overset{ˉ}{g}$ and thus $D_{Y} = 0$ .

As shown in the final remark of the Appendix, the condition of Theorem 3 that either $g = g_{0}$ or $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ can be weakened to $(\overset{ˉ}{g}, \overset{ˉ}{Q})$ having to satisfy $P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) / \overset{ˉ}{g} = 0$ , allowing for the analysis of collaborative double robust TMLE, as discussed in the next section. However, as shown in the next section, if one arranges in the TMLE algorithm that ${\overset{ˉ}{g}}_{n}^{*} = {\overset{ˉ}{g}}_{n}^{r *}$ (i.e. ${\overset{ˉ}{g}}_{n}^{*}$ already non-parametrically adjusts for ${\overset{ˉ}{Q}}_{n}^{*}$ ), then there is no need for the extra targeting in ${\overset{ˉ}{Q}}_{n}^{k}$ , and the influence curve will be $D^{*} (Q, g) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g})$ .

5 Collaborative double robust inference for C-TMLE when using super-learning to fit outcome regression and reduced treatment mechanism

We first review the theoretical underpinning for collaborative estimation of nuisance parameters, in this case, the outcome regression and treatment mechanism. Subsequently, we explain that the desired collaborative estimation can be achieved by applying the previously established template for construction of a C-TMLE to a TMLE that solves certain estimating equations when given an initial estimator of $(Q_{0}, g_{0})$ . This C-TMLE template involves (1) creating a sequence of TMLEs $((g_{n, k}^{*}, Q_{n, k}^{*}) : k = 1, \dots, K)$ constructed in such a manner that the empirical risk of both $g_{n, k}^{*}$ and $Q_{n, k}^{*}$ is decreasing in k, and (2) using cross-validation to select the k for which $Q_{n, k}^{*}$ is the best fit of $Q_{0}$ . Subsequently, we present this TMLE that maps an initial of $(Q_{0}, g_{0})$ into targeted estimators solving the desired estimating equations and establish its asymptotic linearity under appropriate conditions, including that the initial estimator of $(Q_{0}, g_{0})$ is collaboratively consistent. Finally, we present a concrete C-TMLE algorithm that uses this TMLE algorithm as its basis, so that our theorem can be applied to this C-TMLE: a C-TMLE is still a TMLE, but it is a TMLE based on a data adaptively selected initial estimator that is collaboratively consistent, so that we can apply the same theorem to this C-TMLE.

5.1 Motivation and theoretical underpinning of collaborative double robust estimation of nuisance parameters

We note that $P_{0} D^{*} (Q, g) = P_{0} \{\frac{A}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) + \overset{ˉ}{Q} - Ψ (Q)\}$ . If $Q_{W} = Q_{W, 0}$ , this reduces to

P_{0} D^{*} (Q, g) = P_{0} \{\frac{A}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q})\} = Ψ (Q_{0}) - Ψ (Q) + P_{0} \{\frac{A - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q})\} .

Let $G$ be the class of all possible distributions of A, given W, and let $g_{0} \in G$ be the true conditional distribution of A given W. We define the set $G (P_{0}, \overset{ˉ}{Q}) \equiv \{g :\in G : 0 = P_{0} (A - \overset{ˉ}{g}) \frac{{\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}}{\overset{ˉ}{g}}\}$ . For any $g \in G (P_{0}, \overset{ˉ}{Q})$ , we have $P_{0} D^{*} (Q, g) = Ψ (Q_{0}) - Ψ (Q)$ . Suppose we have an estimator $(Q_{n}^{*}, g_{n}^{*})$ satisfying $P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) = 0$ and converging to a $(Q, g)$ so that $g \in G (P_{0}, \overset{ˉ}{Q})$ . Then it follows that $P_{0} D^{*} (Q, g) = 0$ and $P_{0} D^{*} (Q, g) = Ψ (Q_{0}) - Ψ (Q)$ , thereby establishing that $Ψ (Q_{n}^{*})$ is a consistent estimator of $Ψ (Q_{0})$ . Let us state this crucial result as a lemma

Lemma 1(van der Laan and Gruber [33]) If $P_{0} (A - \overset{ˉ}{g}) ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) / \overset{ˉ}{g} = 0$ , and $P_{0} D^{*} (Q, g) = 0$ , then $Ψ (Q) = ψ_{0}$ . More generally, $P_{0} D^{*} (Q, g) = Ψ (Q_{0}) - Ψ (Q) + P_{0} (A - \overset{ˉ}{g}) ({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) / \overset{ˉ}{g}$ .

We note that $G (P_{0}, \overset{ˉ}{Q})$ contains the true conditional distributions $g_{0}^{r}$ of A, given $W^{r}$ , for which $(\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) / {\overset{ˉ}{g}}_{0}^{r}$ is a function of $W^{r}$ , i.e. for which $\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}$ only depends on W through $W^{r}$ . We refer to such distributions as reduced treatment mechanisms. However, it contains many more conditional distributions since any conditional distribution g for which $(A - \overset{ˉ}{g} (W))$ is orthogonal to $({\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}) / \overset{ˉ}{g}$ in $L_{0}^{2} (P_{0})$ is an element of $G (P_{0}, \overset{ˉ}{Q})$ . We refer to van der Laan and Gruber [33] and Gruber and van der Laan [29] for the introduction and general notion of collaborative double robustness.

5.2 C-TMLE

The general C-TMLE introduced in van der Laan and Gruber [33] provides a template for construction of a TMLE $(g_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ satisfying $P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) = 0$ and converging to a $(g, \overset{ˉ}{Q})$ with $g \in G (P_{0}, \overset{ˉ}{Q})$ so that $P_{0} D^{*} (Q, g) = 0$ and thereby $Ψ (Q) - Ψ (Q_{0}) = 0$ . Thus C-TMLE provides a template for construction of targeted MLEs that exploit the collaborative double robustness of TMLEs in the sense that a TMLE will be consistent as long as $(Q_{n}^{*}, g_{n}^{*})$ converges to a $(Q, g)$ for which $g \in G (P_{0}, \overset{ˉ}{Q})$ . The goal is not to estimate the true treatment mechanism, but instead to construct a $g_{n}^{*}$ that converges to a conditional distribution given a reduction $W^{r}$ of W that is an element of $G (P_{0}, \overset{ˉ}{Q})$ . We could state that, just as the propensity score provides a sufficient dimension reduction for the outcome regression, so does, given $\overset{ˉ}{Q}$ , $(\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0})$ provide a sufficient dimension reduction for the propensity score regression in the TMLE. The current literature appears to agree that propensity score estimators are best evaluated with respect to their effect on estimation of the causal effect of interest, not by metrics such as likelihoods or classification rates [45–48], and the above-stated general collaborative double robustness provides a formal foundation for such claims.

The general C-TMLE has been implemented and applied to point treatment and longitudinal data [20, 29–33, 35]. A C-TMLE algorithm relies on a TMLE algorithm that maps an initial $({\overset{ˉ}{Q}}_{n}, g_{n})$ into a TMLE $({\overset{ˉ}{Q}}_{n}^{*}, g_{n}^{*})$ and uses this algorithm in combination with a targeted variable selection algorithm for generating candidate models for the propensity score to generate a sequence of candidate TMLEs $(g_{n}^{* k}, {\overset{ˉ}{Q}}_{n}^{* k})$ , increasingly non-parametric in k, and finally uses cross-validation to select the best TMLE among these candidates estimators of ${\overset{ˉ}{Q}}_{0}$ .

5.3 A TMLE that allows for collaborative double robust inference

Our next theorem presents a TMLE algorithm and a corresponding influence curve under the assumption that the propensity score correctly adjusts for the possibly misspecified $\overset{ˉ}{Q}$ and ${\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q} = E_{0} (Y - \overset{ˉ}{Q} (W) | A = 1, W)$ . The presented TMLE algorithm already arranges that this TMLE indeed non-parametrically adjusts for $\overset{ˉ}{Q}$ . In the next subsection, we will present an actual C-TMLE algorithm that generates a TMLE for which the propensity score is targeted to adjust for $\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}$ , so that this theorem can be applied.

Theorem 4

Definitions: For any given $\overset{ˉ}{g}, \overset{ˉ}{Q}$ , let ${\overset{ˉ}{g}}_{n}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q})$ and ${\overset{ˉ}{Q}}_{n}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q})$ be consistent estimators of ${\overset{ˉ}{g}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ and ${\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ , respectively (e.g. using a super-learner or other non-parametric adaptive regression algorithm). Let ${\overset{ˉ}{Q}}_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ and ${\overset{ˉ}{g}}_{n}^{r *} = {\overset{ˉ}{g}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ denote these estimators applied to the TMLE $({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ defined below.

“Score” equations the TMLE should solve: Below, we describe an iterative TMLE algorithm that results in estimators ${\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{Q}}_{n}^{r *}$ , $g_{n}^{*}$ , ${\overset{ˉ}{Q}}_{n}^{*}$ that solve the following equations:

0 = P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*})

(3)

0 = P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) .

Iterative targeted MLE of $ψ_{0}$ :

Initialization: Let ${\overset{ˉ}{Q}}_{n}$ and $g_{n}$ (e.g. aiming to adjust for ${\overset{ˉ}{Q}}_{n} - {\overset{ˉ}{Q}}_{0}$ ) be initial estimators.

Let ${\overset{ˉ}{Q}}_{n}^{0} = {\overset{ˉ}{Q}}_{n}$ , ${\overset{ˉ}{g}}_{n}^{0} = {\overset{ˉ}{g}}_{n}^{r} ({\overset{ˉ}{g}}_{n}, {\overset{ˉ}{Q}}_{n}^{0})$ , and ${\overset{ˉ}{Q}}_{n}^{r 0} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{0}, {\overset{ˉ}{Q}}_{n}^{0})$ .

\in_{A, n} = arg max_{\in} P_{n} log g_{n}^{k} (\in) .

Define the submodel $L o g i t {\overset{ˉ}{Q}}_{n}^{k} (\in) = L o g i t {\overset{ˉ}{Q}}_{n}^{k} + \in H_{Y} (g_{n}^{k})$ and let $L (\overset{ˉ}{Q})$ be the quasi-log-likelihood loss function for ${\overset{ˉ}{Q}}_{0}$ . Let $\in_{Y, n} = arg {min}_{\in} P_{n} L ({\overset{ˉ}{Q}}_{n}^{k} (\in))$ be the MLE. Let ${\overset{ˉ}{Q}}_{n}^{k + 1} = {\overset{ˉ}{Q}}_{n}^{k} (\in_{Y, n})$ , ${\overset{ˉ}{g}}_{n}^{k + 1} = {\overset{ˉ}{g}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{k} (\in_{A, n}), {\overset{ˉ}{Q}}_{n}^{k + 1})$ , and ${\overset{ˉ}{Q}}_{n}^{r k + 1} = {\overset{ˉ}{Q}}_{n}^{r} ({\overset{ˉ}{g}}_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1})$ .

Iterating till convergence: Now, set $k \leftarrow k + 1$ and iterate this updating process mapping a $(g_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{k}, {\overset{ˉ}{Q}}_{n}^{r k})$ into $(g_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{k + 1}, {\overset{ˉ}{Q}}_{n}^{r k + 1})$ till convergence or till large enough K so that the following estimating equations are solved up till an $o_{P} (1 / \sqrt{n})$ -term:

\begin{matrix} o_{P} (1 / \sqrt{n}) = P_{n} D^{*} (Q_{n}^{K}, g_{n}^{K}) \\ o_{P} (1 / \sqrt{n}) = P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r K}, {\overset{ˉ}{g}}_{n}^{K}) . \end{matrix}

Final substitution estimator: Denote these limits (in k) of this iterative procedure with $g_{n}^{*}$ , ${\overset{ˉ}{Q}}_{n}^{*}$ , ${\overset{ˉ}{Q}}_{n}^{r *}$ . Let $Q_{n}^{*} = (Q_{W, n}, {\overset{ˉ}{Q}}_{n}^{*})$ , where $Q_{W, n}$ is the empirical distribution estimator of $Q_{W, 0}$ . The TMLE of $ψ_{0}$ is defined as $Ψ (Q_{n}^{*})$ .

Assumption on limits $\overset{ˉ}{g}, \overset{ˉ}{Q}$ of ${\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*}$ : Assume that $({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ is consistent for $(\overset{ˉ}{g}, \overset{ˉ}{Q})$ w.r.t. $∥ \cdot ∥_{0}$ -norm, where $\overset{ˉ}{g} (W) = E_{P_{0}} (A | W^{r})$ for some function $W^{r} (W)$ of W for which $\overset{ˉ}{Q}$ only depends on W through $W^{r}$ , and assume that $P_{0} \frac{\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}}{\overset{ˉ}{g}} (A - \overset{ˉ}{g}) = 0$ , where the latter holds, in particular, if $\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}$ only depends on W through $W^{r}$ (e.g. ${\overset{ˉ}{g}}_{n}^{*}$ involves non-parametric adjustment by $\overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{0}$ ). As a consequence, we have $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}^{r}$ .

Empirical process condition: Assume that $D^{*} (Q_{n}^{*}, g_{n}^{*})$ , $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ fall in a $P_{0}$ -Donsker class with probability tending to 1 as $n \mapsto \infty$ .

Negligibility of second-order terms: Define

{\overset{ˉ}{Q}}_{0, n}^{r} \equiv E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*}) .

Assume that the following conditions hold for each of the following possible definitions of ${\overset{ˉ}{g}}_{0, n}^{r}$ : $E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{n}^{*})$ , $E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*})$ , $E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ . Note that ${\overset{ˉ}{g}}_{0}^{r} = E_{0} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{0} (A | \overset{ˉ}{g}) = \overset{ˉ}{g}$ is the limit of each of these choices for ${\overset{ˉ}{g}}_{0, n}^{r}$ .

We assume $\overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*}$ are bounded away from $δ > 0$ with probability tending to one, and

\begin{matrix} ∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} = o_{P} (1) \\ ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0}^{2} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n}) \\ ∥ {\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} = o_{P} (1 / \sqrt{n}) . \end{matrix}

Then,

Ψ (Q_{n}^{*}) - ψ_{0} = (P_{n} - P_{0}) I C (P_{0}) + o_{P} (1 / \sqrt{n}),

where

I C (P_{0}) = D^{*} (Q, g) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) .

Thus, consistency of this TMLE relies upon the consistency of ${\overset{ˉ}{Q}}_{n}^{r *}$ as an estimator of ${\overset{ˉ}{Q}}_{0}^{r}$ , and estimator $({\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{*})$ converging to a $(\overset{ˉ}{Q}, \overset{ˉ}{g})$ for which $\overset{ˉ}{g}$ equals a true conditional mean of A, given $W^{r}$ , and ${\overset{ˉ}{Q}}_{0} - \overset{ˉ}{Q}, \overset{ˉ}{Q}$ only depend on W through $W^{r}$ . Since ${\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r}$ depends on how well ${\overset{ˉ}{g}}_{n}^{*}$ approximates $\overset{ˉ}{g}$ , ${\overset{ˉ}{Q}}_{n}^{r *} - {\overset{ˉ}{Q}}_{0}^{r}$ depends on how well $({\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{*})$ approximates $(\overset{ˉ}{Q}, \overset{ˉ}{g})$ , beyond the behavior of the non-parametric regression defining ${\overset{ˉ}{Q}}_{n}^{r}$ . In addition, ${\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r}$ depends on either how well ${\overset{ˉ}{g}}_{n}^{*}$ approximates $\overset{ˉ}{g}$ or how well ${\overset{ˉ}{Q}}_{n}^{*}$ approximates $\overset{ˉ}{Q}$ . As a consequence, it follows that each of the second-order terms displayed in the theorem involves square differences of approximation errors ${\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}$ and ${\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}$ .

It is also interesting to note that the algebraic form of the influence curve of this TMLE is identical to the influence curve of the TMLE of Theorem 2 that relied on ${\overset{ˉ}{g}}_{n}^{*}$ being consistent for ${\overset{ˉ}{g}}_{0}$ .

5.4 A C-TMLE algorithm

The TMLE algorithm presented in Theorem 4 maps an initial estimator $(Q_{n}^{0}, g_{n}^{0})$ into an updated estimator $(Q_{n}^{*}, g_{n}^{*})$ that solves the two estimating equations (3), allowing for statistical inference with known influence curve if the initial estimator $(Q_{n}^{0}, g_{n}^{0})$ is collaboratively consistent (i.e. the limits of $(Q_{n}^{*}, g_{n}^{*})$ satisfy the condition in the theorem). The updating algorithm results in a $g_{n}^{*}$ that non-parametrically adjusts for ${\overset{ˉ}{Q}}_{n}^{*}$ itself, and thus for its limit $\overset{ˉ}{Q}$ in the limit. The condition on the limit g was that it should non-parametrically adjust not only for $\overset{ˉ}{Q}$ but also for $\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}$ . If the initial estimator $g_{n}^{0}$ already adjusted for an approximation of ${\overset{ˉ}{Q}}_{n}^{0} - {\overset{ˉ}{Q}}_{0}$ , for example, $(g_{n}^{0}, Q_{n}^{0})$ is already a C-TMLE, then this condition might hold approximately. Nonetheless, we want to present a C-TMLE algorithm that simultaneously fits g in response to $\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}$ , but also carries out the non-parametric adjustment by $\overset{ˉ}{Q}$ . The latter is normally not part of the C-TMLE algorithm, but we want to enforce this in order to be able to apply Theorem 3 and thereby obtain a known influence curve. We achieve this goal in this subsection by applying the C-TMLE algorithm as presented by van der Laan and Gruber [49] and to the particular TMLE algorithm presented in Theorem 4.

First, we compute a set of K univariate covariates $W_{1}, \dots, W_{K}$ , i.e. functions of W, which we will refer to as main terms, even though a term could be an interaction term or a super-learning fit of the regression of A on a subset of the components of W. Let $Ω = {W_{1}, \dots, W_{K}}$ be the full collection of main terms. In the previous subsection, we defined an algorithm that maps an initial $(Q, g)$ into a TMLE $(Q^{*}, g^{*})$ . Let $O \mapsto L (Q) (O)$ be the loss function for $Q_{0}$ .

The general template of a C-TMLE algorithm is the following: given a TMLE algorithm that maps any initial $(Q, g)$ into a TMLE $(Q^{*}, g^{*})$ , the C-TMLE algorithm generates a sequence of increasing sets $S^{k} \subset Ω$ of k main terms, where each set $S^{k}$ has an associated estimator $g^{k}$ of $g_{0}$ , and simultaneously it generates a corresponding sequence of $Q^{k}$ , $k = 1, \dots, K$ , where both $g^{k}$ and $Q^{k}$ are increasingly non-parametric in k. Here increasingly non-parametric means that the empirical mean of the loss function of the fit is decreasing in k. This sequence $(g^{k}, Q^{k})$ maps into a corresponding sequence of TMLEs $(g^{k *}, Q^{k *})$ using the TMLE algorithm presented in Theorem 4. In this variable selection algorithm, the choice of the next main term to add, mapping $S^{k}$ into $S^{k + 1}$ , is based on how much the TMLE using the g-fit implied by $S^{k + 1}$ , using $Q^{k}$ as initial estimator, improves the fit of the corresponding TMLE $Q^{k *}$ for $Q_{0}$ . Cross-validation is used to select k among these candidate TMLEs $Q^{k *}$ , $k = 1, \dots, K$ , where the last TMLE $Q^{K *}$ uses the most aggressive bias reduction by being based on the most non-parametric estimator $g^{K}$ implied by $Ω$ .

In order to present a precise C-TMLE algorithm we will first introduce some notation. For a given subset of main terms $S \subset Ω$ , let $S^{c}$ be its complement within $Ω$ . In the C-TMLE algorithm, we use a forward selection algorithm that augments a given set $S^{k}$ into a next set $S^{k + 1}$ obtained by adding the best main term among all main terms in the complement $S^{k, c}$ of $S^{k}$ . Each choice $S$ corresponds with an estimator of $g_{0}$ . In other words, the algorithm iteratively updates a current estimate $g^{k}$ into a new estimate $g^{k + 1}$ , but the criterion for g does not measure how well g fits $g_{0}$ ; it measures how well the TMLE of $Q_{0}$ that uses this g (and as initial estimator $Q^{k}$ ) fits $Q_{0}$ .

Given a set $S^{k}$ , an initial $g^{k - 1}, Q^{k - 1}$ , we define a corresponding $g^{k}$ obtained by MLE-fitting of $β$ in the logistic regression working model

L o g i t {\overset{ˉ}{g}}^{k} = L o g i t {\overset{ˉ}{g}}_{0}^{r} ({\overset{ˉ}{g}}^{k - 1}, {\overset{ˉ}{Q}}^{k - 1}) + \sum_{j \in S^{k}} β_{j} W_{j},

where we remind the reader of the definition ${\overset{ˉ}{g}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{0} (A | \overset{ˉ}{Q} (W), \overset{ˉ}{g} (W))$ . Thus, this estimator $g^{k}$ involves non-parametric adjustment by ${\overset{ˉ}{g}}^{k - 1}, {\overset{ˉ}{Q}}^{k - 1}$ , augmented with a linear regression component implied by $S^{k}$ . This function mapping $S^{k}, g^{k - 1}, Q^{k - 1}$ into a fit $g^{k}$ will be denoted with $g (S^{k}, g^{k - 1}, Q^{k - 1})$ . This also allows us to define a mapping from $(Q^{k}, S^{k}, Q^{k - 1}, g^{k - 1})$ into a TMLE $(Q^{k *}, g^{k *})$ defined by the TMLE algorithm of Theorem 4 applied to initial $Q^{k}$ and $g^{k} = g (S^{k}, g^{k - 1}, Q^{k - 1})$ . We will denote this mapping into $Q^{k *}$ with $T M L E (Q^{k}, S^{k}, Q^{k - 1}, g^{k - 1})$ .

The C-TMLE algorithm defined below generates a sequence $(Q^{k}, S^{k})$ and thereby corresponding TMLEs $(Q^{k *}, g^{k *})$ , $k = 0, \dots, K$ , where $Q^{k}$ represents an initial estimate, $S^{k}$ a subset of main terms that defines $g^{k}$ , and $Q^{k *}, g^{k *}$ the corresponding TMLE that starts with $(Q^{k}, g^{k})$ . These TMLEs $Q^{k *}$ represent subsequent updates of the initial estimator $Q^{0}$ . The corresponding main term set $S^{k}$ that defines $g^{k}$ in this k-specific TMLE, increases in k, one unit at a time: $S^{0}$ is empty, $| S^{k + 1} | = | S^{k} | + 1$ , $S^{K} = Ω$ . The C-TMLE uses cross-validation to select k, and thereby to select the TMLE $Q^{k *}$ that yields the best fit of $Q_{0}$ among the $K + 1$ k-specific TMLEs $(Q^{k *} : k = 0, \dots, K)$ that are increasingly aggressive in their bias-reduction effort. This C-TMLE algorithm is defined as follows and uses the same format as presented in Wang et al. [35]:

Initiate algorithm: Set initial TMLE. Let $k = 0$ , and $Q^{k} = Q^{0}$ , $g^{s t a r t}$ be initial estimates of $Q_{0}$ , $g_{0}$ , and let $S^{0}$ be the empty set. Let $g^{k} = g (S^{0}, Q^{0}, g^{s t a r t})$ . This defines an initial TMLE

Q^{0 *} = T M L E (Q^{0}, S^{0}, Q^{0}, g^{0}) .

Determine next TMLE. Determine the next best main term to add:

S^{k + 1, c a n d} = arg min_{\{S^{k} \cup \{W_{j}\} : W_{j} \in S^{k, c}\}} P_{n} L (T M L E (Q^{k}, S^{k} \cup \{W_{j}\}, Q^{k - 1}, g^{k - 1})) .

P_{n} L (T M L E (Q^{k}, S^{k + 1, c a n d}, Q^{k - 1}, g^{k - 1})) \leq P_{n} L (Q^{k *}),

then $(S^{k + 1} = S^{k + 1, c a n d}, Q^{k + 1} = Q^{k})$ , else $Q^{k + 1} = Q^{k *}$ , and

S^{k + 1} = arg min_{\{S^{k} \cup \{W_{j}\} : W_{j} \in S^{k, c}\}} P_{n} L (T M L E (Q^{k *}, S^{k} \cup \{W_{j}\}, Q^{k - 1}, g^{k - 1})) .

[In words: If the next best main term added to the fit of $E_{P_{0}} (A | W)$ yields a TMLE of $E_{P_{0}} (Y | A, W)$ that improves upon the previous TMLE $Q^{k *}$ , then we accept this best main term, and we have our next $(Q^{k + 1}, S^{k + 1})$ and corresponding TMLE $Q^{k + 1 *}, g^{k + 1 *}$ (which still uses the same initial estimate of $Q_{0}$ as $Q^{k *}$ uses). Otherwise, reject this best main term, update the initial estimate in the candidate TMLEs to the previous TMLE $Q^{k *}$ of $E_{P_{0}} (Y | A, W)$ , and determine the best main term to add again. This best main term will now always result in an improved fit of the corresponding TMLE of $Q_{0}$ , so that we now have our next TMLE $Q^{k + 1 *}, g^{k + 1}$ (which now uses a different initial estimate than $Q^{k *}$ used).]

Iterate. Run this from $k = 1$ to K at which point $S^{K} = Ω$ . This yields a sequence $(Q^{k}, g^{k})$ and corresponding TMLE $(Q^{k *}, g_{k}^{*})$ , $k = 0, \dots, K$ .

This sequence of candidate TMLEs $Q^{k *}$ of $Q_{0}$ has the following property: the estimates $g^{k}$ are increasingly non-parametric in k and $P_{n} L (Q^{k *})$ is decreasing in k, $k = 0, \dots, K$ . It remains to select k. For that purpose we use V-fold cross-validation. That is, for each of the V splits of the sample in a training and validation sample, we apply the above algorithm for generating a sequence of candidate estimates $(Q^{k *} : k)$ to a training sample, and we evaluate the empirical mean of the loss function at the resulting $Q^{k *}$ over the validation sample, for each $k = 0, \dots, K$ . For each k we take the average over the V splits of the k-specific performance measure over the validation sample, which is called the cross-validated risk of the k-specific TMLE. We select the k that has the best cross-validated risk, which we denote with $k_{n}$ . Our final C-TMLE of $Q_{0}$ is now defined as $Q_{n}^{*} = Q^{k_{n} *}$ , and the TMLE of $ψ_{0}$ is defined as $ψ_{n}^{*} = Ψ (Q_{n}^{*})$ .

Fast version of above C-TMLE: We could carry out the above C-TMLE algorithm but replacing the TMLE that maps an initial $(Q, g)$ into $(Q^{*}, g^{*})$ replaced by the first step of the TMLE that maps $(Q, g)$ into $(Q^{1}, g^{1})$ . In that manner, the selection of the sets $S^{k}$ is based on the bias reduction achieved in a first step of the TMLE algorithm, and most bias reduction occurs in the first step. After having selected the final one-step TMLE $Q^{k_{n} 1}$ and corresponding $g^{k_{n}}$ , one should still carry out the full TMLE algorithm so that the final $Q_{n}^{*} = Q^{k_{n} *}, g^{k_{n} *}$ is a real TMLE solving the estimating equations of Theorem 4.

Statistical inference for C-TMLE: Let ${\overset{ˉ}{Q}}_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*}, {\overset{ˉ}{Q}}_{n}^{*})$ be the final estimator of ${\overset{ˉ}{Q}}_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} (\overset{ˉ}{g}, \overset{ˉ}{Q}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ , a by-product of the TMLE algorithm. An estimate of the influence curve of $ψ_{n}^{*}$ is given by

I C_{n} = D^{*} (Q_{n}^{*}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) .

The asymptotic variance of $\sqrt{n} (ψ_{n}^{*} - ψ_{0})$ can thus be estimated with $σ_{n}^{2} = 1 / n \sum_{i = 1}^{n} I C_{n} (O_{i})^{2}$ . An asymptotically valid 0.95-confidence interval for $ψ_{0}$ is given by $ψ_{n}^{*} \pm 1.96 σ_{n}^{*} / \sqrt{n}$ .

6 Discussion

Targeted minimum loss-based estimation allows us to construct plug-in estimators $Ψ (Q_{n}^{*})$ of a path-wise differentiable parameter $Ψ (Q_{0})$ utilizing the state of the art in ensemble learning such as super-learning, while guaranteeing that the estimator $Q_{n}^{*}$ and an estimator $g_{n}^{*}$ of the nuisance parameter the TMLE utilizes in its targeting step solve a set of user-supplied estimating equations, empirical means of estimating functions. These estimating functions can be selected so that the resulting TMLE of $ψ_{0}$ has certain statistical properties such as being efficient, or guaranteed to be more efficient than a given user-supplied estimator [28, 29], and so on. However, most importantly, these estimating equations are necessary to make the TMLE asymptotically linear, i.e. to make the TMLE unbiased enough so that the first-order linear expansion can be used for statistical inference. For example, by selecting the estimating functions to be equal to the canonical gradient of $Ψ : M \mapsto I R$ one arranges that $Ψ (Q_{n}^{*})$ is asymptotically efficient under conditions that assume consistency of $Q_{n}^{*}$ and $g_{n}^{*}$ .

However, we noted that this level of targeting is insufficient if one only relies on consistency of $g_{n}^{*}$ , even when that suffices for consistency of $Ψ (Q_{n}^{*})$ . Under such weaker assumptions, additional targeting is necessary so that a specific smooth functional of $g_{n}^{*}$ is asymptotically linear, which requires that an unknown smooth function of $g_{n}^{*}$ is itself a TMLE. The joint targeting of $Q_{n}^{*}$ and $g_{n}^{*}$ is achieved by a TMLE that also solves the extra equations making this smooth function of $g_{n}^{*}$ asymptotically linear, allowing one to establish asymptotic linearity of $Ψ (Q_{n}^{*})$ under milder conditions that assume that the second-order terms are negligible relative to the first-order linear approximation.

In this article we also pushed this additional level of targeting to a new level by demonstrating how it allows for double robust statistical inference, and that even if we estimate the nuisance parameter in a complicated manner that is based on a criterion that cares about how it helps the estimator to fit $ψ_{0}$ , as used by the C-TMLE, we can still determine a set of additional estimating equations that need to be targeted by the TMLE in order to establish asymptotic linearity and thereby valid statistical inference based on the central limit theorem. This allows us now to use the sophisticated but often necessary C-TMLE while still preserving valid statistical inference under regularity conditions.

It remains to evaluate the practical benefit of the modifications of IPTW, TMLE, and C-TMLE as presented in this article for both estimation and assessment of uncertainty. We plan to address this in future research.

Even though we focussed in this article on a particular concrete estimation problem, TMLE is a general tool and our TMLE and theorems can be generalized to general statistical models and path-wise differentiable statistical target parameters.

We note that this targeting of nuisance parameter estimators in the TMLE is not only necessary to get a known influence curve but also necessary to make the TMLE asymptotically linear. So it does not simply suffice to run a bootstrap as an alternative of influence curve based inference, since the bootstrap can only work if the estimator is asymptotically linear so that it has an existing limit distribution. In addition, the established asymptotic linearity with known influence curve has the important by-product that one now obtains statistical inference with no extra computational cost. This is particularly important in these large semi-parametric models that require the utilization of aggressive machine learning methods in order to cover the model-space, making the estimators by necessity very computer intensive, so that a (disputable) bootstrap method might simply be too computer extensive.

Acknowledgments

This research was supported by an NIH grant R01 AI074345-06. The author is grateful for the excellent, helpful, and insightful comments of the reviewers.

Appendix

Proof of Theorem 1

To start with we note:

\begin{matrix} P_{n} D (g_{n}^{*}) - P_{0} D (g_{0}) = (P_{n} - P_{0}) D (g_{0}) + P_{n} (D (g_{n}^{*}) - D (g_{0})) \\ = (P_{n} - P_{0}) (D (g_{0}) - ψ_{0}) + P_{0} (D (g_{n}^{*}) - D (g_{0})) + (P_{n} - P_{0}) (D (g_{n}^{*}) - D (g_{0})) . \end{matrix}

The first term of this decomposition yields the first component $D (g_{0}) - ψ_{0}$ of the influence curve. Since $g_{n}^{*}$ falls in Donsker class the rightmost term is $o_{P} (1 / \sqrt{n})$ if $P_{0} (D (g_{n}^{*}) - D (g_{0}))^{2} \mapsto 0$ in probability. So it remains to analyze the term $P_{0} (D (g_{n}^{*}) - D (g_{0}))$ . We now note

\begin{matrix} P_{0} (D (g_{n}^{*}) - D (g_{0})) = P_{0} Y A \{1 / g_{n}^{*} - 1 / g_{0}\} = P_{0} Y A \{(g_{0} - g_{n}^{*}) / (g_{n}^{*} g_{0})\} \\ = P_{0} Y A \{g_{0} - g_{n}^{*}\} / g_{0}^{2} + P_{0} Y A {(g_{0} - g_{n}^{*})}^{2} / (g_{0}^{2} g_{n}^{*}) . \end{matrix}

By our assumptions, the last term

P_{0} Y A (g_{0} - g_{n}^{*})^{2} / g_{0}^{2} g_{n}^{*} = P_{0} {\overset{ˉ}{Q}}_{0} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0})^{2} / ({\overset{ˉ}{g}}_{0} {\overset{ˉ}{g}}_{n}^{*}) = o_{P} (1 / \sqrt{n}) .

So it remains to study:

P_{0} Y A \{g_{0} - g_{n}^{*}\} / g_{0}^{2} = P_{0} {\overset{ˉ}{Q}}_{0} ({\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}^{*}) / {\overset{ˉ}{g}}_{0} .

Note that this equals $- {Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0})}$ , where $Ψ_{1} (g) = P_{0} \frac{{\overset{ˉ}{Q}}_{0}}{{\overset{ˉ}{g}}_{0}} \overset{ˉ}{g}$ is an unknown smooth parameter of g. Our strategy is to first approximate this parameter by an easier (still unknown) parameter $Ψ_{1}^{r} (g) = P_{0} {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0} \overset{ˉ}{g}$ resulting in a second-order term: $Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) = Ψ_{1}^{r} (g_{n}^{*}) - Ψ_{1}^{r} (g_{0}) + o_{P} (1 / \sqrt{n})$ . This is carried out in the next lemma. The efficient influence curve of a target parameter $Φ : \overset{ˉ}{g} \mapsto P_{0} H \overset{ˉ}{g}$ (which treats $P_{0}$ as known) at $g_{0}$ is given by $H (A - {\overset{ˉ}{g}}_{0})$ . Thus, one likes to construct ${\overset{ˉ}{g}}_{n}^{*}$ so that it solves the empirical mean of $H_{0}^{r} (A - {\overset{ˉ}{g}}_{n}^{*})$ for $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}$ , so that ${\overset{ˉ}{g}}_{n}^{*}$ targets the parameter $Ψ_{1}^{r} (g_{0})$ . However, $H_{0}^{r}$ is unknown. Therefore, instead ${\overset{ˉ}{g}}_{n}^{*}$ is constructed to solve the empirical mean of an estimate $H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*})$ of the efficient influence curve $H_{0}^{r} (A - {\overset{ˉ}{g}}_{n}^{*})$ , and we will show that this indeed suffices to establish the asymptotic linearity of $Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*})$ .

Lemma 2Define $Ψ_{1} (g) = P_{0} \frac{{\overset{ˉ}{Q}}_{0}}{{\overset{ˉ}{g}}_{0}} \overset{ˉ}{g}$ , $Ψ_{1}^{r} (g) = P_{0} \frac{{\overset{ˉ}{Q}}_{0}^{r}}{{\overset{ˉ}{g}}_{0}} \overset{ˉ}{g}$ , ${\overset{ˉ}{Q}}_{0, n}^{r} \equiv E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{0} (W), {\overset{ˉ}{g}}_{n}^{*} (W))$ , and ${\overset{ˉ}{Q}}_{0}^{r} = E_{P_{0}} (Y | A = 1, {\overset{ˉ}{g}}_{0} (W))$ , where ${\overset{ˉ}{g}}_{n}^{*} (W)$ is treated as a fixed function of W when calculating the conditional expectation. Assume

R_{1, n} \equiv P_{0} ({\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0} = o_{P} (1 / \sqrt{n}) .

Then,

Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) = Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) + R_{1, n} .

Proof of Lemma 2: Note that

\begin{aligned} Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) & = P_{0} Y A \{g_{n}^{*} - g_{0}\} / g_{0}^{2} \\ = P_{0} {\overset{ˉ}{Q}}_{0, n}^{r} A \{g_{n}^{*} - g_{0}\} / g_{0}^{2} \\ = P_{0} {\overset{ˉ}{Q}}_{0, n}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0} \\ = P_{0} {\overset{ˉ}{Q}}_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0} + P_{0} ({\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0} . □ \end{aligned}

Since we assumed $R_{1, n} = o_{P} (1 / \sqrt{n})$ , it remains to prove that $Ψ_{1}^{r} (g_{n}^{*}) - Ψ_{1}^{r} (g_{0}) = P_{0} {\overset{ˉ}{Q}}_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) / {\overset{ˉ}{g}}_{0}$ is asymptotically linear. Recall that $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}$ , and $H_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} / {\overset{ˉ}{g}}_{n}^{*}$ , where ${\overset{ˉ}{Q}}_{n}^{r *}$ is obtained by regressing Y on the initial estimator ${\overset{ˉ}{g}}_{n} (W)$ and $A = 1$ .

The next step of the proof is the following series of equalities

\begin{aligned} P_{0} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = & P_{0} (D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*})) + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \\ = & \int (H_{n}^{r *} - H_{0}^{r}) (W) (A - {\overset{ˉ}{g}}_{n}^{*} (W)) d P_{0} (W, A) + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \\ = & \int (H_{n}^{r *} - H_{0}^{r}) (W) ({\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}^{*}) (W) d P_{0} (W) + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \\ = & R_{2, n} + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}), \end{aligned}

where, by assumption, $R_{2, n} = o_{P} (1 / \sqrt{n})$ . We now note that

\begin{aligned} P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) = & \int H_{0}^{r} (W) (A - {\overset{ˉ}{g}}_{n}^{*} (W)) d P_{0} (A, W) \\ = & \int H_{0}^{r} (W) {\overset{ˉ}{g}}_{0} (W) d P_{0} (W) - \int H_{0}^{r} (W) {\overset{ˉ}{g}}_{n}^{*} (W) d P_{0} (W) \\ \equiv & Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) - Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) . \end{aligned}

Thus, we have

- P_{0} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) - R_{2, n},

from which we deduce that, by Lemma 2 and $P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = 0$ , that

\begin{aligned} Ψ_{1} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1} ({\overset{ˉ}{g}}_{0}) & = - P_{0} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) + R_{1, n} + R_{2, n} \\ = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) + R_{1, n} + R_{2, n} \\ = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0}) + R_{1, n} + R_{2, n} + R_{3, n}, \end{aligned}

where we defined

R_{3, n} = (P_{n} - P_{0}) (D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0})) .

By our assumptions, $R_{3, n} = o_{P} (1 / \sqrt{n})$ , so that it follows that $Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0}) + o_{P} (1 / \sqrt{n})$ . □

Proof of Theorem 2

One easily checks that

\begin{aligned} Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = & - P_{0} D^{*} (Q_{n}^{*}, g_{0}) \\ = & - P_{0} D^{*} (Q_{n}^{*}, g_{n}^{*}) + P_{0} (D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q_{n}^{*}, g_{0})) \\ = & (P_{n} - P_{0}) D^{*} (Q_{n}^{*}, g_{n}^{*}) + P_{0} \{D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q_{n}^{*}, g_{0})\}, \end{aligned}

because $P_{n} D^{*} (Q_{n}^{*}, g_{n}^{*}) = 0$ by eq. (2). If $D^{*} (Q_{n}^{*}, g_{n}^{*})$ falls in a $P_{0}$ -Donsker class and $P_{0} {D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q, g_{0})}^{2} = o_{P} (1)$ for some possibly misspecified limit Q of $Q_{n}^{*}$ , then the first term on the right-hand side equals $(P_{n} - P_{0}) D^{*} (Q, g_{0}) + o_{P} (1 / \sqrt{n})$ , giving us the first component $D^{*} (Q, g_{0})$ of the influence curve of $Ψ (Q_{n}^{*})$ . The second term can be written as $A + B$ with

\begin{matrix} A = P_{0} (\{D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q_{n}^{*}, g_{0})\} - \{D^{*} (Q, g_{n}^{*}) - D^{*} (Q, g_{0})\}) \\ B = P_{0} \{D^{*} (Q, g_{n}^{*}) - D^{*} (Q, g_{0})\} . \end{matrix}

The first term A equals

- P_{0} (H_{Y} (g_{n}^{*}) - H_{Y} (g_{0})) ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}),

where $H_{Y} (g) (A, W) = A / \overset{ˉ}{g} (W)$ . By our assumptions, this term is $o_{P} (1 / \sqrt{n})$ . Thus, it suffices to establish asymptotic linearity of $Ψ_{1} (g_{n}^{*}) = P_{0} D^{*} (Q, g_{n}^{*})$ as an estimator of $Ψ_{1} (g_{0}) = P_{0} D^{*} (Q, g_{0})$ . We have

\begin{matrix} Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) = - P_{0} (Y - \overset{ˉ}{Q}) \frac{A}{{\overset{ˉ}{g}}_{n}^{*} {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} {\overset{ˉ}{Q}}_{0, n}^{r} \frac{A}{{\overset{ˉ}{g}}_{n}^{*} {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} {\overset{ˉ}{Q}}_{0, n}^{r} \frac{1}{{\overset{ˉ}{g}}_{n}^{*}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}), \end{matrix}

where ${\overset{ˉ}{Q}}_{0, n}^{r}$ appeared by writing the expectation w.r.t. $P_{0}$ as an expectation of the conditional expectation, given $A, {\overset{ˉ}{g}}_{n}^{*} (W), {\overset{ˉ}{g}}_{0} (W)$ . Let $H_{0, n}^{r} = {\overset{ˉ}{Q}}_{0, n}^{r} / {\overset{ˉ}{g}}_{n}^{*}$ and recall $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}$ , where ${\overset{ˉ}{Q}}_{0}^{r} = E_{P_{0}} (Y - \overset{ˉ}{Q} (W) | A = 1, {\overset{ˉ}{g}}_{0} (W))$ . The last term can be written as

- P_{0} H_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) - P_{0} (H_{0, n}^{r} - H_{0}^{r}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) .

By our assumptions, the second term above is $o_{P} (1 / \sqrt{n})$ . Thus, in order to establish asymptotic linearity of $Ψ_{1} (g_{n}^{*})$ , it suffices to establish asymptotic linearity of $Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) = P_{0} H_{0}^{r} {\overset{ˉ}{g}}_{n}^{*}$ as an estimator of $Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) = P_{0} H_{0}^{r} {\overset{ˉ}{g}}_{0}$ , where $P_{0}$ and $H_{0}^{r}$ are treated as known.

The estimator ${\overset{ˉ}{g}}_{n}^{*}$ was constructed to target $P_{0} H_{n}^{r *} {\overset{ˉ}{g}}_{0}$ instead where we recall that $H_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} / {\overset{ˉ}{g}}_{n}^{*}$ . That is, our targeted estimator $g_{n}^{*}$ solves the efficient influence curve equation $P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = 0$ for the parameter $P_{0} H_{n}^{r *} {\overset{ˉ}{g}}_{0}$ of ${\overset{ˉ}{g}}_{0}$ . We now note that

\begin{aligned} P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) = & \int H_{0}^{r} (W) (A - {\overset{ˉ}{g}}_{n}^{*} (W)) d P_{0} (A, W) \\ = & \int H_{0}^{r} (W) {\overset{ˉ}{g}}_{0} (W) d P_{0} (W) - \int H_{0}^{r} (W) {\overset{ˉ}{g}}_{n}^{*} (W) d P_{0} (W) \\ \equiv & Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) - Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) . \end{aligned}

We have

\begin{matrix} P_{0} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = P_{0} {D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*})} + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \\ = \int (H_{n}^{r *} - H_{0}^{r}) (W) (A - {\overset{ˉ}{g}}_{n}^{*} (W)) d P_{0} (W, A) + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \\ = \int (H_{n}^{r *} - H_{0}^{r}) (W) ({\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}^{*} (W)) d P_{0} (W) + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}) \equiv R_{2, n} + P_{0} D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{n}^{*}), \end{matrix}

where $R_{2, n} = o_{P} (1 / \sqrt{n})$ , by assumption. Combining the last two equations yields:

\begin{aligned} Ψ_{1}^{r} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1}^{r} ({\overset{ˉ}{g}}_{0}) & = - P_{0} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - R_{2, n} \\ = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - R_{2, n} \\ = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0}) - R_{2, n} + R_{3, n}, \end{aligned}

where we defined

R_{3, n} = (P_{n} - P_{0}) (D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0})) .

We have that $R_{3, n} = o_{P} (1 / \sqrt{n})$ if $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0})$ falls in a $P_{0}$ -Donsker class with probability tending to 1, and $P_{0} {D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0})}^{2} \mapsto 0$ in probability when $n \mapsto \infty$ . Thus, we have proven that $Ψ_{1}^{r} (g_{n}^{*}) - Ψ_{1}^{r} (g_{0}) = (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0}) + o_{P} (1 / \sqrt{n})$ . Thus,

Ψ_{1} (g_{n}^{*}) - Ψ_{1} (g_{0}) = - (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, {\overset{ˉ}{g}}_{0}) + o_{P} (1 / \sqrt{n}) . □

Proof of Theorem 3

As outlined in Section 1, we have

\begin{matrix} Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = - P_{0} D^{*} (Q_{n}^{*}, g_{n}^{*}) + P_{0} ({\overset{ˉ}{Q}}_{0} - {\overset{ˉ}{Q}}_{n}^{*}) \frac{{\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}^{*}}{{\overset{ˉ}{g}}_{n}^{*}} \\ = (P_{n} - P_{0}) D^{*} (Q_{n}^{*}, g_{n}^{*}) + P_{0} ({\overset{ˉ}{Q}}_{0} - {\overset{ˉ}{Q}}_{n}^{*}) \frac{{\overset{ˉ}{g}}_{0} - {\overset{ˉ}{g}}_{n}^{*}}{{\overset{ˉ}{g}}_{n}^{*}} \\ = (P_{n} - P_{0}) D^{*} (Q, g) + P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} + o_{P} (1 / \sqrt{n}), \end{matrix}

if $D^{*} (Q_{n}^{*}, g_{n}^{*})$ falls in a Donsker class with probability tending to 1, and $P_{0} {D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q, g)}^{2} \mapsto 0$ in probability as $n \mapsto \infty$ . The first term on right-hand side gives us the first component $D^{*} (Q, g)$ of the influence curve of $ψ_{n}^{*}$ .

It suffices to analyze the second term. Initially, we note that

P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} + R_{1, n},

where

R_{1, n} = - P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{\overset{ˉ}{g} {\overset{ˉ}{g}}_{n}^{*}} .

By assumption, $R_{1, n} = o_{P} (1 / \sqrt{n})$ .

Now, we note

\begin{matrix} P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} + \overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} + \overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} \\ = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{\overset{ˉ}{g}} + P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} \\ + P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{\overset{ˉ}{g}} + P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} . \end{matrix}

By our assumptions, the first term $R_{2, n} = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{\overset{ˉ}{g}}$ satisfies $R_{2, n} = o_{P} (1 / \sqrt{n})$ . In addition, the last term equals zero by assumption: $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ or $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ .

So it suffices to analyze the second and third terms of this last expression. In order to represent the second and third terms we define

\begin{matrix} Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{Q}}_{n}^{*}) = P_{0} {\overset{ˉ}{Q}}_{n}^{*} \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} \\ Ψ_{1, \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{0}} ({\overset{ˉ}{g}}_{n}^{*}) = P_{0} \frac{\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}}{\overset{ˉ}{g}} {\overset{ˉ}{g}}_{n}^{*} . \end{matrix}

The sum of the second and third terms can now be represented as:

\begin{matrix} I (\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}) \{Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} (\overset{ˉ}{Q})\} \\ + I (\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}) \{Ψ_{1, \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{0}} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1, \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{0}} (\overset{ˉ}{g})\} . \end{matrix}

For notational convenience, we will suppress the dependence of these mappings on the unknown quantities, and thus use $Ψ_{1}, Ψ_{2}$ .

Analysis of $Ψ_{1} ({\overset{ˉ}{g}}_{n}^{*})$ if $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ : Recalling the definition ${\overset{ˉ}{Q}}_{0, n}^{r}$ , we have

\begin{matrix} Ψ_{1} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1} (\overset{ˉ}{g}) = P_{0} \frac{\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} (Y - \overset{ˉ}{Q}) \frac{A}{{\overset{ˉ}{g}}_{0}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} \frac{E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, {\overset{ˉ}{g}}_{0}, {\overset{ˉ}{g}}_{n}^{*})}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} \frac{{\overset{ˉ}{Q}}_{0, n}^{r}}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = - P_{0} \frac{{\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r}}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) - P_{0} \frac{{\overset{ˉ}{Q}}_{0}^{r}}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) . \end{matrix}

By our assumptions,

R_{3, n} \equiv P_{0} \frac{{\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r}}{{\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) = o_{P} (1 / \sqrt{n}),

so that it remains to analyze $- P_{0} H_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0})$ , where $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / {\overset{ˉ}{g}}_{0}$ . Let $H_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} / {\overset{ˉ}{g}}_{n}^{*}$ , and recall that by construction $P_{n} D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = P_{n} H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) = 0$ . We then proceed as follows:

\begin{aligned} P_{0} H_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ = P_{0} H_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) + P_{0} (H_{0}^{r} - H_{n}^{r *}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) \\ \equiv P_{0} H_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) + R_{4, n}, \end{aligned}

where, by our assumptions,

\begin{matrix} R_{4, n} = P_{0} (H_{n}^{r *} - H_{0}^{r}) ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) = o_{P} (1 / \sqrt{n}) . \end{matrix}

In addition,

\begin{matrix} P_{0} H_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}) = - P_{0} H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) \\ = (P_{n} - P_{0}) H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) \\ = (P_{n} - P_{0}) H_{0}^{r} (A - {\overset{ˉ}{g}}_{0}) + R_{4, n} + R_{5, n}, \end{matrix}

where $R_{5, n} = o_{P} (1 / \sqrt{n})$ if $P_{0} {D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}}^{2} = o_{P} (1)$ and $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ falls in a Donsker class with probability tending to 1. This proves that, if $\overset{ˉ}{g} = {\overset{ˉ}{g}}_{0}$ , then $Ψ_{1} ({\overset{ˉ}{g}}_{n}^{*}) - Ψ_{1} ({\overset{ˉ}{g}}_{0}) = - (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) + o_{P} (1 / \sqrt{n})$ .

Analysis of $Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*})$ if $\overset{ˉ}{Q} = {\overset{ˉ}{Q}}_{0}$ : Recall the definitions of $H_{Y} ({\overset{ˉ}{g}}^{r}, \overset{ˉ}{g})$ , ${\overset{ˉ}{g}}_{0, n}^{r} = E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{*}, \overset{ˉ}{Q})$ , ${\overset{ˉ}{g}}_{n}^{r *}$ (an estimator of ${\overset{ˉ}{g}}_{0}^{r} = E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ ), and that, by construction, $P_{n} H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) = 0$ . We have

\begin{aligned} Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2} ({\overset{ˉ}{Q}}_{0}) = & P_{0} \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \\ = & - P_{0} \frac{A - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \\ = & - P_{0} \frac{{\overset{ˉ}{g}}_{0, n}^{r} - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \\ = & P_{0} \frac{A}{{\overset{ˉ}{g}}_{0, n}^{r}} \frac{{\overset{ˉ}{g}}_{0, n}^{r} - \overset{ˉ}{g}}{\overset{ˉ}{g}} (Y - {\overset{ˉ}{Q}}_{n}^{*}) \\ = & P_{0} H_{Y} ({\overset{ˉ}{g}}_{0, n}^{r}, \overset{ˉ}{g}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) \\ = & P_{0} H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) \\ + P_{0} {H_{Y} ({\overset{ˉ}{g}}_{0, n}^{r}, \overset{ˉ}{g}) ({\overset{ˉ}{Q}}_{0} - {\overset{ˉ}{Q}}_{n}^{*}) - H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) ({\overset{ˉ}{Q}}_{0} - {\overset{ˉ}{Q}}_{n}^{*})} \end{aligned}

Here we used that ${\overset{ˉ}{g}}_{0}$ is a conditional expectation of A allowing us to first replace ${\overset{ˉ}{g}}_{0}$ by A and then retake the conditional expectation but now only conditioning on what is needed to fix all other terms within expectation w.r.t. $P_{0}$ . As a result of this trick, we were able to replace the hard to estimate ${\overset{ˉ}{g}}_{0}$ that conditions on all of W by the easier ${\overset{ˉ}{g}}_{0, n}^{r}$ . Similarly, we used this to replace ${\overset{ˉ}{Q}}_{0}$ by Y. The last term is a second-order term involving square differences $({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})$ and $({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) ({\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{n}^{r *})$ . By our assumptions, this last term is $o_{P} (1 / \sqrt{n})$ . We now proceed as follows:

\begin{matrix} - P_{0} H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) = (P_{n} - P_{0}) H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) \\ = (P_{n} - P_{0}) H_{Y} ({\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) (Y - \overset{ˉ}{Q}) + o_{P} (1 / \sqrt{n}), \end{matrix}

where we assumed that $H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*})$ falls in a Donsker class with probability tending to 1, and

P_{0} {\{H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) (Y - {\overset{ˉ}{Q}}_{n}^{*}) - H_{Y} ({\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) (Y - \overset{ˉ}{Q})\}}^{2} \mapsto 0,

in probability. This proves $Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2} ({\overset{ˉ}{Q}}_{0}) = - (P_{n} - P_{0}) D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) + o_{P} (1 / \sqrt{n})$ . □

Proof of Theorem 4

As in the proof of previous theorem, we start with

\begin{matrix} Ψ (Q_{n}^{*}) - Ψ (Q_{0}) = (P_{n} - P_{0}) D^{*} (Q, g) + P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} + o_{P} (1 / \sqrt{n}), \end{matrix}

where we use that $D^{*} (Q_{n}^{*}, g_{n}^{*})$ falls in a Donsker class with probability tending to 1, and $P_{0} {D^{*} (Q_{n}^{*}, g_{n}^{*}) - D^{*} (Q, g)}^{2} \mapsto 0$ in probability as $n \mapsto \infty$ . The first term yields the first component $D^{*} (Q, g)$ of the influence curve of $ψ_{n}^{*}$ .

As in the proof of previous theorem, we decompose this second term as follows:

\begin{matrix} P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} + \overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} + \overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} \\ = P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{{\overset{ˉ}{g}}_{n}^{*}} + P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}} \\ + P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{{\overset{ˉ}{g}}_{n}^{*}} + P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{{\overset{ˉ}{g}}_{n}^{*}}, \end{matrix}

resulting in four terms, which we will denote with Terms 1–4. We will now analyze these four terms.

Term 1: The first term $P_{0} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \frac{{\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}}{\overset{ˉ}{g}} = o_{P} (1 / \sqrt{n})$ , by assumption.

Term 4: Due to our assumption that $P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) / \overset{ˉ}{g} = 0$ this last term equals:

\begin{matrix} - P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) \frac{({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})}{{\overset{ˉ}{g}}_{n}^{*} \overset{ˉ}{g}} \\ = - P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) \frac{({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})}{{\overset{ˉ}{g}}^{2}} + R_{1, n}, \end{matrix}

where, by assumption,

R_{1, n} = - P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) \frac{({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})^{2}}{{\overset{ˉ}{g}}^{2} {\overset{ˉ}{g}}_{n}^{*}} = o_{P} (1 / \sqrt{n}) .

We proceed as follows:

\begin{aligned} - P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) \frac{({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})}{{\overset{ˉ}{g}}^{2}} \\ = - P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) (\overset{ˉ}{g} - A) \frac{({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})}{{\overset{ˉ}{g}}^{2}} \\ = - P_{0} \frac{(\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0})}{\overset{ˉ}{g}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) + P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{A}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) . \end{aligned}

The first term is asymptotically equivalent with minus Term 3, which shows that Term 3 is canceled out by a component of Term 4 up till a second-order term that is $o_{P} (1 / \sqrt{n})$ , by assumption. The second term equals

\begin{matrix} P_{0} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) \frac{A}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) = - P_{0} (Y - \overset{ˉ}{Q}) \frac{A}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) \\ = - P_{0} E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*}) \frac{A}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) \\ = - P_{0} E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*}) \frac{{\overset{ˉ}{g}}_{0, n}^{r}}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) \\ = - P_{0} E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}) \frac{{\overset{ˉ}{g}}_{0}^{r}}{{\overset{ˉ}{g}}^{2}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) - P_{0} (H_{1} ({\overset{ˉ}{g}}_{n}^{*}) - H_{1} (\overset{ˉ}{g})) ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}), \end{matrix}

where $H_{1} ({\overset{ˉ}{g}}_{n}^{*}) \equiv E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*}) \frac{{\overset{ˉ}{g}}_{0, n}^{r}}{{\overset{ˉ}{g}}^{2}}$ approximates $H_{1} (\overset{ˉ}{g}) = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}) \frac{{\overset{ˉ}{g}}_{0}^{r}}{{\overset{ˉ}{g}}^{2}}$ , ${\overset{ˉ}{g}}_{0, n}^{r} = E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*})$ , and ${\overset{ˉ}{g}}_{0}^{r} = E_{P_{0}} (A | \overset{ˉ}{g})$ . Let ${\overset{ˉ}{Q}}_{0, n}^{r} = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{n}^{*})$ and ${\overset{ˉ}{Q}}_{0}^{r} = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ . We assumed $∥ {\overset{ˉ}{g}}_{0, n}^{r} - {\overset{ˉ}{g}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n})$ , and $∥ {\overset{ˉ}{Q}}_{0, n}^{r} - {\overset{ˉ}{Q}}_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} = o_{P} (1 / \sqrt{n})$ , which implies that

R_{2, n} = P_{0} (H_{1} ({\overset{ˉ}{g}}_{n}^{*}) - H_{1} (\overset{ˉ}{g})) ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) = o_{P} (1 / \sqrt{n}) .

By assumption, $E_{0} (A | W_{1})$ for some $W_{1}$ that is a function of W. Therefore, ${\overset{ˉ}{g}}_{0}^{r} (W) = E_{P_{0}} (A | \overset{ˉ}{g} (W)) = E_{P_{0}} (E_{P_{0}} (A | W_{1}) | \overset{ˉ}{g} (W)) = \overset{ˉ}{g} (W)$ . Thus, it remains to analyze

(4)

- P_{0} \frac{E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})}{\overset{ˉ}{g}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) .

This term is analyzed below and it is shown that this term equals

- (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) + o_{P} (1 / \sqrt{n}) .

To conclude, we have then shown that the fourth term equals the latter expression minus the third term.

We now analyze (4) which can be represented as $- P_{0} \frac{{\overset{ˉ}{Q}}_{0}^{r}}{\overset{ˉ}{g}} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g})$ , where ${\overset{ˉ}{Q}}_{0}^{r} = E_{P_{0}} (Y - \overset{ˉ}{Q} | A = 1, \overset{ˉ}{g})$ . In this proof, we will use the notation $H_{0}^{r} = {\overset{ˉ}{Q}}_{0}^{r} / \overset{ˉ}{g}$ , $H_{n}^{r *} = {\overset{ˉ}{Q}}_{n}^{r *} / {\overset{ˉ}{g}}_{n}^{*}$ . Since $\overset{ˉ}{g} (W) = E_{0} (A | W_{1})$ for some $W_{1}$ , and ${\overset{ˉ}{Q}}_{0}^{r}$ is thus also a function of $W_{1}$ , we have

P_{0} \frac{{\overset{ˉ}{Q}}_{0}^{r}}{\overset{ˉ}{g}} \overset{ˉ}{g} = P_{0} \frac{{\overset{ˉ}{Q}}_{0}^{r}}{\overset{ˉ}{g}} A .

We now proceed as follows:

\begin{matrix} - P_{0} H_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) = - P_{0} H_{0}^{r} ({\overset{ˉ}{g}}_{n}^{*} - A) \\ = - P_{0} H_{n}^{r *} ({\overset{ˉ}{g}}_{n}^{*} - A) - P_{0} (H_{0}^{r} - H_{n}^{r *}) ({\overset{ˉ}{g}}_{n}^{*} - A) \\ = P_{0} H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) - P_{0} (H_{0}^{r} - H_{n}^{r *}) ({\overset{ˉ}{g}}_{n}^{*} - E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})) . \end{matrix}

For the second term $R_{4, n}$ , we can substitute

{\overset{ˉ}{g}}_{n}^{*} - E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = ({\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g}) + E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{0}^{r}) - E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}),

by noting that $\overset{ˉ}{g} = E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{0}^{r})$ . Thus, this second term results in two terms, one that can be bounded by $∥ H_{n}^{r *} - H_{0}^{r} ∥_{0} ∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0}$ and the other is bounded by

∥ H_{n}^{r *} - H_{0}^{r} ∥_{0} ∥ E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{0}^{r}) - E_{0} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) ∥_{0} .

By assumption, both terms are $o_{P} (1 / \sqrt{n})$ and thus $R_{4, n} = o_{P} (1 / \sqrt{n})$ .

Since, by construction of $g_{n}^{*}$ , $P_{n} H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) = 0$ , the first term can be written as follows:

\begin{matrix} P_{0} H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) \\ = - (P_{n} - P_{0}) H_{n}^{r *} (A - {\overset{ˉ}{g}}_{n}^{*}) \\ = - (P_{n} - P_{0}) H_{0}^{r} (A - \overset{ˉ}{g}) + R_{5, n}, \end{matrix}

where $R_{5, n} = o_{P} (1 / \sqrt{n})$ if $P_{0} {D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g})}^{2} = o_{P} (1)$ and $D_{A} ({\overset{ˉ}{Q}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ falls in a Donsker class with probability tending to 1, and we are reminded that $D_{A} (\overset{ˉ}{g}, {\overset{ˉ}{Q}}_{0}^{r}) = H_{0}^{r} (A - \overset{ˉ}{g})$ . This completes the proof for the fourth term.

Term 3: Our analysis of Term 4 showed that Term 3 cancels out and thus that the sum of the third and fourth terms equals $- (P_{n} - P_{0}) D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) + o_{P} (1 / \sqrt{n})$ , which yields the second component $- D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g})$ of the influence curve of $ψ_{n}^{*}$ .

Analysis of Term 2: Up till a second-order term that can be bounded by $∥ {\overset{ˉ}{g}}_{n}^{*} - \overset{ˉ}{g} ∥_{0} ∥ {\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q} ∥_{0} = o_{P} (1 / \sqrt{n}$ , we can represent Term 2 as

\begin{matrix} \{Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} (\overset{ˉ}{Q})\} . \end{matrix}

where

\begin{matrix} Ψ_{2, \overset{ˉ}{g}, {\overset{ˉ}{g}}_{0}} ({\overset{ˉ}{Q}}_{n}^{*}) = P_{0} {\overset{ˉ}{Q}}_{n}^{*} \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} . \end{matrix}

We have

\begin{matrix} Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2} (\overset{ˉ}{Q}) = P_{0} \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \\ = - P_{0} \frac{A - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \\ = - P_{0} \frac{E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{*}, \overset{ˉ}{Q}) - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) . \end{matrix}

Recall that, by our assumption, $\overset{ˉ}{g} = E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ . Let ${\overset{ˉ}{g}}_{0, n}^{r} = E_{P_{0}} (A | \overset{ˉ}{g}, {\overset{ˉ}{Q}}_{n}^{*}, \overset{ˉ}{Q})$ . By our assumptions,

(5)

P_{0} \frac{{\overset{ˉ}{g}}_{0, n}^{r} - \overset{ˉ}{g}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) = o_{P} (1 / \sqrt{n}) .

This proves that $Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2} (\overset{ˉ}{Q}) = o_{P} (1 / \sqrt{n})$ . □

Remark: Proof of additional result In this analysis of Term 2, we assumed $\overset{ˉ}{g} = E_{P_{0}} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ , and condition (5). Let us now try to provide a different type of analysis for this Term 2, relying on different conditions. We have

\begin{aligned} Ψ_{2} ({\overset{ˉ}{Q}}_{n}^{*}) - Ψ_{2} (\overset{ˉ}{Q}) = & P_{0} \frac{\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}}{\overset{ˉ}{g}} ({\overset{ˉ}{Q}}_{n}^{*} - \overset{ˉ}{Q}) \\ = & P_{0} \frac{A - \overset{ˉ}{g}}{\overset{ˉ}{g}} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{n}^{*}) \\ = & P_{0} \frac{{\overset{ˉ}{g}}_{0, n}^{r} - \overset{ˉ}{g}}{{\overset{ˉ}{g}}_{0, n}^{r} \overset{ˉ}{g}} A (Y - {\overset{ˉ}{Q}}_{n}^{*}), \end{aligned}

where ${\overset{ˉ}{g}}_{0, n}^{r} = E_{0} (A | \overset{ˉ}{g}, \overset{ˉ}{Q}, {\overset{ˉ}{Q}}_{n}^{*})$ , and if we assume that $P_{0} \frac{{\overset{ˉ}{g}}_{0, n}^{r} - \overset{ˉ}{g}}{{\overset{ˉ}{g}}_{0, n}^{r} \overset{ˉ}{g}} A (Y - \overset{ˉ}{Q}) = 0$ . The latter equality holds if we target in the TMLE algorithm ${\overset{ˉ}{Q}}_{n}^{*}$ with clever covariate $H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*}) = ({\overset{ˉ}{g}}_{n}^{r *} - {\overset{ˉ}{g}}_{n}^{*}) / ({\overset{ˉ}{g}}_{n}^{r *} {\overset{ˉ}{g}}_{n}^{*}) A$ , where ${\overset{ˉ}{g}}_{n}^{r *}$ estimates a non-parametric regression of A on ${\overset{ˉ}{Q}}_{n}^{*}, {\overset{ˉ}{g}}_{n}^{*}$ , exactly as in Theorem 3. Under that assumption one can now show that we obtain another influence curve component $D_{Y}$ defined by

D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g}) = \frac{{\overset{ˉ}{g}}_{0}^{r} - \overset{ˉ}{g}}{{\overset{ˉ}{g}}_{0}^{r} \overset{ˉ}{g}} A (Y - \overset{ˉ}{Q}),

where ${\overset{ˉ}{g}}_{0}^{r} = E_{0} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ . Thus, now we have that $Ψ (Q_{n}^{*})$ is asymptotically linear with influence curve $D^{*} (Q, g) - D_{A} ({\overset{ˉ}{Q}}_{0}^{r}, \overset{ˉ}{g}) - D_{Y} (\overset{ˉ}{Q}, {\overset{ˉ}{g}}_{0}^{r}, \overset{ˉ}{g})$ . However, note that if ${\overset{ˉ}{g}}_{0}^{r} = \overset{ˉ}{g}$ , i.e. if $\overset{ˉ}{g} = E_{0} (A | \overset{ˉ}{g}, \overset{ˉ}{Q})$ , then $D_{Y} = 0$ . To conclude, one can remove the condition that ${\overset{ˉ}{g}}_{n}^{*}$ needs to non-parametrically adjust for ${\overset{ˉ}{Q}}_{n}^{*}$ as arranged by the TMLE algorithm in Theorem 4 by adding the additional clever covariate $H_{Y} ({\overset{ˉ}{g}}_{n}^{r *}, {\overset{ˉ}{g}}_{n}^{*})$ to the submodel for ${\overset{ˉ}{Q}}_{n}^{*}$ in the TMLE algorithm, and the influence curve will now have another component $D_{Y} (P_{0})$ , as in Theorem 3. This results in a generalization of Theorem 3 which does not require that either $Q_{n}^{*}$ or $g_{n}^{*}$ is consistent, but only requires that their limits $\overset{ˉ}{g}, \overset{ˉ}{Q}$ satisfy $P_{0} (\overset{ˉ}{g} - {\overset{ˉ}{g}}_{0}) / \overset{ˉ}{g} (\overset{ˉ}{Q} - {\overset{ˉ}{Q}}_{0}) = 0$ . Thus, this latter generalization of Theorem 3 would provide an appropriate theorem for a C-TMLE that does not enforce the non-parametric adjustment for ${\overset{ˉ}{Q}}_{n}^{*}$ , but still needs to adjust for ${\overset{ˉ}{Q}}_{n}^{*} - {\overset{ˉ}{Q}}_{0}$ .

References

1.↑
Bickel PJ, Klaassen CA, Ritov Y, Wellner J. Efficient and adaptive estimation for semiparametric models. Springer-Verlag, 1997.
- Google Scholar
- Export Citation
2.↑
Gill RD. Non- and semiparametric maximum likelihood estimators and the von Mises method (part 1). Scand J Stat 1989;16:97–128.
- Google Scholar
- Export Citation
3.
Gill RD, van der Laan MJ, Wellner JA. Inefficient estimators of the bivariate survival function for three models. Ann Inst Henri Poincaré 1995;31:545–97.
- Google Scholar
- Export Citation
4.↑
van der Vaart AW, Wellner JA. Weak convergence and empirical processes. New York: Springer-Verlag, 1996.
- Crossref
- Google Scholar
- Export Citation
5.↑
van der Laan MJ. Estimation based on case-control designs with known prevalence probability. Int J Biostat 2008. Available at: http://www.bepress.com/ijb/vol4/iss1/17/.
- Export Citation
6.↑
van der Laan MJ, Rose S. Targeted learning: causal inference for observational and experimental data. New York: Springer, 2012.
- Google Scholar
- Export Citation
7.↑
van der Laan MJ, Rubin D. Targeted maximum likelihood learning. Int J Biostat 2006;20.
- Google Scholar
- Export Citation
8.↑
van der Laan MJ, Dudoit S. Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: finite sample oracle inequalities and examples. Technical report, Division of Biostatistics, University of California, Berkeley, CA, November 2003.
- Export Citation
9.↑
van der Laan MJ, Polley E, Hubbard A. Super learner. Stat Appl Genet Mol Biol 2007;6:Article 25.
- Crossref
- Google Scholar
- Export Citation
10.↑
van der Vaart AW, Dudoit S, van der Laan MJ. Oracle inequalities for multi-fold cross-validation. Stat Decis 2006;240:351–71.
- Google Scholar
- Export Citation
11.↑
Robins JM, Rotnitzky A. Recovery of information and adjustment for dependent censoring using surrogate markers. In Aids epidemiology. Methodological issues. Basel: Bikhäuser, 1992:297–331.
- Crossref
- Google Scholar
- Export Citation
12.
Robins JM, Rotnitzky A. Semiparametric efficiency in multivariate regression models with missing data. J Am Stat Assoc 1995;900:122–9.
- Crossref
- Google Scholar
- Export Citation
13.↑
van der Laan MJ, Robins JM. Unified methods for censored longitudinal data and causality. New York: Springer-Verlag, 2003.
- Crossref
- Google Scholar
- Export Citation
14.
Robins JM, Rotnitzky A, van der Laan MJ. Comment on “on profile likelihood” by S.A. Murphy and A.W. van der Vaart. J Am Stat Assoc – Theory Methods 2000;450:431–5.
- Google Scholar
- Export Citation
15.
Robins JM. Robust estimation in sequentially ignorable missing data and causal inference models. In Proceedings of the American Statistical Association, 2000.
- Export Citation
16.↑
Robins JM, Rotnitzky A. Comment on the Bickel and Kwon article, “inference for semiparametric models: some questions and an answer”. Stat Sin 2001;110:920–36.
- Google Scholar
- Export Citation
17.↑
Scharfstein DO, Rotnitzky A, Robins JM. Adjusting for non-ignorable drop-out using semiparametric nonresponse models, (with discussion and rejoinder). J Am Stat Assoc 1999;940:1096–120 (1121–46).
- Crossref
- Google Scholar
- Export Citation
18.↑
Bembom O, Petersen ML, Rhee S-Y, Fessel WJ, Sinisi SE, Shafer RW, et al. Biomarker discovery using targeted maximum likelihood estimation: application to the treatment of antiretroviral resistant HIV infection. Stat Med 2009;28:152–72.
- Crossref
- PubMed
- Google Scholar
- Export Citation
19.↑
Gruber S, van der Laan MJ. A targeted maximum likelihood estimator of a causal effect on a bounded continuous outcome. Int J Biostat 2010;6:Article 26. Available at: www.bepress.com/ijb/vol6/iss1/26
- PubMed
- Google Scholar
- Export Citation
20.↑
Gruber S, van der Laan MJ. An application of collaborative targeted maximum likelihood estimation in causal inference and genomics. Int J Biostat 2010;60.
- Google Scholar
- Export Citation
21.↑
Gruber S, van der Laan MJ. A targeted maximum likelihood estimator of a causal effect on a bounded continuous outcome. Technical Report 265, UC Berkeley, CA, 2010.
- PubMed
- Export Citation
22.↑
Rosenblum M, van der Laan MJ. Targeted maximum likelihood estimation of the parameter of a marginal structural model. Int J Biostat 2010;60.
- Google Scholar
- Export Citation
23.↑
Sekhon JS, Gruber S, Porter K, van der Laan MJ. Propensity-score-based estimators and C-TMLE. In: MJ van der Laan and S Rose, editors. Targeted learning: prediction and causal inference for observational and experimental data, chapter 21. New York: Springer, 2011.
- Google Scholar
- Export Citation
24.↑
Gruber S, van der Laan MJ. Targeted minimum loss based estimation of a causal effect on an outcome with known conditional bounds. Int J Biostat 2012;8.
- Crossref
- Google Scholar
- Export Citation
25.↑
Zheng W, van der Laan MJ. Asymptotic theory for cross-validated targeted maximum likelihood estimation. Technical Report 273, Division of Biostatistics, University of California, Berkeley, CA, 2010.
- Export Citation
26.↑
Zheng W, van der Laan MJ. Cross-validated targeted minimum loss based estimation. In: MJ van der Laan and S Rose, editors. Targeted learning: causal inference for observational and experimental data, chapter 21. New York: Springer, 2011:459–74.
- Crossref
- Google Scholar
- Export Citation
27.↑
van der Vaart AW. Asymptotic statistics. New York: Cambridge University Press, 1998.
- Crossref
- Google Scholar
- Export Citation
28.↑
Rotnitzky A, Lei Q, Sued M, Robins J. Improved double-robust estimation in missing data and causal inference models. Biometrika 2012;99:439–56.
- Crossref
- PubMed
- Google Scholar
- Export Citation
29.↑
Gruber S, van der Laan MJ. Targeted minimum loss based estimator that outperforms a given estimator. Int J Biostat 2012;80:Article 11.
- Crossref
- Google Scholar
- Export Citation
30.
Gruber S, van der Laan MJ. Marginal structural models. In: MJ van der Laan and S Rose, editors. C-TMLE of an additive point treatment effect, chapter 19. New York: Springer, 2011.
- Google Scholar
- Export Citation
31.
Porter KE, Gruber S, van der Laan MJ, Sekhon JS. The relative performance of targeted maximum likelihood estimators. Int J Biostat 2011;70:1–34.
- Crossref
- Google Scholar
- Export Citation
32.
Stitelman OM, van der Laan MJ. Collaborative targeted maximum likelihood for time to event data. Int J Biostat 2010:Article 21.
- Google Scholar
- Export Citation
33.↑
van der Laan MJ, Gruber S. Collaborative double robust penalized targeted maximum likelihood estimation. Int J Biostat 2010;60.
- Google Scholar
- Export Citation
34.
van der Laan MJ, Rose S. Targeted learning: prediction and causal inference for observational and experimental data. New York: Springer, 2011.
- Crossref
- Google Scholar
- Export Citation
35.↑
Wang H, Rose S, van der Laan MJ. Finding quantitative trait loci genes. In: MJ van der Laan and S Rose, editors. Targeted learning: causal inference for observational and experimental data, chapter 23. New York: Springer, 2011.
- Google Scholar
- Export Citation
36.↑
Hernan MA, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology 2000;110:561–70.
- Crossref
- Google Scholar
- Export Citation
37.↑
Györfi L, Kohler M, Krzyżak A, Walk H. A distribution-free theory of nonparametric regression. New York: Springer-Verlag, 2002.
- Crossref
- Google Scholar
- Export Citation
38.↑
van der Laan MJ, Dudoit S, van der Vaart AW. The cross-validated adaptive epsilon-net estimator. Stat Decis 2006;240:373–95.
- Google Scholar
- Export Citation
39.↑
van der Laan MJ, Dudoit S, Keles S. Asymptotic optimality of likelihood-based cross-validation. Stat Appl Genet Mol Biol 2004;3:Article 4.
- Crossref
- Google Scholar
- Export Citation
40.↑
Dudoit S, van der Laan MJ. Asymptotics of cross-validated risk estimation in estimator selection and performance assessment. Stat Methodol 2005;20:131–54.
- Crossref
- Google Scholar
- Export Citation
41.↑
Polley EC, Rose S, van der Laan MJ. Super learning. In: MJ van der Laan and S Rose, editors. Targeted learning: causal inference for observational and experimental data, chapter 3. New York: Springer, 2011.
- Google Scholar
- Export Citation
42.↑
Polley EC, van der Laan MJ. Super learner in prediction. Technical report 200. Division of Biostatistics, UC Berkeley, Working Paper Series, 2010.
- Export Citation
43.↑
van der Laan MJ, Petersen ML. Targeted learning. In: Zhang C, Ma Y, editors. Ensemble machine learning. New York: Springer, 2012:117–56. ISBN 978-1-4419-9326-7.
- Crossref
- Google Scholar
- Export Citation
44.↑
van der Laan MJ. Efficient and inefficient estimation in semiparametric models. Center for Mathematics and Computer Science, CWI-tract 114. 1996.
- Google Scholar
- Export Citation
45.↑
Lee BK, Lessler J, Stuart EA. Improved propensity score weighting using machine learning. Stat Med 2009;29:337–46.
- Google Scholar
- Export Citation
46.
Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 2009;20:512–22. DOI: 10.1097/EDE.0b013e3181a663cc.
- Crossref
- PubMed
- Google Scholar
- Export Citation
47.
Vansteelandt S, Bekaert M, Claeskens G. On model selection and model misspecification in causal inference. Stat Methods Med Res 2010;21:7–30. DOI:10.1177/0962280210387717.
- Crossref
- PubMed
- Google Scholar
- Export Citation
48.↑
Westreich D, Cole SR, Funk MJ, Brookhart MA, Sturmer T. The role of the c-statistic in variable selection for propensity scores. Pharmacoepidemiol Drug Saf 2011;20:317–20.
- Crossref
- PubMed
- Google Scholar
- Export Citation
49.↑
van der Laan MJ, Gruber S. Collaborative double robust penalized targeted maximum likelihood estimation. Int J Biostat 2009;6.
- Google Scholar
- Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

ProCite

RefWorks

Reference Manager

BibTeX

Zotero

EndNote

Article information

Published Online:: 2014-02-11

Published in Print:: 2014-05-01

Citation Information:: The International Journal of Biostatistics, Volume 10, Issue 1, Pages 29–57, eISSN 1557-4679, ISSN 2194-573X, DOI: https://doi.org/10.1515/ijb-2012-0038.

FREE ACCESS

Journal + Issues

The International Journal of Biostatistics

IJB publishes biostatistical models and methods, statistical theory, as well as original applications of statistical methods, for important practical problems arising from various sciences. It covers the entire range of biostatistics, from theoretical advances to relevant and sensible translations of a practical problem into a statistical framework, including advances in biostatistical computing.

Online ISSN:: 1557-4679

First published:: 01 May 2005

Language:: English

Publisher:: De Gruyter

Search

Abstract
1 Introduction and overview
2 Statistical inference for IPTW-estimator when using super-learning to fit treatment mechanism
3 Statistical inference for TMLE when using super-learning to consistently fit treatment mechanism
- 3.1 Asymptotic linearity of a TMLE using a targeted estimator of the treatment mechanism
- 3.2 Using a δ$$\bf\it \delta $$-specific submodel for targeting g that guarantees the positivity condition
4 Double robust statistical inference for TMLE when using super-learning to fit outcome regression and treatment mechanism
5 Collaborative double robust inference for C-TMLE when using super-learning to fit outcome regression and reduced treatment mechanism
6 Discussion
Proof of Theorem 1
Proof of Theorem 2
Proof of Theorem 3
Proof of Theorem 4

Subjects

Open Access

Services

About us

Targeted Estimation of Nuisance Parameters to Obtain Valid Statistical Inference

Abstract

1 Introduction and overview

1.1 The role of nuisance parameter estimation

1.2 Targeting the fit of the nuisance parameter: general approach

1.3 Concrete example covered in this article

1.4 Relation to current literature on targeted nuisance parameter estimators

1.5 Organization

1.6 Notation

2 Statistical inference for IPTW-estimator when using super-learning to fit treatment mechanism

2.1 An IPTW-estimator using super-learning to fit the treatment mechanism

2.2 Asymptotic linearity of a targeted data-adaptive IPTW-estimator

2.3 Comparison of targeted data-adaptive IPTW and an IPTW using parametric model

3 Statistical inference for TMLE when using super-learning to consistently fit treatment mechanism

3.1 Asymptotic linearity of a TMLE using a targeted estimator of the treatment mechanism

3.2 Using a δ-specific submodel for targeting g that guarantees the positivity condition

4 Double robust statistical inference for TMLE when using super-learning to fit outcome regression and treatment mechanism

5 Collaborative double robust inference for C-TMLE when using super-learning to fit outcome regression and reduced treatment mechanism

5.1 Motivation and theoretical underpinning of collaborative double robust estimation of nuisance parameters

5.2 C-TMLE

5.3 A TMLE that allows for collaborative double robust inference

5.4 A C-TMLE algorithm

6 Discussion

Proof of Theorem 1

Proof of Theorem 2

Proof of Theorem 3

Proof of Theorem 4

References

Article information

Journal + Issues

The International Journal of Biostatistics

Details

Search

Sections

Cited By

3.2 Using a $δ$ -specific submodel for targeting g that guarantees the positivity condition