Jump to ContentJump to Main Navigation
Show Summary Details
More options …

The International Journal of Biostatistics

Ed. by Chambaz, Antoine / Hubbard, Alan E. / van der Laan, Mark J.

2 Issues per year

IMPACT FACTOR 2017: 0.840
5-year IMPACT FACTOR: 1.000

CiteScore 2017: 0.97

SCImago Journal Rank (SJR) 2017: 1.150
Source Normalized Impact per Paper (SNIP) 2017: 1.022

Mathematical Citation Quotient (MCQ) 2016: 0.09

See all formats and pricing
More options …

A Generally Efficient Targeted Minimum Loss Based Estimator based on the Highly Adaptive Lasso

Mark van der Laan
Published Online: 2017-10-12 | DOI: https://doi.org/10.1515/ijb-2015-0097


Suppose we observe n independent and identically distributed observations of a finite dimensional bounded random variable. This article is concerned with the construction of an efficient targeted minimum loss-based estimator (TMLE) of a pathwise differentiable target parameter of the data distribution based on a realistic statistical model. The only smoothness condition we will enforce on the statistical model is that the nuisance parameters of the data distribution that are needed to evaluate the canonical gradient of the pathwise derivative of the target parameter are multivariate real valued cadlag functions (right-continuous and left-hand limits, (G. Neuhaus. On weak convergence of stochastic processes with multidimensional time parameter. Ann Stat 1971;42:1285–1295.) and have a finite supremum and (sectional) variation norm. Each nuisance parameter is defined as a minimizer of the expectation of a loss function over over all functions it its parameter space. For each nuisance parameter, we propose a new minimum loss based estimator that minimizes the loss-specific empirical risk over the functions in its parameter space under the additional constraint that the variation norm of the function is bounded by a set constant. The constant is selected with cross-validation. We show such an MLE can be represented as the minimizer of the empirical risk over linear combinations of indicator basis functions under the constraint that the sum of the absolute value of the coefficients is bounded by the constant: i.e., the variation norm corresponds with this L1-norm of the vector of coefficients. We will refer to this estimator as the highly adaptive Lasso (HAL)-estimator. We prove that for all models the HAL-estimator converges to the true nuisance parameter value at a rate that is faster than n1/4 w.r.t. square-root of the loss-based dissimilarity. We also show that if this HAL-estimator is included in the library of an ensemble super-learner, then the super-learner will at minimal achieve the rate of convergence of the HAL, but, by previous results, it will actually be asymptotically equivalent with the oracle (i.e., in some sense best) estimator in the library. Subsequently, we establish that a one-step TMLE using such a super-learner as initial estimator for each of the nuisance parameters is asymptotically efficient at any data generating distribution in the model, under weak structural conditions on the target parameter mapping and model and a strong positivity assumption (e.g., the canonical gradient is uniformly bounded). We demonstrate our general theorem by constructing such a one-step TMLE of the average causal effect in a nonparametric model, and establishing that it is asymptotically efficient.

Keywords: asymptotic linear estimator; canonical gradient; cross-validated targeted minimum loss estimation (CV-TMLE); Donsker class; efficient influence curve; efficient estimator; empirical process; entropy; highly adaptive Lasso; influence curve; one-step TMLE; super-learning; targeted minimum loss estimation (TMLE)


  • 1.

    Bickel PJ, Klaassen CAJ, Ritov Y, Wellner, J. Efficient and adaptive estimation for semiparametric models. Berlin/ Heidelberg/New York: Springer, 1997.Google Scholar

  • 2.

    Robins JM, Rotnitzky A. Recovery of information and adjustment for dependent censoring using surrogate markers. In: AIDS epidemiology. Basel: Birkhauser, 1992: 297–331.Google Scholar

  • 3.

    van der Laan MJ, Robins JM. Unified methods for censored longitudinal data and causality. New York: Springer, 2003.Google Scholar

  • 4.

    van der Laan MJ. Estimation based on case-control designs with known prevalance probability. Int J Biostat. 2008;4(1):Article17.Google Scholar

  • 5.

    van der Laan MJ, Rose S. Targeted learning: Causal inference for observational and experimental data. Berlin/Heidelberg/New York: Springer, 2011.Web of ScienceGoogle Scholar

  • 6.

    van der Laan MJ, Rubin Daniel B. Targeted maximum likelihood learning. Int J Biostat. 2006;2(1):Article 11.Google Scholar

  • 7.

    Gruber S, van der Laan MJ. An application of collaborative targeted maximum likelihood estimation in causal inference and genomics. Int J Biostat. 2010;6(1).Google Scholar

  • 8.

    Porter KE, Gruber S, van der Laan MJ, Sekhon JS. The relative performance of targeted maximum likelihood estimators. Int J Biostat. Jan 1, 2011;7(1): Article 31., 2011. Published online Aug 17, 2011. doi: . Also available at: U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 279, http://www.bepress.com/ucbbiostat/paper279.CrossrefGoogle Scholar

  • 9.

    Sekhon JS, Gruber S, Porter KE, van der Laan MJ. Propensity scorebased estimators and c-tmle. In: van der Laan MJ, Rose S, editors, Targeted learning: Causal inference for observational and experimental data. New York/Dordrecht/ Heidelberg/London: Springer, 2012.Web of ScienceGoogle Scholar

  • 10.

    Polley EC, Rose S, van der Laan MJ. Super learner. In: van der Laan MJ, Rose S, editors, Targeted learning: Causal inference for observational and experimental data. New York/Dordrecht/Heidelberg/London: Springer, 2011.Web of ScienceGoogle Scholar

  • 11.

    van der Laan MJ, Gruber S. One-step targeted minimum loss-based estimation based on universal least favorable one-dimensional submodels. Int J Biostat. 2016;12(1):351–378. doi: .CrossrefPubMedWeb of ScienceGoogle Scholar

  • 12.

    van der Laan MJ, Polley EC, Hubbard AE. Super learner. Stat Appl Genet Mol. 2007;6(1):Article 25.

  • 13.

    van der Vaart AW, Dudoit S, van der Laan MJ. Oracle inequalities for multi-fold cross-validation. Stat Decis. 2006;24(3):351–371.Google Scholar

  • 14.

    Polley EC, Rose Sherri, van der Laan MJ. Super learning. In: van der Laan MJ, Rose S, editors, Targeted learning: Causal inference for observational and experimental data. New York/Dordrecht/Heidelberg/London: Springer, 2012.Google Scholar

  • 15.

    Zheng W, van der Laan MJ. Cross-validated targeted minimum loss based estimation. In: van der Laan MJ, Rose S, editors, Targeted learning: Causal inference for observational and experimental studies. New York: Springer, 2011.Web of ScienceGoogle Scholar

  • 16.

    van der Laan MJ. A generally efficient targeted minimum lossbased estimator. Technical Report 300, UC Berkeley, 2015. http://biostats.bepress.com/ucbbiostat/paper343.Google Scholar

  • 17.

    Neuhaus G. On weak convergence of stochastic processes with multidimensional time parameter. Ann Stat 1971;42:1285–1295.CrossrefGoogle Scholar

  • 18.

    van der Vaart AW, Wellner JA. A local maximal inequality under uniform entropy. Electr J Stat. 2011;5:192–203. ISSN: 1935–7524, DOI: .CrossrefGoogle Scholar

  • 19.

    van der Vaart AW, Wellner JA. Weak convergence and empirical processes. Berlin/Heidelberg/New York: Springer, 1996.Google Scholar

  • 20.

    Gill RD, van der Laan MJ, Wellner JA. Inefficient estimators of the bivariate survival function for three models. Annales de l’Institut Henri Poincaré 1995;31:545–597.Google Scholar

  • 21.

    van der Laan MJ, Dudoit S, van der Vaart AW. The cross-validated adaptive epsilon-net estimator. Stat Decis. 2006;24(3):373–395.Google Scholar

  • 22.

    Benkeser D, van der Laan MJ. The highly adaptive lasso estimator. In: Proceedings of the IEEE Conference on Data Science and Advanced Analytics, 2016. To appear.

  • 23.

    Chambaz A, van der Laan MJ. Targeting the optimal design in randomized clinical trials with binary outcomes and no covariate, theoretical study. Int J Biostat. 2011a;7(1):1–32. Working paper 258, www.bepress.com/ucbbiostatWeb of Science

  • 24.

    Chambaz A, van der Laan MJ. Targeting the optimal design in randomized clinical trials with binary outcomes and no covariate, simulation study. Int J Biostat. 2011b;7(1):33.Working paper 258,www.bepress.com/ucbbiostat.Web of ScienceGoogle Scholar

  • 25.

    van der Laan MJ. Causal inference for networks. Technical Report 300, UC Berkeley, 2012. http://biostats. bepress.com/ucbbiostat/paper300, to appear in Journal of Causal Inference.Google Scholar

  • 26.

    van der Laan MJ, Balzer LB, Petersen ML. Adaptive matching in randomized trials and observational studies. J Stat Res. 2013;46(2):113–156.Google Scholar

  • 27.

    Gruber S, van der Laan MJ. Targeted maximum likelihood estimation, R package version 1.2.0-1, Available at http://cran.rproject.org/web/packages/tmle/tmle.pdf, 2012.Google Scholar

  • 28.

    Petersen M, Schwab J, Gruber S, Blaser N, Schomaker M, van der Laan MJ. Targeted maximum likelihood estimation of dynamic and static marginal structural working models. J Causal Inf. 2013;2:147–185.Google Scholar

  • 29.

    Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61:962–972.CrossrefPubMedWeb of ScienceGoogle Scholar

  • 30.

    Iván Díaz, van der Laan MJ. Sensitivity analysis for causal inference under unmeasured confounding and measurement error problems. Int J Biostat. In press.Web of ScienceGoogle Scholar

  • 31.

    van der Laan MJ, Petersen ML. Targeted learning. In: Zhang C, Ma Y, editors, Ensemble machine learning: methods and applications. New York: Springer, 2012.Google Scholar

  • 32.

    van der Laan MJ, Petersen ML. Causal effect models for realistic individualized treatment and intention to treat rules. Int J Biostat. 2007;3(1):Article 3.Google Scholar

  • 33.

    van der Laan MJ, Dudoit S. Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: finite sample oracle inequalities and examples. Technical Report 130, Division of Biostatistics, University of California, Berkeley, 2003.Google Scholar

  • 34.

    van der Laan MJ, Dudoit S, Keles S. Asymptotic optimality of likelihood-based cross-validation. Stat Appl Genet Mol. 2004;3(1):Article4.Google Scholar

About the article

Received: 2015-12-13

Accepted: 2017-06-29

Published Online: 2017-10-12

Citation Information: The International Journal of Biostatistics, Volume 13, Issue 2, 20150097, ISSN (Online) 1557-4679, DOI: https://doi.org/10.1515/ijb-2015-0097.

Export Citation

© 2017 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in