Abstract
We study the identification of channels of policy reforms with multiple treatments and different types of selection for each treatment. We disentangle reform effects into policy effects, selection effects, and time effects under the assumption of conditional independence, common trends, and an additional exclusion restriction on the non-treated. Furthermore, we show the identification of direct- and indirect policy effects after imposing additional sequential conditional independence assumptions on mediating variables. We illustrate the approach using the German reform of the allocation system of vocational training for unemployed persons. The reform changed the allocation of training from a mandatory system to a voluntary voucher system. Simultaneously, the selection criteria for participants changed, and the reform altered the composition of course types. We consider the course composition as a mediator of the policy reform. We show that the empirical evidence from previous studies reverses when considering the course composition. This has important implications for policy conclusions.
*This study is part of the project “Regional Allocation Intensities, Effectiveness and Reform Effects of Training Vouchers in Active Labor Market Policies”, IAB project 1155. This is a joint project of the Institute for Employment Research (IAB) and the University of Freiburg. We gratefully acknowledge financial and material support from the IAB. The paper was presented at ESPE in Aarhus, CAFE Workshop in Børkop, SOLE in Washington, EALE in Ljubljana, Joint Research Centre of the European Commission, Centre for European Economic Research, and the University of Bern. We thank participants for helpful comments, in particular Hugo Bodory, Bernd Fitzenberger, Hans Fricke, Michael Lechner, Michael Knaus, Thomas Kruppe, Marie Paul, and Gesine Stephan. We are particularly grateful for detailed comments and remarks from Conny Wunsch. Furthermore, we thank two anonymous referees. The usual disclaimer applies.
References
Abadie, A. 2005. “Semiparametric Difference-in-Differences Estimators.” The Review of Economic Studies 72 (1): 1–19. https://doi.org/10.1111/0034-6527.00321.Search in Google Scholar
Baron, R. M., and D. A. Kenny. 1986. “The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51: 1173–82. https://doi.org/10.1037/0022-3514.51.6.1173.Search in Google Scholar
Biewen, M., B. Fitzenberger, A. Osikominu, and M. Paul. 2014. “The Effectiveness of Public Sponsored Training Revisited: The Importance of Data and Methodological Choices.” Journal of Labor Economics 32 (4): 837–97. https://doi.org/10.1086/677233.Search in Google Scholar
Bruttel, O. 2005. “Delivering Active Labour Market Policy Through Vouchers: Experiences with Training Vouchers in Germany.” International Review of Administrative Sciences 71 (3): 391–404. https://doi.org/10.1177/0020852305056809.Search in Google Scholar
Card, D., and D. R. Hyslop. 2005. “Estimating the Effects of a Time-Limited Earnings Subsidy for Welfare-Leavers.” Econometrica 73 (6): 1723–70. https://doi.org/10.1111/j.1468-0262.2005.00637.x.Search in Google Scholar
Cox, D. R. 1958. “The Regression Analysis of Binary Sequences.” Journal of the Royal Statistical Society: Series B (Methodological) 20: 215–42. https://doi.org/10.1111/j.2517-6161.1958.tb00292.x.Search in Google Scholar
Deuchert, E., M. Huber, and M. Schelker. 2019. “Direct and Indirect Effects Based on Difference-in-Differences with an Application to Political Preferences Following the Vietnam Draft Lottery.” Journal of Business and Economic Statistics 37 (4): 710–20. https://doi.org/10.1080/07350015.2017.1419139.Search in Google Scholar
Doerr, A., B. Fitzenberger, T. Kruppe, M. Paul, and A. Strittmatter. 2017. “Employment and Earnings Effects of Awarding Training Vouchers in Germany.” Industrial and Labor Relations Review 70 (3): 767–812. https://doi.org/10.1177/0019793916660091.Search in Google Scholar
Felfe, C., N. Nollenberger, and N. Rodriguez-Planas. 2014. “Can’t Buy Mommy’s Love? Universal Childcare and Children’s Long-Term Cognitive Development.” Journal of Population Economics 283 (2): 393–422. https://doi.org/10.1007/s00148-014-0532-x.Search in Google Scholar
Flores, C., and A. Flores-Lagunes. 2009. “Identification and Estimation of Causal Mechanisms and Net Effects of a Treatment under Unconfoundedness.” IZA Discussion Paper, 4237.10.2139/ssrn.1423353Search in Google Scholar
Fricke, H. 2017. “Identifcation Based on Difference-in-Differences Approaches with Multiple Treatments.” Oxford Bulletin of Economics and Statistics 79 (3): 426–33. https://doi.org/10.1111/obes.12178.Search in Google Scholar
Graham, B. S., C. C. De Xavier Pinto, and D. Egel. 2016. “Efficient Estimation of Data Combination Models by the Method of Auxiliary-to-Study Tilting.” Journal of Business & Economic Statistics 34 (2): 288–301. https://doi.org/10.1080/07350015.2015.1038544.Search in Google Scholar
Gundersen, C., B. Kreider, J. Pepper, and V. Tarasuk. 2017. “Food Assistance Programs and Food Insecurity: Implications for Canada in Light of the Mixing Problem.” Empirical Economics 52 (3): 1065–87. https://doi.org/10.1007/s00181-016-1191-4.Search in Google Scholar
Havnes, T., and M. Mogstad. 2011a. “Money for Nothing? Universal Child Care and Maternal Employment.” Journal of Public Economics 95 (11–12): 1455–65. https://doi.org/10.1016/j.jpubeco.2011.05.016.Search in Google Scholar
Havnes, T., and M. Mogstad. 2011b. “No Child Left Behind: Subsidized Child Care and Children’s Long-Run Outcomes.” American Economic Journal: Economic Policy 3 (2): 97–129. https://doi.org/10.1257/pol.3.2.97.Search in Google Scholar
Heckman, J. J., H. Ichimura, and P. E. Todd. 1997. “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme.” The Review of Economic Studies 64 (4): 605–54. https://doi.org/10.2307/2971733.Search in Google Scholar
Huber, M. 2014. “Identifying Causal Mechanisms (Primarily) Based on Inverse Probability Weighting.” Journal of Applied Econometrics 29 (6): 920–43. https://doi.org/10.1002/jae.2341.Search in Google Scholar
Huber, M., M. Lechner, and G. Mellace. 2017. “Why Do Tougher Caseworkers Increase Employment? The Role of Programme Assignment as a Causal Mechanism.” The Review of Economics and Statistics 99 (1): 180–3. https://doi.org/10.1162/rest_a_00632.Search in Google Scholar
Huber, M., M. Lechner, and A. Strittmatter. 2018. “Direct and Indirect Effects of Training Vouchers for the Unemployed.” Journal of the Royal Statistical Society, Series A 181 (2): 441–63. https://doi.org/10.1111/rssa.12279.Search in Google Scholar
Huber, M., M. Schelker, and A. Strittmatter. 2020. “Direct and Indirect Effects Based on Changes-in-Changes.” Journal of Business and Economic Statistics forthcoming, https://doi.org/10.1080/07350015.2020.1831929.Search in Google Scholar
Imai, K., L. Keele, D. Tingley, and T. Yamamoto. 2011. “Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” American Political Science Review 105 (4): 765–89. https://doi.org/10.1017/s0003055411000414.Search in Google Scholar
Imai, K., L. Keele, and T. Yamamoto. 2010. “Identification, Inference and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25: 51–71. https://doi.org/10.1214/10-sts321.Search in Google Scholar
Imbens, G. 2000. “The Role of the Propensity Score in Estimating Dose-Response Functions.” Biometrika 87 (3): 706–10. https://doi.org/10.1093/biomet/87.3.706.Search in Google Scholar
Kikuchi, N. 2017. “Intergenerational Transmission of Education in Japan: Nonparametric Bounds Analysis with Multiple Treatments.” ISER Discussion Paper No. 1011.10.2139/ssrn.3039898Search in Google Scholar
Lechner, M. 1999. “Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification.” Journal of Business and Economic Statistics 17 (1): 74–90. https://doi.org/10.1080/07350015.1999.10524798.Search in Google Scholar
Lechner, M. 2001. “Identification and Estimation of Causal Effects of Multiple Treatments under the Conditional Independence Assumption,” In Econometric Evaluation of Labour Market Policies, edited by M. Lechner, and F. Pfeiffer, 43–58. ZEW Economic Studies 13. New York: Springer-Verlag.10.1007/978-3-642-57615-7_3Search in Google Scholar
Lechner, M. 2010. “The Estimation of Causal Effects by Difference-in-Difference Methods.” Foundations and Trends in Econometrics 4 (3): 165–224. https://doi.org/10.1561/0800000014.Search in Google Scholar
Lechner, M., R. Miquel, and C. Wunsch. 2011. “Long-run Effects of Public Sector Sponsored Training.” The Journal of the European Economic Association 9 (4): 742–84. https://doi.org/10.1111/j.1542-4774.2011.01029.x.Search in Google Scholar
Lechner, M., and J. Smith. 2007. “What is the Value Added by Caseworkers?” Labour Economics 14 (2): 135–51. https://doi.org/10.1016/j.labeco.2004.12.002.Search in Google Scholar
Lechner, M., and A. Strittmatter. 2019. “Practical Procedures to Deal with Common Support Problems in Matching Estimation.” Econometric Reviews 38 (2): 193–207. https://doi.org/10.1080/07474938.2017.1318509.Search in Google Scholar
Lechner, M., and C. Wunsch. 2013. “Sensitivity of Matching-Based Program Evaluations to the Availability of Control Variables.” Labour Economics 21 (C): 111–21. https://doi.org/10.1016/j.labeco.2013.01.004.Search in Google Scholar
Manski, C. 1997. “The Mixing Problem in Programme Evaluation.” The Review of Economic Studies 64 (4): 537–53. https://doi.org/10.2307/2971730.Search in Google Scholar
McCall, B., J. A. Smith, and C. Wunsch. 2016. “Government-Sponsored Vocational Education for Adults.” Handbook of the Economics of Education 5: 479–652. https://doi.org/10.1016/b978-0-444-63459-7.00009-9.Search in Google Scholar
Pearl, J. 2001. “Direct and Indirect Effects.” In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, 411–20. San Francisco: Morgan Kaufmann.10.1145/3501714.3501736Search in Google Scholar
Perez-Johnson, I., Q. Moore, and R. Santillano. 2011. Improving the Effectiveness of Individual Training Accounts: Long-Term Findings from an Experimental Evaluation of Three Service Delivery Models. Final Report. Princeton, NJ: Mathematica Policy Research.Search in Google Scholar
Petersen, M. L., S. E. Sinisi, and M. J. van der Laan. 2006. “Estimation of Direct Causal Effects.” Epidemiology 17: 276–84. https://doi.org/10.1097/01.ede.0000208475.99429.2d.Search in Google Scholar
Rinne, U., A. Uhlendorff, and Z. Zhao. 2013. “Vouchers and Caseworkers in Training Programs for the Unemployed.” Empirical Economics 45 (3): 1089–127. https://doi.org/10.1007/s00181-012-0662-5.Search in Google Scholar
Robins, J., and S. Greenland. 1992. “Identifiability and Exchangeability for Direct and Indirect Effects.” Epidemiology 3: 143–55. https://doi.org/10.1097/00001648-199203000-00013.Search in Google Scholar
Rosenbaum, P., and D. Rubin. 1983. “The Central Role of Propensity Score in Observational Studies for Causal Effects.” Biometrica 70 (1): 41–55. https://doi.org/10.1093/biomet/70.1.41.Search in Google Scholar
Rubin, D. B. 1974. “Estimating the Causal Effect of Treatments in Randomized and Non-Randomized Studies.” Journal of Educational Psychology 66 (5): 688–701. https://doi.org/10.1037/h0037350.Search in Google Scholar
Strittmatter, A. 2016. “What Effect Do Vocational Training Vouchers Have on the Unemployed?” IZA World of Labor 316, https://doi.org/10.15185/izawol.316.Search in Google Scholar
Tomini, F., W. Groot, and H. Maassen van den Brink. 2016. “The Effectiveness of the Voucher Training Programs: A Systematic Review of the Evidence from Evaluations.” TIER Working Paper Series, 16/08.Search in Google Scholar
Twinam, T. 2017. “Complementarity and Identification.” Econometric Theory 33 (5): 1154–85. https://doi.org/10.1017/s0266466616000359.Search in Google Scholar
Van der Weele, T. J. 2009. “Marginal Structural Models for the Estimation of Direct and Indirect Effects.” Epidemiology 20: 18–26. https://doi.org/10.1097/ede.0b013e31818f69ce.Search in Google Scholar
Article note
This study is part of the project “Regional Allocation Intensities, Effectiveness and Reform Effects of Training Vouchers in Active Labor Market Policies”, IAB project 1155. This is a joint project of the Institute for Employment Research (IAB) and the University of Freiburg. We gratefully acknowledge financial and material support from the IAB. The paper was presented at ESPE in Aarhus, CAFE Workshop in Børkop, SOLE in Washington, EALE in Ljubljana, Joint Research Centre of the European Commission, Centre for European Economic Research, and the University of Bern. We thank participants for helpful comments, in particular Hugo Bodory, Bernd Fitzenberger, Hans Fricke, Michael Lechner, Michael Knaus, Thomas Kruppe, Marie Paul, and Gesine Stephan. We are particularly grateful for detailed comments and remarks from Conny Wunsch. Furthermore, we thank two anonymous referees. The usual disclaimer applies.
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/jem-2019-0012).
© 2020 Walter de Gruyter GmbH, Berlin/Boston
