Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter June 20, 2015

Creating Opportunities for Rural Producers: Impact Evaluation of a Pilot Program in Colombia

Sandra V. Rozo, Veronica Gonzalez, Carlos Morales and Yuri Soares

Abstract

This paper presents the impact evaluation of a pilot program that treated 57 small organizations of agricultural producers with high risk of getting involved in illegal drug production in Colombia. The program supported producers mainly by facilitating the commercialization of their new licit alternative sources of income. We combine propensity score matching, regression discontinuity, and Bayesian decision theory, with unique and rich panel data to assess the economic impact of the program. Our results suggest that the program was successful on increasing total sales and improving the product’s quality for the treated producers. The intervention was more successful when combined with other programs that gave producers incentives to abandon illegal drug production definitely.

JEL Classification: O13; O33; O54, and Q18

Funding statement: Funding: This paper was part of an evaluation that the Office of Evaluation and Oversight (OVE) of the Inter-American Development Bank carried out for the projects supported by the Multilateral Investment Fund II (MIF II).

Acknowledgements

We would like to thank the United Nations Office of Drugs and Crime (UNODC) in Colombia for their invaluable collaboration and for making the data available for these study. The usual disclaimer applies.

Appendix

A Beneficiaries Selection

Once a potential group of beneficiaries was selected UNODC carried out deeper analysis to identify the beneficiaries. In particular, the organizations in the groups were classified according to a final score constructed based on 5 categories. Each category was calculated as the sum of the scores of an extensive group of variables summarized in Table 8 below. The weight that each variable had on the total score is included in parenthesis next to each variable. The program treated a total of 57 organizations of producers which were chosen as the ones that had the best final score until resources lasted. [13]

Figure 7: Development score used for selecting beneficiaries.

Figure 7:

Development score used for selecting beneficiaries.

B Outcomes

Table 8:

Outcomes analyzed by Línea Productiva.

Name of VariableVariables Description
SalesTotal sales (Colombian pesos)
Utility MarginUtility Margin (Colombian pesos)
Loan Application=1 if applied to loan
Production CapacityProductive Capacity of Members (Ha of land)
Productive Capacity AreaProductive Capacity in Production (ha of land)
Stage=1 if stage of establishment and sustainability, 2=commercialization and production
Sales Personnel=1 if Org has Sales Personnel
TI Security=1 if transportation issue security
TI Highways=1 transportation issue highways condition or no roads
TI weather=1 if transportation issue weather
TI Costs=1 if transportation issue high costs
TI No Transp=1 if transportation issue no transport
Publicity=1 if org makes product’s publicity
Trade Fund=1 if org has commercialization fund
Organization’s qualityNumber of quality certifications of Organization
Sales through OrgPercentage of Members who sale through Organization
Number of ProductsNumber of Products
Product’s qualityNumber of Quality Certifications/Number of Products
Producer’s QualityNumber of certifications of producers for main product

C Variables Included in the Propensity Score

Table 9:

Variables included in the estimation of the pscores.

Variables included in the probit modelDescription
YearsNumber of years since foundation
N. membersNumber of members
N. DonorsNumber of donors
Municipality=1 if members live in same municipality
Created com=1 if created to commercialize products
Created PP=1 if created to participate in “Proyectos Productivos”
Created coll=1 if created to work collectively
Number of ServicesNumber of Services
Copy of rules=1 if more than 50% of members have copy of the rules
Newspaper com.=1 if organization communicates to members through newspaper
Radio com.=1 if organization communicates to members through radio
Web com.=1 if organization communicates to members though the web
Complaints system=1 if org has a complaint system in place
N. InvitationsNumber of invitations needed to coordinate a member’s meeting
Years boardYears between board elections
Control Institutions=1 if org has a control institution
Financial Contributions=1 if members pay an economic contribution
Capital Property=1 if org has capital property
Financial Statements=1 if org has financial statements
N. CreditsNumber of credits being processed
Balance Sheet=1 if org has balance sheets
Petty Cash=1 if org has petty cash
Strategic Plan“=1 if org has a strategic plan
N. of ProjectsNumber of projects being executed by org
Supports Production=1 if org supports production
Supports Intermediation=1 if org supports intermediation
Supports Packaging=1 if org supports packaging
Supports Transformation=1 if org supports transformation
Supports Commercialization=1 if org supports commercialization
Technical Personnel=1 if org has technical personnel
Administrative Personnel=1 if org has administrative personnel

Table 10:

Probit model estimates.

VariableCoefficientSt. Error
Years0.050.02
N. members0.000.00
N. Donors–0.610.44
Municipality–0.070.25
Created com21.380.99
Created PP21.520.95
Created coll21.460.96
Number of Services–0.110.07
Copy of rules0.160.26
Newspaper com.0.040.30
Radio com.1.150.42
Web com.0.840.47
Complaints system–0.270.25
N. Invitations–0.200.29
Years board–0.340.16
Control Institutions0.290.42
Financial Contributions0.270.34
Capital Property–0.880.32
Financial Statements0.330.46
N. Credits0.010.04
Balance Sheet0.890.39
Petty Cash0.120.26
Strategic Plan–0.200.28
N. of Projects–0.090.10
Supports Production0.270.27
Supports Intermediation0.260.32
Supports Packaging–0.400.48
Supports Transformation–0.350.52
Supports Commercialization0.350.27
Technical Personnel0.090.26
Administrative Personnel0.380.28
Pseudo R20.26
N. Observations438

D Constructing the Predictive Distribution of Sales

D.1 Bayesian Decision Theory

Bayesian Decision Theory is concerned with identifying the best decision rule a planner can make under certain assumptions. According to Bayesian Theory the optimal decision rule is that which minimizes the expected loss function for the relevant population, in this case organizations of producers. In general, the problem to be solved could be written as:

MinDE[U(Y,D)/data

where D represents the available choice of parameters, and U() represents the objective function to be minimized (usually represented as the sum of the squared error). What makes this theory different from other optimization problems is that the expected value to be minimized is calculated based on the posterior distribution of the parameters conditional on the data that we observe, which we will denote as P(θ/Y). Here θ=(β,σ) will represent the parameters of the program. In particular β denotes the coefficients of a regression of total sales on the treatment dummy (for each policy to be analyzed), and other relevant covariates; and σ represents the variance of the mean squared error. In other words, Bayesian theory assumes that the parameters of the model are uncertain and unknown, and hence, they have a distribution of their own. The uncertainty of the parameters (i.e., the posterior distribution of the parameters) will affect the outcome distribution and should be taken into account when trying to derive the predicted distribution of sales.

Our interest in the evaluation will be emphasized in obtaining the predictive distribution of sales for each of the organizations in the sample. This distribution embodies all of the uncertainty of the model, and for this reason we first need to derive the posterior distribution of the parameters of the model and from it we can derive the predictive distribution of sales.

In our data we observe the sales of those organizations that are the most organized, and hence, keep financial statements. Since for those organizations that did not keep records sales are likely positive but just not observed sales will be modelled using a censored normal likelihood: a Tobit model. Define the latent variable yit as a latent variable of observed sales such that:

yit={yitifyit>00otherwise

here yit will be observed if the organization kept financial statements. For this Tobit model it will be assumed that Yit|{Xit=xit,β,σ}N(xitβ,σ). This distribution will be called the predictive distribution of sales. The objective is to construct a predicted distribution of sales that embodies the uncertainty of the parameters of the model. Following Dehejia (2004) we will employ the Gibbs sampling algorithm to construct the posterior distribution of the parameters. The algorithm is described in detail in Appendix D.

Using this algorithm we will obtain 1,000 draws of the posterior distribution {β(j),σ(j)2}j=11000, of the parameters and with them we will be able to construct the simulated distribution of sales for each organizations under the different policies to be analyzed. These distributions are what we called the predictive distribution of sales. The process to simulate the predictive distributions of sales is described in detail in Appendix D.

The results of these simulations will only hold if our choice of likelihood was correct. In other words, the fit of the model should be tested. Figures 8 and 9 confirm that the empirical distributions of total sales for treated and control units are well approximated by the mean distribution of our simulations.

Figure 8: Observed sales and simulated mean sales distribution – control group.

Figure 8:

Observed sales and simulated mean sales distribution – control group.

Figure 9: Observed sales and simulated mean sales distribution – treated group.

Figure 9:

Observed sales and simulated mean sales distribution – treated group.

D.2 Constructing the Posterior Distribution of Parameters

We first estimated the posterior distribution of the parameters through a Gibs sampling method. The Gibs algorithm consists of the following steps:

  1. Let yitz equal yit for the uncensored observations, i.e., {i,t|yit>0}, and for the censored observations {i,t|yii=0}, draw yitz from the negative portion of a truncated normal distribution with mean xitβ and variance σ2.

  2. Draw for β from N(βˆ,σ2(xx)1) where βˆ=(xx)1xyz

  3. Draw for σ2 from a Gamma(8840,||yzxβ||2/2)

  4. Iterate on this algorithm 5,000 times and keep only the last 1,000. This completes the estimation of the posterior distribution of θ.

D.3 Simulating the predictive distribution or sales

To simulate the predictive distribution of earnings we followed the following steps:

  1. For each organization in the sample consider Xit1 and Xit0 to be a vector of covariates that represents the observed characteristics of the organization. Both vectors differ only in that Xit1(Xit0) has the treatment dummy equal to 1 (to 0).

  2. Use the stored draws of the posterior distribution {β(j),σ(j)2}j=11000, and based on the parameters draw for the predictive distribution of earnings when the organization received the treatment and when it did not from a normal distribution:

yit1(j)|{Xit=xit1,β,σ}N(xit1β(j),σ(j)2)
yit0(j)|{Xit=xit0,β,σ}N(xit0β(j),σ(j)2)
  1. From the previous estimation we obtained y, from it we recover the simulates sales as:

yit={yitifyit>00otherwise
  1. Store the predictive distribution of outcomes for each organization

References

Abadie, A., J.Angrist, and G.Imbens. 2010. “On the Failure of Bootstrap for Matching Estimators.” Econometrica76 (6):153757.Search in Google Scholar

Abadie, A., and G.Imbens. 2006. “Large Sample Properties of Matching Estimators for Average Treatment Effects.” Econometrica74 (1):23567.Search in Google Scholar

Angrist, J., and A.Kugler2008. “Rural Windfall or a New Resource Curse? Coca, Income, and Civil Conflict in Colombia”. The Review of Economics and Statistics90(2): 191215.Search in Google Scholar

Angrist, J., and J.Pischke2009. Mostly Harmless Econometrics: an Empiricist’s Companion. Oxfordshire: Princeton University Press.Search in Google Scholar

Caliendo, M., and S.Kopeinig. 2005. Some Practical Guidance for the Implementation of Propensity Score Matching. Institute for the Study of Labor, Discussion Paper N. 1588.Search in Google Scholar

Dehejia, R.2004. “Program Evaluation as a Decision Problem.” Journal of Econometrics125 (1):14173.Search in Google Scholar

Dell, M.2011. Trafficking Networks and the Mexican Drug War. Forthcoming, American Economic Review.Search in Google Scholar

Dube, O., and J.FVargas. 2013. “Commodity Price Shocks and Civil Conflict: Evidence from Colombia”. The Review of Economic Studies80(4): 1384421.Search in Google Scholar

Farrell, G.1998. “Global Empirical Review of Drug Crop Eradication and United Nations’ Crop Substitution and Alternative Development Strategies.” Journal of Drug Issues28 (2):395436.Search in Google Scholar

Gaviria, A., and D.Mejia2011. Politicas Antidriga en Colombia: Exitos, Fracasos y Extravios. Ediciones Uniandes. Colombia.Search in Google Scholar

Imbens, G., and K.Kalyanaraman2012. “Optimal Bandwidth Choice for the Regression Discontinuity Estimator”. Review of Economic Studies79:933–59.Search in Google Scholar

Imbens, G., and J.Wooldridge. 2007. What’s new in Econometrics: Estimation of Average Treatment Effects Under Uncounfoundedness. National Bureau of Economic Research, Summer 2007.Search in Google Scholar

Imberns, G., and J.Wooldridge Recent Developments in the Econometrics of Program Evaluation. National Bureau of Economic Research Working Paper No. 14251.Search in Google Scholar

Lechner, M.1999. “Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification”. Journal of Business & Economic Statistics17(1): 7490.Search in Google Scholar

Lee, D., and T.Lemieux. 2009. Regression Discontinuity Design in Economics. National Bureau of Economic Research, Working Paper N. 14723.Search in Google Scholar

Ludwig, J., and D.Miller. 2005. Does Head Start Improve Children’s Life Chances? Evidence from a Regression Discontinuity Design. National Bureau of Economic Research, Working Paper N. 11702.Search in Google Scholar

Mansfiled, D.2011. Assessing Supply-Side Policy and Practice: Eradication and Alternative Development. Global Commission on Drug Policies. Working Paper.Search in Google Scholar

McCrary, J.2008. “Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test.” Journal of Econometrics142 (2):698714.Search in Google Scholar

Rabassa, A., and P.Chalk2001. Colombian Labyrinth. Arlington, Virginia: RAND.Search in Google Scholar

Rosenbaum, P., and D.Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika70:4150.Search in Google Scholar

Rosembaum, P., and D.BRubin. 1985. “Constructing a Control Group using Multivariate Matched Sampling Methods that Incorporate the Propensity Score”. The American Statistician39(1): 338.Search in Google Scholar

Rubin, D.B. 1974. “Estimating Causal Effects of Treatments in Randomized and Non-randomized Studies”. J. Ed. Psychol. 66:688–701.Search in Google Scholar

Sianesi, B.2004. “An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s”. Review of Economics and Statistics86(1): 13355.Search in Google Scholar

Stuart, E. 2010. “Matching Methods for Casual Inference: A Review and a Look Forward”. Statistical Science25 (1):121.Search in Google Scholar

UNODC. 2010. Informe de Gestión Diciembre de 2010. Proyecto de Fortalecimiento Comercial y Agroindustrial de Productores de Desarrollo Alternativo. Bogotá.Search in Google Scholar

UNODC. 2011. World Drug Report 2011. Unted Nations. New York.Search in Google Scholar

Vargas, M., and R.Renato. 2005. Crop Substitution and Diversification Strategies: Empirical Evidence from Selected Brazilian Municipalities. Economics of Tobacco Control, Paper N. 28.Search in Google Scholar

Zirnite, P.1998. “Militarization of the US Drug Control Program”. Foreign Policy in Focus3(27): 13.Search in Google Scholar

Published Online: 2015-6-20
Published in Print: 2015-12-1

©2015 by De Gruyter