Jump to ContentJump to Main Navigation
Show Summary Details
More options …

The Economists’ Voice

Ed. by Belke, Ansgar / Schnabl, Gunther

1 Issue per year


CiteScore 2017: 0.15

SCImago Journal Rank (SJR) 2017: 0.104
Source Normalized Impact per Paper (SNIP) 2017: 0.105

Online
ISSN
1553-3832
See all formats and pricing
More options …
Ahead of print

Issues

The Unbearable Errorlessness of Global Poverty Estimates

Michail Moatsos
  • Corresponding author
  • International Institute of Social History, 1019 AT Amsterdam, Netherlands
  • Department of History and Art History, Utrecht University, Utrecht, The Netherlands, Mobile: +0031-616302686
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2018-10-13 | DOI: https://doi.org/10.1515/ev-2018-0004

Abstract

In October 2015 the World Bank initiated the Atkinson Commission on Global Poverty seeking advise on (1) keeping the international poverty line (iPL) constant in real terms, and (2) what else the Bank should make available to complement the dollar-a-day estimates. The Commission’s Report bears a set of 21 key recommendations, largely covering the most important voiced worries of the research community over the Bank’s methods and estimates. In response the Bank adopted fully and unconditionally only one–out of ten–recommendations regarding point one above, and three–out of nine–recommendations to the second point. In addition the Bank accepted one of the two overarching recommendations. Among the remaining 16 sidelined or partially accepted recommendations lies arguably the most obvious and important one: the urge that the Bank publishes the error terms of its estimates. Without them these estimates are supported by little else other than the administrative authority of the Bank.

Keywords: commission on poverty; global poverty; international poverty line; sustainable development goals; world bank

Introduction to a Weighted Commentary

Reading the Atkinson Commission Report in its entirety provides a gateway to all the big issues and important details of measuring poverty on a global scale. The immediate and unconditional adoption of all recommendations found therein is arguably highly recommended. The World Bank, however, does not seem to be ready to do so. Despite the rather unenthusiastic reception of the Atkinson Committee report by the World Bank, this piece of work has, in my opinion, all the credentials to become a reference tool for researchers in the field of global poverty measurement for years to come. The importance of the topic cannot be overstated. Also in terms of the United Nations agenda it is the case for both the Millennium and the Sustainable Development Goals, where the first goal targets global poverty. Thus it is a topic that deserves our fullest attention.

What follows is not an overall balanced review of the report. That would be a stretch too far. Rather, it offers a weighted commentary based on the infeasible mission statement of the Atkinson Commission, as well as the type of response each point received from the Bank.1 Despite the–nothing but–strong criticism I offer below, I am bound to acknowledge the openness of the Bank in publishing a prompt cover note on the Commission’s report (The World Bank 2016), and the initiative of the former Bank’s chief economist Kaushik Basu to form this Commission in the first place.

A Mandate Bearing a Catch-22

The Commission was assembled in October 2015 by Kaushik Basu, the former chief economist and senior vice-president of the Bank. The most prominent members of the international community in poverty research constituted this Commission.2 Its mission was embodied into addressing the following two short, yet very important, questions:

  1. What should be the interpretation going forward of the definition of extreme poverty, set in 2015 at 1.90 PPP-adjusted dollars a day per person, in real terms?

  2. What choices should the World Bank make regarding complementary poverty measures to be tracked and made available to policy makers?

The mandate defining the task of the Commission appears general, but it is rather restrictive in turn. It is easy to rationalize these questions within the current context while reading through them. But the (implicit) imperative is rather clear: It sets the current method of the Bank as an uncontested standard, and asks the Commission advice for what is the best that can be done with it, but not instead of it. Given the outspoken criticism that the method has received (Aten and Heston 2010; Deaton 2010; Reddy and Pogge 2010; Srinivasan 2010; Subramanian 2015), this mandate can be seen as very restrictive.

Apparently the Commission agreed in providing answers and advice to both questions. Such advice, though, leaves the most important part of the investigation (which concerns the assembly of an appropriate methodology to estimate the level and the evolution of global poverty) for another time. Conversely, that part would be to investigate whether the Bank’s method is appropriate, as well as consistent; and by which standards.

The Commission is careful in clarifying this point, and tries to avoid directly commenting positively or negatively upon the Bank’s method: “[t]he Commission was not asked to evaluate the procedures by which the World Bank arrived at the October 2015 estimates” (Atkinson 2016, p. 14).3

However, the Commission does not discuss whether or not the first point of its mandate (that is: keeping the iPL constant in real terms) is actually possible. Doing so would violate its mandate to keep the definition in PPP terms as stipulated in the question itself. This presents itself as a classic catch-22 clause. There are good empirical grounds that cast doubt about the feasibility of the task, broadly speaking. Moatsos (2016) shows that the dollar a day method buys a different basket each year in each country, therefore it is impossible to be held constant in PPP dollar and real welfare terms overall.4 It is very unlikely that future price trends (as captured by average CPI) will evolve is such differentials globally as to conspire and allow for an iPL to buy a welfare equivalent consumption basket all around the globe. Thus the Commission ought to be both diplomatic and creative.

Succeeding on a Mission Impossible

The report is delivered with two glosses: (1) how to determine the change in poverty over time, and (2) what is the margin of error in the estimates. The important issue here is that the two have to be both present. The first one cannot materialize without the second. One cannot monitor the path towards fulfilling SDG 1.15 without the first gloss, and one cannot say if the SDG 1.1 has been fulfilled without the second gloss, since uncertainty needs to be accounted for.

The recommendations–although stated independently–must be applied in combination to bear fruit. For example, the Commission regarding the first question offers five methodological recommendations that should be read jointly.6 It advices the Bank to first convert and report iPL in local currency units (recommendation 1) and explain the differences with the National Poverty Line (recommendation 2). Then the Bank should apply an appropriate price index for the poor (recommendation 9), and never touch the dollar denominated value of iPL (recommendation 10).7 This is the only way to maintain the real value of the $1.90-a-day (in 2011 PPP terms) and only for each country separately. Implicitly, what is supported by the Commission is that there is no way to build another iPL in dollar terms that would be welfare equivalent to that of the 2011 ICP round. Instead if we convert the dollar value to local currencies and apply an appropriate price index, we won’t have welfare equivalence among countries, but we will have between years and within each country separately. If this is applied to all countries individually, the welfare content of the iPL on a global scale would be as constant as possible.8 Take good notice that this does not mean that the iPL will be welfare equivalent among countries. It is a method that holds the welfare content of the iPL within each country–whatever that is–constant between years; thus keeping the recommendations within the Commission’s strict mandate.

Therefore the Commission implicitly accepts the substantial empirical issue–that the iPL buys a different basket each year and each country–and advices the Bank to only use improved versions of domestic CPI index in order for the iPL to maintain the same real value–at least–in domestic (only) terms. This is a very important point in reading the report as it is the way that the Commission rejects the premise that it is good practice to re-estimate the iPL in every ICP round, and hoping to find comparable and consistent results. Thus the validity of the Bank’s dollar a day method is critically undermined. It is with great concern that one reads the Bank’s response on this issue:

“[…] we plan to follow this recommendation, but leave open the possibility that future PPP rounds might be used again to inform the construction of the IPL, even before 2030, if and only if we are satisfied that the ICP methods have substantially stabilized over at least two ICP rounds. This would require that our own researchers, and the broader scholarly community, are largely of the view that changes in PPPs are then driven by real changes in cost-of-living parities, rather than in data collection or index-number methodology.” (The World Bank 2016, p. 3).

Quick reading this passage gives the impression that it consists a reasonable response to the Commission. However, in light of the clarifications presented above, it rather cancels the rationale of the entire Report.

No Estimate Without Error, Period

Regardless of any other kind or type of recommendation one can offer to the Bank, the issue of measurement error is second to none. Not only in measuring poverty, but in measuring anything. There exists no meaningful real world estimate or measurement without being accompanied by its margin of error. The entire body of natural and social sciences builds upon this.

Naturally acknowledging this, the Commission advices in recommendation 5 the following (italics added):

The World Bank poverty estimates should be based on a “total error” approach, evaluating the possible sources, and magnitude, of error, particularly nonsampling error and the error introduced by the process of determining the International Poverty Line.

The Bank offers in return (italics added):

In our minds, this is one of the Commission’s most important recommendations. World Bank economists have often felt uneasy about the misleading precision with which our poverty estimates appear to become imbued in public debate, despite our best efforts to document in detail the very significant uncertainty involved in each of the various steps leading up to the final numbers. Presenting formal estimates of statistical confidence on the numbers, analogous to the confidence intervals common in usual econometric inference, would enhance the public understanding of the true nature of these estimates. That said, we feel that we do not currently possess the in-house statistical capacity to correctly produce estimates of “total error” arising from the multiplicity of possible sources of error listed above. Given resources, we will invest in building such capacity, with a view to computing “total” confidence intervals for our poverty statistics as soon as it is feasible.

Apparently the Bank asserts that if one cannot perfectly specify the margin of error, it is better not to offer any. Nothing can be further from the truth^. As specified in the recommendation a key element in estimating the “total error” is to account for the error introduced by the very method that the iPL is estimated. This–not surprisingly–is quite elementary, and it also has a very large value. Which makes the response of the Bank very questionable.

Using a Monte-Carlo approach to account for uncertainty, Moatsos and Lazopoulos (2018) estimate that the confidence interval of the Dollar-a-day in estimating the number of people living in poverty in 1990 spans from 500 million to almost 3 billion people.9 These margins of error consist a severe limitation built-in the dollar-a-day methodology that is rarely acknowledged, both by the Bank and its critics. The Atkinson Commission Report aims at correcting this, but the Bank seems not too keen to listen, by pleading over the complexity of the issue in its rather most extreme form.

Researchers ought not to be kind to the Bank over this critical issue.

Post-Recommendations

Firstly, the Bank should refrain from concluding points from the Report–even implicitly–that are strictly not contained in the Report. For example, the Bank writes (stress added):

“Recommendation 11 suggests that the Bank’s traditional focus on the incidence of absolute income or consumption poverty (or “money-metric deprivation”) should be augmented–but not replaced–by attention to various alternative indicators”.

To some, this text might imply that the Commission endorses the method of the Bank for monitoring global poverty, by not asking to replace it. As already mentioned, this is clearly not the case as stated in the report: “[t]he Commission was not asked to evaluate the procedures by which the World Bank arrived at the October 2015 estimates” (Atkinson 2016, p. 14). Thus the Bank should not–in any possible way–attempt to imply that the Commission did.

Secondly, cyclical arguments are not very useful in promoting a mutually beneficial debate. For example, on recommendation 15 (regarding the promotion of the basic needs approach) the Bank claims that it resists from sponsoring “a technocratic alternative led by various international agencies”, and instead support that the “best definition of poverty is one that reflects a measure of societal agreement within the country”. This of course goes directly against the main point of that “the first-order issue is to demand welfare consistency [...] when measuring global poverty” (Ravallion 2015, p. 4). In effect though, the iPL is in itself a “technocratic alternative” in the form of methodological obscurity and inconsistency, leaving large room for an array of measurement errors. As it can be logically concluded from the mandate, the Commission was strictly instructed not to investigate the Bank’s method.

Finally, it is true that standing corrected is a virtue, not a sin. This holds for both people and legal entities.

Acknowledgement

In writing this text I have benefited greatly from discussions with Bas van Bavel, Auke Rijpma, and Jan Luiten van Zanden. The usual disclaimer applies.

Appendix

Recommendations as Appreciated by the Bank

What follows is both an augmented and curtailed version of the 21 recommendations of the Commission on Global Poverty. They are augmented in the sense that comments with respect to ramifications introduced by the Bank are provided in italics based on The World Bank (2016). And they are curtailed in the sense that those recommendations that the Bank does “not currently plan to pursue” and those sections actually excluded from the recommendations that the Bank only partially endorses, are noted with a strike-through typesetting. Also those recommendations fully and without any qualifications are marked as bold.

Before each recommendation a categorization is offered according to three different methods. The first follows that of the The World Bank (2016) in terms of the type of verbal acceptance from the side of the Bank, with (1) accepted for immediate incorporation, (2) conditional according to prevailing circumstances such as financial resources, and (3) not pursued at this point. The second categorization follows that of Atkinson (2016) in terms of correspondence with (1) monitoring the development of SDG 1.1, (2) monitoring poverty beyond the definition of SDG1.1, and (3) overarching recommendations in order to “[make] it happen”. The third categorization transcends the entire report, and splits the set of recommendations simply among those that address issues of methodology (13 recommendations), and those dealing with data issues (8).

Please note that there is more distance in opinion between the Bank’s response and the Report than what is listed here. For example, with respect to the second recommendation, the Commission is specific that the statistics on the National Poverty Statistics Reports (NPSR) must be given with caution: “To this end, there needs to be more explanatory text and fewer graphs; and there should be clear warnings about the margins of error that surround the estimates”. Thus inspire that the Bank informs as that it accepts recommendation 2, the careful reader understands that this is only partially so. The error terms required in the Report where this recommendation is explained will not be provided as the Bank will not implement recommendation 5 in the foreseeable future.

  1. (WB 1; CoP 1; Method) The global extreme poverty standard should be cited in general terms as “the International Poverty Line”, and expressed in each country in terms of the currency of that country. Reference to the $1.9 a day will still be used as “ancillary explanation”.

  2. (WB 1; CoP 1; Method) There should be National Poverty Statistics Reports (NPSR) for each country, giving the Global Poverty estimates, explaining the local currency value of the International Poverty Line and the relation to the official poverty line(s) in that country (where they exist), considering how the trends in poverty measured according to the International Poverty Line relate to those shown by national statistics, and incorporating a set of World Bank Complementary Indicators, as proposed in chapter 2 of this Report.

  3. (WB 2; CoP 1; Data) There should be an investigation of the extent to which people are “missing” from the global poverty count, and proposals made for adjustments where appropriate at the national level for survey underrepresentation and noncoverage by surveys; more generally, the World Bank should carry out a review, in conjunction with other members of the UN statistical system, of the fitness for purpose of the baseline population data for each country, and the methods used to update from the baseline to the years covered by the global poverty estimates. “Given sufficient resources […] will endeavor to set up each of these partnerships […]

  4. (WB 2; CoP 1; Data) The World Bank should take the lead in a standing Joint Statistical Working Group for household consumption statistics, with a remit to set guidelines for the measurement of household consumption, to examine the relation between consumption and income, and to investigate the relation among household survey, national accounts, and other data sources. “Given sufficient resources […] will endeavor to set up each of these partnerships […]

  5. (WB 2; CoP 1; Method) The World Bank poverty estimates should be based on a “total error” approach, evaluating the possible sources, and magnitude, of error, particularly nonsampling error and the error introduced by the process of determining the International Poverty Line. “Given resources, we will invest in building such capacity, with a view to computing “total” confidence intervals for our poverty statistics as soon as it is feasible.

  6. (WB 1; CoP 1; Data) The World Bank should make public the principles according to which household survey data are selected for use in the global poverty count; and there should be an assessment at national level of the availability and quality of the required household survey data, and a review of possible alternative sources and methods of ex post harmonization.

  7. (WB 2; CoP 1; Data) The World Bank, in conjunction with national statistical agencies and other statistical bodies, should explore the construction of an annual national accounts–based indicator of household living standards, as measured by consumption defined in a way that matches as far as possible household survey practice. “Given sufficient resources […] will endeavor to set up each of these partnerships […]

  8. (WB 2; CoP 1; Data) There should be an investigation for a small number of countries by the World Bank of alternative methods of providing up-to-date poverty estimates using scaled-down surveys, or the SWIFT or other surveys, plus modeling, where the appropriate methods may vary across countries. “To the extent that generalizable lessons arise from those pilots, it may become possible to integrate them into the mainstream global poverty undertaking”.

  9. (WB 2; CoP 1; Method) The World Bank, as a user of Consumer Price Indexes (CPI), should, in conjunction with the responsible international bodies and with the national statistical agencies, seek to improve the quality of the domestic CPI, with particular references to those aspects most relevant to global poverty measurement; this should include examination of the likely magnitude of any bias, and exploration of special price indexes for the poor. “Given sufficient resources […] will endeavor to set up each of these partnerships […]

  10. (WB 1; CoP 1; Method) The global poverty estimates should be updated up to 2030 on the basis of the International Poverty Line for each country set in local currency, and updated in line with the change in the national CPI or, where available, national index of prices for the poor; the estimates would not be revised in the light of new rounds of the ICP.“[W]e plan to follow this recommendation, but leave leave open the possibility that future PPP rounds might be used again to inform the construction of the IPL, even before 2030, if and only if we are satisfied that the ICP methods have substantially stabilized over at least two ICP rounds.

  11. (WB 1; CoP 2; Method) The Bank should publish, alongside the global poverty count, a portfolio of Complementary Indicators, including a multidimensional dashboard of outcome indicators, where the number of such indicators should be sufficiently small that they can receive prominence in public debate and in policy making; the selection of the Complementary Indicators should be based on an explicit set of principles, and the implementation of these principles should follow external consultation, including with the proposed external audit body.

  12. (WB 1; CoP 2; Method) The portfolio of Complementary Indicators should include the mean poverty gap, relative to the International Poverty Line measured over the whole population and expressed as a percentage of the poverty line.

  13. (WB 1; CoP 2; Data) The global poverty figure, and the counterpart national figures, should be accompanied by estimates of the numbers of women, children, and young adults living in households with consumption below the International Poverty Line, as well as the number of female–headed households below the International Poverty Line. “[M]ore work is needed before a gender profile of global poverty that is truly meaningful can be presented.

  14. (WB 3; CoP 2; Data) The World Bank should explore the use of subjective assessments of personal poverty status (in “quick” surveys of poverty), and of the minimum consumption considered necessary to avoid extreme poverty, as an aid to interpreting the conclusions drawn from the global poverty estimates.

  15. (WB 3; CoP 2; Method) The World Bank should develop a program of work, in conjunction with other international agencies, on a basic needs–based estimate of extreme poverty; these estimates would, when developed, form an alternative indicator to be included in the portfolio of Complementary Indicators, and serve to provide an interpretation of what the International Poverty Line would buy.

  16. (WB 1; CoP 2; Method) The World Bank should introduce a “societal” head count measure of global consumption poverty that takes account, above an appropriate level, of the standard of living in the country in question, thus combining fixed and relative elements of poverty.

  17. (WB 3; CoP 2; Method) The indicator for the shared prosperity goal should be unambiguously stated as raising the living standards of the bottom 40 percent in each country (not confounded with their relative share), and extended to include an indicator identifying the growth of per capita real consumption of the bottom 40 percent of the world distribution of consumption.

  18. (WB 1; CoP 2; Method) The World Bank should establish its own requirements with regard to the measurement of nonmonetary poverty, for inclusion in the Complementary Indicators (including the overlapping poverty measure) and in other World Bank uses, and ensure that these are fully represented in the activities of the international statistical system, particularly with regard to the proposed SDG indicators.

  19. (WB 1; CoP 2; Method) The Complementary Indicators should include a multidimensioned poverty indicator based on the counting approach.

  20. (WB 1; CoP 3; Data) There should be a major investment in statistical sources and analysis, with these activities being accorded a high priority in the work of the World Bank.

  21. (WB 3; CoP 3; Method) The International Poverty Line estimates and the proposed Complementary Indicators should be audited on a regular basis by a body fully external to the World Bank, and this body should be consulted about major changes in methodology.

References

  • Aten, B., and A. Heston. 2010. “Use of Country Purchasing Power Parities for International Comparisons of Poverty Levels: Potential and Limitations.” In Debates on the Measurement of Global Poverty, edited by S. Anand, P. Segal, and J. E. Stiglitz. Oxford: Oxford University Press. Google Scholar

  • Atkinson, A. B. 2016. Monitoring Global Poverty: Report of the Commission on Global Poverty. Washington, DC: The World Bank. Google Scholar

  • Deaton, A. 2010. “Measuring Poverty in a Growing World (or Measuring Growth in a Poor World).” In Debates on the Measurement of Global Poverty, edited by S. Anand, P. Segal, and J. E. Stiglitz, 187– 222. Oxford: Oxford University Press . Google Scholar

  • Ferreira, F. H. G., S. Chen, A. Dabalen, D. Jolliffe, A. Narayan, E. B. Prydz, A. Revenga, P. Sangraula, and N. Yoshida. 2015. “A Global Count of the Extreme Poor in 2012: Data Issues, Methodology and Initial Results.” Policy Research Working Paper 7432. Washington, DC: The World Bank. Google Scholar

  • Moatsos, M. 2016 . “Global Absolute Poverty: Behind the Veil of Dollars.” The Journal of Globalization and Development 7 (2): 1– 28 . DOI: . CrossrefGoogle Scholar

  • Moatsos, M., and A. Lazopoulos. 2018. “Not Beyond Reasonable Doubt: The First Millennium Development Goal.” Working Paper in PreparationGoogle Scholar

  • Ravallion, M. 2015. “Toward Better Global Poverty Measures.” The Journal of Economic Inequality 24 (2): 227–248. DOI: . CrossrefGoogle Scholar

  • Reddy, S. G., and T. Pogge. 2010. “How not to Count the Poor.” In Debates on the Measurement of Global Poverty, edited by S. Anand, P. Segal, and J. E. Stiglitz, 42–51. Oxford: Oxford University Press. Google Scholar

  • Srinivasan, T. N. 2010. “Irrelevance of the $1‐a‐Day Poverty Line.” In Debates on the Measurement of Global Poverty, edited by S. Anand, P. Segal, and J. E. Stiglitz, 143–151. New York: Oxford University Press. Google Scholar

  • Subramanian, S. 2015 . “Once More Unto The Breach.” Economic & Political Weekly 50 (45): 35– 40. Google Scholar

  • The World Bank. 2016. A cover note to the Report of the Commission on Global Poverty. Google Scholar

Footnotes

  • 1

    Do note that I am discussing here only the recommendations related to issues of method, as the data related recommendations are hardly debatable. See Appendix for the categorization and an entire list of the recommendations 

  • 2

    The Commission is not entirely external to the Bank, as out of its 24 members six are directly linked with the Bank by holding key positions there either currently or recently. 

  • 3

    It was then that the Bank made public the new $1.90 dollar a day line (Ferreira et al. 2015). 

  • 4

    Overall in the sense of both time (between years) and space (between different countries). 

  • 5

    The United Nations SDG 1.1 states that: “By 2030, eradicate extreme poverty for all people everywhere, currently measured as people living on less than $1.25 a day” and its accompanying indicator is the “Proportion of population below the international poverty line, by sex, age, employment status and geographical location (urban/rural)”. 

  • 6

    This is something that the report does not do, as it treats each recommendation mostly in isolation. 

  • 7

    On the above, recommendation 5 needs to be added about including the margin of error in the estimates. This is so fundamental that deserves special attention. See section below. 

  • 8

    This is also the rationale behind figure 1.2 in the report. 

  • 9

    This research has been presented in PEGNet 2017 in Zurich under the title “Halving Global Poverty: Reality or Artifact?”, and is available by the authors upon request. 

About the article

Published Online: 2018-10-13


Citation Information: The Economists’ Voice, 20180004, ISSN (Online) 1553-3832, DOI: https://doi.org/10.1515/ev-2018-0004.

Export Citation

©2018, Michail Moatsos published by DeGruyter, Berlin/Boston. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in