Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access May 22, 2019

Cite This Economics Paper! It Is Time for the House of Cards to Fall Down

  • Steven Payson EMAIL logo
From the journal Open Economics


This paper takes a fresh look at citation counts and publications in top-rank journals, which the academic economics profession uses to evaluate and promote its members. It first examines how and why citations are mentioned in an article, and what this implies about their counts. The discussion then examines how average citation counts to articles are used to rank journals, and the paper reviews the concerns that have been expressed about this practice. These concerns identify the large variance in citation counts among articles of the same journal, implying that those articles themselves must vary greatly in quality (Engemann and Wall 2009). To address these concerns, the paper proposes the classification of citations into three categories: Fodder Citations (for references that contribute only trivially to a paper), Relevant Citations (which substantively contribute to the paper, though the paper would remain roughly the same without them), and Essential Citations (which have a major influence). The paper argues that counts of citations by the last two categories offers greater credibility in the application of citation counts to evaluate economic literature. Finally, the paper provides an opportunity for economists to participate in a new project that solicits information on citations by these categories.


In accordance with the classification of citations in this paper {“Payson (2019)” as it should be listed in future papers}, the Essential Citations for this paper were Colussi (2018), Engemann and Wall (2009), Heckman and Moktan (2018), and Payson (2017), and the additional Relevant Citations were Akerlof (2018) and Stern (2013).Search in Google Scholar

Akerlof, G. A. (2018, January 5-7). Sins of Omission and the Practice of Economics. Paper presented on Annual Meetings of the American Economic Association, Philadelphia, PA.Search in Google Scholar

Bergstrom, C. (2007). Eigenfactor: Measuring the Value and Prestige of Scholarly Journals. College and Research Libraries News, 68 (5), 314–16.10.5860/crln.68.5.7804Search in Google Scholar

Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A Principal Component Analysis of 39 Scientific Impact Measures. PLoS One, 4 (6), e6022.10.1371/journal.pone.0006022Search in Google Scholar

Bostaph, S. (2018) Book Review: How Economics Professors Can Stop Failing Us. Quarterly Journal of Austrian Economics, 21 (1), 79-86.Search in Google Scholar

Card, D., & DellaVigna, S. (2013). Nine Facts About Top Journals in Economics. Journal of Economic Literature, 51 (1), 144-161.10.1257/jel.51.1.144Search in Google Scholar

Colussi, T. (2018). Social Ties in Academia: A Friend is a Treasure. The Review of Economics and Statistics, 100 (1), 45-50.10.1162/REST_a_00666Search in Google Scholar

Engemann, K. M., & Wall, H. (2009). A Journal Ranking for the Ambitious Economist. Federal Reserve Bank of St. Louis Review, 90 (3), 127-39.10.20955/r.91.127-140Search in Google Scholar

Hamermesh, D., (2015). Citations in Economics: Measurement, Uses and Impacts. Working Paper No. 21754. National Bureau of Economic Research.10.3386/w21754Search in Google Scholar

Heckman, J. & Moktan, S. (2018). “Publishing and Promotion in Economics: The Tyranny of the Top Five,” National Bureau of Economic Research Working Paper10.3386/w25093Search in Google Scholar

Kalaitzidakis, P, Mamuneas, T. P., & Stengos, T. (2003). Rankings of Academic Journals and Institutions in Economics. Journal of the European Economic Association 1 (6), 1346–66.10.1162/154247603322752566Search in Google Scholar

Laband, D. N., & Piette, M. J. (1994). The Relative Impacts of Economics Journals: 1970–1990. Journal of Economic Literature, 32 (2), 640–66.Search in Google Scholar

Liebowitz, S. J., & Palmer, J. P. (1984). Assessing the Relative Impacts of Economics Journals. Journal of Economic Literature, 22 (1), 77–88.Search in Google Scholar

Moed, H. F. (2010). Measuring Contextual Citation Impact of Scientific Journals. Journal of Informetrics, 4 (3), 265–77.10.1016/j.joi.2010.01.002Search in Google Scholar

Ӧnder, A., & Terviӧ, M. (2015). Is Economics a House Divided? Analysis of Citation Networks. Economic Inquiry. 53 (3), 1491-1505.10.1111/ecin.12164Search in Google Scholar

Oswald, A. J. (2007). An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers. Economica, 74 (293), 21-31.10.1111/j.1468-0335.2006.00575.xSearch in Google Scholar

Palacios-Huerta, I., & Volij, O. (2004). The Measurement of Intellectual Influence. Econometrica, 72 (3): 963–77.10.1111/j.1468-0262.2004.00519.xSearch in Google Scholar

Payson, S. (2017). How Economics Professors Can Stop Failing Us: The Discipline at a Crossroads. Lanham, MD: Rowman and Littlefield.Search in Google Scholar

Powdthavee, N., Riyanto, Y. E., & Knetsch, J. L. (2017). Impact of Lower Rated Journals on Economists’ Judgments of Publication Lists: Evidence from a Survey Experiment. IZA Discussion Papers 10752, Institute for the Study of Labor (IZA).10.2139/ssrn.2964161Search in Google Scholar

Powdthavee, N., Knetsch, J., & Riyanto, Y., (2018, January 5-7). Negative Impact of Lower Rated Journals on Economists’ Judgments of Publication Lists: Evidence from a Survey Experiment. Paper presented on Annual Meetings of the American Economic Association, Philadelphia, PA. in Google Scholar

Pinski, G., & Narin, F. (1976). Citation Influence for Journal Aggregates of Scientific Publications: Theory, with Application to the Literature of Physics. Information Processing and Management, 12 (5), 297–312.10.1016/0306-4573(76)90048-0Search in Google Scholar

RePEc. (2018a). Ideas now has a fantasy League. The RePEc Blog. Accessed August 5, 2018. in Google Scholar

RePEc. (2018b). Ideas Fantasy League. Accessed August 5, 2018. in Google Scholar

RePEc. (2018c). Top 10% Authors, as of July 2018. Accessed August 11, 2018. in Google Scholar

Stern, D. I. (2013). Uncertainty Measures for Economics Journal Impact Factors. Journal of Economic Literature, 51 (1), 173-189.10.1257/jel.51.1.173Search in Google Scholar

Wall, H. J. (2009). Journal Rankings in Economics: Handle with Care. Working Paper No. 2009-014A. Federal Reserve Bank of St. Louis.10.20955/wp.2009.014Search in Google Scholar

Zimmerman, C. (2012). Academic Rankings with RePEc. Working Paper 2012-023A. Federal Reserve Bank of St. Louis. Open Linguistics 2019; 5:94–10810.20955/wp.2012.023Search in Google Scholar

Received: 2018-08-14
Accepted: 2018-12-31
Published Online: 2019-05-22

© 2019 Steven Payson, published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 9.12.2023 from
Scroll to top button