Abstract
This paper takes a fresh look at citation counts and publications in top-rank journals, which the academic economics profession uses to evaluate and promote its members. It first examines how and why citations are mentioned in an article, and what this implies about their counts. The discussion then examines how average citation counts to articles are used to rank journals, and the paper reviews the concerns that have been expressed about this practice. These concerns identify the large variance in citation counts among articles of the same journal, implying that those articles themselves must vary greatly in quality (Engemann and Wall 2009). To address these concerns, the paper proposes the classification of citations into three categories: Fodder Citations (for references that contribute only trivially to a paper), Relevant Citations (which substantively contribute to the paper, though the paper would remain roughly the same without them), and Essential Citations (which have a major influence). The paper argues that counts of citations by the last two categories offers greater credibility in the application of citation counts to evaluate economic literature. Finally, the paper provides an opportunity for economists to participate in a new project that solicits information on citations by these categories.
References
In accordance with the classification of citations in this paper {“Payson (2019)” as it should be listed in future papers}, the Essential Citations for this paper were Colussi (2018), Engemann and Wall (2009), Heckman and Moktan (2018), and Payson (2017), and the additional Relevant Citations were Akerlof (2018) and Stern (2013).Search in Google Scholar
Akerlof, G. A. (2018, January 5-7). Sins of Omission and the Practice of Economics. Paper presented on Annual Meetings of the American Economic Association, Philadelphia, PA.Search in Google Scholar
Bergstrom, C. (2007). Eigenfactor: Measuring the Value and Prestige of Scholarly Journals. College and Research Libraries News, 68 (5), 314–16.10.5860/crln.68.5.7804Search in Google Scholar
Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A Principal Component Analysis of 39 Scientific Impact Measures. PLoS One, 4 (6), e6022.10.1371/journal.pone.0006022Search in Google Scholar
Bostaph, S. (2018) Book Review: How Economics Professors Can Stop Failing Us. Quarterly Journal of Austrian Economics, 21 (1), 79-86.Search in Google Scholar
Card, D., & DellaVigna, S. (2013). Nine Facts About Top Journals in Economics. Journal of Economic Literature, 51 (1), 144-161.10.1257/jel.51.1.144Search in Google Scholar
Colussi, T. (2018). Social Ties in Academia: A Friend is a Treasure. The Review of Economics and Statistics, 100 (1), 45-50.10.1162/REST_a_00666Search in Google Scholar
Engemann, K. M., & Wall, H. (2009). A Journal Ranking for the Ambitious Economist. Federal Reserve Bank of St. Louis Review, 90 (3), 127-39.10.20955/r.91.127-140Search in Google Scholar
Hamermesh, D., (2015). Citations in Economics: Measurement, Uses and Impacts. Working Paper No. 21754. National Bureau of Economic Research.10.3386/w21754Search in Google Scholar
Heckman, J. & Moktan, S. (2018). “Publishing and Promotion in Economics: The Tyranny of the Top Five,” National Bureau of Economic Research Working Paper10.3386/w25093Search in Google Scholar
Kalaitzidakis, P, Mamuneas, T. P., & Stengos, T. (2003). Rankings of Academic Journals and Institutions in Economics. Journal of the European Economic Association 1 (6), 1346–66.10.1162/154247603322752566Search in Google Scholar
Laband, D. N., & Piette, M. J. (1994). The Relative Impacts of Economics Journals: 1970–1990. Journal of Economic Literature, 32 (2), 640–66.Search in Google Scholar
Liebowitz, S. J., & Palmer, J. P. (1984). Assessing the Relative Impacts of Economics Journals. Journal of Economic Literature, 22 (1), 77–88.Search in Google Scholar
Moed, H. F. (2010). Measuring Contextual Citation Impact of Scientific Journals. Journal of Informetrics, 4 (3), 265–77.10.1016/j.joi.2010.01.002Search in Google Scholar
Ӧnder, A., & Terviӧ, M. (2015). Is Economics a House Divided? Analysis of Citation Networks. Economic Inquiry. 53 (3), 1491-1505.10.1111/ecin.12164Search in Google Scholar
Oswald, A. J. (2007). An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers. Economica, 74 (293), 21-31.10.1111/j.1468-0335.2006.00575.xSearch in Google Scholar
Palacios-Huerta, I., & Volij, O. (2004). The Measurement of Intellectual Influence. Econometrica, 72 (3): 963–77.10.1111/j.1468-0262.2004.00519.xSearch in Google Scholar
Payson, S. (2017). How Economics Professors Can Stop Failing Us: The Discipline at a Crossroads. Lanham, MD: Rowman and Littlefield.Search in Google Scholar
Powdthavee, N., Riyanto, Y. E., & Knetsch, J. L. (2017). Impact of Lower Rated Journals on Economists’ Judgments of Publication Lists: Evidence from a Survey Experiment. IZA Discussion Papers 10752, Institute for the Study of Labor (IZA).10.2139/ssrn.2964161Search in Google Scholar
Powdthavee, N., Knetsch, J., & Riyanto, Y., (2018, January 5-7). Negative Impact of Lower Rated Journals on Economists’ Judgments of Publication Lists: Evidence from a Survey Experiment. Paper presented on Annual Meetings of the American Economic Association, Philadelphia, PA. https://tinyurl.com/ycy69mbe.10.2139/ssrn.2964161Search in Google Scholar
Pinski, G., & Narin, F. (1976). Citation Influence for Journal Aggregates of Scientific Publications: Theory, with Application to the Literature of Physics. Information Processing and Management, 12 (5), 297–312.10.1016/0306-4573(76)90048-0Search in Google Scholar
RePEc. (2018a). Ideas now has a fantasy League. The RePEc Blog. Accessed August 5, 2018. https://tinyurl.com/yamn5wye.Search in Google Scholar
RePEc. (2018b). Ideas Fantasy League. Accessed August 5, 2018. https://ideas.repec.org/fantasy.html.Search in Google Scholar
RePEc. (2018c). Top 10% Authors, as of July 2018. Accessed August 11, 2018. https://ideas.repec.org/top/top.person.all.html.Search in Google Scholar
Stern, D. I. (2013). Uncertainty Measures for Economics Journal Impact Factors. Journal of Economic Literature, 51 (1), 173-189.10.1257/jel.51.1.173Search in Google Scholar
Wall, H. J. (2009). Journal Rankings in Economics: Handle with Care. Working Paper No. 2009-014A. Federal Reserve Bank of St. Louis.10.20955/wp.2009.014Search in Google Scholar
Zimmerman, C. (2012). Academic Rankings with RePEc. Working Paper 2012-023A. Federal Reserve Bank of St. Louis. Open Linguistics 2019; 5:94–10810.20955/wp.2012.023Search in Google Scholar
© 2019 Steven Payson, published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.
