! It Is Time for the House of Cards to Fall Down

This paper takes a fresh look at citation counts and publications in top-rank journals, which the academic economics profession uses to evaluate and promote its members. It first examines how and why citations are mentioned in an article, and what this implies about their counts. The discussion then examines how average citation counts to articles are used to rank journals, and the paper reviews the concerns that have been expressed about this practice. These concerns identify the large variance in citation counts among articles of the same journal, implying that those articles themselves must vary greatly in quality (Engemann and Wall 2009). To address these concerns, the paper proposes the classification of citations into three categories: Fodder Citations (for references that contribute only trivially to a paper), Relevant Citations (which substantively contribute to the paper, though the paper would remain roughly the same without them), and Essential Citations (which have a major influence). The paper argues that counts of citations by the last two categories offers greater credibility in the application of citation counts to evaluate economic literature. Finally, the paper provides an opportunity for economists to participate in a new project that solicits information on citations by these categories.


Introduction
The more familiar one is with the publication process in academic economics, the more one is inclined to be amused by the title of this paper. Citing someone's work is often seen as a "favor to them." Thus, the audacity of this paper to suggest that it be cited by anyone who happens to read it is perhaps analogous to the reaction that an attractive person might have to a perfect stranger approaching them and asking for an intimate relationship. However, whatever the topic of one's research in economics, if there is any effort to get it published (as presumably there is), then that work will necessarily be integrally involved in a "citation system" -a system that drags with it the baggage of whatever references that research will cite, and whatever references will cite it. So, actually, there is a good reason for anyone doing research in economics to cite this paper, because this paper has everything to do with the citation system upon which their work relies.
Many authors maintain the logical and correct perspective that they should cite references that are relevant to their own work. Unfortunately, others take a different perspective, where they might respond to the title of this paper by asking "Why should I cite your paper? I don't even know you, and I certainly don't owe you any favors, nor do I have anything to gain from citing your paper." This type of reaction -or mentality about citations (for lack of a better term), points to the horrific effect that the citation system has had on the academic economics profession. By the citation system I do not mean citation counts alone, but everything that is downstream from them as well. This system thus includes publication counts that The simple IF is defined as the mean number of citations received in a given year by articles published in a journal in a number of preceding years. … Recursive or iterative impact factors ... use an iterative process to compute the factors while taking into account the prestige of the citing journals. Recursive IFs better reflect journal prestige while simple IFs reflect popularity (Bollen et al. 2009). Other publicly available indicators of journal quality are the source-normalized impact ... provided by Elsevier that takes into account the differing citation potential in different fields and the journal h-index provided by Google Scholar Metrics.
Note, also, that the ninth citation in the last sentence of the earlier quotation (to Moed 2010) was not needed either, since that idea associated with the citation only described an easily observable fact about what the publisher Elsevier happens to do.
Adding to this absurdity is the fact that there was absolutely no requirement on the part of the author to know anything about these nine citations that he wrote into his paper, beyond the tiniest of the arbitrary ideas with which those citations could be affiliated, such as being one of the hundreds of articles that mention impact factors. (This is not to say that the author had not studied these cited works, only that there was no need for him or her to do so.) This contrasts sharply with the heroic, implicit assumption made by the profession that the authors cite only what they believe are valuable contributions to knowledge.
In reality, authors need not have ever seen the article, and thus may know nothing about what their cited work has to offer. In fact, all they need to do is to copy the citation from any another source that lists it. To demonstrate how easy this is, I have just done it myself in this paper! That is, since the nine citations that Stern provided in the long quotation above are mentioned in this paper, I could argue that, for thoroughness, I need to cite them as well, and so I have added them to this paper's reference section. Now, I, too, have played the game of appearing to have performed a thorough literature review in writing this paper! Let us now further examine the one citation, out of the nine referred to above, that was uniquely legitimate (though barely, one might add) in terms of it being associated with a particular research finding that was actually mentioned - Bollen et al., 2009. Again, the one idea for which Bollen et al. were cited was expressed in the following 11 words: "Recursive IFs better reflect journal prestige while simple IFs reflect popularity." This is the only citation of Bollen et al. in Stern's paper. What one takes away from these 11 words is some degree of endorsement for the IF measure. That is, it appears from the citation that Bollen et al., on the basis of their study, acknowledge that the IF measure is generally useful -as a measure of prestige when it is a recursive IF and as a measure of popularity when it is a simple IF. However, when one actually looks at the Bollen et al. article itself, one finds that the article's main thesis is that citation counts, and impact factors in particular, are not adequate measures of "scholarly impact" or "scientific impact." In fact, the abstract of their paper states: Our results indicate that the notion of scientific impact is a multi-dimensional construct that cannot be adequately measured by any single indicator, although some measures are more suitable than others. The commonly used citation Impact Factor is not positioned at the core of this construct, but at its periphery, and should thus be used with caution.
The above quotation is a fancy and polite statement that the true intellectual importance or contribution of published work is not well captured by the impact factor. This is a contradiction of the 11 words that Stern attributed to Bollen et al.'s work in his citation to them, where he implied that their article endorsed the IF as a useful measure.
To recap, the initial long quotation in Stern's article, which happened to have been only a footnote in his paper, contained nine citations, only one of which was legitimate in the context of referencing a particular research finding as opposed to being an arbitrary draw of possible citations from hundreds that simply mention a particular topic. That one legitimate citation was attributed to an idea that was not only peripheral to the main theme of the cited paper, but actually contradicted it. This situation, of citations meaning essentially nothing, and thus not reflecting the contribution to knowledge that they are alleged to reflect, is not uncommon at all in economic papers. Indeed, it is much more the rule than the exception. The citation system, in short, is a "house of cards," which is already common knowledge in the academic economics profession. Fact #3. Citations and publication counts are, themselves, the express end goalthe final objectiveof most of the academic economics profession. In countless articles and books on the topic, researchers have measured terms like "value" and "quality" of research, and even "value" and "quality" of researchers themselves, as simply monotonic functions of the publication counts (usually weighted by the ranks of journals) to their credit, or monotonic functions of their citation counts (see for example Önder and TerviÖ 2015 and Kalaitzidakis et al. 2003).
The evidence that the profession treats citation and publication counts as an end goal in itself, in contrast to the promoting genuine discovery, is pervasive. For example, in David Card's Presidential Lecture during the 2016 Annual Meeting of the Western Economic Association International, he "advised members of his audience to write papers intended to receive a multitude of citations if they desired publication in top-ranked journals" (Bostaph 2018, p. 81). Similarly, "at the annual conference of the Southern Economics Association, keynote speaker Andrei Schleifer was introduced as the most cited economist in the world, as if this was his key accomplishment." (Bostaph 2018, p. 81).
At the annual meetings of the American Economic Association in 2018, a paper was presented by Powdthavee et al. that greatly showed how much publications, rather than genuine discovery itself, was the end goal of academic economists. The premise of their paper was that, in applying for teaching positions, academic economists need to choose which of their publications they should list on their CV: Should they list all their publications, or only the ones in top-ranked journals, with the idea that their publications in low-ranked journals would have a negative effect on their applications? The authors concluded, "adding publications in lower-rated journals … does have a significant negative impact on economists' judgments of the value of the author's contribution." This is troubling of course, because it begs the question of why such lower-ranked journals even exist, i.e., why people write for them, and why people read them, if they adversely affect their author's careers. It speaks as well to a related problem -the effective "cast system" that resides implicitly in the academic economics culture. You are either one of the "cool kids" with impressive publications, or a commoner who faces an upward battle for any recognition of your work.
The very first slide of their presentation (after the title slide) contained only nine words in huge letters -the question: "What represents an ideal CV for an academic economist?" Their answer on the next slide quoted from D. Card and S. DellaVigna (2013): "Publications in the top journals have a powerful influence on the direction of research in economics, on the career paths of young researchers, and on the pay of academic economists." As an aside, this quotation attributed to Card and DellaVigna is an idea that is obvious and known to virtually all academic economists. Yet, the authors of the paper felt a need to verify it, or perhaps glorify it, by creating a citation to prominent economists from an article in the prestigious Journal of Economic Literature. Here is an example of the "reverse causality" that will be discussed as Fact #6 below: The citation, in this case, does not offer any evidence for why Card and DellaVigna's paper is published in a high ranking journal -the citation only came to be because it could attribute an obvious statement to an article in a high-ranking journal. In other words, the prominence of the journal and the authors themselves caused the citation to be made, as opposed to the citation serving as independent evidence for journal's and the authors' prominence. Here we see the "cart pulling the horse" in contrast to the story that the profession likes to tell about citations being supposedly pulled by advances in knowledge.
In their very next slide, they offered the following: "The reason why most academic economists judge their own and their peers' achievements by numbers of publications in top journals is simple: the signals of achievement require very little effort in gathering information and necessitate almost no thought. Hamermesh (2015)." What was particularly amazing about this quotation -which was offered quite seriously, not as a joke -was the complacency with which the presenters of the study, as well as the audience in the room, simply accepted this explanation without objection or reservation. That is, no one said (or appeared to think) that it was a shame that this situation exists -quite the contrary -it was generally received as a blessing -because, golly gee, "look at how citation counts make our lives so easy!" Let us ask ourselves what the above quotation by Hamermesh (2015) ultimately means about the commitment (or the lack thereof) of the profession to pursuing genuine and valuable scientific discovery. What these particular words imply is indisputable: they imply that no such commitment exists -in fact, no commitment to "thinking" exists. Why "think" if you do not have to -if publication counts can do all the thinking for you? Thanks to citations and publication counts, the absence of interest in the validity and value of scientific inquiry among academic economists has become trumped by an increasingly important (for their careers that is) intellectual game of competing to get yourself published in a top journal.
The study by Powdthavee et al. surveyed professors in economics departments and asked them whether they would be willing to hire prospective job candidates on the basis of hypothetical lists of journals in which those candidates had been published. The only information provided to the survey participants, for them to base their ranking decision, was the names of the journals themselves, and nothing else -not even titles of the hypothetical articles in those journals. In other words, what the survey did was equivalent to asking participants something like the following: Here is somebody with one article in the Journal of Labor Economics, one in the Pacific Economic Bulletin, and one in Empirical Economics, so tell us how good you think this person is as an applicant, on a scale of 1 to 10? The survey participants were expected to respond to something like this, and they actually did. The underlying assumption in this study was that, for a hiring decision, all that a department needs to know is the journals in which the applicant has published, regardless of what they might have written in those articles.
According to the Powdthavee's presentation, only one person officially replied to the survey with an objection, who wrote the following to them: Wrong guy to ask, I am afraid! I simply can't answer the questions based on the publication lists. I have no idea what the paper discovers … Lots of papers in good journals are cited by nobody! Many good papers are in lesser journals. If I had citations that would be another matter.
This objection was actually presented as a source of amusement at the conference session, in the context of: "Hey look! There's always got to be a party pooper in every crowd!" Besides being a conference paper, the study by Powdthavee et al. was published in a couple other outlets, most notably a year earlier in IZA Discussion Papers which is a perfectly good economics journal. Yet, here is a question for the authors of the "Negative Impact of Lower Rated Journals": If you truly believe your own findings then how could you, yourselves, be proud of your own published work in the IZA Discussion Papers, which is, by your own definition, a lower rated journal? Such non sequiturs again reflect the house of cards that is the citation-system in academic economics (see Figure 1).

Fact #4. It is true that the most important works in economics have received the most citations, but this is irrelevant to the issue at hand.
There is no such thing as an extremely important, pivotal article in economics that does not receive an enormous number of citations. However, as in all things, what happens at the extremes is generally not what happens under normal circumstances. Consider two hypothetical extremes as a heuristic: (1) the articles that are among the top 1 percent in quality (assuming heroically for the moment that they can be objectively identified), but not among the top 0.5 percent in quality, which we could call the "one-in-a-hundred articles" and (2) those that are in the top 0.01 percent, or "one-inten-thousand articles." Everyone would then certainly expect the vast majority of the one-in-ten-thousand articles to have many more citations than the vast majority of one-in-a-hundred articles. However, this says almost nothing about what we should expect of articles that are not in the top 1 percent in quality. Among all the 99 percent of other articles, our knowing that one article received 10 citations and another 20 citations says absolutely nothing about which of these is "higher in quality." It is a safer bet to suppose that the paper with 20 citations instead of 10 was written by an author who happens to have more friends willing to cite them, or wrote on a slightly more popular topic, etc.
Fact #5. Based on the supposedly "all knowing" citation counts, publication counts (weighted by journal ranks) are widely recognized (by experts on the topic) as being invalid. Below is a passage from Engemann and Wall (2009, p. 128) where they argue, essentially, that one cannot tell the quality of an article on the basis of the rank of the journal that publishes it, because articles from the same high-ranking journal vary enormously in quality, as measured by citations: It would be a mistake, for example, to think that a journal ranking is anything like a definitive indicator of the relative quality of individual papers within the journals. First, any journal's citation distribution is heavily skewed by a small number of very successful papers, and even the highest ranked journals have large numbers of papers that are cited rarely, if at all (Oswald, 2007;Wall, 2009). Where does this place us? The profession has defined top-ranked journals as those journals with the highest average number of citations per article because it is simply assumed that the worth of a journal is measurable in this way. This is regardless of the fact that among the articles of any one journal, the number of citations received per article has a highly skewed distribution in which nearly all of the articles receive very few citations, and only a tiny fraction receives a very large number of citations. Yet, academic economists are evaluated by their institutions based on their authorship of articles weighted by the ranks of journals, and not on the number of citations to their articles, "even though citation counts was our only measure of importance in comparing the journals, and the citation counts of the individual publications are readily available." Certain excuses have been used for evaluating academics on the basis of publication counts rather than citations counts, such as citation counts requiring a lag for the work to be discovered, but overall, these excuses do not hold water (Payson 2017). Fact #6. There is an obvious reverse causality which the profession must surely realize, but pretends it does not exist. The most prominent journals, by virtue of their already-known prominence, and their high circulation, will most certainly receive more citations on average for any article they publish, all else being equal. In other words, if we were to take the same article in a minor journal and publish it in a prominent journal, it would, of course, receive many more citations from this switch alone, simply because of the much greater circulation and exposure of the high ranking journal, plus the assumption that most of the profession will make (whether it is true or not) that whatever is published in a high-ranking journal is of higher quality to begin with, and is thus more worthy of being cited. In short, supply creates its own demand with regard to high-ranking journal articles, and this is already obvious to everyone. It follows then that, for the editors and publishers of top-ranked journals to speak proudly of how often their journals are cited (as if those citation counts reflect a purely independent assessment of their journal's high quality) is a laughable and shameful example of the fallacy that association is causation. Fact #7. In spite of its importance in determining (more than anything else) the careers of academic economists, the citation system is very often seen as an adolescent source of amusement. As an example, let us consider a project that was implemented by RePEc (Research Papers in Economics), which is the one organization that does more than any other to present statistics on citations and publications in economics, and draw inferences from them. In its own promotion of adolescent amusement, RePEc (2018a) proudly instituted a "fantasy league" game, analogous to fantasy football, which they describe as follows (where "IDEAS" is a forum that is part of RePEc): IDEAS now has a fantasy league ... It all started with an April Fool's post on this blog that many found to be a great idea. ... As we do not have a publishing season, and publishing is very slow in economics, the rules had to be adapted from the standard fantasy leagues in other "sports." They may need to be amended in the future, but players will be alerted well ahead of any changes. And if you are uncomfortable with being traded in a fantasy league, you can opt out. ... Have fun playing! The point of this discussion is not to begrudge people of having fun, but the devil, here, is in the details. The "trading" that takes place in the game involves the names of actual, real people, and their real-life status, who are identified as authors in the RePEc system. The default is that people playing the game can choose those real people as being in their "fantasy economics department" while the onus is on the people who were chosen to "opt out" if they do not wish to have their real-life publication record serve as a component of this game. Here are additional details of the game (RePEc 2018b): Scoring ... fantasy departments are ranked every month at the same time the IDEAS/RePEc rankings are updated. The scoring is performed in the same way as for real departments: the scores are added over all department members for each criterion, departments are ranked for each criterion, and a harmonic mean of the resulting ranks is computed.
In other words, all players in the game are pretending to be chairs of academic economics departments, where all economics departments compete along the same set of rules. Players score points in accordance with the actual points that RePEc assigns to real people for their real-life publications. Players of the game will thus be happy with the members of the fantasy team that they created when those members accumulate publications in top-ranked journals.
This game illustrates the incredibly simplistic, assumed role of economic researchers, and the assumed role of economics departments. Yet, more telling is the idea that, through this game, actual people are being viewed as celebrities whose success at publishing, in reality, is seen as a competitive sport. This situation reflects the assumption that the work of academic economists, and the hiring decisions of their departments, are, indeed, run in the same way as a competitive sport.
Let us imagine now how insensitive this game would be if it were played in other professions that house professionals who actually take their work seriously. Consider, for example, a fantasy hospital game where each hospital administrator gets to pick their dream doctors based on the published records that actual doctors have regarding their success rates with patient care. Would hospital administrators ever play such a game? Of course not, because, among other things, their involvement in any such game would belittle and insult the very important work that they do. What this all implies is that the chairs of academic economic departments do much less serious work, so that there is much less to belittle. It can further be argued that the less-serious work performed in academic economics departments is all because of the citation system, which has simply made the work of economics departments a "publishing game" more than anything else.
In Powdthavee et al.'s presentation at the 2018 AEA meetings, one of their slides mentioned a finding from a different study. That other study was entitled "Your Right Arm for a Publication in the AER?" and the finding was, "[P]eople are willing to sacrifice more than half a thumb for an AER {American Economic Review} publication (Attema et al., 2014)." The most useful measure of the value of an AER article to the author (as a credential in their CV) would be the article's value in monetary terms. In contrast, how much the article is worth in body parts is outrageously arbitrary and obscure. Moreover, the question itself of how much someone would sacrifice in exchange for an AER article to their name is irrelevant. It would only be relevant if someone had already written a paper and presumably aspires to have it published in the AER. However, in that case the relevant measure of interest would be the differential effect of publishing the article in the AER versus publishing it in another outlet for which there is a reasonable certainty of it being accepted for publication. Moreover, if the author were truly proud of their own paper, then they should believe that, wherever it is published, its quality would eventually shine through (in theory, by the citations it would receive), and that paper should therefore eventually receive the recognition it deserves. So in the end, there should be no reason for anyone to be willing to lose half a thumb over it, one way or another. However, the entire context of Attema et al.'s study, and the reference to that study in the AEA presentation by Powdthavee et al., is completely different. It is simply the context of the author effectively winning a game of chance if their paper is accepted to the AER, analogous to winning the lottery, and thus the AER article is highly valued independent of anything else.
Given all of these considerations, it should be obvious that Attema et al.'s paper served only one main purpose, which was to find amusement in the ridiculously high value that the profession places on AER articles. The true objective of the paper was not to question seriously that value of an AER article, but only to find a way to poke fun at it, which is an indirect form of acceptance of something that is genuinely troubling at many levels. In other words, the paper offers therapeutic, comic relief, for the troubling situation that most academic economists face -the pressure to publish in top-ranked journals. To the extent that others will also be amused by the notion of an AER article being worth half-a-thumb, Attema et al.'s paper will be well cited (as it was by Powdthavee et al.) as if they offered an important intellectual achievement.
Fact #8. Citations have been used to rank academic economists against each other, along with a single, linear, rating scale, analogous to the rankings of sports contestants. This has had a terrible influence on the culture of the profession, causing many of its members to see themselves as contestants in a simplistic game, rather than being devoted researchers committed to contributing to valuable knowledge. One sees this, in particular, in the RePEc website, where many thousands of named individuals are uniquely ranked among each other, ordinally, e.g., from 1 to 2,696 for the first 5 percent of all academic economists in their system, followed by lists of authors for each percentile from 6 percent to 10 percent (RePEc 2018c). If you are not in the top 10 percent of their list, you are obviously not in the "in crowd" of academic economic publishing. This situation is characteristic of similar displays of people who are honored to appear on a public list, like making the honor roll in high school or the dean's list in college. In addition to distorting incentives away from the honest pursuit of knowledge, and encouraging an unhealthy competition, this published list acts to legitimize the obsession the profession now has with publication and citation counts, which serves as the primary basis upon which the profession evaluates and values its own members. Without such a list, one could facetiously ask, what are academic economists supposed to do to evaluate each other's work, actually read each other's work and make their own individual judgements based on their own opinions about what they read? Then where would we be!
The deeper we examine this overall situation, the more capable we are to realize its absurdity. In other fields of study, the general absence of such superficial and linear rankings enables the scientists to value their own ideas exclusively and not value, or devalue, themselves in the process. In contrast, because of these rankings in academic economics, it is that much easier for the academic economist to fall into believing automatically that they are superior (in their thinking) if they are ranked higher, and inferior if they are ranked lower. Those who are not even "on the charts" (to borrow a term from the music entertainment industry) will have much more trouble believing in themselves when their own profession appears to have decided not to believe in them. In addition, the public, which has complete access to these published ranks, will reinforce them by naively assuming that the higher the rank of the economist listed, the more brilliant they must be.
This ranking system is wonderfully convenient and beneficial to the economists in the highest ranks, who may very wrongly develop inflated egos, thereby harming the profession in countless ways. Those who are truly the most brilliant among the top-ranked economists should surely see the wisdom in enabling the public to know that these rankings display winners in a publication game, much more than they display the people who have the most superior understanding of the field of economics. So these leaders should stop being silent about all this, in spite of the benefits they derive from it (for themselves or by way of public relations for their departments), and accept the ethical, public responsibility that comes with their status, by helping to expose this house of cards.
In direct opposition to what many academic economists wishfully believe, other fields of science and social science do not have the same obsession with citation counts that academic economists haveinstead, other fields admirably focus on genuine discovery as their goal. However, those researchers, in any field, who do not have an ability to advance knowledge through genuine discovery are condemned to a life, instead, of producing as many "publication credentials" as they can, simply to make up for it. As argued in Payson (2017), those who produce more publications are often those who contribute the least to useful knowledge -they simply substitute one for the other as alternative routes to advancing their career.
Fact #9. Many of the most prominent economists who have achieved full professorship at a highranking institution refrain from publishing in top-ranked journals in order to allow more junior professionals to have a better chance to advance their careers. While this may appear to be "noble" on the part of the prominent professors not to "crowd out" their more junior colleagues for the scarce resource of top-journal articles, we need to ask what this really says about top-ranked journal publications.
Some prominent economics professors have openly espoused that they are, in a way, ethically bound to reduce or even terminate their own contributions of publications in top-ranked journals, and they have likewise expressed contempt for other well-known professors who continue to exploit (in their view) their prominent status by extensively publishing in those journals as sole authors. Their contempt is based on the premise that these other famous professors have no need for any more publications to advance their careers, so why should they make it any harder for more-junior academic economists to get a top-ranked article, who may desperately need one in order to acquire tenure or to land a first teaching position?
This desperation for a top-ranked publication is especially prevalent among the many thousands of foreign students who come to the United States to acquire a Ph.D. in economics, aspiring to then immigrate to the U.S. to become a professor. For them, their entire life path rests essentially on being able to acquire a top-rank journal publication, without which there would probably be no other work available to them in the United States (as discussed in Payson, 2017). Table 1 below explores the logic, or lack thereof, behind the notion that full professors should restrain from publishing (as sole authors) articles in top-ranked journals, in order to give more junior academic economists a better chance at it. As shown, there is either something wrong with the idea, because it will weaken the quality of the journals' articles, or the idea is fine but implies that full professors should not have the recognition they are generally given. There is one other possibility that is not shown in the table however, which is that full professors do have the most knowledge, and their contributions would therefore increase the value of the journals, but that is of lesser importance (in some people's way of thinking). That is, some might argue that, since the journals play such a crucial role in career development regardless of their contribution to knowledge, it would be better for them to support more junior researchers even if it means reducing the amount of new knowledge they provide.
All of this confusion could be resolved, perhaps, if the profession went the extra mile of assessing the true value and contribution made by journal articles and their authors, and considering the process by which junior members should rise in their careers, rather than simply counting citations as the only measure of quality.

Question Possible Answer Follow-up Question(s) Possible Answers and Their Implications
Are full professors in economics the most knowledge-able in their field?
Yes Then if they abstain from contributing to top-ranked journals, would that then weaken the quality of the journal's articles?
Yes. So therefore, the argument that they abstain from contributing is not justified. The people making that judgment must simply be wrong -how they even got the idea is then a mystery.
No, so they should abstain as recommended.
But why then do they have the status of full professors, and why would they be able presumably to contribute to journal articles if they did not abstain?
They just accomplished what they needed to, in order to acquire the credential, and so publication acceptance must be based, in part, on their status in the profession independent of the quality of their current contributions.

Fact 10. Strong, vested interests play an important role in supporting the profession's effective obsession with citation counts and publication counts in top-ranked journals.
The glorification of citation counts and top-ranked publication counts not only justifies the profession's existing leadership positions, but they play a major role in securing grant money for those leaders' institutions, and better enable those leaders' own graduate students to secure preferable teaching positions (through co-authorship with them), thereby preserving their institutions' elite status. The ranking of institutions, on the basis of the citations their members have received in economic literature, has been commonly recognized and studied. See, for example, Önder and TerviÖ (2015, p. 1496) which provides a list of the "top 50 institutions by citations." Needless to say, all of these benefits would be in jeopardy if the validity of citation counts and topranked publication counts as a measure of the value of knowledge were seriously challenged. Moreover, it is important to understand that the journals are not, by any means, independent of the elite members of the profession -they are in fact controlled by them. A very high proportion (and often the majority) of topranked publications are for authors in the highest-ranked departments. Those same departments populate the leadership of those journals.
Along these lines, a very important, and very recent study by Colussi (2018, p. 49) found that "a large fraction of the published papers" in top-ranked journals "are authored by scholars connected to an editor of the journal at the time of the publication. The share is particularly large for editors' former Ph.D. students and faculty colleagues." The perception of potential cronyism is also the brunt of jokes among members of the profession. One of these jokes, for example, is the statement that the acceptance of papers in top-ranked journals is determined primarily on the basis of only one numerical parameter -the zip code of the applicant (where the zip codes of the most prominent universities guarantee acceptance). Likewise, it is also a well-known fact that most academic economics, who believe they have good papers, will simply not bother to apply to top-ranked journals under the assumption that they do not have the necessary "connections" for their paper to be accepted, or that they will be discriminated against by not being affiliated with a top-ranked institution. Though this is quite well known, the profession's leaders make little or no effort to change this perception, begging the question of why they apparently have no interest in changing it. One could easily guess the answer to this question. Furthermore, the express policy by journals like the AER to exercise the option of rejecting paper applications "without review" (as difficult as this may be to believe) only exacerbates this perception.
In defense of this apparent incestuous relationship between the top-ranked journals and the institutions that employ the authors of those journals' articles, the leaders of both the top-ranked journals and the top-ranked economics departments would simply claim (and have claimed) that this is all to be expected because they are simply the smartest people in the field of economics. Therefore, by this argument, the best journals and the best departments should simply have the smartest people (and it follows that those who object to the situation must only be jealous for not being recognized as being as smart as they are).
Unfortunately, as explained in detail in Payson (2017), the situation is not quite so simple. Needless to say, to those who have these vested interests, this paper and related works are likely to be a thorn in their side, from a public-relations perspective, for their journals or their own academic institutions. Therefore, strategic efforts will surely be made by some of these leaders to ignore (if they can), and then discredit (as much as they can), the facts and conclusions that are presented in this paper and other papers like it.
Lately, however, the evidence for, and outcry against, the house of cards that is the citation system in academic economics, has built up to the point that even the profession's tight-knit leadership cannot continue to pretend the problem does not exist. To handle the situation, one of the latest public-relations strategies by the profession's leadership has actually been to acknowledge the problem and purport to address it. They are now beginning to address it by a broad appeal for economic departments to rediscover their ability to read and evaluate people's research. Specifically, as recently as September 2018, Professors James J. Heckman and Sidharth Moktan wrote a National Bureau of Economic Research Working Paper, entitled "Publishing and Promotion in Economics: The Tyranny of the Top Five," with the "top five" meaning the top five journals in economics. In it they note (with my own underlining): In light of the many adverse and potentially severe consequences associated with reliance on the T5, we believe it unwise for the discipline to continue using publication in the T5 as a measure of research achievement and as a predictor of future scholarly potential. The need for change is made ever more apparent by the T5's inadequacy as a predictor of individual article quality, much less the quality of a person. … Akerlof (2018) sounds the alarm about the practice of relying on external rankings rather than individual reading of papers. The appropriate solution to the problem will require a significant shift from the current publications-based system of deciding tenure, to a system that emphasizes departmental peer-review of a candidate's work. Such a system would give serious consideration to unpublished working papers and the quality and integrity of a scholar's work. (p. 53-54) While this is progress, for which Heckman, Moktan, and Akerlof deserve some credit, it is also incredibly ironic, and for some of us disheartening, how Heckman and Moktan describe Akerlof's article as "sounding the alarm" about all this. As a point, in fact, the same alarm has been sounded already in numerous publications on this topic by economists for decades, but just not by anyone in the profession's most elite circle, which until just now has apparently collectively agreed to remain silent on the issue. So now that the house of cards is finally beginning to fall, those who have had the power to build, maintain, and promote that house of cards this entire time, and have ignored all the criticisms made of it for generations, are now apparently trying to take credit for the idea of finally deciding to criticize it as if such criticism were uniquely their own. It is a shame that there was never in place a more legitimate citation system that would have required these authors to give credit in their paper to those other authors who were outside their prominent circle, but who have been actively promoting and publishing the same ideas all along.

Getting Back to Why Citations and Journal Articles Should Exist
As mentioned above, there are two main reasons why papers in economics are published. I dare say that the greater of the two reasons, with regard to the motivation among academic economists, is so that these published papers can be cited. Some may say that another reason is to acquire a publication credential, which is obvious, but, as D. Card and many others have argued, a paper in a top-ranked journal will only be published if it has a strong promise of being heavily cited. So in this sense, as an intermediate goal, papers are still written for the purpose of being cited (or at least for the purpose of being perceived by journal editors as likely to be cited, which is pretty much the same thing).
It should also be obvious and assumed that there is one other reason for why papers in academic economics are written, which is, on occasion, espoused in the profession's "party line." This is the noble, intellectual, and philanthropic reason of advancing important and valuable knowledge in the field of economics for the benefit of society. The running joke within the vast majority of the profession -or the house of cards that needs to fall down -is the naïve belief, or the pathetic excuse (depending on one's perspective), that the measure of the fulfilment of first reason -accumulating citations -is a valid and reasonably accurate measure of the fulfilment of the second reason -the extent to which the paper advances valuable knowledge. This very wrongful association between the two, as discussed in the previous section on ten facts, is what has greatly harmed the academic economics profession. As if this is not bad enough, in itself, this problem has poisoned the very nature of how academic economists see their responsibilities in life, and how they select their leaders, all of which has further deteriorated the profession.
Along similar lines, there are three main reasons why academic economics papers are read. My use of the term "read" here is actually a euphemism -in the vast majority of cases they are not "read" in the normal sense of the word. In most cases they are "scanned" or, even worse, only their abstracts are read, and worst of all, the citations are simply found, and then cut and pasted, into one's own reference section, as I have demonstrated in the preparation of Fact #2 above.
The three reasons for why the academic economics papers are demanded are presented below: (1) Readers scan (or in some cases even read) the paper to see if it should be cited in a paper that they, themselves, are working on, to demonstrate that their own paper contains a comprehensive literature review of whatever topic they are exploring.
(2) Readers scan the paper so they can later give the appearance to their colleagues and students that they are current with the latest literature.
(3) Readers study the paper actually to learn something important to improve their knowledge as an end in itself, and to apply that knowledge to their own work.
Of all these reasons, the most important one, by far, is #3, because that is the very purpose -or at least the major purpose -for which articles are supposed to be written and supposed to be read. Yet, academic economists accomplish both #1 and #2 by scanning, and #3 only by careful study. Since academic economists are constrained by the amount of time they can spend, the more the incentive system encourages scanning, the less time will be spent on studying. Scanning has much more to do with playing a game, while studying has much more to do with what economic research is supposed to be all about. So the question is, how do we get out of the professional scanning business, and back into the studying business? The answer is to remove the profession's obsession with citation counts, which has been causing the practice of scanning to take precedence over the process of studying.
Here is a hypothetical question we should all be asking ourselves as contributors to economic literature: Would we rather have our work read and seriously studied by, say, 25 people in the world, or scanned for a minute by, say, 25,000 people in the world who scanned it just enough to decide to include the work as a citation in their own work. If your answer is the latter, my next question would be, "Then why did you write the paper in the first place?" The obvious answer would be so that it will be heavily cited, even though the people citing it never read it enough to understand it. If that is what you really wanted, then why are you doing economic research, to begin with? Was it to get paid? As an economist I can understand that, but only partially.

Ending the Madness
Because the current system of top-ranked publications has been so upheld by the most prominent members of the academic economics profession, who rely on the same system to justify their status and authority, the vast majority of the academic economics community have felt hopeless about it for several decades. Fortunately, that this can all end soon!

Redefining Citations
The first two facts presented at the beginning of this paper explored the reality of what the vast majority of citations truly are, which might be described briefly as "payment of homage" (to those who are already the most prominent in the subfield of the paper), but in any case they are citations to work that has had no real effect on the findings of the paper, nor do they back up the ideas of the paper to any extent. For this reason, we can call them "citation fodder." Of the remaining minority of citations, I would like to identify three main categories. There will always be some gray areas among these categories, requiring some judgment calls, which is the nature of nearly everything that requires categorization. They are: 1. Essential Citations: Citations to studies that played a key, substantive role in affecting the outcome of the paper or the paper's ability to justify its methods and findings. The main test for this category is the hypothetical, counterfactual, controlled-experiment-type test that asks how different the paper would be if this reference were never used. If the paper would be substantially different (or not exist at all) then the citation meets the criterion. 2. Relevant Citations: These citations have a significant effect on the paper in terms of its findings, methods, or justification. Without any one of these citations, the paper would be about the same, but the cited reference did contribute to the paper, so the paper would be at least slightly different without it. 3. Negative Citations: These are citations to work that is being criticized in the paper as being invalid, factually wrong, misleading, or generally harmful to economic discourse (in one way or another). Though these citations tend to be rare in most papers, they should still be identified so that positive credit is not given to the authors of these works.
See Appendix A for a more complete set of guidelines for the classification of citations. This will include guidance on how the gray areas associated with these categories can be approached. The categories of citation fodder, relevant citations, and essential citations are necessarily exhaustive and mutually exclusive, i.e., every citation, with regard to a specific paper in which it is cited, should be classified as belonging to one and only one of these three categories. The vast majority will likely be citation fodder, followed in number by relevant citations, and then essential citations. However, there is no rule regarding these relative frequencies. Furthermore, there is no set rule that there must be at least one citation in each category. For example, if the research is highly original, it is possible that there is not a single essential citation in the paper, which need not reflect negatively on the paper.
The negative citation may then be viewed as a negative sign in front of any one of these three types of citations. As an example, in this paper itself there was a substantial discussion of Powdthavee et al. (2018), but with the conclusion that the paper had an overall negative effect on discourse in this area, by glorifying something, both directly and indirectly, that should not be glorified. The separate identification of negative citations would add useful information but may cause more harm than good in a variety of ways, so it is recommended here that such citations not be identified. Thus, these citations will remain under the fodder category, even if they are essential (in a negative way) to the paper at hand.

Proposed Course of Action
It is hereby proposed that every published paper provide, essentially, three lists of citations, though two of the lists would require a minuscule amount of space, and the main, large list (with one additional entry) would be the same as it has always been. The specifics are as follows: 1. The reference section of any paper would appear the same way as it always has, with the complete set of all citations (fodder citations, relevant citations, and essential citations, whether positive or negative). The only exception would be that the reference section also includes a citation to this paper, since this paper will be mentioned in a footnote (as described below). 2. A footnote or endnote will be added somewhere in the paper. The suggested location would be right at the title of the Reference section, e.g., instead of the title "References" there would be the title "References n " where n is the n'th footnote (or endnote) in the paper. If the journal disallows its placement there, but allows its placement somewhere else, then that would still be fine.
If the journal does not allow footnotes, then a single sentence can be added in-between the section heading "References" or "Bibliography" and the beginning of the reference section. This is what was done for this article (see below).
3. The footnote will list only non-negative essential and relevant citations, broken out separately. Readers who are then familiar with this method will be able to assume, then, that all of the other citations listed in the reference section are either citation fodder, or essential and relevant citations that were negative. If they are not familiar with the method then they would still probably have a general understanding of what is being displayed from the basic definitions of "essential" and "relevant." 4. The footnote would contain a reference to this paper for the purpose of enabling readers to understand why the footnote was added and to have a reference for the definitions used for essential citations and relevant citations. The footnote can simply refer to this paper as "Payson (2019) since the full citation to this paper should be in the reference section. 5. An example of the footnote, based on this paper, is as follows: In accordance with the classification of citations in Payson (2019), the Essential Citations for this paper were Colussi (2018), Engemann andWall (2009), Heckman andMoktan (2018), and Payson (2017), and the additional Relevant Citations were Akerlof (2018) and Stern (2013).
6. It is expected that, in the vast majority of cases (except for papers that are about the citation system itself), the "Payson (2019)" citation would qualify as citation fodder. It should still be included in the reference section. However, for all of the reasons provided here, and the fact that it is referred to in the paper.

The Benefits of the Redefined Citations
The redefined citations that include separate categories for essential and relevant citations will clarify those citations that rightfully do what citations are purported to do. That is, they will give credit to prior publications where credit is due, for contributing valuable knowledge that is recognized and used in the current paper, which is unique knowledge that could not be acquired by any one of a hundred alternative sources. Currently, the profession gives as much credit to the multitude of fodder citations and negative citations, which those works simply do not deserve. Through literature searches and direct, voluntary assistance from authors who add the new footnote to their papers, I will perform a follow-up analysis of the data provided by these new footnotes and would welcome all others who would like to study these data to do the same. Using these data I will first publish another research paper on this topic, and then compile my findings (and extended arguments) into a book. The next article and book will measure, analyze, and publicize the following variables: 1. The proportions of all citations in articles that are essential citations and relevant citations. 2. The beginnings of data on essential citations and relevant citations by: a. the authors being cited, and their institutions, b. the journal (or other outlets) being cited, c. the subfield being cited, given the subfield of the paper (for example, how many essential citations are there to econometric papers in empirical economics?).
As an appendix to the next article, and in the book, I will list, i.e., cite, all of the papers that contributed to the experiment as straightforward evidence for how the data were collected, and as encouragement for others to participate. Moreover, as this paper itself becomes cited by those who participate in the project, and as those participants become cited as well, this will provide additional direct evidence of how citation counts are primarily counts of fodder citations. Such findings could, in turn, strengthen the argument in support of the profession identifying and counting only essential and relevant citations. These findings could also provide evidence that others share the concerns of this paper and agree that essential citations and relevant citations should receive more credit than fodder citations. Ultimately, if the movement receives enough support, some of the leaders of journals will promote the idea as well, and at that point, the house of cards will surely fall. Those who are most familiar with the citation system, and the games that are played in it, might rightfully expect this major revision of the system to be gamed as well. For example, what is to stop close friends, who are already inclined to cross an ethical line and cite each other as much as they can for their own mutual gain, to cite each other as essential citations, thereby obtaining even greater benefits from their collusion? Or, equivalently, what would stop (other than ethical behavior!) a journal referee from requiring that their own work is cited as an essential citation in the paper they are reviewing? Indeed, if precautions are not taken, the new categories can invite more problems associated with cronyism and deception than what had already existed before them. Fortunately, "Protective Measures," discussed in Appendix B, will be implemented to prevent these problems.
Likewise, for a wide variety of reasons, at many levels, some journal editors might directly object to the added footnote, and say that their journal will not allow it. Appendix B on Protective Measures examines this topic further and explains how deliberate efforts to suppress participation in the project will be exposed and countered, and are then likely to fail in the end.
The academic economics profession badly needs to rid itself of its effective obsession with citation counts and top-ranked publication counts as "the only game in town," and focus, instead, on advancing economics in a manner that will at least try to make the world a better place. Getting rid of this obsession will most certainly not solve all of the profession's problems, but it is a good start.