Skip to content
BY-NC-ND 4.0 license Open Access Published by De Gruyter December 1, 2013

Peer Review Guidance: How Do You Write a Good Review?

  • Thomas Wesley Allen

Abstract

Peer review is essential to the quality of scientific publications. As the volume of submissions to these journals increases, the need for an expanding cadre of trained reviewers also increases. Many reviewers do not receive formal training on conducting a peer review or education on the peer review process, however. The author presents an overview of the peer review process and provides guidance for conducting high-quality reviews of manuscripts submitted to The Journal of the American Osteopathic Association and other scientific journals.

It is the mission of The Journal of the American Osteopathic Association (JAOA) “to advance medicine through the timely publication of peer-reviewed osteopathic medical research.” Accompanying the mission statement are several goals including, “to ensure the professionalism of the publication process.”

To achieve its mission and goals, The Journal must rely on its volunteer peer reviewers. According to former JAOA Associate Editor Felix J. Rogers, DO, “the JAOA requires an ever-expanding pool of peer reviewers committed to the prompt and comprehensive evaluation of each manuscript.”1

All scientific journals, including the JAOA, are constantly recruiting new peer reviewers. But, many people who are being called on to conduct a peer review may be unfamiliar with the requirements and expectations of the process.2 Most physicians, scientists, and other experts who volunteer their time to review receive little training—formal or informal—in the critical review of research articles or in the peer review process.3 Black et al4 suggest that to ensure the quality of peer reviews, journals should train their reviewers.

In the present article, I provide an overview of the peer review process and describe in detail the general and specific tasks required of peer reviewers.

Why Peer Review?

According to the International Committee of Medical Journal Editors,5 “peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff.” Evidence of the early use of peer review has been found in ancient Greek writings.6 Prepublication review of scientific papers by experts began in the 17th century, but it was not until the 1940s that peer review became standard journal policy.7 Today, the peer review process is a standard part of research publication: As the established method of validating one's work, it is essential to the continued growth of knowledge.6

Who Are Peer Reviewers?

By definition, peer reviewers are peers of the authors.8 The Merriam-Webster Dictionary defines peer as “one that is of equal standing with another.”9 When selecting peer reviewers for a manuscript, editors typically choose reviewers who have demonstrated expertise in the subject. Reviewers with a background or facility in statistics may also be selected if a study's methods and results require sophisticated evaluation.

Characteristics of an Excellent Reviewer

Arguably, the most important characteristic of a good peer reviewer is to be an advocate for the author. Benos et al10 put it best when they wrote, “The most important rule is to follow the golden rule: treat all manuscripts in the same manner that you would want your own treated.”

Black et al4 found little association between reviewer characteristics and the quality of reviews they produced. The authors did find that training in epidemiology or statistics was statistically significantly associated with higher quality reviews. Younger age and more time spent on a review (up to 3 hours) were also associated with better reviews.

Benefits to the Reviewer

There are several reasons why reviewers should volunteer their time. For one, being selected as a reviewer is an honor. The selection signifies an acknowledgment of one's expertise in the field and of one's standing in his or her career. In addition, peer reviewing benefits the profession. The reviewer contributes to the accumulated body of knowledge and facilitates the distribution of new information to our colleagues and to the public. Finally, volunteering as a peer reviewer can help advance one's career. Reviewers can list the experience on their curriculum vitae, and many journals—including the JAOA—offer continuing medical education credits for completed reviews.

Overview of the Peer Review Process

Most scientific journals, including the JAOA, require reviewers to complete peer reviews using a Web-based manuscript tracking system. When a manuscript is submitted to a journal, it is first reviewed by the editor in chief and associate editors, who assign the manuscript to 2 or more peer reviewers.

Review requests are typically sent by means of e-mail. If reviewers accept a peer review request, they are given a timeframe for reviewing the manuscript. Peer reviewers of the JAOA are asked to complete reviews within 15 days. If reviewers decline a peer review request, the editors of the journal appreciate recommendations for alternative reviewers.

After completing a review of a manuscript, peer reviewers submit comments and recommendations for both the author and the editor. The editor in chief will then consider comments from all reviewers and accept the manuscript for publication without revision, accept the manuscript pending minor or major revisions, or reject the manuscript. In the event that the manuscript is accepted pending revision, authors are asked to address reviewer comments. When the authors submit a revised manuscript, the revised version is usually sent to the original peer reviewers for a second review.

At the end of the peer review process, the editor in chief and associate editors rate reviewers on the basis of the quality of their reviews and responsiveness. A reviewer's collective rating will often determine whether he or she is asked to review a future manuscript for that journal.

Types of Peer Review

Journals typically use 1 of 2 peer review models: single blind or double blind. The single-blind review model is easily the most common and is the model used by the JAOA. In the single-blind review process, the identity of the reviewer is concealed, but the identities of the authors are not. An advantage of the single-blind review process is that the reviewer is able to identify and assess previous work by the authors and determine whether advances from previous work are reported. A disadvantage of the single-blind review process is the potential for the anonymous reviewer to be biased—positively or negatively—in the review.

In the double-blind model of peer review, the identities of both the reviewer and the authors are concealed. An advantage of the double-blind review process is that potential for reviewer bias on the basis of authors' previous work or institution prestige is lessened. The disadvantage of this model is that truly blind reviews are uncommon—authors frequently refer to their prior publications in a manuscript, and reviewers can easily search for authors' previous work in the subject.11

Additional pros and cons of each peer review model are discussed by Keenum and Shubrook.8 It is clear that no peer review process is perfect. Although peer reviewers have no control over the peer review process used by journals, they should ensure every review they conduct is fair, thorough, and unbiased.

How I Review an Original Research Manuscript

Several excellent references are helpful in guiding peer reviewers in the review process.2,3,8,10,12,13 In this section, I describe how I approach a peer review and include considerations for each section type. Of note, I focus the majority of my recommendations on the review of original research articles, so some points may not be applicable to all article types. Before beginning a peer review, it is important to familiarize yourself with the guidelines of the journal for which you are reviewing.

Before I agree to review a manuscript, I ask myself several questions:

  • Do I have time to dedicate to the review, considering a typical review will require approximately 3 hours?

  • Am I qualified to review this particular manuscript?

  • Have I any expertise in the subject area?

  • Do I have a conflict of interest (eg, Do I have a financial interest in the research? Have I collaborated with the author[s] in the past?)? When I believe I may have a conflict of interest, I discuss them with the editor before agreeing to review the manuscript.

After agreeing to review a manuscript and before I begin the review, I remind myself of my responsibilities as a peer reviewer. Paramount in my mind is my responsibility both to the author and to the editor. To the author, I will be an advocate. I will be collegial and offer constructive criticism. I will maintain confidentiality and keep in mind that the manuscript belongs to the author.

To the editor, I will provide my best, unbiased evaluation on the basis of my opinion of the validity of the author's conclusions. For additional general guidelines, see Figure 1.

Figure 1. 
          General considerations of peer review. Adapted from the ACS Style Guide.14
Figure 1.

General considerations of peer review. Adapted from the ACS Style Guide.14

First Reading

During my first reading of a manuscript, I focus on “big picture” issues. I consider the importance and significance of the research question and the originality of the research. Is this topic timely and appropriate for the journal's audience? Has the research question been posed before? If so, is the manuscript adding anything new to the literature?

I also look at the writing style and structure of the manuscript. Will the reader who is unfamiliar with the topic be able to clearly understand the research? Are ethical considerations (eg, financial disclosures, patient confidentiality) addressed?

I make marginal notes if I find a problem with the presentation or science of the manuscript. After completing the first read, I try to summarize the paper in a few sentences or a short paragraph. Many readers quickly skim articles; by briefly summarizing the manuscript, I can get a sense of the impression readers will get. I do not return to a detailed and comprehensive second reading until a day or so has passed.

Second Reading

When reading the paper a second time, I consider each section carefully and develop specific comments and recommendations for the author and the editor (Figure 2).

Figure 2. 
            Considerations of peer review by manuscript section.
Figure 2.

Considerations of peer review by manuscript section.

It is important to read the manuscript with a critical and creative eye. In addition to assessing the paper's logic and assumptions, reviewers should look for possible improvements to clarify the presentation. It is essential, of course, to provide specific suggestions to the author. For example, when a manuscript contains references to older, outdated studies, I will not simply state, “References are outdated.” Rather, I identify more recent studies for the authors to consider citing. Of note, I focus my comments on the content of the manuscript. If I find an otherwise valuable paper difficult to read because of writing style, grammatical errors, or labored syntax, I will suggest that the authors seek writing assistance. It is not necessary for reviewers to detail grammatical errors.

Conclusion

Volunteering as a peer reviewer can be a valuable and rewarding experience. By following basic guidelines, reviewers can ensure their comments are professional, thorough, and helpful and improve the quality of scientific publications.


Dr Allen served as the American Osteopathic Association's editor in chief from 1987 to 1998.

Address correspondence to Thomas Wesley Allen, DO, MPH, 1111 S St Louis Ave, University of Oklahoma College of Medicine, Tulsa, OK 74120-5440. E-mail:

References

1 Rogers FJ . Saying what you mean and meaning what you say [editorial]. J Am Osteopath Assoc.2008;108(6):281-282.Search in Google Scholar

2 Einerson TR Koren G . To accept or reject? a guide to peer reviewing of medical journals[published online August 10, 2012]. J Popul Ther Clin Pharmacol.2012;19(2):e328-e333.Search in Google Scholar

3 Lovejoy TI Revenson TA France CR . Reviewing manuscripts for peer-review journals: a primer for novice and seasoned reviewers. Ann Behav Med.2011;42(1):1-13. doi:10.1007/s12160-011-9269-x.10.1007/s12160-011-9269-xSearch in Google Scholar PubMed

4 Black N van Rooyen S Godlee F Smith R Evans S . What makes a good reviewer and a good review for a general medical journal?JAMA. 1998;280(3):231-233.10.1001/jama.280.3.231Search in Google Scholar PubMed

5 International Committee of Medical Journal Editors . Uniform requirements for manuscripts submitted to biomedical journals: ethical considerations in the conduct and reporting of research: peer review. International Committee of Medical Journal Editors website. http://www.icmje.org/ethical_3peer.html. Accessed October 15, 2013.Search in Google Scholar

6 Kronick DA . Peer review in 18th-century scientific journalism. JAMA. 1990;263(10):1321-1322.10.1001/jama.1990.03440100021002Search in Google Scholar

7 Burnham JC . The evolution of editorial peer review. JAMA. 1990;263(10):1323-1329.10.1001/jama.1990.03440100023003Search in Google Scholar

8 Keenum A Shubrook J . How to peer review a scientific or scholarly article. Osteopath Fam Physician. 2012;4(6):176-179.10.1016/j.osfp.2012.06.003Search in Google Scholar

9 Peer. The Merriam-Webster Unabridged Dictionary website. http://unabridged.merriam-webster.com/unabridged/peer. Accessed October 21, 2013.Search in Google Scholar

10 Benos DJ Kirk KL Hall JE . How to write a review paper. Adv Physiol Educ.2003;27(2):47-52. doi:10.1152/advan.00057.2002.10.1152/advan.00057.2002Search in Google Scholar PubMed

11 AMA Manual of Style: A Guide for Authors and Editors . 10th ed.New York, NY: Oxford University Press; 2007.Search in Google Scholar

12 Hoppin FG Jr. How I review an original scientific article. Am J Respir Crit Care Med.2002;166(8):1019-1023.10.1164/rccm.200204-324OESearch in Google Scholar PubMed

13 Council of Science Editors . CSE's White Paper on Promoting Integrity in Scientific Journal Publications, 2012 Update. Wheat Ridge, CO: Council of Science Editors; 2012. http://www.councilscienceeditors.org/files/public/entire_whitepaper.pdf. Accessed November 5, 2013.Search in Google Scholar

14 Peer review. In: The ACS Style Guide: Effective Communication of Scientific Information. 3rd ed.New York, NY: Oxford University Press; 2006:71-76.Search in Google Scholar

Received: 2013-06-25
Revised: 2013-07-15
Accepted: 2013-07-19
Published Online: 2013-12-01
Published in Print: 2013-12-01

© 2013 The American Osteopathic Association

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Downloaded on 8.6.2023 from https://www.degruyter.com/document/doi/10.7556/jaoa.2013.070/html
Scroll to top button