Peer Review Guidance: How Do You Write a Good Review?

Why Peer Review?

According to the International Committee of Medical Journal Editors,⁵ “peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff.” Evidence of the early use of peer review has been found in ancient Greek writings.⁶ Prepublication review of scientific papers by experts began in the 17th century, but it was not until the 1940s that peer review became standard journal policy.⁷ Today, the peer review process is a standard part of research publication: As the established method of validating one's work, it is essential to the continued growth of knowledge.⁶

Who Are Peer Reviewers?

By definition, peer reviewers are peers of the authors.⁸ The Merriam-Webster Dictionary defines peer as “one that is of equal standing with another.”⁹ When selecting peer reviewers for a manuscript, editors typically choose reviewers who have demonstrated expertise in the subject. Reviewers with a background or facility in statistics may also be selected if a study's methods and results require sophisticated evaluation.

Overview of the Peer Review Process

Most scientific journals, including the JAOA, require reviewers to complete peer reviews using a Web-based manuscript tracking system. When a manuscript is submitted to a journal, it is first reviewed by the editor in chief and associate editors, who assign the manuscript to 2 or more peer reviewers.

Review requests are typically sent by means of e-mail. If reviewers accept a peer review request, they are given a timeframe for reviewing the manuscript. Peer reviewers of the JAOA are asked to complete reviews within 15 days. If reviewers decline a peer review request, the editors of the journal appreciate recommendations for alternative reviewers.

After completing a review of a manuscript, peer reviewers submit comments and recommendations for both the author and the editor. The editor in chief will then consider comments from all reviewers and accept the manuscript for publication without revision, accept the manuscript pending minor or major revisions, or reject the manuscript. In the event that the manuscript is accepted pending revision, authors are asked to address reviewer comments. When the authors submit a revised manuscript, the revised version is usually sent to the original peer reviewers for a second review.

At the end of the peer review process, the editor in chief and associate editors rate reviewers on the basis of the quality of their reviews and responsiveness. A reviewer's collective rating will often determine whether he or she is asked to review a future manuscript for that journal.

Types of Peer Review

Journals typically use 1 of 2 peer review models: single blind or double blind. The single-blind review model is easily the most common and is the model used by the JAOA. In the single-blind review process, the identity of the reviewer is concealed, but the identities of the authors are not. An advantage of the single-blind review process is that the reviewer is able to identify and assess previous work by the authors and determine whether advances from previous work are reported. A disadvantage of the single-blind review process is the potential for the anonymous reviewer to be biased—positively or negatively—in the review.

In the double-blind model of peer review, the identities of both the reviewer and the authors are concealed. An advantage of the double-blind review process is that potential for reviewer bias on the basis of authors' previous work or institution prestige is lessened. The disadvantage of this model is that truly blind reviews are uncommon—authors frequently refer to their prior publications in a manuscript, and reviewers can easily search for authors' previous work in the subject.¹¹

Additional pros and cons of each peer review model are discussed by Keenum and Shubrook.⁸ It is clear that no peer review process is perfect. Although peer reviewers have no control over the peer review process used by journals, they should ensure every review they conduct is fair, thorough, and unbiased.

How I Review an Original Research Manuscript

Several excellent references are helpful in guiding peer reviewers in the review process.^{2,3,8,10,12,13} In this section, I describe how I approach a peer review and include considerations for each section type. Of note, I focus the majority of my recommendations on the review of original research articles, so some points may not be applicable to all article types. Before beginning a peer review, it is important to familiarize yourself with the guidelines of the journal for which you are reviewing.

Before I agree to review a manuscript, I ask myself several questions:

• Do I have time to dedicate to the review, considering a typical review will require approximately 3 hours?

• Am I qualified to review this particular manuscript?

• Have I any expertise in the subject area?

• Do I have a conflict of interest (eg, Do I have a financial interest in the research? Have I collaborated with the author[s] in the past?)? When I believe I may have a conflict of interest, I discuss them with the editor before agreeing to review the manuscript.

After agreeing to review a manuscript and before I begin the review, I remind myself of my responsibilities as a peer reviewer. Paramount in my mind is my responsibility both to the author and to the editor. To the author, I will be an advocate. I will be collegial and offer constructive criticism. I will maintain confidentiality and keep in mind that the manuscript belongs to the author.

To the editor, I will provide my best, unbiased evaluation on the basis of my opinion of the validity of the author's conclusions. For additional general guidelines, see Figure 1.

Figure 1.

General considerations of peer review. Adapted from the ACS Style Guide.¹⁴

Second Reading

When reading the paper a second time, I consider each section carefully and develop specific comments and recommendations for the author and the editor (Figure 2).

Figure 2.

Considerations of peer review by manuscript section.

It is important to read the manuscript with a critical and creative eye. In addition to assessing the paper's logic and assumptions, reviewers should look for possible improvements to clarify the presentation. It is essential, of course, to provide specific suggestions to the author. For example, when a manuscript contains references to older, outdated studies, I will not simply state, “References are outdated.” Rather, I identify more recent studies for the authors to consider citing. Of note, I focus my comments on the content of the manuscript. If I find an otherwise valuable paper difficult to read because of writing style, grammatical errors, or labored syntax, I will suggest that the authors seek writing assistance. It is not necessary for reviewers to detail grammatical errors.

Conclusion

Volunteering as a peer reviewer can be a valuable and rewarding experience. By following basic guidelines, reviewers can ensure their comments are professional, thorough, and helpful and improve the quality of scientific publications.