1 & 2. Sue Porter: should also provide specific suggestions wherever possible about how to solve the issues identified in the review. Most important, a good review raises issues that, when successfully addressed by the author, increase the paper's likelihood of publication.3. With respect to taste, I am sensitive to this issue as my own personal area of research is not mainstream within tax or even within accounting. Although we all have distinct tasteswith respect to research, reviewers should provide an objective examination of the merits of a paper rather than an opinion regarding its palatability. This issue is of particular troublewhen a paper is rejected for taste (or contribution) on later rounds. Issues of taste (and contribution) should be dealt with on a paper's first round, and should generally not beissues on later rounds.5. Struggle to find a way to make your review positive in tone. Even when you find fault with the paper, favor the use of adjectives like interesting, imaginative, and ambitious. If, by some miracle, you happen to like the paper, don't shrink from being enthusiastic in your praise. Recall that authors hardly ever hear kind remarks about their work. The human tendency toward criticizing others dwells so strongly in our constitutions that we find it much easier to recognize flaws than to discover virtues. Try to repress that tendency. Allow yourself the pleasure of saying something nice or paying a compliment once in a while. 6.
One page would be too shallow— not enough thought on my part. Drafting more than four pages either signals a rejection (there is too much wrong in the paper) or that I'm writing a microscopic to-do list. I trim this down either way, and I always re-read the report before I send it to make sure it is as mannerly as possible. (Lil Mills)Begin with the major issues – important for the authors to know what the deal-breakers are
With respect to taste, I am sensitive to this issue as my own personal area of research is not mainstream within tax or even within accounting. Although we all have distinct tastes with respect to research, reviewers should provide an objective examination of the merits of a paper rather than an opinion regarding its palatability. This issue is of particular trouble when a paper is rejected for taste (or contribution) on later rounds. Issues of taste (and contribution) should be dealt with on a paper's first round, and should generally not be issues on later rounds. (Yetman)Try to accept the author's stated purpose and, instead of attacking the avowed objectives of the re-search, concentrate on evaluating how successfully it accomplishes those goals. (Holbrook 1986)
I generally devote two days to writing a review. First, I start by looking at the tables and try to infer what hypotheses are being tested. That way I am better preparedto assess whether the front and back ends of the paper match. Otherwise, I can be convinced by good storytelling on the part of the authors and miss disconnects in the analysis. Second, after looking over the tables and identifying the hypotheses that I think have been tested, I go back and read the manuscript from the beginning. At this point, I write detailedcomments in the margins and question each sentence. I will also read background papers if necessary. Third, I reread the paper and reconsider my margin comments in light of having seen the entire paper and having read other papers. Finally, I begin to write myreview, using the structure Richard suggests below by synthesizing multiple margin comments into a few (hopefully) coherent review points. Assessing the quality of analysis is generally time consuming, but not difficult. I find the most difficult part of writing a review is assessing the contribution.
Sample - We can be rigorous without being unreasonable.
How to be a better reviewer
How to be a better reviewer
Mary B. Curtis
University of North Texas
Mechanics of the review process
• You will be contacted by an editor about reviewing a paper and given a
– You are typically told the title and provided an abstract. You are never told the
author’s name and it is unethical to seek to learn the author’s name.
– A good way to begin your career as a reviewer is by volunteering to review for
• You review the paper and turn in your review – typically within a month to
six weeks. All AAA reviews are facilitated by the online system AllenTrack.
– To volunteer, go to AllenTrack and register. You will provide key words that
editors will use to select you.
• Your recommendation will be: accept (never on first round), minor
revision, major revision, reject
• You will generally learn of the editor’s decision to the author and often
receive the other reviewer’s comments, as well.
Characteristics of a good review
• The reviewer has two audiences: the author and the editor.
– For the author, you are trying to help this person get this paper
published – if not here, then somewhere
– For the editor, you are trying to help them decide whether to go
forward with the paper
A helpful review is one that is constructive rather than destructive.
Try to clearly communicate what the issue is, why it is a
problem, and what can be done about it.
An important aspect of the review process is the communication
of review points in a manner that is both clear and concise.
Do not communicate your recommendation directly to the
authors. It is the editor's role to make the decision and
communicate that decision to the authors.
Be tactful and kind!
Structure of a good review
• The review report should be typed, 12-point type, line
spacing maybe 1.15.
• Summarize the manuscript in 1 or 2 sentences.
• Find something to complement the authors on.
• Then begin with major issues – this section should not
be more than a page, two at max.
• Then cover two to three pages of questions and less
important concerns. (Lil Mills)
• Should you speak in third person (The authors state …)
or first person (You state that …) JATA group said first
Characteristics of a bad review
• Rejecting a paper because you are not interested in it Review the paper, not whether the idea is to your
taste; Do not try to turn the paper into something you
are more interested in – you are not the author (Bob
• Reviewer identifies a way to address a concern, then is
not satisfied when the results are unexpected (Anne
• Similarly, it is frustrating when a reviewer simply does
not believe your theory, but can offer no alternative
explanation for your results (Anne Magro)
Characteristics of a good reviewer
Read quickly through the paper right away, before accepting the assignment, to
make sure you are knowledgeable enough to complete it and that double-blind
anonymity hasn’t been broken (you don’t know who the author is). Also make sure
that everything you need (instrument, response document, etc.) has been
How to read the paper:
– Multiple times, in layers: One approach is for the first read to focus primarily on the theory
and logic of the arguments; Read next time, focusing on adequacy of experimental design and
– Multiple times, in layers: Another approach is to start by looking at the tables and try to infer
what hypotheses are being tested; then go back and read the manuscript from the beginning
– this way can assess whether the front and back ends of the paper match (Anne Magro)
– Set aside enough time to read the paper in depth and to provide a full set of comments
in one sitting. If you do your review over a period of time, your comments will likely
appear incoherent, unsympathetic, and self-contradictory (Holbrook 1986)
Don’t forget to compare theory and design to the instrument the author turned in.
Construct your comments – after you brainstorm the major issues, go back and
organize the review comments around the major issues.
• don’t tell a behavioral researcher to re-run their
experiment. Help them fix the one they have.
• do consider whether the sample is
appropriate, not whether it is experienced –
generalizability comes from the theory, not the
subjects (Libby et al. 2002; Peecher and Solomon
• do tell the editor if you suspect any untoward
problems with the manuscript – published
elsewhere, presented elsewhere (if for
conference review), etc.
Sources this material and
for more reading
• Campion, Michael A. 1993. Article review checklist: A criterion
checklist for reviewing research article. Personnel Psychology 46 (3):
• Feldman, D. C. 2003. Sense and Sensibility: Balancing the Interests
of Authors, Reviewers, and Editors. Journal of Management 29(1):
• Libby, R., R. Bloomfield, and M. Nelson. 2002. Experimental
Research in Financial Accounting. Accounting, Organizations, and
Society (27): 775-810.
• Omer, T. C., S. L. Porter, R. J. Yetman, A. M. Magro, L. F. Mills, R. C.
Sansing, and B. C. Ayers. 2004. A Discussion with Reviewers:
Insights from the Midyear ATA Meetings. JATA 26 (supplement):
• Peecher, M. E. and I. Solomon. 2001. Theory and Experimentation in
studies of audit judgments and decisions: Avoiding common
research traps. International Journal of Auditing 5: 193-203.