Downloaded 11 times
More Related Content
Similar to How to be a better reviewer
How to be a better reviewer
- 1. How to bea better reviewer Mary B. Curtis University of North Texas
- 2. Mechanics of thereview process • You will be contacted by an editor about reviewing a paper and given a deadline. – You are typically told the title and provided an abstract. You are never told the author’s name and it is unethical to seek to learn the author’s name. – A good way to begin your career as a reviewer is by volunteering to review for conferences. • You review the paper and turn in your review – typically within a month to six weeks. All AAA reviews are facilitated by the online system AllenTrack. – To volunteer, go to AllenTrack and register. You will provide key words that editors will use to select you. • Your recommendation will be: accept (never on first round), minor revision, major revision, reject • You will generally learn of the editor’s decision to the author and often receive the other reviewer’s comments, as well.
- 3. Characteristics of agood review • The reviewer has two audiences: the author and the editor. (Richard Sansing) – For the author, you are trying to help this person get this paper published – if not here, then somewhere – For the editor, you are trying to help them decide whether to go forward with the paper 1. 2. 3. 4. 5. A helpful review is one that is constructive rather than destructive. Try to clearly communicate what the issue is, why it is a problem, and what can be done about it. An important aspect of the review process is the communication of review points in a manner that is both clear and concise. Do not communicate your recommendation directly to the authors. It is the editor's role to make the decision and communicate that decision to the authors. Be tactful and kind!
- 4. Structure of agood review • The review report should be typed, 12-point type, line spacing maybe 1.15. • Summarize the manuscript in 1 or 2 sentences. • Find something to complement the authors on. • Then begin with major issues – this section should not be more than a page, two at max. • Then cover two to three pages of questions and less important concerns. (Lil Mills) • Should you speak in third person (The authors state …) or first person (You state that …) JATA group said first person
- 5. Characteristics of abad review • Rejecting a paper because you are not interested in it Review the paper, not whether the idea is to your taste; Do not try to turn the paper into something you are more interested in – you are not the author (Bob Yetman) • Reviewer identifies a way to address a concern, then is not satisfied when the results are unexpected (Anne Magro) • Similarly, it is frustrating when a reviewer simply does not believe your theory, but can offer no alternative explanation for your results (Anne Magro)
- 6. Characteristics of agood reviewer • • Read quickly through the paper right away, before accepting the assignment, to make sure you are knowledgeable enough to complete it and that double-blind anonymity hasn’t been broken (you don’t know who the author is). Also make sure that everything you need (instrument, response document, etc.) has been submitted How to read the paper: – Multiple times, in layers: One approach is for the first read to focus primarily on the theory and logic of the arguments; Read next time, focusing on adequacy of experimental design and analysis. – Multiple times, in layers: Another approach is to start by looking at the tables and try to infer what hypotheses are being tested; then go back and read the manuscript from the beginning – this way can assess whether the front and back ends of the paper match (Anne Magro) – Set aside enough time to read the paper in depth and to provide a full set of comments in one sitting. If you do your review over a period of time, your comments will likely appear incoherent, unsympathetic, and self-contradictory (Holbrook 1986) • • Don’t forget to compare theory and design to the instrument the author turned in. Construct your comments – after you brainstorm the major issues, go back and organize the review comments around the major issues.
- 7. Please …. • don’ttell a behavioral researcher to re-run their experiment. Help them fix the one they have. • do consider whether the sample is appropriate, not whether it is experienced – generalizability comes from the theory, not the subjects (Libby et al. 2002; Peecher and Solomon 2001). • do tell the editor if you suspect any untoward problems with the manuscript – published elsewhere, presented elsewhere (if for conference review), etc.
- 8. Sources this materialand for more reading • Campion, Michael A. 1993. Article review checklist: A criterion checklist for reviewing research article. Personnel Psychology 46 (3): 705-718. • Feldman, D. C. 2003. Sense and Sensibility: Balancing the Interests of Authors, Reviewers, and Editors. Journal of Management 29(1): 1-4. • Libby, R., R. Bloomfield, and M. Nelson. 2002. Experimental Research in Financial Accounting. Accounting, Organizations, and Society (27): 775-810. • Omer, T. C., S. L. Porter, R. J. Yetman, A. M. Magro, L. F. Mills, R. C. Sansing, and B. C. Ayers. 2004. A Discussion with Reviewers: Insights from the Midyear ATA Meetings. JATA 26 (supplement): 135-141. • Peecher, M. E. and I. Solomon. 2001. Theory and Experimentation in studies of audit judgments and decisions: Avoiding common research traps. International Journal of Auditing 5: 193-203.
Editor's Notes
- #4 1 & 2. Sue Porter: should also provide specific suggestions wherever possible about how to solve the issues identified in the review. Most important, a good review raises issues that, when successfully addressed by the author, increase the paper's likelihood of publication.3. With respect to taste, I am sensitive to this issue as my own personal area of research is not mainstream within tax or even within accounting. Although we all have distinct tasteswith respect to research, reviewers should provide an objective examination of the merits of a paper rather than an opinion regarding its palatability. This issue is of particular troublewhen a paper is rejected for taste (or contribution) on later rounds. Issues of taste (and contribution) should be dealt with on a paper's first round, and should generally not beissues on later rounds.5. Struggle to find a way to make your review positive in tone. Even when you find fault with the paper, favor the use of adjectives like interesting, imaginative, and ambitious. If, by some miracle, you happen to like the paper, don't shrink from being enthusiastic in your praise. Recall that authors hardly ever hear kind remarks about their work. The human tendency toward criticizing others dwells so strongly in our constitutions that we find it much easier to recognize flaws than to discover virtues. Try to repress that tendency. Allow yourself the pleasure of saying something nice or paying a compliment once in a while. 6.
- #5 One page would be too shallow— not enough thought on my part. Drafting more than four pages either signals a rejection (there is too much wrong in the paper) or that I'm writing a microscopic to-do list. I trim this down either way, and I always re-read the report before I send it to make sure it is as mannerly as possible. (Lil Mills)Begin with the major issues – important for the authors to know what the deal-breakers are
- #6 With respect to taste, I am sensitive to this issue as my own personal area of research is not mainstream within tax or even within accounting. Although we all have distinct tastes with respect to research, reviewers should provide an objective examination of the merits of a paper rather than an opinion regarding its palatability. This issue is of particular trouble when a paper is rejected for taste (or contribution) on later rounds. Issues of taste (and contribution) should be dealt with on a paper's first round, and should generally not be issues on later rounds. (Yetman)Try to accept the author's stated purpose and, instead of attacking the avowed objectives of the re-search, concentrate on evaluating how successfully it accomplishes those goals. (Holbrook 1986)
- #7 I generally devote two days to writing a review. First, I start by looking at the tables and try to infer what hypotheses are being tested. That way I am better preparedto assess whether the front and back ends of the paper match. Otherwise, I can be convinced by good storytelling on the part of the authors and miss disconnects in the analysis. Second, after looking over the tables and identifying the hypotheses that I think have been tested, I go back and read the manuscript from the beginning. At this point, I write detailedcomments in the margins and question each sentence. I will also read background papers if necessary. Third, I reread the paper and reconsider my margin comments in light of having seen the entire paper and having read other papers. Finally, I begin to write myreview, using the structure Richard suggests below by synthesizing multiple margin comments into a few (hopefully) coherent review points. Assessing the quality of analysis is generally time consuming, but not difficult. I find the most difficult part of writing a review is assessing the contribution.
- #8 Sample - We can be rigorous without being unreasonable.