2. To paraphrase Winston Churchill…
“Peer review is the worst form of scientific
evaluation, except for all others that have
been tried” (Roediger, 1987, p. 239).
3. CL’s perspective…
First published in 1986
As of Thursday 12 NOV 10, 2 books and 77
articles in refereed journals (PsycINFO)
Associate Editor (AE) @ Organizational
Research Methods 2005-2015
Editorial Board, PPsych, Human Performance,
& GOM (previously, PPRP and HRMR)
Reviewer for scores of other journals,
including methods, I/O, HR, OB and lots of
other stuff (e.g., exercise, medicine, pharmacy)
Teach “Behind the Scenes @ ORM”
4. Pre-publication steps
Plan the research
Do the research
Write it up
Pick a publication outlet
Send stuff out for review
5. Step 1: Initial submission
Author
Editor
1.Cover Letter
2.Your paper
3. Prayer(?)
6. Step 2: Reviewer selection
and dissemination of stuff
-Manuscript
-Instructions
-Rating form
-Review form
Editor
R1
R2
R3
7. Step 3: Reviews get returned
Editor
R1
R2
R3
-Completed
rating form
-Detailed
review
-Recom-
mendation
8. Step 4: Receive Reviews and
Decision
Author
Editor -Editor’s decision
letter
-(Completed
rating forms)
-Reviewers’
comments
9. Steps 5
Prepare a revised version of your
paper for publication reconsideration,
and
A LONG letter in response to the
Editor’s and reviewers’ comments
- OR -
Prepare a revised version of your
paper for publication consideration at
another journal
R&R
Reject
10. Steps 6, 7, 8 and 9
6. Resubmit revised manuscript for publication
reconsideration along with your loooong letter
7. Original reviews, revised manuscript
and letter sent to original reviewers
8. Reviewers judge the adequacy of
your responses and revisions and
perform a second review
9. Editor adjudicates the success of your efforts
and communicates the editorial decision
11. Overview of the whole process
Submit paper Desk reject
Initial Review Reject
Revise & Resubmit*
Re-review Reject
Conditional
Accept
Re-review
Accept
30%
15%
55%
(rare)**
(twice)
~5%
*
About as good
as it gets
**CL – 4 times, JAP,
SEM, JVB, ORM
13. Pre-submission: I will not discuss
Planning the research
Executing the research study
Data analysis
Interpretation
I assume that you’ve completed your
research, and you know what you
have (for better or worse)* – how do
you sell it and get it published?
*(HP 2007)
14. Does it matter where you publish?
Thousands of journals and hundreds of different specialties
Multiple tiers – the higher the tier, the more your paper is
“worth” (Gomez-Mejia & Balkin [1992] – the value of a top-tier paper was
US$1,210 in 1988, a future value of $9,589, and a cumulative annuity of
$84,134 in 2011)
Higher citation rates (Gomez-Mejia & Balkin [1992] –each citation worth $192)
Greater prestige for your Department and University
Some Universities pay on a “piece rate” basis
Increased chances of P&T and mobility, more student applications
Impact factor – reflects citation rates (see ISI Web of
Science)
North American vs. European, Asian and
Australasian journals
15.
16.
17. How do you decide?
Fit between your paper and the journal’s mission
I/HR vs. O/OB
Theoretical vs. methodological vs. empirical
Read journals’ mission statements
Peruse previously published articles & TOCs
Ask colleagues or the Editor
Strength of your paper’s findings/ relative
contribution
“Trickle-down” model
18. Manuscript preparation
Appropriate style guide, e.g.,
APA Publication Manual
Lots of practical guides, e.g.
Silva, P. (2007).How to write a lot: A
practical guide to productive academic
writing. Washington, DC: American
Psychological Association.
o Bem, D. J. (1987). Writing the empirical journal
article. In M. Zanna & J. Darley (Eds.), The
compleat academic: A practical guide for the
beginning social scientist (pp. 171-201). New
York: Random House.
19. Manuscript preparation
“Hourglass” structure
Simple declarative sentences
Edit, edit, and edit
Colleagues’ informal review
I start with Tables, then the
references, then the introduction, etc.
Reviewers write my discussion
sections
20. 1. Initial submission
The cover letter
Does it matter?
Human subjects
Exclusivity
Research ethics
Has no practical bearing
I’ve read one at ORM as AE by
accident!
21.
22.
23.
24.
25. Step 2: Reviewer selection
and dissemination of stuff
Editor
R1
R2
R3
- (Typically 3) reviewers
selected on the basis
of their expertise
- 1 or 2 board members
- 1 or 2 ad hoc reviewers
- Editor or AE may select
reviewers
- Reviews are intended to be “double-blind”
- Authors can suggest reviewers or “un-suggest” them
- Aim for 45-day turnaround at ORM
- Editor will gently prod tardy reviewers – can drop them
26. What you do in the mean time
Sit back and wait 6 weeks to 2 months
– WRONG!
Work on your other R&Rs or your next
research or your next paper
Should always have multiple things “in
the pipeline”
27. A VERY active colleague of mine…
Blair, C.A. & Hoffman, B.J. (under review). Situational judgment tests vs. assessment centers:
A comparison of high and low fidelity managerial simulations. Personnel Psychology.
Bynum, B. Hoffman, B.J., Meade, A.W., & Gentry, W. (under review). More evidence for the
meaningfulness of multisource performance ratings: A comparison of across-source
and within-source measurement equivalence. Human Performance.
Eby, L.E., Allen, T., Baranik, L., Hoffman, B.J., Curtis, S., Baldwin, S., & Morrison, A. (revise
and resubmit status). An interdisciplinary meta-analysis of the antecedents, correlates,
and outcomes of mentoring. Psychological Bulletin.
Hoffman, B.J. & Meade, A. (revise and resubmit status). Rethinking the psychometric properties
of AC dimensions: New evidence using invariance constraints. Human Performance.
Hoffman, B.J., Gorman, A., Atchley, E.K., Blair, C., Meriac, J., & Overstreet, B. (under review).
When practicality and utility collide: Evidence for the effectiveness of an alternate
multi-source feedback measurement methodology. Journal of Applied Psychology.
Miller, J., Hoffman, B., & Harding, H. (revise and resubmit status). The structure and nomological
network of narcissism. Journal of Personality.
Sutton, A.W., Baldwin, S., Wood, L., & Hoffman, B.J. (under review). A meta-analysis of the
influence of rater affect on performance ratings. Journal of Applied Psychology.
30. The narrative review
Review sandwich
Reviewers’ job to find fault – Editor’s
job to publish a journal
SLAMing (Stressing the Limiting
Aspects of Manuscripts)
How blind are “blind reviews?”
Nowadays – critical yet constructive
31. Are reviewers biased?
Four perspectives:
Gatekeeping – filter out unworthy research from
that worthy of publication
Particularism – decisions based on personal
preferences or prejudices
Universalism – decisions based on universally
held criteria of high quality academic research
Accumulated advantage – published authors
accumulate advantage and become even more
successful
32. Why are papers rejected?
Lack of incremental contribution –
what’s new? What unanswered
question does this research address
and does it answer it?
Methodological flaws
Preparation/organization/coherence
33. Lance’s rejections 2009-2010
Rejected 11 papers
Average of 3.55 reasons for the
rejection
Reasons:
41% Methodological/design flaws
26% Lack of incremental contribution
21% Preparation/organization issues
8% Omitted literature/lack of theory
5% Lack of fit with ORM’s mission
34. Characteristics of good papers
Firmly grounded in relevant theory
Previously unanswered questions are
clearly identified
Methods appropriate to answer the research
questions
Well organized and clearly written
Find significant results
Tie findings back to implications for theory
and practice
35. Integration…
Reviewers’ comments and
recommendations will overlap, but
Inter-reviewer agreement is notoriously low
– disagreement is common and expected
Reviewers chosen for their complementary
perspectives
Editor must integrate, reconcile, and
communicate the publication decision to the
author(s)
36. As AE @ ORM, I…
Work hard to try to make the right decision
Am reasonably “hands on”
I admit to having made one mistake and it
was satisfactorily rectified
Have tried to maintain respect for all
concerned (not always easy)
Dedicate about ½ day per week as AE
Tried to avoid “ghostwriting”
37. Your chances with Lance at ORM
2005-2010 Average/yr
Accept/CA 0 0
Reject 63 10.8
R&R:
Accept 29 5
Reject 3 <1
Overall acceptance rate to date (CL): 30.5%*
Overall rejection rate: ~88%
*26.7% excuding 5 “invited” papers
39. If you get a “desk reject”
You have not done your homework, or
Your paper was not judged to be of
high enough quality to take reviewers’
time to send it out for review
SO:
You should re-evaluate the quality of
the work, revise it in light of the
Editor’s comments and send it to
another journal
40. If you get a “reject after review”
Do not disparage – most papers are
publishable somewhere
Do not simply resubmit the paper to another
journal
The reasons for the paper’s rejection are still there
Decent chance of getting the same reviewers at
another journal – it’s a small world!
Revise it in light of the Editor’s and reviewers’
comments and send it to another journal
(perhaps a “lower tier”)
41. If you get a “Conditional accept”
Update your vita immediately
to read “(in press)”
Throw a huge party and invite
everyone you know
You have just won the
publication lottery
Cherish the moment because it’s not
likely to happen again!
42. If you get an “Unconditional accept”
Don’t worry, it’s not going to
happen
A miracle has occurred
Consider establishing a new
religion
43. If you get a “R&R…”
This is what you’re hoping for!
Your work has just begun…
“High risk” vs. “low risk” R&R and reading
between the lines
You should eventually get your paper accepted
for publication (the door is open)
You want to minimize the number of cycles in…
