A couple of days ago, I outlined my thoughts on two of a set of seven ideas that have been proposed for improving scholarly journals — that articles should be shorter and that more of their content should be laid off to Web sites. Although the merits of those two suggestions are obvious, I have some qualms about both of them.
Now I’ll turn to the third suggestion.
Editors should decline to send out a significant number of submitted articles for review. Any editor worth his or her salt recognizes an article at the time of submission with no probability or a low probability of success.
I suppose we could all agree with this suggestion in principle, but here as so often is the case the devil may lie in the details. What’s “a significant number”? 5%? 20%? 40%? When we move from principle to practice my “in principle” concurrence begins to wear a bit thin.
The premise underlying this suggestion is unquestionably correct. A large proportion – even, though I hesitate to say this publicly, more than half – of the papers that a journal – even a top-tier journal – receives are instantly recognizable as non-starters. In some instances, that’s because the submission is substantively inappropriate for the journal; no matter how broadly the journal’s scope is defined, the paper simply falls outside of it. In some other instances, it’s because the paper wasn’t ready to be sent out; not enough sweat has been invested in it and not enough pairs of eyes have seen it. At the APSR, we received many papers that were riddled with typos (even in the title), full of basic grammatical errors, and so on – in short, papers that hadn’t even been spell-checked, let alone carefully edited and re-edited. More commonly, though, we received papers that weren’t ready to be seen because the author hadn’t yet decided what the paper was about. The author may, for example, have run some regressions and provided a detailed report of the results without even attempting to define a research issue crisply or to make a case for its importance. At least at the APSR, though, the most common shortcoming was none of the above. Rather, it was just that the paper wasn’t “big” enough, in a theoretical and/or substantive sense, to warrant publication in a top journal. It may have been a competently executed analysis, but it really belonged to a subfield journal, given its incremental contribution.
So yes, lots of papers – many, even most – could be rejected by the editor after a quick scan. (Though I’d be less comfortable about having an editor reject a paper for the last-named reason than for the preceding ones, given that the paper may center on issues far removed from the editor’s own expertise.)
But let’s ask ourselves why a larger portion of papers should be rejected without being sent out for review, and why they shouldn’t be.
One reason to bypass the review process is to save time for the author. Who wants to sit and wait for month after month for a rejection letter, especially when it turns out that the paper wasn’t a bona fide contender in the first place? This is a valid concern, but it’s much less valid than it used to be, because (horror stories aside, and everyone has one) journal turnaround times are so much faster than they used to be. At the leading political science journals these days, the turnaround time for papers that go through the full review process tends to be two to three months. That’s not a long wait.
Another reason to bypass the review process is to save time and effort for reviewers. It’s not much fun to be sent a lousy paper and asked to review it, and reviewing lots and lots of papers (many of which are lousy) puts a burden on reviewers. By the way, one major reason why the burden on reviewers has gotten so heavy is that turnaround times have gotten so short; now an enterprising author can get a paper rejected by three or four different journals in a single year. On the other hand, when a reviewer complained to me about the poor quality of a paper I had sent for review, I would ask which type of paper requires more time and effort on the reviewer’s part: a good one that could be better, or a an irredeemably lousy one. Almost invariably the answer would be “the good one,” because it doesn’t require a close reading to determine, or a lengthy review to describe, what’s wrong with a lousy paper. Now assume that I, as editor, am going to send you no more than a certain number of papers per year to review – say, three or four. In terms of conserving your time and energy, I’m actually doing you a favor by sending you lousy papers.
Do you think I’m kidding about that last point? Well, to some extent I am, but only to a very limited extent. Consider the following: The best review I ever saw, of the thousands I read while editing the APSR, began as follows: “I love this paper.” Then the reviewer went on to present 22 single-spaced pages full of specific and constructive suggestions about the paper could be improved. The reviewer must have spent at least a week preparing that review. That’s obviously an extreme case, but it dramatically illustrates the point.
Along the same lines, I occasionally received complaints from reviewers to the effect that they were so overburdened with writing reviews that they didn’t have time to do their own work. In many instances, when the reviewer told me how many reviews he or she had completed recently, I sympathized and let him or her off the hook. But in many cases I didn’t. Here’s the way such conversations often went:
Reviewer: I’m way too busy writing reviews. This is a real burden.
Me: I’m sorry to hear that. How many reviews did you write last year?
Reviewer: Let’s see: Eight or ten, I guess. Maybe one every month. That’s terrible.
Me: And how many papers of your own did you submit for review?
Reviewer: I finished three new papers last year.
Me: And were they all accepted?
Reviewer: Well, no. In fact, all three were rejected.
Me: And did you then send them to another journal?
Reviewer: Yup.
Me: And were they then accepted?
Reviewer: Hmm. Two got revise-and resubmits, and one got rejected.
Me: And did you revise and resubmit the two and submit the other one to a different journal?
Reviewer: Yes.
Me: And did you have any older papers that were out for review last year?
Reviewer: Yah, two or three of them that I’d finished the year before and revised.
Me: Okay, now let’s do the math. Last year you reviewed eight or ten papers, maybe twelve. During the same year, you generated a need for, let’s do the math, three reviews each of your three new papers on the first time around – that’s nine reviews – plus three more apiece when you sent them out for the second time – that’s nine more reviews – plus three more apiece on the third round – that’s nine more reviews – plus, say, six to nine for the older papers you were still circulating from the previous year. So you wrote somewhere between eight and twelve reviews while creating the need for somewhere around 35 reviews. It sounds to me like you’re not carrying your weight. You’d better get busy writing some more reviews.
Reviewer: [Sputtering noise heard – expletive deleted.]
Now, two other points:
The top journals are overrun with submissions. It’s unrealistic to expect the editor of such a journal to provide anything beyond a cursory explanation of why a paper wasn’t even sent out for review. Authors, for better or worse, expect to have their work appraised in more than a cursory way – you do, I do, and so do they. Having your paper returned with a summary rejection is not at all a happy occasion, and tends to breed considerable ill will for the journal and to foster the sense that one hasn’t gotten a fair hearing. What do you, as an author, learn from having an editor tell you, in virtually so many words, that he or she isn’t going to open the review process on your paper because it doesn’t appear to be a strong candidate for publication?
And here’s the double-whammy part of that reservation: A disproportionate number of the lousy papers are submitted by junior scholars. For them, getting their work reviewed by those who are knowledgeable in their field can be an important socialization experience from which they learn a lot. By contrast, getting their work summarily dismissed by an editor gives them almost nothing to go on. The peer review process is supposed to produce better scholarship, in the long term if not always immediately; but that goal gets short-circuited if one doesn’t get one’s work reviewed.
Journals vary. Editors vary. Situations vary. In many instances, more papers should be rejected without being sent out for review. Prominent among those are papers in the first couple of categories I mentioned above: papers that clearly fall outside the scope of the journal and those that are so sloppy that they shouldn’t have been submitted. If a paper meets the minimum criteria of pertinence and care in preparation, though, I tend to believe that the advantages of sending it out for review outweigh the disadvantages.