Peer Review Problems

I am not going to pretend that I have a solution to this problem, I want to make that clear upfront.

As an academic who has a niche research area, I tend to get sent a number of manuscripts to review for journals every week. Depending on my workload, I will either agree to do the peer review, or I will politely decline (and offer names of others who might be suitable to the journal editors). This is fairly standard practice.

On the occasions that I agree to undertake the manuscript review, I give it the time that it deserves. After all, this is someone’s livelihood. I expect the same in return when I submit work to a journal, though I’m not naive enough to believe I always get it, especially given some of the reviews I’ve received back in the past.

The issue however seems to be that the reviews aren’t always taken seriously by the journal editors. I’m not going to name names here, but there are a couple of journals in particular that seem to crop up time and time again. I have rejected papers outright for having made mistakes in the mathematics, thus rendering the results and subsequent interpretations meaningless. I have rejected papers outright for making invalid assumptions about the data or mathematical model, again rendering the results and interpretations meaningless. I detail these issues in my report to the journal editor, along with my recommendation to reject the manuscript. Sometimes papers just need some tidying up, or a rethink of their interpretations. But fundamental problems with the assumptions or the mathematics are unrecoverable without rewriting the entire paper. It’s not a “major revision required”, it’s “you need to rewrite the entire paper”.

Yet I have lost count of the number of times that I have rejected papers for having mathematics that is just plain wrong, having results that are mathematically impossible to obtain, or making assumptions that are statistically invalid. Papers that I have subsequently seen published in the journal with the problems still staring me in the face. I know they send the manuscripts to 2 or 3 reviewers for assessment, and I may have been the only one to pick up on the issue (reviewers have different areas of expertise). But when I explicitly detail where the problems are and how they invalidate the results and interpretations, yet the paper is published pretty much as is anyway?

What’s the point of sending manuscripts out to peer review if the journal editors are just going to ignore the reviews and recommendations of the expert reviewers? Isn’t the whole point of peer review to send the manuscript to someone who knows more about the topic than the journal editor in order to make an assessment on the validity of the science? If there’s a problem with the science that the reviewer picks up on, then that needs to be dealt with, as inconvenient as it might be within the academic political sphere.

Not to mention I’ve seen some manuscript authors take what I’ve written in a review and copy large parts of it verbatim into their revised manuscript without any attribution. You know what? That’s plagiarism. Call it a pers. comm., that doesn’t bother me. But that kind of behaviour is repugnant. I discussed this particular case with a few people, without mentioning the journal or culprits, and was basically told that “it happens, the editors know about it, but it’s too much hassle to deal with”.

The problem is that you usually don’t know about these problems with your review being ignored or your review being plagiarised until it’s been published and is in the public domain. While you can submit a complaint to the journal editor, it’s really too late to do anything about it. The journal can issue a retraction, but once it’s out, it’s out. People will have seen it, and you can’t really take that back.

While I’m all for journals publishing work from a more diverse range of authors, this need not occur at the expense of good science or good academic practice. I now refuse to publish anything in these journals, it’s happened far too often to be put down to a simple oversight. How can I publish in a journal that I know for a fact is ignoring reviews and publishing bad science? I don’t want my own work tarred with the same brush. And it’s also reaching the point where I no longer want to agree to undertake reviews for these journals at all if all my effort is simply going to be ignored. We don’t get paid for doing peer review, we do it to be good academic citizens, and these journals are just wasting my time.

As I said from the start, I don’t really have a solution to these problems that is workable. Maybe journals should be required to send the final corrected manuscript proof to the reviewers before putting it on their “In Press” section on their websites. It’d give a final chance for the reviewers to intervene. It might not get us anywhere if the reviews are being intentionally disregarded by the editors though. I don’t know. It’s just frustrating me.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s