Today on Slate, I do one of my things where I have a really quick and easy idea that will fix a huge problem in academia with little or no new effort (in this case, it is simply redistributed effort), and everyone laughs at it and yells at me.

Today the subject of my things-fixing is academic peer review, specifically in the humanities (though some in the sciences will find familiarity too–though I hope not in the 18-month wait time!). In fact, peer review is such a passionate subject amongst academics that I have a follow-up column for an academic audience coming out in Vitae on the heels of this one! ACADEMICS HAVE A LOT TO SAY ABOUT PEER REVIEW, and I want to help the system be better for them, even though I plan never to participate in it again.

So, what do you guys think? Is this a good idea–or, more importantly, will its good far outweigh its harm? Is it yet more unpaid academic labor (or, rather, “more” labor for people who should be doing it in the first place–bigwigs who submit their old recycled stuff to top-tier journals and simply expect them to be accepted?)? I want to hear from you!

ADDENDUM:

Here, in case you’re wondering, is a full list of my experiences with peer review.

  1. My first article ever, “The Mirror and the Tower,” on 18th Century Sturm und Drang (wot? yes! I love Sturm und Drang! I do, I do!), was submitted to the Women in German Yearbook in 2007 when I was still a grad student. It got a revise-and-resubmit that was helpful but pretty harsh (though I can’t blame the reviewer because feminist Germanists do NOT like David Welberry and my stuff was based almost entirely on one thing he wrote once). I revised and resubmitted, and then it was rejected solely on the basis that I used one quote from one source that they did not deem sufficiently scholarly. I was pretty bummed, but one of my mentors, the luminously brilliant Gail Hart (sorry Gail, now everyone knows I love you), said “Nah, it was good! Just send it somewhere else!” So I sent it to a comp-lit journal called Sympoisum that only accepted submissions on paper (!), and received an acceptance, also on paper (!) while I was living in Vienna in 2008. Whee!
  2. My second article, which began its life in 2009, “Wittgenstein’s Tractatus, Kafka’s Verwandlung and the Limits of Metaphor,” was rejected out of hand without being sent out for peer review by Philosophy and Literature. A few months later, the editor of Phil & Lit died, and I was probably not as sad as I should have been. Because what the fuck, not even sending it out? So I sent it next to Modern Austrian Literature, where it underwent a two-year R & R epic saga under the eye of an increasingly irate reviewer who–it became increasingly clear–had not so much as skimmed the Ludwig Wittgenstein Wikipedia page. I made all of his/her changes dutifully and with good humor, and my tenacity was rewarded when the article finally came out in 2011.
  3. My third article, which was solicited (ha!), “Unerschütterlich: Kafka’s Proceß, Wittgenstein’s Tractatus and the Law of Logic,” was accepted enthusiastically on first pass at The German Quarterly, with only minor revisions. Huzzah!
  4. My book, Kafka and Wittgenstein, came about because an editor at Northwestern noticed I was presenting a paper on Robert Walser at the 2011 GSA, and asked to meet with me (and I was like HELL YES OK WHAT IS THIS A JOKE?). In advance of the meeting, I sent him a galley proof of the German Quarterly article. On the basis of that German Quarterly article alone (no proposal), I was offered an Advance Contract on a book (which, one of my former faculty members and many Chronicle commentators were quick to remind me, “doesn’t mean much”). I finished the manuscript of Kafka and Wittgenstein in May of 2013 and was given the readers’ reports (WHICH WERE AWESOME and not even a little bit like the ones I pick on today), later that year. The book then went to Acquisitions, where it was accepted for publication and is now forthcoming.
  5. My final academic article, “Nichts zu sagen,” which I am just gonna go ahead and upload here in draft form (because, why not?), was submitted to Modern Language Notes in October of 2012. I have yet to hear a single, solitary peep back about it.
Advertisements

40 thoughts on “Peer Review Review

  1. One of the consequences of this fucked-up system is that people are now sending stuff to “journals” (I.e., scams) where you’re pretty much guaranteed publication if you pay the editing “fees” that can run up to $1k.

    Like

  2. But I am Dick. There are thousands of journals. Any idea or group can be the start another one and ‘be the peer’ reviewers. But these peers are of a small slice and most often incapable of critical reading and consideration. When I get my 3 friends together I can let them “peer review” my work. It has become utterly meaningless.

    Like

  3. I once had an article spend, oh, about four years in Revise and Resubmit Purgatory. The very last report basically told me all the reasons my article sucked and didn’t cite the correct literature–and then recommended publication.

    Like

  4. I agree with many of the Slate commenters. I think there’s a grain of truth to your criticisms, but overall you give a cartoonish and clichéd picture of the peer review process. So it’s not very useful.

    Like

    1. Well, you are an experienced academic with a lot of publications and thus extensive experience on both sides of peer review, so you know what you’re talking about. Oh wait, no, you’re not, and you don’t.

      Like

      1. First of all, you don’t know anything about me. For example: that I use my married name on Facebook, but not in my professional life. Secondly, if you’re saying that a person needs extensive experience and a lot of publications to know what they’re talking about, then you must not know what you’re talking about.

        Like

      2. I have a fair amount of publications and extensive experience interviewing and talking to academics of all levels. I have no idea who you are but assume you are a grad student or recent PhD because of how passionately you seem to dislike me. Most experienced academics benignly ignore me or know I’m actually doing good. Only someone very deluded with inexperience would act like you.

        Like

      3. I don’t dislike you, Rebecca. I don’t even know you. I just remarked that your Slate article makes some good points but also trades in cartoons and clichés about the peer review process. You tried to pull rank, so I called you out on that as well.

        I notice you tend to do that – you cast yourself as an academic outsider when it suits you, but when you sense that you’re talking to someone even more marginal than you, you wave your supposedly superior credentials at them. You behave like a bully. You did it on that grad student’s blog, and it was gross. You tried to do it to me, and it didn’t work.

        You might be a perfectly nice person – I don’t know. I just find your mode of arguing hypocritical and high-handed.

        Like

      4. Ugh, I DEEPLY regret engaging with those shits over at Meta Whatever. That I will blame, however, 100% on pregnancy. I was having THE WORST morning sickness and moodiness at that time, I mean THE WORST. I was not myself. I apologized to everyone (citing “health issues” because my pregnancy was too early to be open about), and will duly apologize to you. SchuFetus, consider yourself blamed for all of my misdeeds in the months of May and June once again.

        But, that’s an interesting point and I do see where you’re coming from, but…when I’m talking to someone “more marginal” than me? Not possible. Literally not possible to be more marginal than me.

        However, I think it’s more nuanced than you put it. I DO tend to pull rank on people IFF (if and only if, in logic-talk) they are shilling for academia and I don’t understand why–then I simply remind them that they haven’t seen the movie all the way through and I have (when necessary).

        That’s what you seem to be doing, and I legit don’t understand why. Why go to bat for a system that will almost certainly exploit you and chew you up and destroy you?

        Like

  5. the comments on that piece are like… so much of peer review itself. Phew.

    I like your suggestion (though I think the point made about people potentially gaming the system might affect its workability). I wonder if another alternative would be to have each reviewer also have access to the others’ reviews (even if anonymously) before these are returned to the submitting author. I think people would quickly realize how “biting cleverness” comes off as “being an ass” when reading others’ versions of it (rather than their own, which they privately no doubt chortle over and imagine are just delightful), and also just how well a constructive review reflects on the reviewer. It can be a critical constructive review — doesn’t have to gush — but demonstrates the character and intellectual generosity of the person writing it so clearly. If people knew they were going to have an audience bigger than the author whom they are going to JUST CRUSH HA HA HA HA HA and the editor whom (I think) they imagine chortling along with them, and had a few rounds of reading what other people’s “being an ass” vs. “being a colleague” look like, they might get it together.

    Like

      1. 😦 I hope your blog counteracts the asshole mc trollies. I’d send you cupcakes if I could, because cupcakes always help.

        Like

    1. This and Rebecca’s are good ideas worth considering. Even anonymously, it would also encourage reviewers to give real feedback, rather than three vague comments and “revise and resubmit.”

      Like

  6. I like the part of your proposal where you only get considered to publish if you, yourself, have turned in reviews in a timely fashion. I just had an article rejected because one reviewer recommended “reject” and the other one just went AWOL (at least the editors admitted it.)

    Like

  7. I like the idea, and I especially like the idea of doing something, anything, to make submitting a journal article less painful in terms of waiting, reading snide comments, etc. Still, I am not certain that everyone should be reviewing articles who are writing them. Aside from the problem of expecting that everyone even knows how to give constructive (or heck, negative but at least helpful) criticism, I wonder whether insisting on doing a review in order to be considered will end up with the same group of people reviewing and then submitting/publishing. I do think that setting a time limit and meaning it (and pulling the person off the review and then a “time-out” period for being asked to review) would help authors enormously. Adding names to reviews? I thought I liked that idea until I watched too many colleagues one-up each other with negative criticism for committees of various kinds (much like the conference situations you describe). In the end, it is part of service that doesn’t really get recognized – there are neither offers of books or $$ as there usually are for reviewing book manuscripts nor any meaningful way to list the service on a c.v. With so many journals owned by the same publishing houses, what about a point system? Free year of journal subscription to one of the journals or a book from the publishing house if you do x number of reviews over a 2-year period? Wait, let me guess, too expensive. (Because the time we all wait for reviewers to get back to us has no value, of course. 😉 ).

    Like

  8. I work on a prestigious humanities, peer-reviewed publication and I can say that your assessment is SPOT ON. I also think your solution is a good one. I have maybe one or two people (out of dozens) I know will always get back to me in a timely fashion. Something I’m trying to work into the system is dumping reviewers if they take past a certain amount of time, though of course that means we have to dig up someone else…

    Like

      1. DUDE, it is totally true. A friend had a ms returned to her, from a grad student journal, with comments such as “What the hell kind of piece of shit argument is this? This is barely even an argument.” and “This is bullshit.” peppered through her ms.

        Like

  9. You get shitty comments for suggesting this?! Geez. As Kathleen Lowrey said above, they unintentionally give support to your proposal.

    I’m now curious to find out what you could suggest that would not receive rabid criticism. “I think professors should try hard to show up to class on time.”

    Like

      1. Yeah, I just don’t get that. If you have tenure – if you are untouchable, – why are you so invested in the system? I guess if you criticize the system that gave you a stamp of approval, you’re casting doubt on your own quality.

        If I was good at anything else and didn’t love my job like I do, I’d feel like I don’t belong in academia.

        Like

  10. Your suggestion about establishing eligibility to publish in a journal through being a responsible referee first is a good one, and is a variation on a notion I use myself: I don’t referee articles for journals to which I do not subscribe. That is, if I have demonstrated my commitment to a journal by subscribing to it, I have an interest in the quality of the work published there, and this suggests that I will be a responsible referee; another, more mercenary way to put it is that I have bought the right to have a say in the journal’s work. Several editors have expressed surprise when I decline to review a submission on the grounds that I am not a subscriber to his/her journal, but I cannot, in good conscience, impose an opinion where I have not earned the right to do so. In fact, it seems a little arrogant to imagine that everyone will want my opinion….

    Dave Taylor

    Like

    1. Ha, right? I think the opposite holds true, too–if you’ve never read a journal and wouldn’t be qualified to review anything in it, you shouldn’t submit there (but that doesn’t stop a lot of people!).

      Like

  11. I considered waiting six months to post this comment in the spirit of peer review time says:

    I have a few rejoinders to offer in the spirit of friendly comments. One, nothing makes a professor more compassionate with his/her students than the peer-review process. When a prof receives a negative report back from a blind reviewer telling him, “the author’s thesis is muddled” he remembers how hard it is to articulate a clear, precise, and intriguing claim. (Professors who have stopped publishing are the ones who complain the most about students’ writing.) Two, the peer-review process saves many junior faculty/graduate students from having an embarrassing article appear in print. Third, the articles that are published are fabulous – erudite, convincing, and boundary breaking. In terms of outcomes of research, there is no problem with peer-review. I also think students with errant dissertation advisors can use the peer-review process as their secret weapon. If one’s director won’t offer feedback on chapters, then send it in to a journal. A student will likely receive two thorough reports with ample feedback on what needs to be fixed. (In my experience advisors are even slower than peer-reviewers.) If the article gets R&R’d, then bonus!

    Like

    1. There is absolutely no reason for any of what you say to necessitate meanness for its own sake, which is what I am critiquing. There is a difference between “this thesis is a bit muddled,” which is fine, and “this person has no business in the discipline because she did not cite the two sentences of Foucault I have read,” which happens a lot. Also: “The articles that are published are fabulous – erudite, convincing, and boundary breaking.” I’d have to disagree, at least in my discipline and similar. I’d say that 90% of what appears in journals today is staggeringly medicore and wholly uneccessary–and has the readership of exactly zero to prove it (and that is a proven fact). Finally, while some of my experiences with peer review made my work better, there was one experience, with Modern Austrian Literature, where my capitulation to the increasingly nasty whims of a reader that knew demonstrably less about my subject (Wittgenstein) than I did made the article considerably worse. Nothing made me happier than un-doing that reader’s asinine requirements when I converted that article to a chapter of my book–which was accepted as-is. The defense of the status quo of peer review by academics has been, to put it mildly, flabbergasting. I know literally nobody who has not had an experience so painful it scarred them for life. I am called on the phone by weeping friends; emailed by distraught colleagues. Not because someone said their thesis was muddled, but because someone who doesn’t know them and probably didn’t read their work very carefully has said they are worthless as scholars. There is simply no place for that in a healthy discipline, and advocating on its behalf simply demonstrates internalization of a culture of abuse.

      Like

      1. Any time an abusive report goes through that is the fault of the journal editor. Most reviewers are extremely kind. I submit a ton. The one time a reviewer tried to bully me after a long RandR process, I went “Kohlhaas” on the journal editor. Burned bridged? — don’t care because it goes both ways: I no longer read, cite, or assign from that journal. BUT the sad truth: most tears over reviews (mine, my friends) reveal an inability to take constructive criticism. The presupposition to the Slate article’s thesis is that the research is meaningless anyway — prompting social sci and hard sci people to say “oh, here’s more proof the humanities are worthless.”

        Like

      2. It’s not that the humanities are useless by any measure–but there is seriously something wrong with anyone who thinks the amount of publication necessary even to get a job is in any way OK or beneficial. I mean, look at me. Multiple articles and a book and NOTHING. And there are hundreds like me. Ten years ago I would have enough to get tenure. Now, it’s not even worthy of a job. And nobody–NOBODY–reads ANY OF IT. You think people are reading your stuff? They are not. They. Are. Not. They are too busy cranking out their own, and then a peer review or two when they’re in their worst possible mood. Humanities publishing is important–but in order not to become a further cliche of itself and stay important, the expectation for how much we’re supposed to do needs to be be cut in like a fourth.

        As far as taking constructive criticism, to an extent your’e right–academics are generally pretty bad at hearing anyone tell them their genius isn’t perfect. It’s especially amusing to me now, given that every article I write comes back COVERED–and I mean CO. VERED. in changes and directives to rewrite, sometimes paragraphs, sometimes the whole article. I am expected to do it all and in a matter of hours, and I always do. (That said, I still prefer working with Slate’s most notoriously “mean” editor to acaemic peer review any day.)

        Constructive criticism is great. I never said it wasn’t. Gatekeeping for its own sake and saying “Why isn’t this what I would have written?” needs to change. Even if it’s only 10% of peer reviews that’s too many.

        Like

      3. I strongly disagree that the research is bad and unread. I read it! My students read it! Scholars cite it! Frequently, I read a gem and the author bio lists the person as off the tenure track (grad student, lecturer, recent PhD). The weird result of the cruel job market is excellent research. For many, it becomes the last piece of evidence that they were scholars. Why diminish that work? True, I mostly read the top journals (PMLA) but in my undergraduate literature classes, I assign critical articles for every text from what I’d call a “tier 2 journal.” Still, the research in these articles is exciting and exacting. I would say in the past four years we have had a renaissance in literary criticism. The excesses of theory have been shaken off and the historicism/ cultural studies turn is much more exacting in terms of evidence. I see only one major problem–not mean reader reports, but this: Research requires extensive archival work and without institutional affiliation (research money/vast libraries/subscription-only databases of primary sources) it will be near impossible to write anything that will get published. Institutional barriers to entry are excessive. No Erich Auerbach could emerge in this era. (If ethos is important: I’m a minion in this field and a dedicated reader of PKK. I earned my PhD right in the heart of the recession–thanks, Goldman Sachs. RS and I have identical CVs except I’m in Am lit.) Schuman, you talk tough, but in my heart of hearts, I know you will be a professor again one day. I hope you get the worst service assignments (wink!).

        Like

      4. Just to be abundantly clear: You believe the current status quo, in which thousands of scholars labor unfunded and unpaid for NO REASON WHATSOEVER (that research will never get them jobs, much less tenure), is all right? You think that it shouldn’t change at all, because you like the articles in PLMA? Do you understand how unbelievably miserable the lives of most adjuncts in your discipline are, just to get their basic necessities met–and yet, on top of all that, they crank out articles that nobody will read (because PMLA? like 1% of the shit in your discipline), in the vain hope that someone like you on some search committee somewhere will take mercy upon them and hire them, which you never will, because after all, if they haven’t gotten a job after this long, there must be something wrong with it. Yes, the articles in PMLA are good. They never weren’t. Some other articles might be good too. That in no way excuses the current back-breaking publication requirements that lead to nothing. It’s not just that adjuncts don’t have access to archives. It’s that adjuncts should not have to waste a cent more of their labor unpaid! If you’re really a devoted reader of this blog, then open your eyes and actually read some of the comments from people and what they are actually going through. The extent to which academics have come out of the woodwork to defend the status quo since I started writing is staggering.

        Like

  12. If you want cruel, you should have seen the review I did for a Chinese group’s paper that lifted entire paragraphs from a paper I wrote the year before. A paper they didn’t even cite, I might add. And they cited one of my other papers. And spelled my name wrong.

    Like

  13. How about, as a solution, that referee names are attached to articles almost like co-authors. Since knowledge is a collective endeavour, then adding the names of referees alongside the authors would indicate who contributed to the production of the article (you could also add editors). I know we have acknowledgements, but these are too ‘subjective’ (for want of a better word) and don’t provide any form of prestige for reviewers. Imagine how good it would feel being a referee formally attached to a well read and well regarded article. It might encourage referees to be more constructive, rather than dismissive.

    Another suggestion is that editors need to take charge more and tell referees that they cannot “reject” a paper but must offer comments to make the paper better. It is then up to the editor whether they choose to publish it or not.

    Following a suggestion by Brian Martin, I now reveal my name in reviews and refuse to review for journals that don’t let me do that. I put this at the top of my reviews:

    “Reviewer: Kean Birch

    [It is my policy as a referee to identify myself to authors for (a) transparency reasons (e.g. authors can judge my knowledge and can therefore judge the validity of my comments) and (b) for good practice reasons (e.g. it forces me to be positive in my reviews to help the authors improve their papers rather than write negative or vague comments)].”

    Like

Hello. I "value" your comment. (No, really, I do!) Please don't be a dick, though.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s