My post on “academic editing 2.0” generated some 
heat in the comments, and some misunderstandings. What I was proposing is actually really conservative, and aimed at preserving and strengthening the profession. Among the gamut of digital possibilities it would have to be counted as timid, modest and cautious. Let me try to be more clear.
Imagine you spend three years researching a topic, and send the article that results to the AHR. Instead of the editor saying “this is interesting, I’m going to send it to Profs. X, Y, and Z for review,” the editor would instead say “this is interesting, I’m going to post it on the AHR web site and solicit initial reviews from Profs X, Y, and Z.” The three reviewers would then post their responses. They could be anonymous, as now, or maybe not.
It’s exactly the same process, with all the same credentialing and standards and depth of scholarship, except that the peer review is visible. By making it visible, you broaden the discussion and you gain authority.
Right now, non-practiti0ners distrust and disregard peer review because it seems like a private club of people who only talk to each other–which is kind of what it is. If you make peer review open, you make it clear how historical argument is grounded, and how it’s revised–on other words, you make the things which make a profession a profession MORE clear. You strengthen the profession, not weaken it.
What I’m imagining here would be vetted by the editor–the editor would function like a moderator. I would open the review up to interested persons, and allow them to comment, but I’d have the editor moderate those comments, for civility and value.
The advantage to the three initial reviewers would be that their work does not go unrecognized–instead of vanishing, it would form part of an ongoing discussion/argument about meaning and evidence. It would “count” in a way that doing a peer review now does not “count,” because it’d be visible and public.
Let me stress again that in this model, the editor is still exercising discretion about what gets posted–he/she is not posted every submission, only the ones he/she likes. This is not a proposal for every single paper being posted. You still have hierarchy, in that the editor is choosing who he thinks is worthy of comments, both the initial three reviewers and the subsequent discussants.
This model preserves depth in scholarship, and it preserves peer review, and it preserves hierarchy. But it makes peer review a much more productive and fertile field.
There are lots of other possibilities. There are other forms of publication possible, other ways to imagine peer review. I’d like to offer this as a very modest and limited way of making what we do more interesting, more lively, and more rewarding.
From a student’s prospective, this sounds like a useful learning tool. One of the things I’ve realized lately is that I’m learning as much from the process of creating historical work as I am from the work itself. I would like to see an article that hasn’t yet been reviewed, how reviewers handle their part, and how the work transforms (or doesn’t) based on the process. I am currently just an outsider looking in, but it makes history seem like a much more collaborative process. (Or it makes an already collaborative process more transparent.) By the way I read your post, it sounds like this would still be peer review for a printed journal, so the prestige of having your work in the AHR, for example, would still exist? Not that I don’t advocate online journals, but it seems to me that print journals are still more prized locations in which to publish your work. Moving journals online will be a slower process to be widely accepted.
Yes–this was something I was fortunate to have access to during my graduate coursework because an instructor was a formal journal editor and a published scholar visiting campus generously agreed to share an initial submission and the comments she received from reviewers. One of the many advantages of what Mike has described is that it would make this process more transparent as a matter of course, and it would, I think, increase the value of journals and the frequency with which they were read and used in teaching. An “article” would become a broader process of scholarly inquiry that could be assigned in toto or in a variety of different smaller pieces depending on the demands of the course and the instructor’s goals.
I’ve received comments from reviewers on work that were very mean spirited, and in at least one case intentionally obstructionist because of interpretive differences. I think an open review process like this would diminish the unproductive and bad behavior of turf-protecting and insulting because most people wouldn’t act like that if their name was actually attached. I’m all for it.
It’d be cool for a journal to have both a works-in-progress review site, and an official published site. But, I’d like for both of them to have comment possibilities.
As you present it here, this model alleviates some of my initial concerns. However, there is another question which may strike post-print scholars as quaint or irrelevant, but still concerns me: What is the final or “authoritative” version of the article? Is the first version of the article posted along with the reviewers’ comments? Or is a revised version of the piece posted along with the reviewers’ comments? This is particularly an issue, I think, in history where one interpretation builds on (or rejects or modies) a previous one. But if there is no settled understanding of what is being advanced, and if the critique is instantaneous, how does that re-shape the intellectual discourse of the field?
The important issue that Rosie raises about identifying the “final” or “authoritative” model is one that’s hardly confined to history, and there are already good models for doing this in the sciences, e.g., arXiv.org. See, for example, the multiple versions and comments associated with the following two research papers:
http://arxiv.org/abs/0901.0198
http://arxiv.org/abs/0909.3853v3
You can actually witness the progress of each piece of scholarship. If we go even further by adding to arXiv.org some of the review process, what Mike’s proposing not only doesn’t undermine scholarly authority, it actually strengthens it in the sense that building authority and gaining the consensus of one’s peers would actually be exposed to a little healthy daylight.
You could certainly have a version history available, as at arXiv.org. I assume at some point the editor would have to step in and call it “done,” at least in some cases.
I’m convinced. like Sean, that this will strengthen professional authority by making it clear to skeptics, that is, members of the public who use history to argue–how academic argument takes place–what peer review actually looks like.
Imagine someone, not a professional historian, who wants to argue for the religiosity of the founding fathers. Right now the review process looks like “we all met in secret, and we decided you’re wrong.” His skepticism is fully warranted: the whole process is hidden and exclusive. Making the review process more public might make it clear that academic history is based on serious, deep research and analysis, rather than on clubby insiderism.
But what happens if I spend three years researching an article, send it to the AHR, and the editor says, “Nah, not gonna post it nor solicit reviews — it’s not interesting enough”? That would be bad. For the editor, more than for me, because I as well as every other author would immediately write in protest, and then the editor would die of e-mail, with e-mail corroding her lungs, e-mail trickling from her mouth, and e-mail seeping from her suppurating wounds.
Which is just to say that I think that some 2.0 version of a journal had best accept and post every submitted article, even the truly wacky or sloppy ones, and leave them open to open review. Though the editor could still have a selecting function, choosing to feature certain articles and to bring particular articles to the attention of particular reviewers.
I’m not convinced this is actually true (although I totally buy that description of death by email!). The only difference between what Mike is proposing and current practice is that the review is open. Decisions about what gets reviewed are the same: some submissions make it past that initial hurdle, some do not. And, speaking from my experience as an associate editor of a journal, those emails come in anyway!
Posting every submission to an open review might slow the otherwise inevitable email deluge, but I think it would cause another one: a deluge of submissions to be reviewed runs the danger of obscuring the work that is potentially publishable as well as turning off reviewers willing to participate in the process. As I said in my comments on 2.0, it’s a lot of work for a journal to get scholars willing to review this way. It’s also a lot of work for the authors, who are asked to stay on top of the continuing assessments–for that reason, Shakespeare Quarterly closes the open reviews after 6 to 8 weeks, so that authors have a chance to revise and so that comments aren’t being left unanswered.
Thanks for clarifying that, Sarah — I actually wondered after I posted whether my assumption that all submitted articles go through peer review was accurate. Having never been a journal editor, I didn’t know. Makes sense that they’re not. I guess I’d still say that a 2.0 journal should post (publish) more submissions than a print journal does, since space issues aren’t so pressing.
Dear Mike,
In the 2.0 version, you complained of a “crushingly slow, turgid process.” In this version, you promise “exactly the same process, with all the same credentialing and standards and depth of scholarship, except that the peer review is visible.”
In other words, you would have a process that is visibly, crushingly slow and turgid.
The reason that peer review can take so long is that conscientious scholars will let a manuscript sit until they can give it their full attention, which often means until the next break in teaching. Speeding it up could mean sacrificing quality or shortchanging students.
Making the process visible may indeed have some benefits, but it is not itself an answer to the speed problem.
I think the process would be more lively AND more thoughtful, because it would have to speak to someone other than the disembodied non-person it speaks to now–that is, not the fact that the original author is anonymous, but the fact that the original author is sealed away behind a blind wall.
At the moment, writing a peer review is not unlike tossing something down a well It’s true, there is something poetic and nostalgic about wells, and old oaken buckets, and sometimes well buckets pulled up interesting things. But if it were more akin to a good discussion at an oral exam and less akin to dropping something down a well, it would be no less “deep” and possibly more rewarding.
I have to admit that your objection here is not entirely wrong–I wrote this post in response to claims from Rosie that I was proposing some kind of radical, hierarchy destroying sack of the academy. It was designed to say “look, you can have the digital and the familiar.”
I still maintain that we would find the peer review process less of a chore to be put off and more of a pleasure eagerly sought if it were part of a dialogue, and also, that making that dialogue public, or semi-public, would strengthen the hand of academics in public discourse
[…] Update: Further Thoughts after hostile comments […]
[…] it clear that AmericanHistoryNow will have an editor, and the editor (me) will be choosing articles for posting, and managing the peer review process, as described here. But it will also have sections which don’t have a conventional editorial […]