The authors of this blog have been discussing what a truly open computational journal would look like for many years. It now seems likely that we will see some honest attempts in the near future. Thus, this seemed like a good time to lay out in a series of posts the key issues faced by such a journal. It is not hard to sketch out what such a journal might look like, and this has been done on numerous occasions. There is even a proof of concept, Open Research Computation. What we would like to address are very specific issues which determine whether the journal will accomplish its goal of enabling useful comparisons between different algorithms and codes, or be relegated to the dustbin of history.
In the first installment, I will discuss what I see as a key feature for a journal handling code: tools for code archiving and redistribution. I do not believe any current journal handles this in a satisfactory way, whereas there are numerous, free, useful implementations on the web. Also, I believe these are the proper tools to deal with paper source as well.
The first job of an author will be to submit both his paper and code to the journal for review. I imagine that papers will be initially screened, and only papers that go to review will have code submissions, but this is a small point. I imagine that each paper would be assigned a repository into which could be deposited both a paper and source code. This organization has many advantages:
- Administration using SSH public keys
- Easy updating by authors
- Full record of changes by both editors and authors
- Reviewers could place comments directly in the source using a branch
- Easy anonymous retrieval of source code by readers
We could, of course, use two repositories to prevent retrieval of the paper directly, but this seems superfluous for an open access journal. I think the advantage for reviewer comments is especially nice. The todonotes package for LaTeX could be used to directly insert reviewer comments without the cumbersome intermediary of email listings with page numbers, etc. Several other PDF markup technologies would also apply here.
In order to proceed with this organization, the journal must have a few tools in place. Some version control must be in place, and for this purpose distributed version control would be necessary I think. I recommend the usual suspects: Git, Mercurial, and Bazaar. Moreover, some key management architecture must be in place. This can be as simple as using hg-wrapper.sh in the SSH authorized_keys files, to the sophisticated schemes used by hosting sites. In fact, all these services are currently provided, free of charge, on sites such as BitBucket, Git Hub, Launchpad, and Google Code. One can easily imagine a mashup of the arXiv and a hosting site which could accomplish exactly this. Without these code hosting capabilities, I do not believe any journal could truly facilitate open computational research. Moreover, with them I think the process of reviewing and updating work would become vastly more efficient.
Upcoming posts in this series will deal with:
- Transparency of journal finances
- Persistent anonymization of reviews and comments
- A tiered pricing model for submissions